Python advanced, part 3 of 5
Python Threads and Processes
A normal Python program does one thing at a time, from the first line to the last. Threads and processes let it work on several things in the same period of time. This page explains the ideas in plain words first, then shows the tools, and spends most of its time on the hard part: what goes wrong when two pieces of code touch the same data. The running example is a small bank whose customers are the Agrawani family: Rishikesh, Sandhya, Subhash, Malinikesh and Hemkesh.
The ideas
Part 1 of 5 · the words you need before any code
Most confusion about this topic comes from a handful of words that sound alike. This part defines each one with an everyday picture, and explains the one rule that decides which tool you should pick.
- Concurrency and parallelism: one cook switching dishes, or several cooks
- Waiting work and thinking work: I/O-bound and CPU-bound
- The three tools: threading, multiprocessing, asyncio
- The GIL: why Python threads take turns
Concurrency and parallelism
Start with four words.
- A CPU core is the part of the computer that actually carries out instructions. One core can do one thing at one moment. A laptop today usually has 4 to 16 cores.
- A process is one running program. It has its own private memory. When you type
python3 bank.py, the operating system starts one process. - A thread is one line of work inside a process. Every process starts with one thread, called the main thread. It can start more. All threads of a process share the same memory, so they see the same variables.
- A task here just means one job you want done, such as "fetch Sandhya's statement".
Now the two big words. Concurrency means several tasks are in progress during the same period. They may take turns. Parallelism means several tasks are running at the very same instant, which needs more than one core.
Picture a kitchen. One cook making three dishes puts rice on to boil, chops vegetables while it boils, then stirs the dal. Only one pair of hands, but three dishes are in progress. That is concurrency. Three cooks each making one dish at the same moment is parallelism.
| Concurrency | Parallelism | |
|---|---|---|
| Meaning | Dealing with many tasks in the same period | Doing many tasks at the same instant |
| Kitchen | One cook switching between dishes | Several cooks, one dish each |
| Bank | One teller who serves the next customer while the first fills in a form | Several counters open at once |
| Cores needed | One is enough | Two or more |
| Helps when | Tasks spend most of their time waiting | Tasks spend most of their time calculating |
| In Python | threading, asyncio | multiprocessing |
Concept: Parallelism is one way to get concurrency, but not the only way. A single cook is concurrent without being parallel.
Waiting work and thinking work
I/O means input and output: reading a file, asking a database, calling a web service. During I/O the program mostly waits for something outside it to answer. A task that spends most of its time like this is I/O-bound. The cook waiting for rice to boil is I/O-bound.
A task that spends most of its time calculating is CPU-bound: adding up a million transactions, resizing an image, checking a password hash. The cook chopping onions is CPU-bound. The hands are busy the whole time.
This one question decides everything else on this page. A waiting task can step aside and let another task use the core, so one core is enough and threads work well. A calculating task never steps aside, so the only way to go faster is more cores, which in Python means more processes.
Remember: On this page time.sleep() always stands in for slow I/O. It behaves the same way as a real network call: the thread waits and uses no CPU.
The three tools
The standard library has three ways to run tasks concurrently. This page covers the first two. The third has its own page.
threading | multiprocessing | asyncio | |
|---|---|---|---|
| Unit of work | Thread | Process | Coroutine (a function that can pause itself) |
| Who switches tasks | The operating system, at any moment | The operating system, at any moment | Your code, only at await |
| Runs Python code in parallel | No (because of the GIL, next chapter) | Yes | No, it uses one thread |
| Memory | Shared | Separate for each process | Shared |
| Cost of one more task | Small | Large | Tiny |
| Best for | I/O-bound work, tens of tasks | CPU-bound work | I/O-bound work, thousands of tasks |
| Main danger | Race conditions on shared data | Slow start, data must be copied | One slow call blocks everything |
The GIL
CPython is the standard Python that you download from python.org. Inside it there is one lock called the GIL, the Global Interpreter Lock. A thread must hold the GIL to run Python code, and only one thread can hold it at a time. Think of a kitchen with several cooks but a single knife: whoever holds the knife works, the others wait.
Why it exists. CPython keeps a counter on every object to know when the object can be deleted. If two threads changed such a counter at the same instant, it could end up wrong and crash the interpreter. One big lock protects all of that. It keeps the interpreter simple and keeps single-threaded programs fast.
What it means for you:
- Threads do not run Python code in parallel. Two threads doing arithmetic take turns on one core, so CPU-bound work does not get faster with threads.
- A thread gives up the GIL whenever it waits: during
time.sleep(), file and network I/O, and many slow calls into C code. That is why threads do speed up I/O-bound work. - A running thread is also asked to give up the GIL every few milliseconds, so that others get a turn. This can happen between any two small steps of your code.
- The GIL does not make your code thread-safe. It protects Python's own internals, not your data. The whole third part of this page is about that.
The next block counts down twice, first one after the other, then in two threads. It is marked as not runnable here because timings differ on every machine. The output shown is from one run on a 10-core laptop with standard Python 3.9.
Two threads were no faster, even though nine other cores were free.
Note: Python 3.13 and later can be installed as an optional "free-threaded" build that has no GIL. It is not the default build. Everything on this page describes standard CPython with the GIL.
Threads
Part 2 of 5 · several lines of work in one program
A thread is cheap to start and shares the program's variables. Use threads when tasks spend their time waiting. In this part no two threads change the same data, so nothing can go wrong yet.
- Starting a thread:
start(),join()and arguments - Waiting at the same time: three slow calls in the time of one
- Daemon threads: background threads that do not keep the program alive
- Getting results back:
ThreadPoolExecutor - Thread-local data: one private copy for each thread
Starting a thread
Create a threading.Thread, tell it which function to run with target, and call start(). The function now runs in a new thread while the main thread carries on. join() makes the main thread wait until that thread has finished.
Mistake to avoid: target=print_statement("Sandhya") calls the function straight away in the main thread and hands its return value, None, to the thread. Pass the function and its arguments separately. Also note the comma in ("Sandhya",): without it the brackets do not make a tuple.
Waiting at the same time
Three statements take 0.3, 0.2 and 0.1 seconds to fetch. One after the other that is 0.6 seconds. In three threads the waits overlap, so the total is about the longest single wait.
The shortest wait finished first, whatever order the threads were started in. The exact time changes a little on every run, which is why the code rounds it.
Mistake to avoid: Calling start() and join() in the same loop runs the threads one after the other: each one is started and then waited for before the next begins. Start them all, then join them all.
Daemon threads
A Python program ends only when all its normal threads have finished. A daemon thread is a background helper that does not count: when the main thread and all other normal threads are done, the program exits and daemon threads are simply stopped.
The program ends here. With daemon=False, the default, it would never end.
Careful: A daemon thread is cut off wherever it happens to be, with no clean-up. Do not use one for work that must finish, such as writing a file.
Getting results back
A thread's target function cannot hand its return value back to you. The easy answer is a pool: a fixed group of worker threads that you give jobs to. ThreadPoolExecutor.map() works like the built-in map(), runs the calls in the pool, and returns the results in the same order as the inputs.
Five calls of 0.1 seconds finished in well under 0.5 seconds. Leaving the with block waits for all the workers, so no join() is needed.
submit and as_completed
submit() schedules one call and returns a future straight away. A future is a ticket for a result that is not ready yet. future.result() waits for the value. as_completed() gives you the futures as each one finishes, fastest first.
Concept: An exception inside a worker does not crash the program or print anything. It is stored in the future and raised again when you call result(). If you never ask for the result, you never see the error.
Thread-local data
Sometimes each thread needs its own value under the same name, for example "the customer I am serving now". A threading.local() object gives every thread a private set of attributes.
With an ordinary global variable, all three tellers would have seen the last name that was written.
Race conditions and locks
Part 3 of 5 · when threads share data
Threads share memory. That is what makes them convenient, and it is where almost every threading bug comes from. A shared resource is anything more than one thread can reach: a global variable, an object, a file. A race condition is a bug where the result depends on which thread happens to get there first. It is the worst kind of bug, because the program usually works and fails only now and then.
- Check, then act: two withdrawals from one account
- The lost update: why
balance += 1is three steps - Fixing it with a lock: one thread at a time
- Critical sections: lock as little as possible
- RLock: a lock the same thread can take twice
- Deadlock: two threads waiting for each other for ever
- Semaphore: at most N at a time
- Event and Condition: waiting for a signal
- Queue: handing work over safely
- What is thread-safe: a short list
Check, then act
Sandhya and Subhash share one account with 1000 in it. Both try to withdraw 800 at the same moment, from two different counters. The code checks the balance first, so it looks safe.
Both were paid and the account is 600 overdrawn. Each thread did its check while the balance was still 1000, and each then acted on an answer that was no longer true. The order of events, called the interleaving, was:
| Time | Sandhya's thread | Subhash's thread | balance |
|---|---|---|---|
| 1 | checks: 1000 >= 800, yes | 1000 | |
| 2 | counting notes | checks: 1000 >= 800, yes | 1000 |
| 3 | subtracts 800 | counting notes | 200 |
| 4 | subtracts 800 | -600 |
Honest note: The time.sleep(0.05) between the check and the update is there to make the bug appear on every run. Without it the gap is a few millionths of a second and the bug shows up perhaps once in many thousand runs. That is not safer. It is the same bug, harder to find.
The lost update
There is a race even with no if. The line balance += 1 looks like one action, but the computer does it in three steps. This pattern is called read-modify-write.
| Step | What happens | Written out |
|---|---|---|
| 1. Read | Fetch the current value of balance | current = balance |
| 2. Modify | Add 1 to that private copy | new = current + 1 |
| 3. Write | Store the result back in balance | balance = new |
If a second thread reads between another thread's step 1 and step 3, both start from the same old value and one deposit is lost. The block below writes the three steps out and puts a short sleep in the middle, so that all five threads read before any of them writes.
Five deposits, and the balance is 1. All five threads read 0 and all five wrote 1.
The real one-line version is below. It is marked as not runnable because its result varies: it depends on the Python version, the machine and luck. On the Python 3.9 used to check this page it printed a different wrong number on almost every run. On Python 3.10 and later, a simple loop like this one often prints the right answer, because the interpreter switches threads at fewer points. The line is still not safe there. In a larger program, where the switch points fall differently, updates can still be lost.
Fixing it with a lock
A lock is an object that only one thread can hold at a time. Its other name is mutex, short for mutual exclusion: if I am in, you are out. Picture a cabin at the bank with one key. Whoever has the key goes in. Everyone else waits at the door until the key comes back.
with lock: takes the key at the top of the block and gives it back at the bottom, even if an exception is raised inside.
The check and the update now happen as one unit. The second thread cannot even look at the balance until the first has finished changing it. Which of the two gets there first is still up to the operating system, but either way exactly one is paid.
The same lock repairs the lost update:
Remember: A lock only works if every piece of code that touches the data uses the same lock. Python does not connect the lock to the variable. One function that forgets the lock brings the race back. You can also call lock.acquire() and lock.release() yourself, but with is safer because it can never forget to release.
Critical sections
A critical section is a piece of code that must be run by only one thread at a time, because it uses a shared resource. In the last chapter it was the lines inside with lock:.
While one thread is inside, every other thread that wants the lock stands still. So keep the critical section small: lock the shared read and write, and do slow work outside. Below, four tellers each print a receipt (0.1 seconds) and add 100 to the day's total.
Both versions are correct. The first one makes the four tellers queue up, so the threads gave no speed-up at all.
Rule of thumb: Inside a lock, do only what must not be interrupted. Never wait for the network, a file or a person while holding one.
RLock
A plain Lock does not know who holds it. If the thread that already holds it asks for it again, that thread waits for itself for ever. The block shows this safely by giving up after 0.1 seconds.
An RLock, a re-entrant lock, remembers its owner and lets that same thread take it again. Other threads still have to wait. You need it when one locked method calls another locked method on the same object:
With threading.Lock() in place of RLock(), add_interest() would hang on the call to deposit() and never return.
Deadlock
A deadlock is when threads wait for each other and none can ever continue. The classic case needs two locks. Each account has its own lock, and a transfer must hold both. Rishikesh sends money to Hemkesh at the same moment as Hemkesh sends money to Rishikesh.
This block is marked as not runnable because it hangs for ever.
Thread 1 holds Rishikesh's lock and waits for Hemkesh's. Thread 2 holds Hemkesh's lock and waits for Rishikesh's. Neither will let go. It is two people in a narrow corridor, each waiting for the other to step back.
The fix is simple: always take locks in the same fixed order. Here, the lower account number first, whichever way the money flows. Then two threads can never hold one lock each.
Rule of thumb: Code that holds one lock at a time cannot deadlock in this way. If you need two, decide the order once and follow it everywhere.
Semaphore
A lock lets one thread in. A semaphore lets up to N in. It is a counter of free places: taking a place lowers it, leaving raises it, and at zero you wait. The bank has two tellers and five customers arrive together.
Never more than two customers were at a counter. The usual real use is the same idea: at most N open connections to a database or a web service.
Note: The semaphore limits how many threads are inside, but two threads are still inside together. The shared counters therefore still need their own lock.
Event and Condition
An Event is a flag that threads can wait for. It starts off. wait() blocks until some other thread calls set(). Here the customers wait until the doors open.
A Condition is a lock plus a way to wait until the shared data reaches some state. Below, a withdrawal of 500 waits until enough money has been deposited. wait_for() gives up the lock while it waits and takes it back before it returns. notify_all() tells the waiting threads to look again.
In practice: You will rarely write a Condition yourself. The queue in the next chapter is built from one, and it covers most needs.
Queue
The safest way for threads to work together is not to share data at all, but to pass it along. queue.Queue is a line of items with all the locking already done inside. One thread puts work in, this is the producer. Another takes work out, this is the consumer. get() waits when the queue is empty.
To tell the consumer there is no more work, the producer puts in a special value called a sentinel. None is the usual choice.
No lock appears in this code. The balances are changed by one thread only, and the queue itself is safe to use from many threads. Items come out in the order they went in.
Note: With several consumers, put one sentinel in the queue for each of them, otherwise some will wait for ever. And if several consumers change the same data, that data needs a lock again.
What is thread-safe
Code is thread-safe when it gives correct results however the threads happen to interleave. A short guide:
| Safe to use from several threads | Not safe without a lock |
|---|---|
queue.Queue: put() and get() | x += 1, x = x + 1, d[key] += 1 |
Lock, RLock, Semaphore, Event, Condition | Check then act: if balance >= amount: balance -= amount |
One single list.append(x) or d[key] = value in CPython | if key not in d: d[key] = value |
| Reading data that no thread changes | Two or more updates that must stay in step, such as both sides of a transfer |
| Local variables inside a function (each thread has its own) | Looping over a list or dict while another thread changes it |
The logging module | Several print() calls that must appear together as one block |
The third row of the left column is why the examples on this page could use finished.append(owner) without a lock. It is a property of CPython, not a promise of the language, and it covers one single operation only. As soon as you read a value and then write based on it, you need a lock.
Rule of thumb: If two threads can reach the same data and at least one of them changes it, protect it with a lock or pass it through a queue.
Processes
Part 4 of 5 · real parallelism on several cores
Threads take turns because they share one interpreter and one GIL. A separate process is a whole separate Python: its own interpreter, its own GIL and its own memory. Four processes can calculate on four cores at the same instant. This is the several-cooks kitchen, with each cook in their own kitchen.
- Starting a process:
Processand the__main__guard - A pool of processes: CPU-bound work on every core
- Nothing is shared: a global that does not change
- Sharing on purpose:
Value,Queue,Manager - What processes cost: start-up and pickling
Starting a process
multiprocessing.Process is used just like threading.Thread: target, args, start(), join(). Two things are different. The target must be a function defined at the top level of the file, and the code that starts processes must sit under if __name__ == "__main__":.
Why the guard is required
There are different ways to start a child process. On macOS and Windows, Python uses spawn: it starts a brand-new Python and that new Python imports your file to find the target function. Importing a file runs all its top-level code. Without the guard, the child would reach p.start() again and try to start its own child. Python stops this with a RuntimeError.
Inside the child, __name__ is not "__main__", so everything under the guard is skipped there. You can see the second import happen:
That is the output with spawn. On Linux the long-standing default is fork, which copies the running parent instead of importing the file, so the first line appears once. Write the guard anyway. It is harmless with fork, and your code then works on every system.
Remember: Every block in this part has the guard and top-level functions. Save the code in a .py file and run that file. Functions typed straight into the interactive >>> prompt cannot be found by a spawned child.
A pool of processes
For CPU-bound work you usually want "run this function on every item, using all my cores". ProcessPoolExecutor has the same map() and submit() as the thread pool, but its workers are processes. Here each account gets a checksum, which is pure arithmetic.
Leave out max_workers and the pool uses one worker for each core of the machine. The workload here is kept tiny so that the page's examples run quickly. With so little work, the pool is slower than a plain loop. Processes pay off when each call takes a good fraction of a second or more.
multiprocessing.Pool
The older multiprocessing.Pool does the same job and you will meet it in existing code. This one works out a year's interest at 4 percent.
Sharing on purpose
multiprocessing has three main ways to share. Each one must be created in the parent and passed to the children as an argument.
Value: one shared number
Value("i", 0) is one integer kept in memory that all the processes can see. It is truly shared, so the lost update is back, and this time the processes really do run at the same instant. A Value carries its own lock. Use it.
Queue: passing messages
multiprocessing.Queue works like queue.Queue, but between processes. Children put results in, the parent takes them out.
Manager: shared dicts and lists
A Manager starts one more process that holds real Python objects. The other processes get stand-ins that send every operation to it. It is the most flexible of the three and the slowest.
Careful: Shared data between processes has the same races as shared data between threads. accounts[owner] += 100 on a manager dict is still read, modify, write. Guard it with a lock from manager.Lock(), or better, let each worker return its result and combine the results in the parent.
What processes cost
Parallelism is not free. Two costs matter.
Start-up. Starting a process means starting a new Python interpreter and, with spawn, importing your file again. That takes far longer than starting a thread.
Pickling. Processes share no memory, so every argument and every return value is turned into bytes, sent across, and rebuilt on the other side. Python does this with the pickle module. Sending a large list to a worker copies the whole list. And some things cannot be pickled at all, such as a lambda, an open file or a lock from threading.
A top-level function is pickled as just its name, which the child then looks up after importing the file. That is why targets must be defined at the top level.
Rule of thumb: Use processes for a small number of heavy jobs. Send little data in, get little data back, and reuse a pool so the workers are started only once.
Choosing
Part 5 of 5 · which tool for which job
The choice comes back to the question from the first part: does the task wait, or does it calculate?
- Which one to use: a decision table
- One API for both pools:
concurrent.futures - Common mistakes: a checklist
Which one to use
| Your work | Use | Why |
|---|---|---|
| Fast enough already | Nothing. A plain loop | Concurrent code is harder to write, read and debug |
| I/O-bound, a few tasks to a few hundred | Threads, through ThreadPoolExecutor | The GIL is released while waiting. Works with ordinary blocking code |
| I/O-bound, thousands of tasks | asyncio | A waiting coroutine costs far less memory than a waiting thread |
| CPU-bound | Processes, through ProcessPoolExecutor | Each process has its own GIL, so all cores are used |
| CPU-bound, but each call is very short | A plain loop | Start-up and pickling cost more than the work |
| A mix of both | Threads or asyncio for the waiting, a process pool for the calculating | Each part gets the tool that suits it |
Rule of thumb: Waiting: threads. Waiting on a very large scale: asyncio. Calculating: processes.
One API for both pools
concurrent.futures gives threads and processes the same interface: map(), submit(), futures, as_completed(). You can write the code once and switch between the two by changing one name. For new code it is the best place to start.
Start with whichever pool matches your work. If you guessed wrong, the change is one word.
Common mistakes
| Mistake | What you see | Fix |
|---|---|---|
| Sharing data without a lock | Wrong totals, now and then, never when you are watching | One lock around every read and write of that data, or a queue |
Forgetting join() | The main thread reads results before the workers have finished | Join every thread, or use a pool in a with block |
| CPU-bound work in threads | No speed-up, sometimes slower | Use processes |
Forgetting the __main__ guard | RuntimeError on macOS and Windows | Put the starting code under if __name__ == "__main__": |
| Creating too many threads | Memory use climbs, everything slows down, the server you call refuses you | A pool with a fixed max_workers, or a semaphore |
| Slow work inside a lock | Threads queue up, no speed-up | Keep the critical section small |
| Taking two locks in different orders | The program hangs | One fixed order everywhere |
target=work() with brackets | The work runs in the main thread, the new thread does nothing | target=work, args=(...) |
Never calling future.result() | Errors in workers vanish silently | Ask every future for its result |
Threads and processes are run by the operating system, which may switch between them at any moment. The other way to handle many waiting tasks is to let the code itself decide when to switch, and that is the subject of the next page: Python Async IO.