Learnings

Python advanced, part 3 of 5

Python Threads and Processes

A normal Python program does one thing at a time, from the first line to the last. Threads and processes let it work on several things in the same period of time. This page explains the ideas in plain words first, then shows the tools, and spends most of its time on the hard part: what goes wrong when two pieces of code touch the same data. The running example is a small bank whose customers are the Agrawani family: Rishikesh, Sandhya, Subhash, Malinikesh and Hemkesh.

The ideas

Part 1 of 5 · the words you need before any code

Most confusion about this topic comes from a handful of words that sound alike. This part defines each one with an everyday picture, and explains the one rule that decides which tool you should pick.

Concurrency and parallelism

Start with four words.

  • A CPU core is the part of the computer that actually carries out instructions. One core can do one thing at one moment. A laptop today usually has 4 to 16 cores.
  • A process is one running program. It has its own private memory. When you type python3 bank.py, the operating system starts one process.
  • A thread is one line of work inside a process. Every process starts with one thread, called the main thread. It can start more. All threads of a process share the same memory, so they see the same variables.
  • A task here just means one job you want done, such as "fetch Sandhya's statement".

Now the two big words. Concurrency means several tasks are in progress during the same period. They may take turns. Parallelism means several tasks are running at the very same instant, which needs more than one core.

Picture a kitchen. One cook making three dishes puts rice on to boil, chops vegetables while it boils, then stirs the dal. Only one pair of hands, but three dishes are in progress. That is concurrency. Three cooks each making one dish at the same moment is parallelism.

ConcurrencyParallelism
MeaningDealing with many tasks in the same periodDoing many tasks at the same instant
KitchenOne cook switching between dishesSeveral cooks, one dish each
BankOne teller who serves the next customer while the first fills in a formSeveral counters open at once
Cores neededOne is enoughTwo or more
Helps whenTasks spend most of their time waitingTasks spend most of their time calculating
In Pythonthreading, asynciomultiprocessing

Concept: Parallelism is one way to get concurrency, but not the only way. A single cook is concurrent without being parallel.

Waiting work and thinking work

I/O means input and output: reading a file, asking a database, calling a web service. During I/O the program mostly waits for something outside it to answer. A task that spends most of its time like this is I/O-bound. The cook waiting for rice to boil is I/O-bound.

A task that spends most of its time calculating is CPU-bound: adding up a million transactions, resizing an image, checking a password hash. The cook chopping onions is CPU-bound. The hands are busy the whole time.

This one question decides everything else on this page. A waiting task can step aside and let another task use the core, so one core is enough and threads work well. A calculating task never steps aside, so the only way to go faster is more cores, which in Python means more processes.

Remember: On this page time.sleep() always stands in for slow I/O. It behaves the same way as a real network call: the thread waits and uses no CPU.

The three tools

The standard library has three ways to run tasks concurrently. This page covers the first two. The third has its own page.

threadingmultiprocessingasyncio
Unit of workThreadProcessCoroutine (a function that can pause itself)
Who switches tasksThe operating system, at any momentThe operating system, at any momentYour code, only at await
Runs Python code in parallelNo (because of the GIL, next chapter)YesNo, it uses one thread
MemorySharedSeparate for each processShared
Cost of one more taskSmallLargeTiny
Best forI/O-bound work, tens of tasksCPU-bound workI/O-bound work, thousands of tasks
Main dangerRace conditions on shared dataSlow start, data must be copiedOne slow call blocks everything

The GIL

CPython is the standard Python that you download from python.org. Inside it there is one lock called the GIL, the Global Interpreter Lock. A thread must hold the GIL to run Python code, and only one thread can hold it at a time. Think of a kitchen with several cooks but a single knife: whoever holds the knife works, the others wait.

Why it exists. CPython keeps a counter on every object to know when the object can be deleted. If two threads changed such a counter at the same instant, it could end up wrong and crash the interpreter. One big lock protects all of that. It keeps the interpreter simple and keeps single-threaded programs fast.

What it means for you:

  • Threads do not run Python code in parallel. Two threads doing arithmetic take turns on one core, so CPU-bound work does not get faster with threads.
  • A thread gives up the GIL whenever it waits: during time.sleep(), file and network I/O, and many slow calls into C code. That is why threads do speed up I/O-bound work.
  • A running thread is also asked to give up the GIL every few milliseconds, so that others get a turn. This can happen between any two small steps of your code.
  • The GIL does not make your code thread-safe. It protects Python's own internals, not your data. The whole third part of this page is about that.

The next block counts down twice, first one after the other, then in two threads. It is marked as not runnable here because timings differ on every machine. The output shown is from one run on a 10-core laptop with standard Python 3.9.

Two threads were no faster, even though nine other cores were free.

Note: Python 3.13 and later can be installed as an optional "free-threaded" build that has no GIL. It is not the default build. Everything on this page describes standard CPython with the GIL.

Threads

Part 2 of 5 · several lines of work in one program

A thread is cheap to start and shares the program's variables. Use threads when tasks spend their time waiting. In this part no two threads change the same data, so nothing can go wrong yet.

Starting a thread

Create a threading.Thread, tell it which function to run with target, and call start(). The function now runs in a new thread while the main thread carries on. join() makes the main thread wait until that thread has finished.

Mistake to avoid: target=print_statement("Sandhya") calls the function straight away in the main thread and hands its return value, None, to the thread. Pass the function and its arguments separately. Also note the comma in ("Sandhya",): without it the brackets do not make a tuple.

Waiting at the same time

Three statements take 0.3, 0.2 and 0.1 seconds to fetch. One after the other that is 0.6 seconds. In three threads the waits overlap, so the total is about the longest single wait.

The shortest wait finished first, whatever order the threads were started in. The exact time changes a little on every run, which is why the code rounds it.

Mistake to avoid: Calling start() and join() in the same loop runs the threads one after the other: each one is started and then waited for before the next begins. Start them all, then join them all.

Daemon threads

A Python program ends only when all its normal threads have finished. A daemon thread is a background helper that does not count: when the main thread and all other normal threads are done, the program exits and daemon threads are simply stopped.

The program ends here. With daemon=False, the default, it would never end.

Careful: A daemon thread is cut off wherever it happens to be, with no clean-up. Do not use one for work that must finish, such as writing a file.

Getting results back

A thread's target function cannot hand its return value back to you. The easy answer is a pool: a fixed group of worker threads that you give jobs to. ThreadPoolExecutor.map() works like the built-in map(), runs the calls in the pool, and returns the results in the same order as the inputs.

Five calls of 0.1 seconds finished in well under 0.5 seconds. Leaving the with block waits for all the workers, so no join() is needed.

submit and as_completed

submit() schedules one call and returns a future straight away. A future is a ticket for a result that is not ready yet. future.result() waits for the value. as_completed() gives you the futures as each one finishes, fastest first.

Concept: An exception inside a worker does not crash the program or print anything. It is stored in the future and raised again when you call result(). If you never ask for the result, you never see the error.

Thread-local data

Sometimes each thread needs its own value under the same name, for example "the customer I am serving now". A threading.local() object gives every thread a private set of attributes.

With an ordinary global variable, all three tellers would have seen the last name that was written.

Race conditions and locks

Part 3 of 5 · when threads share data

Threads share memory. That is what makes them convenient, and it is where almost every threading bug comes from. A shared resource is anything more than one thread can reach: a global variable, an object, a file. A race condition is a bug where the result depends on which thread happens to get there first. It is the worst kind of bug, because the program usually works and fails only now and then.

Check, then act

Sandhya and Subhash share one account with 1000 in it. Both try to withdraw 800 at the same moment, from two different counters. The code checks the balance first, so it looks safe.

Both were paid and the account is 600 overdrawn. Each thread did its check while the balance was still 1000, and each then acted on an answer that was no longer true. The order of events, called the interleaving, was:

TimeSandhya's threadSubhash's threadbalance
1checks: 1000 >= 800, yes1000
2counting noteschecks: 1000 >= 800, yes1000
3subtracts 800counting notes200
4subtracts 800-600

Honest note: The time.sleep(0.05) between the check and the update is there to make the bug appear on every run. Without it the gap is a few millionths of a second and the bug shows up perhaps once in many thousand runs. That is not safer. It is the same bug, harder to find.

The lost update

There is a race even with no if. The line balance += 1 looks like one action, but the computer does it in three steps. This pattern is called read-modify-write.

StepWhat happensWritten out
1. ReadFetch the current value of balancecurrent = balance
2. ModifyAdd 1 to that private copynew = current + 1
3. WriteStore the result back in balancebalance = new

If a second thread reads between another thread's step 1 and step 3, both start from the same old value and one deposit is lost. The block below writes the three steps out and puts a short sleep in the middle, so that all five threads read before any of them writes.

Five deposits, and the balance is 1. All five threads read 0 and all five wrote 1.

The real one-line version is below. It is marked as not runnable because its result varies: it depends on the Python version, the machine and luck. On the Python 3.9 used to check this page it printed a different wrong number on almost every run. On Python 3.10 and later, a simple loop like this one often prints the right answer, because the interpreter switches threads at fewer points. The line is still not safe there. In a larger program, where the switch points fall differently, updates can still be lost.

Fixing it with a lock

A lock is an object that only one thread can hold at a time. Its other name is mutex, short for mutual exclusion: if I am in, you are out. Picture a cabin at the bank with one key. Whoever has the key goes in. Everyone else waits at the door until the key comes back.

with lock: takes the key at the top of the block and gives it back at the bottom, even if an exception is raised inside.

The check and the update now happen as one unit. The second thread cannot even look at the balance until the first has finished changing it. Which of the two gets there first is still up to the operating system, but either way exactly one is paid.

The same lock repairs the lost update:

Remember: A lock only works if every piece of code that touches the data uses the same lock. Python does not connect the lock to the variable. One function that forgets the lock brings the race back. You can also call lock.acquire() and lock.release() yourself, but with is safer because it can never forget to release.

Critical sections

A critical section is a piece of code that must be run by only one thread at a time, because it uses a shared resource. In the last chapter it was the lines inside with lock:.

While one thread is inside, every other thread that wants the lock stands still. So keep the critical section small: lock the shared read and write, and do slow work outside. Below, four tellers each print a receipt (0.1 seconds) and add 100 to the day's total.

Both versions are correct. The first one makes the four tellers queue up, so the threads gave no speed-up at all.

Rule of thumb: Inside a lock, do only what must not be interrupted. Never wait for the network, a file or a person while holding one.

RLock

A plain Lock does not know who holds it. If the thread that already holds it asks for it again, that thread waits for itself for ever. The block shows this safely by giving up after 0.1 seconds.

An RLock, a re-entrant lock, remembers its owner and lets that same thread take it again. Other threads still have to wait. You need it when one locked method calls another locked method on the same object:

With threading.Lock() in place of RLock(), add_interest() would hang on the call to deposit() and never return.

Deadlock

A deadlock is when threads wait for each other and none can ever continue. The classic case needs two locks. Each account has its own lock, and a transfer must hold both. Rishikesh sends money to Hemkesh at the same moment as Hemkesh sends money to Rishikesh.

This block is marked as not runnable because it hangs for ever.

Thread 1 holds Rishikesh's lock and waits for Hemkesh's. Thread 2 holds Hemkesh's lock and waits for Rishikesh's. Neither will let go. It is two people in a narrow corridor, each waiting for the other to step back.

The fix is simple: always take locks in the same fixed order. Here, the lower account number first, whichever way the money flows. Then two threads can never hold one lock each.

Rule of thumb: Code that holds one lock at a time cannot deadlock in this way. If you need two, decide the order once and follow it everywhere.

Semaphore

A lock lets one thread in. A semaphore lets up to N in. It is a counter of free places: taking a place lowers it, leaving raises it, and at zero you wait. The bank has two tellers and five customers arrive together.

Never more than two customers were at a counter. The usual real use is the same idea: at most N open connections to a database or a web service.

Note: The semaphore limits how many threads are inside, but two threads are still inside together. The shared counters therefore still need their own lock.

Event and Condition

An Event is a flag that threads can wait for. It starts off. wait() blocks until some other thread calls set(). Here the customers wait until the doors open.

A Condition is a lock plus a way to wait until the shared data reaches some state. Below, a withdrawal of 500 waits until enough money has been deposited. wait_for() gives up the lock while it waits and takes it back before it returns. notify_all() tells the waiting threads to look again.

In practice: You will rarely write a Condition yourself. The queue in the next chapter is built from one, and it covers most needs.

Queue

The safest way for threads to work together is not to share data at all, but to pass it along. queue.Queue is a line of items with all the locking already done inside. One thread puts work in, this is the producer. Another takes work out, this is the consumer. get() waits when the queue is empty.

To tell the consumer there is no more work, the producer puts in a special value called a sentinel. None is the usual choice.

No lock appears in this code. The balances are changed by one thread only, and the queue itself is safe to use from many threads. Items come out in the order they went in.

Note: With several consumers, put one sentinel in the queue for each of them, otherwise some will wait for ever. And if several consumers change the same data, that data needs a lock again.

What is thread-safe

Code is thread-safe when it gives correct results however the threads happen to interleave. A short guide:

Safe to use from several threadsNot safe without a lock
queue.Queue: put() and get()x += 1, x = x + 1, d[key] += 1
Lock, RLock, Semaphore, Event, ConditionCheck then act: if balance >= amount: balance -= amount
One single list.append(x) or d[key] = value in CPythonif key not in d: d[key] = value
Reading data that no thread changesTwo or more updates that must stay in step, such as both sides of a transfer
Local variables inside a function (each thread has its own)Looping over a list or dict while another thread changes it
The logging moduleSeveral print() calls that must appear together as one block

The third row of the left column is why the examples on this page could use finished.append(owner) without a lock. It is a property of CPython, not a promise of the language, and it covers one single operation only. As soon as you read a value and then write based on it, you need a lock.

Rule of thumb: If two threads can reach the same data and at least one of them changes it, protect it with a lock or pass it through a queue.

Processes

Part 4 of 5 · real parallelism on several cores

Threads take turns because they share one interpreter and one GIL. A separate process is a whole separate Python: its own interpreter, its own GIL and its own memory. Four processes can calculate on four cores at the same instant. This is the several-cooks kitchen, with each cook in their own kitchen.

Starting a process

multiprocessing.Process is used just like threading.Thread: target, args, start(), join(). Two things are different. The target must be a function defined at the top level of the file, and the code that starts processes must sit under if __name__ == "__main__":.

Why the guard is required

There are different ways to start a child process. On macOS and Windows, Python uses spawn: it starts a brand-new Python and that new Python imports your file to find the target function. Importing a file runs all its top-level code. Without the guard, the child would reach p.start() again and try to start its own child. Python stops this with a RuntimeError.

Inside the child, __name__ is not "__main__", so everything under the guard is skipped there. You can see the second import happen:

That is the output with spawn. On Linux the long-standing default is fork, which copies the running parent instead of importing the file, so the first line appears once. Write the guard anyway. It is harmless with fork, and your code then works on every system.

Remember: Every block in this part has the guard and top-level functions. Save the code in a .py file and run that file. Functions typed straight into the interactive >>> prompt cannot be found by a spawned child.

A pool of processes

For CPU-bound work you usually want "run this function on every item, using all my cores". ProcessPoolExecutor has the same map() and submit() as the thread pool, but its workers are processes. Here each account gets a checksum, which is pure arithmetic.

Leave out max_workers and the pool uses one worker for each core of the machine. The workload here is kept tiny so that the page's examples run quickly. With so little work, the pool is slower than a plain loop. Processes pay off when each call takes a good fraction of a second or more.

multiprocessing.Pool

The older multiprocessing.Pool does the same job and you will meet it in existing code. This one works out a year's interest at 4 percent.

Nothing is shared

Each process has its own memory. A child gets its own copy of the module's variables, and whatever it changes stays in the child. The parent never sees it.

The same code with a thread would print 200 twice. This is good news and bad news. Two processes cannot have a race on an ordinary variable, because they do not share it. But they also cannot hand results back by changing one. Results must travel another way: as the return value from a pool, or through one of the tools in the next chapter.

Sharing on purpose

multiprocessing has three main ways to share. Each one must be created in the parent and passed to the children as an argument.

Value: one shared number

Value("i", 0) is one integer kept in memory that all the processes can see. It is truly shared, so the lost update is back, and this time the processes really do run at the same instant. A Value carries its own lock. Use it.

Queue: passing messages

multiprocessing.Queue works like queue.Queue, but between processes. Children put results in, the parent takes them out.

Manager: shared dicts and lists

A Manager starts one more process that holds real Python objects. The other processes get stand-ins that send every operation to it. It is the most flexible of the three and the slowest.

Careful: Shared data between processes has the same races as shared data between threads. accounts[owner] += 100 on a manager dict is still read, modify, write. Guard it with a lock from manager.Lock(), or better, let each worker return its result and combine the results in the parent.

What processes cost

Parallelism is not free. Two costs matter.

Start-up. Starting a process means starting a new Python interpreter and, with spawn, importing your file again. That takes far longer than starting a thread.

Pickling. Processes share no memory, so every argument and every return value is turned into bytes, sent across, and rebuilt on the other side. Python does this with the pickle module. Sending a large list to a worker copies the whole list. And some things cannot be pickled at all, such as a lambda, an open file or a lock from threading.

A top-level function is pickled as just its name, which the child then looks up after importing the file. That is why targets must be defined at the top level.

Rule of thumb: Use processes for a small number of heavy jobs. Send little data in, get little data back, and reuse a pool so the workers are started only once.

Choosing

Part 5 of 5 · which tool for which job

The choice comes back to the question from the first part: does the task wait, or does it calculate?

Which one to use

Your workUseWhy
Fast enough alreadyNothing. A plain loopConcurrent code is harder to write, read and debug
I/O-bound, a few tasks to a few hundredThreads, through ThreadPoolExecutorThe GIL is released while waiting. Works with ordinary blocking code
I/O-bound, thousands of tasksasyncioA waiting coroutine costs far less memory than a waiting thread
CPU-boundProcesses, through ProcessPoolExecutorEach process has its own GIL, so all cores are used
CPU-bound, but each call is very shortA plain loopStart-up and pickling cost more than the work
A mix of bothThreads or asyncio for the waiting, a process pool for the calculatingEach part gets the tool that suits it

Rule of thumb: Waiting: threads. Waiting on a very large scale: asyncio. Calculating: processes.

One API for both pools

concurrent.futures gives threads and processes the same interface: map(), submit(), futures, as_completed(). You can write the code once and switch between the two by changing one name. For new code it is the best place to start.

Start with whichever pool matches your work. If you guessed wrong, the change is one word.

Common mistakes

MistakeWhat you seeFix
Sharing data without a lockWrong totals, now and then, never when you are watchingOne lock around every read and write of that data, or a queue
Forgetting join()The main thread reads results before the workers have finishedJoin every thread, or use a pool in a with block
CPU-bound work in threadsNo speed-up, sometimes slowerUse processes
Forgetting the __main__ guardRuntimeError on macOS and WindowsPut the starting code under if __name__ == "__main__":
Creating too many threadsMemory use climbs, everything slows down, the server you call refuses youA pool with a fixed max_workers, or a semaphore
Slow work inside a lockThreads queue up, no speed-upKeep the critical section small
Taking two locks in different ordersThe program hangsOne fixed order everywhere
target=work() with bracketsThe work runs in the main thread, the new thread does nothingtarget=work, args=(...)
Never calling future.result()Errors in workers vanish silentlyAsk every future for its result

Threads and processes are run by the operating system, which may switch between them at any moment. The other way to handle many waiting tasks is to let the code itself decide when to switch, and that is the subject of the next page: Python Async IO.