Python in practice, part 7 of 9
Python Performance and Memory
This page is about making Python code faster and lighter, and about knowing when not to bother. It shows how to measure, which changes usually pay off, how Python stores and frees objects, and what the interpreter does with your code. The running example is a small bank. Every example runs on the standard library alone, so nothing needs pip install. Timings differ between machines and runs, so the examples print comparisons such as "was the first way slower?" and never raw times.
Measure first
Part 1 of 5 · find the slow part before changing anything
Guesses about speed are wrong surprisingly often. These chapters cover the tools that replace guessing with numbers.
- The rule: do not guess
- Timing a small piece with timeit: compare two ways of doing one thing
- Timing a block with perf_counter: a stopwatch inside your program
- Profiling with cProfile: where a whole program spends its time
- Other profilers: names worth knowing
The rule
Most code is fast enough. A script that runs in half a second does not need to run in a quarter of a second. Making code faster usually makes it longer and harder to read, so only pay that price where it buys something.
When a program really is too slow, the time is almost never spread evenly. A few lines, often inside a loop, take most of it. The rule has three steps:
- Make it work and make it clear. Correct code comes first.
- Measure. Find the few lines where the time goes. Do not guess.
- Fix that part only, then measure again. If the number did not improve, undo the change.
Term: the slowest part of a program is called its bottleneck or hot spot. Speeding up anything else changes almost nothing.
Timing a small piece with timeit
The timeit module runs a small piece of code many times and reports how long it took. Running it many times evens out the random noise of a busy computer. From the command line, -s gives setup code that runs once and is not timed.
Your numbers will differ. What matters is the gap: a usec is a millionth of a second and a nsec is a thousandth of that, so the set was a few hundred times faster here.
The same tool works from code. timeit.repeat returns several totals, and the smallest is the one to use, because the larger ones only show that the computer was busy with something else.
The example asks "was the list at least 10 times slower?". The real gap is in the thousands, so the answer is True on every run.
Habit: compare two versions on the same machine in the same session. A time taken yesterday, or on another computer, tells you little.
Timing a block with perf_counter
To time one part of a running program, read a clock before and after. time.perf_counter() is the clock made for this. It has the best precision available and never jumps backwards, which time.time() can do when the system clock is adjusted.
In your own code you would print or log the times, for example with print(f"{label}: {seconds:.3f} s"). The value returned by perf_counter() means nothing alone. Only the difference between two readings is useful.
Profiling with cProfile
A profiler watches a whole program and records how much time each function took. cProfile comes with Python. The program below checks 20,000 transactions against a list of flagged amounts, and is slower than it should be.
The program found the function with the most time spent inside it: is_suspicious, called 20,000 times. To see the full table, sort the statistics and print the top rows. This block is not run by the page's checker, because every time in its output changes from run to run.
That is a real table from one run on Python 3.9. Your times will differ, but the shape will be the same. Read it like this:
| Column | Meaning |
|---|---|
ncalls | How many times the function was called. |
tottime | Seconds spent inside the function itself, not counting the functions it called. |
cumtime | Cumulative seconds: the function plus everything it called. |
percall | The column to its left divided by the number of calls. |
main and count_suspicious have a high cumtime but almost no tottime. They are slow only because of what they call. is_suspicious has nearly all the tottime, so it is the bottleneck. Its one line is amount in flagged on a list. Changing flagged to a set, set(range(0, 5000, 7)), fixes the program without touching anything else.
You can also profile a script without editing it:
Detail: the profiler slows the program down, and it slows code with many small function calls the most. Use it to find where the time goes, and use timeit to compare the old and new version of the part you change.
Other profilers
cProfile reports whole functions. Other tools answer other questions. These are third-party packages, installed with pip, and are listed here as names to look up when you need them.
- line_profiler reports time line by line inside the functions you mark.
- py-spy attaches to a Python program that is already running, without changing or restarting it, and shows where it spends its time.
- memory_profiler and memray do for memory what the tools above do for time.
Faster Python
Part 2 of 5 · the changes that usually pay off
Once you know where the time goes, these are the fixes to try, roughly from the largest gain to the smallest.
- Pick the right data structure: the biggest win
- Avoid repeated work: cache it or move it out of the loop
- Let built-ins do the looping: C speed for free
- Building strings:
joinand+= - Local names are quicker: a small gain for hot loops
- Stream large data with generators: one item at a time
- Avoid needless copies: slices and
sortedbuild new lists - When the answer is a different tool: NumPy, processes, async
Pick the right data structure
No small trick beats choosing a structure whose operations are cheap. x in a_list compares x with every item. x in a_set uses hashing and goes almost straight to the answer. Inside a loop, that difference is the gap between a program that finishes at once and one that does not finish.
Both give the same answers. The list needed 33,334 comparisons for the first lookup alone. The set needs about one for each. The same thinking applies to looking records up by id (use a dict), taking items from the front of a queue (use collections.deque) and keeping the top few items (use heapq). The page Python Data Structures and Algorithms covers each of them with its costs.
Avoid repeated work
The fastest code is code that does not run. Look for work that gives the same answer every time and is still done again and again. There are two standard fixes. If the work sits inside a loop but does not depend on the loop, move it out. If a function is called many times with the same arguments, cache its results with functools.lru_cache.
Careful: only cache a function whose result depends on its arguments and nothing else. A cached "current balance" would keep returning the old balance.
Let built-ins do the looping
Built-in functions such as sum, min, max, sorted, any, all and str.join are written in C inside the interpreter. When you call sum(amounts), the loop over the items runs in C. When you write the loop yourself, the interpreter must carry out every step of it as separate Python instructions, which is several times slower.
A list comprehension sits in between. [a * 2 for a in amounts] is still a Python loop, but it is usually somewhat faster than a for loop that calls append, because it skips looking up and calling the append method on every round. It is also shorter and clearer, which is the better reason to use it.
Building strings
Strings are immutable, so text += piece cannot change text. In principle it builds a new string and copies everything written so far, which makes a long loop of += quadratic. "".join(pieces) works out the final size first and copies each piece once.
Honest note: CPython can often extend the string in place when nothing else refers to it, so += in a simple loop is frequently not as slow as the theory says. That is an implementation detail, not a promise. join is fast everywhere, so collect the pieces in a list and join them once.
Local names are quicker
Python finds a local variable, one assigned inside the function, faster than a global name or an attribute such as math.sqrt. A global is looked up in a dict by name, and math.sqrt needs two lookups. A local is read from a fixed slot.
The gain is small. On the Python 3.9 used to test this page the second version was roughly one and a half times as fast in a tight loop, and Python 3.11 and later made global and attribute lookups cheaper, so expect less there. Use the trick only in a loop that a profiler showed to be hot. The more useful lesson is that code at the top level of a script uses global names throughout, so putting it inside a main() function makes it a little faster for free.
Stream large data with generators
A list holds all of its items in memory at once. A generator produces one item, hands it over, and forgets it. For a large file or a long calculation that means the memory used stays small however much data flows through. Generators are explained in full on Python Iterators and Generators.
Limit: a generator can be read once, from start to finish. If you need the data twice, or need len() or indexing, you need a list.
Avoid needless copies
Some innocent-looking operations build a whole new container. A slice such as items[1:] copies every item it covers. sorted(items) builds a new list, and a + b builds a third list from two. On small data none of this matters. On a list of a million items inside a loop it does.
Balance: a copy is sometimes exactly what you want, because it protects the caller's data from being changed. Remove a copy for speed only when a measurement says it matters.
When the answer is a different tool
Some jobs cannot be made fast with plain Python loops, and the right move is to change tools.
| The work is | Reach for | Why |
|---|---|---|
| Arithmetic on large arrays or tables of numbers | NumPy, pandas (third-party) | They store numbers compactly and run the loop in compiled code. This is called vectorisation. |
| Heavy calculation that keeps a CPU core busy | multiprocessing or concurrent.futures.ProcessPoolExecutor | Several processes use several cores at once. |
| Waiting for many network calls, files or databases | asyncio or threads | The program does other work while it waits. |
| A query over stored data | The database itself | Filtering and summing in SQL beats fetching every row into Python. |
An honest sentence about NumPy and pandas: they can be tens of times faster than a Python loop when you work on whole columns at once, and no faster at all if you loop over their items one by one in Python.
Processes and threads are covered on Python Threads and Processes, and asyncio on Python Async IO.
How memory works
Part 3 of 5 · names, objects and when they are freed
Python manages memory for you. You never free an object by hand. Knowing how it decides to free one explains several surprises, and tells you what to do when a program's memory keeps growing.
- Names refer to objects:
id(),isand== - Mutable and immutable arguments: what a function can change
- Shared small values: a detail not to rely on
- Reference counting: how most objects are freed
- The cycle garbage collector: objects that refer to each other
- Weak references: refer to an object without keeping it alive
- What del does: it removes a name
- Smaller objects with __slots__: for millions of small records
- Measuring one object with getsizeof: shallow sizes
- Measuring a program with tracemalloc: which lines allocate memory
- Memory leaks in practice: things that only ever grow
Names refer to objects
A variable in Python is not a box that holds a value. It is a name attached to an object that lives somewhere in memory. Assignment attaches a name. It never copies. Two names can be attached to the same object, which is called aliasing.
== asks "do these have equal values?". is asks "are these the very same object?". id() returns a number that identifies an object for as long as it lives.
Rule: use == to compare values. Use is only for None, True and False, as in if result is None.
Mutable and immutable arguments
A mutable object can be changed after it is made: lists, dicts, sets and most objects of your own classes. An immutable object cannot: numbers, strings, tuples, frozenset.
Calling a function attaches the parameter name to the same object the caller passed. Nothing is copied. So a function can change a mutable argument and the caller will see it. With an immutable argument there is nothing to change. An operation such as += makes a new object and attaches the local name to that.
Habit: decide for each function whether it changes its argument or returns a new value, and say so in its name or docstring. Doing both is a common source of bugs.
Shared small values
To save memory and time, CPython reuses one object for some values that occur very often, such as small whole numbers and short strings that look like names. This is called interning. It means is sometimes returns True for two equal numbers or strings.
Which values are shared depends on the Python version, on how the value was produced, and even on whether the code is in a file or typed at the prompt. This block is marked as not run for that reason: the lines with is have no single correct output.
Rule: interning is an implementation detail. It can never change what == returns, so code that compares values with == is always right. Python 3.8 and later print a SyntaxWarning for code such as x is 1000.
Reference counting
CPython keeps a counter on every object: the number of references to it. A name, a list slot, a dict value and an attribute each count as one reference. When the counter falls to zero, nothing can reach the object any more, and CPython frees its memory at once.
sys.getrefcount() shows the counter. The exact number is not meaningful: the call itself adds a temporary reference, and the interpreter holds references of its own. The example therefore prints how much the count changes.
Detail: reference counting is how CPython works. Other Python implementations, such as PyPy, manage memory differently and may free an object some time after its last reference is gone. That is one reason to close files with a with block and not wait for the object to be freed.
The cycle garbage collector
Reference counting has one blind spot. If two objects refer to each other, each keeps the other's counter above zero, even when nothing else in the program can reach them. This is a reference cycle.
Python has a second mechanism for this, the cycle garbage collector in the gc module. From time to time it searches for groups of objects that refer only to each other and frees them. gc.collect() runs it immediately. The example watches the objects with a weak reference, explained in the next chapter, which reports whether an object is still alive.
You rarely need to call gc.collect() yourself, and you should not normally disable the collector. The example does both only to make the order of events visible. Cycles are common and harmless: a parent that knows its children, each of which knows its parent, is one.
Weak references
A weak reference refers to an object without adding to its reference count. It does not keep the object alive. When the object is freed, the weak reference simply reports that it is gone.
The main use is a cache. A normal dict used as a cache keeps every object in it alive for ever. A weakref.WeakValueDictionary drops an entry by itself when nothing else uses the object any more.
Detail: not every object can be weakly referenced. Objects of your own classes can. Plain int, str, list, dict and tuple objects cannot. The entry disappears straight away here because CPython frees the object as soon as its count reaches zero. On other implementations it may take a little longer.
What del does
del name removes a name. It does not destroy an object. The object is freed only if that name was its last reference.
You seldom need del on a name. A local variable disappears when its function returns. del is occasionally useful in a long function or at the top level of a script, to let go of a very large object before the next large one is built.
Smaller objects with __slots__
A normal object stores its attributes in a dict of its own, called __dict__. That is flexible, because you can add any attribute at any time, but a dict for each object costs memory. Listing the attribute names in __slots__ tells Python to reserve fixed places for exactly those attributes and to create no __dict__.
How much is saved depends on the Python version, so the example prints only that the slotted class used less. The saving matters when a program holds hundreds of thousands of small objects. For a few hundred it is not worth losing the flexibility. __slots__ is covered further on Python Power Features.
Measuring one object with getsizeof
sys.getsizeof(obj) returns the size of one object in bytes. It is shallow: for a container it counts the container's own table of references and not the objects those references lead to. A list holding one enormous string looks tiny.
The exact byte counts differ between Python versions and between 32-bit and 64-bit builds, so the example prints comparisons.
The last line shows the trade-off from the data structures page: a set of the same thousand numbers takes more room than the list, because a hash table keeps spare slots.
Careful: adding up sizes by hand counts an object twice if two containers share it, and misses objects nested more deeply. For a whole program, use tracemalloc.
Measuring a program with tracemalloc
tracemalloc records every block of memory Python allocates after you start it. get_traced_memory() returns two numbers in bytes: what is in use now, and the peak, the highest point reached so far.
A snapshot goes further and tells you which lines of code allocated the memory. The output holds exact sizes that change between Python versions, so this block is marked as not run. The output below it is from a real run on Python 3.9, with the folder part of the file path removed.
Each row gives the file and line, the total size allocated there and still in use, and the number of memory blocks. Line 4, the list of tuples, holds about 9 MB. To find a leak, take one snapshot early and one later, then call later.compare_to(earlier, "lineno"). The lines at the top of that list are the ones whose memory grew.
Detail: tracing makes the program slower and uses memory itself. Turn it on while you investigate, not all the time.
Memory leaks in practice
Python frees what nothing refers to, so a classic leak, memory that was simply lost, is rare in pure Python. What does happen is that a program keeps references it no longer needs. The memory is still reachable, so Python must keep it. In a long-running program such as a web server it grows until the machine runs out. There are three usual causes.
A cache with no limit
A dict used as a cache gains an entry for every new key and never loses one. Give a cache a maximum size. lru_cache(maxsize=...) throws out the least recently used entry when it is full.
A global list that is only appended to
A module-level list such as all_requests = [] that every call appends to, "for debugging", lives as long as the program does. Use a collections.deque(maxlen=1000) to keep only the most recent items, or write them to a log file.
A reference that lingers in a closure
A closure is an inner function that remembers variables of the function that made it. It keeps those variables alive for as long as the inner function itself exists, even if it only needed one small fact about them.
Also check: callbacks registered with a long-lived object and never removed, and exceptions stored in a variable. A stored exception holds its traceback, which holds every local variable of every function that was running when it was raised.
The interpreter
Part 4 of 5 · what runs your code
"Python" is a language. The program that runs it is an interpreter. The one you download from python.org is called CPython, because it is written in C. A little knowledge of how it works explains most of the advice in this page.
- Bytecode: what your code is turned into
- The GIL and performance: one thread at a time
- Faster interpreters: newer CPython releases and PyPy
Bytecode
CPython does not run your source text directly. It first compiles each function into bytecode, a list of small instructions for an imaginary machine. Then a loop written in C carries those instructions out one by one. Each instruction is cheap, but there are many of them, and that is the main reason a Python loop is slower than the same loop in C.
The dis module shows the bytecode of a function. The instruction names and their number change between Python versions, so this block is marked as not run. The output is from Python 3.9.
Read it from the top. The instructions push the values of balance, balance and rate, multiply the top two, push 100, divide, add, and return the result. LOAD_FAST is the instruction for reading a local variable from its fixed slot, which is why local names are quick. Python 3.11 and later show different names, such as a single BINARY_OP for all three calculations.
The parts that do not change between versions can be read from the function's code object.
Detail: when a module is imported, CPython saves its bytecode in a __pycache__ folder as a .pyc file. The next start can skip the compile step if the source has not changed. This makes programs start faster. It does not make them run faster.
The GIL and performance
The standard CPython has a global interpreter lock, the GIL, which lets only one thread run Python bytecode at any moment. So threads do not make calculation-heavy Python code faster, although they still help with work that mostly waits, such as network calls, because a waiting thread lets go of the lock.
For calculation-heavy work, use several processes. The full story, with examples, is on Python Threads and Processes.
Changing: Python 3.13 added an optional, experimental build of CPython that runs without the GIL. It is a separate download or build option, not the default, and not every third-party package supports it yet.
Faster interpreters
Sometimes the cheapest speed-up is to run the same code on a faster interpreter.
- A newer CPython. Python 3.11 was a large step. Its developers measured it as about 1.25 times as fast as 3.10 on average over their standard set of benchmarks, with some programs gaining much more and some little. Later releases have continued that work in smaller steps. Upgrading costs nothing in code changes, so try it before rewriting anything.
- The experimental JIT. A JIT, or just-in-time compiler, turns code that runs often into machine code while the program runs. Python 3.13 includes an experimental one. It is switched off by default and must be chosen when Python is built, and its developers describe the gains so far as modest. Treat it as something to watch, not something to depend on.
- PyPy. PyPy is a separate implementation of Python with a mature JIT. Long-running programs made of plain Python loops often run several times faster on it. The costs are that it supports a somewhat older version of the language than the newest CPython, and that packages built on C extensions may run slower or not be available.
Check: python3 --version shows the version you are running, and platform.python_implementation() returns the name of the interpreter. Whatever you switch to, measure your own program before and after. Benchmark averages do not tell you how your code will behave.
Checklist and reference
Part 5 of 5 · the page in two tables
- Before you optimise: questions to ask first
- Quick reference: symptom to tool or fix
Before you optimise
Go through this list in order. Stop at the first point that solves the problem.
- Is it really too slow? Decide what "fast enough" means in numbers, for example "the report must finish in under two seconds". If the code already meets that, leave it alone.
- Is it correct, and do you have tests? You will be changing working code. Tests tell you when a faster version gives a different answer.
- Have you measured? Profile with realistic data, not a ten-line sample. Find the function with the highest
tottime. - Is the algorithm the problem? Look for a loop inside a loop, or
inon a list. Fixing the Big-O beats every small trick. - Is work being repeated? Cache it or move it out of the loop.
- Can a built-in or the library do it?
sum,sorted,join,Counterandbisectrun at C speed. - Is the program waiting or calculating? Waiting calls for threads or
asyncio. Calculating calls for processes or NumPy. - Did the change help? Measure again with the same data. Keep the change only if the gain is real and the code is still readable.
Quick reference
| Symptom or question | Tool or fix |
|---|---|
| Which of two small snippets is faster? | python3 -m timeit, or timeit.repeat and take the min |
| How long does this part of my program take? | time.perf_counter() before and after |
| The whole program is slow and I do not know where | python3 -m cProfile -s cumulative script.py, then read tottime |
| One function is slow and I do not know which line | A line profiler such as line_profiler |
| A running server is slow and cannot be restarted | py-spy |
| Membership tests or lookups inside a loop | A set or dict in place of a list |
| The same function is called with the same arguments | functools.lru_cache, with a maxsize |
| The same value is worked out on every round of a loop | Work it out once, before the loop |
| A hand-written loop that sums, counts, sorts or joins | sum, Counter, sorted, str.join |
| Removing from the front of a list | collections.deque and popleft() |
| Memory jumps when a large file is read | Loop over the file, or use a generator, so one line is held at a time |
| Millions of small objects use too much memory | __slots__, tuples, or NumPy arrays for numbers |
| How big is this one object? | sys.getsizeof, remembering that it is shallow |
| Memory keeps growing over hours | tracemalloc snapshots and compare_to. Look for unlimited caches and global lists |
| A cache keeps objects alive that nothing else uses | weakref.WeakValueDictionary |
| Heavy calculation uses one core only | multiprocessing, or NumPy for arrays of numbers |
| The program spends its time waiting for the network or disk | Threads or asyncio |
| Everything is a bit slow and the Python version is old | Upgrade to a newer CPython, or try PyPy |
Slow programs are very often slow because of how they talk to stored data, which is the subject of the next page: Python Databases.