Learnings

Python in practice, part 5 of 9

Python Standard Library Tour

The standard library is the set of modules that is installed together with Python. You only have to import them. This page is a guided tour of the ones worth knowing, each with a small example from a bank and a line that says when to reach for it. Nothing on this page needs pip install, and every example runs on Python 3.9 or later.

Text

Part 1 of 7 · finding, checking and shaping text

Programs spend a lot of time reading text that people wrote: account numbers, bank codes, SMS alerts. String methods such as split() and startswith() handle the simple cases. When the text follows a pattern, the re module is the tool.

re: character classes and quantifiers

A character class matches one character out of a set. Square brackets list the set: [A-Z] is any capital letter, [0-9] any digit. Common sets have short names: \d is a digit, \w a letter, digit or underscore, \s a space, tab or new line.

A quantifier says how many times the thing before it may repeat: {4} exactly four, {9,18} nine to eighteen, + one or more, * zero or more, ? zero or one.

Put the pieces together to check a whole value. An Indian bank branch code (IFSC) is four capital letters, then a zero, then six letters or digits. re.fullmatch succeeds only if the whole string fits the pattern.

The same idea checks an account number: digits only, between 9 and 18 of them.

Concept: Inside square brackets, ^ at the start means "not". [^0-9] is any character that is not a digit. Outside brackets, ^ means the start of the text and $ the end.

re: groups and named groups

Round brackets make a group: a part of the match that you can ask for separately. Groups are numbered from 1, left to right. group(0), or just group(), is the whole match.

Numbers are easy to mix up. A named group, written (?P<name>...), lets you ask by name, and groupdict() returns them all as a dictionary.

Two more things appear in that pattern. The bar | means "or". And (?:...) is a group that only holds things together and is not captured, so (?:from|to) does not get a number.

Tip: Python joins string literals that sit next to each other, which is how the long pattern above is split over two lines.

re: the functions

The pattern language is the same everywhere. What differs is where the function looks and what it gives back.

FunctionWhere it looksReturns
re.search(p, text)Anywhere in the textThe first match, or None
re.match(p, text)Only at the startA match, or None
re.fullmatch(p, text)The whole text must fitA match, or None
re.findall(p, text)EverywhereA list of strings (or of tuples, if the pattern has several groups)
re.finditer(p, text)EverywhereMatch objects, one at a time
re.sub(p, new, text)EverywhereA new string with each match replaced
re.split(p, text)EverywhereA list of the pieces between matches

match found nothing for \d+ because the text does not start with a digit. fullmatch found nothing because the text holds more than digits.

findall collects every match. With one group in the pattern it returns just that group, which is ideal for pulling the amounts out of an alert.

sub replaces. The replacement can use \1, \2 to put groups back, or it can be a function that receives each match.

compile

re.compile(pattern) turns a pattern into an object with the same functions as methods. Give it a name once and reuse it. The code reads better, and the pattern is prepared only one time.

re: greedy, lazy and flags

Quantifiers are greedy: they take as much text as they can. That often swallows more than you meant. Adding ? after a quantifier makes it lazy: it takes as little as it can.

The greedy .* ran from the first quote to the very last one. The lazy .*? stopped at the nearest closing quote. The third form says exactly what is wanted, "anything that is not a quote", and is the clearest.

Flags

A flag is an extra argument that changes how the pattern is read.

FlagShortEffect
re.IGNORECASEre.ICapital and small letters count as the same
re.MULTILINEre.M^ and $ match at the start and end of every line, not only of the whole text
re.DOTALLre.SThe dot also matches a new line
re.VERBOSEre.XSpaces and # comments inside the pattern are ignored, so it can be laid out and explained

Combine flags with a bar: re.IGNORECASE | re.MULTILINE.

re: cheat sheet

PatternMatchesExample
abcExactly that textdebited
.Any one character except a new lineA.c fits A/c
\d \w \sA digit, a word character, a space character\d\d fits 26
\D \W \SThe opposite of each\D fits x
[abc] [A-Z]One character from the set or range[A-Z]{4} fits HDFC
[^abc]One character not in the set[^"]*
* + ?Zero or more, one or more, zero or one of the thing before\d+ fits 4821
{4} {9,18} {2,}Exactly 4, from 9 to 18, 2 or more\d{9,18}
*? +?The lazy forms: as few as possible".*?"
^ $Start and end of the text (or of a line with re.M)^DEBIT
\bA word boundary: the edge between a word and what is next to it\bRs\b
a|bEither onedebited|credited
(...)A group that is captured(\d{4})
(?P<name>...)A captured group with a name(?P<last4>\d{4})
(?:...)A group that is not captured(?:from|to)
\. \( \+The real character, with its special meaning switched offRs\.

Watch out: A regex can check the shape of a value, not whether it is real. HDFC0999999 has the right shape for a branch code and may still not exist. Keep patterns short. A pattern nobody can read is a bug waiting to happen.

string and textwrap

The string module holds ready-made sets of characters and a simple template class. Template fills $name placeholders. It is safer than f-strings when the template text comes from a file or a user, because it cannot run code.

textwrap fits long text into a fixed width, which is useful for letters, emails and terminal output.

Reach for it when: you need the alphabet or digits as a string (string), a template that other people may edit (string.Template), or text wrapped to a width (textwrap). textwrap.dedent() removes the common indentation from a triple-quoted string.

Files and system

Part 2 of 7 · paths, folders and the world around the program

These modules let a program find its files, tidy up after itself, and learn how it was started. Reading and writing files with open() is covered in Python Errors and Files.

pathlib

A path is the address of a file or folder. pathlib.Path turns that address into an object. The / operator joins parts, and it uses the right separator on every operating system.

A Path can also do the work: make folders, write and read a whole file in one call, and check what is there.

glob() finds files by a wildcard pattern. * stands for any run of characters, and ** for any depth of folders. The order is not guaranteed, so sort the result.

Reach for it when: you touch a file path at all. Prefer it to joining strings with "/" and to the older os.path functions. Useful extras: Path.cwd() is the current folder, Path.home() the user's home folder, and path.resolve() the full absolute path.

os and shutil

os talks to the operating system: list a folder, rename or delete one file. shutil ("shell utilities") does the bigger jobs: copy files, move them, delete or archive a whole folder tree.

JobCall
Current folderos.getcwd()
Walk a folder treeos.walk("bank"), or Path("bank").rglob("*")
Copy a fileshutil.copy(src, dst). copy2 also keeps the file times
Copy a folder treeshutil.copytree(src, dst)
Make a zip of a foldershutil.make_archive("backup", "zip", "bank")
Find a programshutil.which("git") gives its path, or None
Free disk spaceshutil.disk_usage(".")

Watch out: shutil.rmtree() and os.remove() delete for good. Nothing goes to a recycle bin. Print the path and check it before you run such a line on real data.

tempfile

A temporary file is one you need only for a moment: a download being checked, a report being built, a file for a test. tempfile creates it in a safe place with a name that cannot clash, and deletes it for you when the with block ends.

The folder name is different on every run, so it is not printed here. There is a single-file form too.

Reach for it when: a file is needed for a short time and must not be left behind, above all in tests. Use tempfile.NamedTemporaryFile() when another program has to open the file by its name.

sys

sys holds information about the running Python and the way the program was started.

  • sys.argv is the list of words typed on the command line. sys.argv[0] is the script name. Running python3 pay.py Rishikesh 500 gives ['pay.py', 'Rishikesh', '500']. Every item is a string.
  • sys.stdout and sys.stderr are two separate output streams. Results go to the first and problems to the second, so that a user can save the results to a file and still see the errors.
  • sys.executable is the full path of the Python that is running. It is used later on this page to start another Python.

Exit codes

Every program hands a number back when it ends, called the exit code. Zero means success. Anything else means failure. sys.exit(n) ends the program with that code. It works by raising SystemExit, which the block below catches so that it can keep running.

Reading standard input

sys.stdin is the input stream. Another program can be connected to it with a pipe (|), so your script can be one step in a chain. This script is not runnable here because it waits for input.

Reach for it when: you need the raw command line, an exit code, or the input and output streams. For anything more than one or two arguments, use argparse, shown below.

Environment variables

An environment variable is a named setting that lives outside the program and is handed to it when it starts. It is the usual place for things that differ between your laptop and the server: database addresses, modes, secret keys. os.environ is a dictionary of them.

In a terminal you set one for a single run by writing it before the command:

Reach for it when: a value must not be written in the code, either because it is a secret or because it changes from machine to machine. Read all settings in one place at the start of the program, and fail early with a clear message if a required one is missing.

Command-line programs

Part 3 of 7 · scripts that take arguments

A command-line program is one you run from a terminal with words after its name, such as bank deposit Rishikesh 500. argparse reads those words, checks them, converts them and writes the help text for you.

argparse: positional and optional arguments

You describe the arguments, and parse_args() returns an object with one attribute for each. There are two kinds:

  • A positional argument is known by its place. It has no dashes and is required.
  • An optional argument starts with -- and may be left out. Its value comes after it.

Called with no argument, parse_args() reads the real command line from sys.argv. The examples below pass a list instead, so that each block runs as it stands. The list holds exactly what you would type after the script name.

Two things to notice. Every value arrives as a string, and an optional argument that was not given is None. The next chapter fixes both.

Concept: The attribute name comes from the argument name with the dashes removed. --rate becomes args.rate, and --min-balance becomes args.min_balance.

argparse: types, choices, defaults and flags

Extra settings on add_argument() do the checking and converting that you would otherwise write by hand.

SettingEffect
type=floatConvert the text with this function. Bad text becomes a clear error
default=4.0The value when the argument is not given
choices=[...]Only these values are allowed
action="store_true"A flag: it takes no value. True if present, False if not
"-v", "--verbose"A short and a long name for the same option
required=TrueMake an option compulsory
nargs="+"Accept one or more values and collect them in a list
help="..."The text shown for this argument by --help

nargs="+" collects several values, for example a list of account holders.

argparse: help and errors for free

Here is the same parser as a real script that reads the command line. It is not runnable in the page because it needs arguments typed after its name.

Nobody wrote a help screen, yet -h and --help work. This is the real output of python3 interest.py --help:

Square brackets in the usage line mark what may be left out. Normal use, and then three mistakes:

On an error, argparse prints the message to the error stream and ends the program with exit code 2. Your own code never sees the bad value.

Version note: This output is from Python 3.9. From Python 3.10 the heading optional arguments: is printed as options:. Everything else is the same.

argparse: subcommands

Larger tools group their work into subcommands, the way git has git commit and git push. Each subcommand is a small parser of its own with its own arguments. Here is a bank with deposit and withdraw. It is not runnable in the page because it reads the command line.

dest="command" stores the chosen subcommand in args.command. prog="bank" sets the name shown in the help. The real output:

Options of the main parser, such as -v, go before the subcommand. Options of a subcommand go after it. Because main() accepts a list, a test can call main(["deposit", "Rishikesh", "500"]) without a terminal. Python Project Setup shows how to install a script so that it runs as plain bank.

getpass

Never take a password as a command-line argument. It would be saved in the shell history and shown in the list of running programs. getpass.getpass() asks for it and hides what is typed. Not runnable here because it waits for the keyboard.

Reach for it when: a script takes more than one or two arguments, or other people will run it. The help text alone is worth it.

Running other programs

Part 4 of 7 · starting a command and reading its result

Sometimes the job is already done by another program: git, a PDF tool, a backup command. The subprocess module starts it, waits for it and gives you what it printed.

subprocess.run

subprocess.run() takes the command as a list: the program first, then each argument as its own item. It waits until the program ends and returns a result object.

The examples run a second Python, because that exists on every machine that can run this page. sys.executable is the path of the Python that is running now, and -c runs the code given in the next item.

ArgumentMeaning
capture_output=TrueCollect the output in result.stdout and result.stderr. Without it, the output goes straight to the screen
text=TrueDecode the output to strings. Without it you get bytes
check=TrueRaise an exception if the exit code is not zero
timeout=10Stop waiting, and stop the program, after this many seconds
input="text"Send this text to the program's standard input
cwd="folder"Run the program in that folder
env={...}Give the program these environment variables

input feeds the other program. Here the second Python adds up the lines it receives, like total.py in the sys chapter.

Reach for it when: a command-line tool already does what you need. If a Python module can do the same job, prefer the module: shutil.copy() instead of running cp.

Failures and timeouts

By default run() does not complain when the other program fails. It only reports the exit code, and it is easy to forget to look. check=True turns a non-zero exit code into a CalledProcessError, so a failure cannot pass unnoticed.

A program that never ends would make yours wait for ever. timeout sets a limit in seconds. When it is passed, the other program is stopped and TimeoutExpired is raised.

If the program itself cannot be found, Python raises FileNotFoundError before anything runs.

Habit: For commands whose success matters, always write check=True and a timeout.

Pass a list, avoid shell=True

There is a second way to give the command: one string with shell=True. The string is handed to the system shell, the program that reads the commands you type in a terminal. The shell gives special meaning to characters such as ;, |, & and $. If any part of that string comes from a user, they can add a command of their own. This is called shell injection.

Below, a "name" typed by a user contains ;, which means "and then run this" to the shell. The example uses echo, which only prints its arguments, so nothing harmful happens. It needs macOS or Linux.

With the string, the shell ran two commands, and the second one was written by the user. With the list there is no shell at all. Each item goes to the program exactly as it is, so the whole name is just text.

  • Pass a list. It is safe, and names with spaces need no quoting.
  • If you have a command as one string, shlex.split("git log -n 5") turns it into a list.
  • If you really need a shell feature such as a pipe, build the string only from values you wrote yourself, or wrap each outside value in shlex.quote().

Data formats

Part 5 of 7 · saving, sending, hiding and counting data

Data has to leave the program sooner or later: into a file, to another system, into a password table. And money has to be counted exactly. These modules cover the common formats and the common traps.

json

JSON is a text format for data that almost every language and web service understands. json.dumps() turns Python data into a JSON string, and json.loads() turns it back. The versions without the s, dump() and load(), work with an open file.

Python's True and None became JSON's true and null. JSON knows only strings, numbers, true, false, null, lists and objects. Other types need help: default=str tells dumps to turn anything it does not know into a string.

Reach for it when: data goes to or comes from a web API, a config file or another language. Two details: dictionary keys always become strings, and tuples come back as lists.

csv

CSV (comma-separated values) is the format spreadsheets import and export: one row per line, columns separated by commas. Do not split the lines yourself. A value may contain a comma inside quotes, and the csv module handles that.

The writer added quotes around the names because they contain a comma, and the reader took them off again. Every value read from a CSV file is a string, so the balance had to go through int().

Reach for it when: data moves to or from a spreadsheet or a bank export. Always open the file with newline="", as the module asks, or blank lines can appear on Windows.

configparser

Settings files often use the INI format: section names in square brackets and key = value lines below them. configparser reads and writes it. Values are strings, and helper methods convert them.

Reach for it when: people need to edit simple settings by hand. Newer projects often use TOML, the format of pyproject.toml. The module that reads it, tomllib, is in the standard library only from Python 3.11, so it is not shown here.

pickle

pickle turns almost any Python object, including objects of your own classes, into bytes and back again. JSON cannot do that. The price is that only Python can read the result.

Security warning: Loading a pickle can run any code that the file's author put into it. Never call pickle.load() or pickle.loads() on data from the internet, from a user, or from anywhere you do not fully control. For data that crosses a boundary, use JSON.

Reach for it when: your own program saves its own Python objects for a short time, for example a cache on the same machine. Not for files you send to others, and not for long-term storage.

base64 and hashlib

base64 rewrites any bytes using only letters, digits and a few signs, so that they can travel through places that accept only text, such as JSON or email. It is an encoding, not a lock. Anyone can turn it back.

A hash is a short fixed-length fingerprint of some data. The same input always gives the same hash, a tiny change gives a completely different one, and there is no way back from the hash to the data. hashlib provides the common hash functions.

One common use is a checksum: hash a file before sending it and again after receiving it. If the two values match, the file arrived unchanged.

Reach for it when: bytes must pass through a text-only channel (base64), or you need to know whether data has changed (hashlib). Use sha256. The older md5 and sha1 are not safe for security work.

Storing passwords

Never store a password as plain text. If the table is ever copied, every account is open. Store a hash instead, and when someone logs in, hash what they typed and compare.

A plain sha256 is not enough, for two reasons. It is very fast, so an attacker can try billions of guesses. And two people with the same password get the same hash. The fix has two parts:

  • A salt: random bytes, different for every user, mixed into the hash. It is stored next to the hash and is not secret.
  • A slow hash made for passwords. hashlib.pbkdf2_hmac() repeats the hashing many thousands of times.

The salt and the hash are different on every run, so only their lengths are printed. The last line shows the salt at work: the same password hashed twice gives two different results. hmac.compare_digest() compares in a way that does not leak, through timing, how many bytes were right.

In real projects: Use a library made for this job, such as argon2-cffi or bcrypt (both need pip install), or the login system of your web framework. The example shows the idea with the standard library alone.

secrets and uuid

secrets makes random values that are safe for security: reset links, API keys, one-time codes. The values are different on every run, so the examples print their length or type instead.

A UUID (universally unique identifier) is a 128-bit number written as 36 characters. uuid.uuid4() makes a random one. The chance of two being equal is so small that separate machines can create ids without asking each other.

Reach for it when: you need a token nobody can guess (secrets) or an id that is unique without a central counter (uuid). Do not use the random module for either. Its numbers can be predicted.

decimal for money

Python's float stores numbers in binary. Most decimal fractions, such as 0.1, have no exact binary form, just as one third has no exact decimal form. The stored value is a tiny bit off, and the errors show up in sums.

For a bank that is not acceptable. decimal.Decimal stores numbers in base ten, exactly as written, the way people do sums on paper.

Always build a Decimal from a string or an int. Building it from a float copies the float's error.

Reach for it when: the numbers are money, or any quantity where every digit must be right. Keep float for measurements and science, where a tiny error does not matter and speed does.

decimal: quantize and rounding

Interest and tax produce more digits than a currency has. quantize() rounds a Decimal to the same number of places as the pattern you give it: Decimal("0.01") means two places. You also choose how to round.

The float version rounds 2.675 down, because the float is really 2.67499999... The decimal version rounds it the way you learned at school.

The rounding mode decides what happens to a value exactly half-way between two results.

ModeRuleTypical use
ROUND_HALF_UPA half goes away from zeroWhat most people and most invoices expect
ROUND_HALF_EVENA half goes to the nearest even digit"Banker's rounding". The default. Over many values, the ups and downs cancel out
ROUND_DOWNCut the extra digits offNever pay out more than was earned
ROUND_UPAny extra digit pushes away from zeroFees that are always rounded up

A classic problem: split 100 rupees between three people without losing a paisa. Round each share down, then give the remainder to one of them.

Tip: Another common approach is to store money as a whole number of the smallest unit, paise or cents, in an int. Integers are always exact.

fractions and statistics

fractions.Fraction keeps a number as an exact numerator and denominator. Shares of ownership are a natural fit.

statistics has the everyday summary numbers, so you do not have to write them yourself.

The mean is the average. The median is the middle value when the list is sorted, and one very large balance does not pull it up. stdev (standard deviation) says how spread out the values are. The mode is the most common value.

Reach for it when: you need exact ratios (fractions) or a quick summary of a list of numbers (statistics). For large tables of data, people use the third-party package pandas.

random

random makes random choices for games, test data and simulations. The numbers come from a formula, so they only look random. Giving the formula a starting value, called a seed, makes it produce the same sequence every time. That is how a test or a demo can be repeated exactly.

For test data it is tidier to keep a generator of your own, so that seeding it does not affect other code.

Reach for it when: you need sample data, a shuffle or a simulation. Never for passwords, tokens or one-time codes. Use secrets for those.

Time

Part 6 of 7 · dates, time zones and stopwatches

Dates look simple and are full of traps: months of different lengths, leap years, time zones, clocks that change in summer. Let the library do the arithmetic.

datetime

datetime has a type for a day (date), for a moment (datetime) and for a length of time (timedelta). They are introduced in Python Errors and Files. Here is the whole set in one example: a fixed deposit opened on one day that matures 90 days later.

strftime turns a date into text ("f" for format) and strptime reads text into a date ("p" for parse). date.today() and datetime.now() give the current day and moment. They are not used here because their output changes.

Reach for it when: anything involves a date. Never do date arithmetic on strings or by counting days yourself.

zoneinfo

A datetime without a time zone is called naive: "10:30" with no idea where. Attach a zone and it becomes aware, and can be converted to any other zone. zoneinfo, new in Python 3.9, knows the world's time zones by name, including their summer-time rules.

All four printed values of paid are the same moment, written for different places. In July, London is on summer time, so the gap to India is one hour smaller. The library knows that. You do not have to.

Watch out: zoneinfo reads the time zone database of the operating system. macOS and Linux have one. Windows does not, so there you need pip install tzdata, the one exception to "no installs" on this page. A good rule for programs: store and compare times in UTC, and convert to a local zone only when showing them to a person.

time.perf_counter

To measure how long a piece of code takes, read a clock before and after. time.perf_counter() is the clock made for this: very fine-grained, and it never jumps backwards. Its value means nothing alone. Only the difference between two readings matters.

The time taken differs from run to run, so the line that prints it is left as a comment.

FunctionUse
time.perf_counter()Timing code
time.time()Seconds since 1 January 1970, as a float. A timestamp to store or send
time.sleep(0.5)Pause the program for half a second
time.monotonic()A clock that never goes backwards, for timeouts

Reach for it when: you want a quick "how long did that take". Careful measuring, with the timeit module, is in Python Performance and Memory.

calendar

calendar answers questions about months and weekdays, and can print a month as text.

Reach for it when: you need the last day of a month (for interest that is paid monthly), the weekday of a date, or a month laid out as text.

Handy helpers

Part 7 of 7 · small modules that save a lot of code

The last group is a mix. Some of these modules have a full chapter on another page, so they get only a pointer here. The rest are shown briefly.

Covered on other pages

These modules are just as much part of the standard library. Each has a proper treatment elsewhere on this site.

ModuleIn one lineRead more
collectionsCounter counts things, defaultdict fills in missing keys, deque is a fast queue, namedtuple is a tuple with field namesPython Power Features
itertoolsBuilding blocks for loops: chain, islice, groupby, combinations, accumulatePython Iterators and Generators
functoolslru_cache remembers results, partial fixes some arguments, wraps keeps a decorated function's namePython Power Features and Python Decorators
dataclasses@dataclass writes __init__, __repr__ and __eq__ for a class that mostly holds dataPython Classes
contextlib@contextmanager turns a small function into something usable with withPython Power Features
heapq, bisectAlways get the smallest item quickly, and search or insert in a sorted listPython Data Structures and Algorithms
sqlite3, shelveA real SQL database in one file, and a dictionary that is saved to diskPython Databases
urllib, http.serverFetch a web address, and serve a folder over HTTP for local testingPython Web APIs
logging, pdb, traceback, warningsRecording what a program does and finding out why it went wrongPython Debugging and Logging
unittestThe built-in test frameworkPython Testing
threading, multiprocessing, asyncioDoing several things at oncePython Threads and Processes and Python Async IO

A one-block taste of the first three, so the names mean something:

enum

An enumeration is a fixed set of named values. Without it, code passes strings such as "savings" around, and a typing mistake like "saving" goes unnoticed. With an Enum, a wrong name is an error at once.

AccountType("fixed") looks a member up by its value, which is how text from a file or a form becomes an enum member.

Reach for it when: a value can only be one of a short, known list: account types, transaction states, days of the week. Compare members with is.

copy

Assigning with = does not copy anything. It gives the same object a second name. copy.copy() makes a shallow copy: a new outer object that still shares the inner ones. copy.deepcopy() copies everything, all the way down.

The shallow copy kept its own owner, but its history is the very same list as the original's, so the new entry shows up in both.

Reach for it when: you need to change a nested structure and keep the original as it was, for example to try a transfer and throw the result away if it fails. Use deepcopy for anything nested.

pprint and operator

pprint ("pretty print") lays out nested data over several lines so that it can be read. It is a good partner for debugging.

operator offers small ready-made functions for use as the key of sorted(), max() and min(). itemgetter("balance") does the same as lambda a: a["balance"], and attrgetter does it for attributes of objects.

Reach for it when: a printed dictionary runs off the screen (pprint), or you sort by a field and want something shorter than a lambda (operator).

zipfile and gzip

zipfile reads and writes .zip archives, which pack many files into one. gzip compresses a single file, and is common for log files and data downloads.

gzip.open() works like open(). Use mode "wt" and "rt" for text.

33,000 characters of repeated text shrank to a few hundred bytes. Real data compresses less than this, but log files often shrink to a tenth of their size.

Reach for it when: you send or store many files as one (zipfile), or want a big text file to take less room (gzip). shutil.make_archive() zips a whole folder in one call.

Quick reference

I need to ...Module
Check or pull out text by its patternre
Wrap long text to a widthtextwrap
Fill a simple template from a filestring.Template
Build, inspect or search file pathspathlib
Copy, move or delete files and foldersshutil, os
Use a file for a moment and have it cleaned uptempfile
End the program with an exit code, or read piped inputsys
Read a setting from outside the codeos.environ
Take arguments on the command lineargparse
Ask for a password without showing itgetpass
Run another programsubprocess
Exchange data with a web service or another languagejson
Read or write spreadsheet datacsv
Read a settings fileconfigparser (or tomllib from Python 3.11)
Save my own Python objects for my own programpickle, never on untrusted data
Send bytes through a text-only channelbase64
Fingerprint data or check a downloadhashlib
Store a passwordhashlib.pbkdf2_hmac with a salt, or a dedicated library
Make a token, key or one-time codesecrets
Make a unique iduuid
Do sums with moneydecimal
Keep exact ratiosfractions
Get the mean, median or spread of some numbersstatistics
Make sample data or shuffle a listrandom
Work with dates and lengths of timedatetime
Convert between time zoneszoneinfo
Time a piece of codetime.perf_counter
Find the last day of a month or print a monthcalendar
Name a fixed set of choicesenum
Copy a nested structurecopy.deepcopy
Print nested data readablypprint
Sort by a field without a lambdaoperator.itemgetter
Count things, or use a queuecollections
Combine or slice loopsitertools
Cache the results of a functionfunctools.lru_cache
Pack or compress fileszipfile, gzip
Keep data in a small databasesqlite3

Before you write a helper of your own or install a package, search the library index in the official Python documentation. There is a good chance the module already exists.

Next in the practice path: Python Data Structures and Algorithms.