Python in practice, part 9 of 9
Python Web APIs
An API is a way for one program to ask another program to do something. On the web, the asking is done over HTTP. This page uses the family bank from Python Databases to show both sides: how to call an API from Python, and how to build one with FastAPI. You need to know Python, and nothing about the web. No example on this page touches the internet.
How the web talks
Part 1 of 4 · the vocabulary: requests, responses, methods, status codes
Before any code, the words. Every web API, in any language, is built from the same few ideas. This part needs nothing installed.
- HTTP in plain words: client, server, request, response
- Methods: GET, POST, PUT, PATCH, DELETE
- Status codes: 200, 404 and friends
- Headers and JSON
- What REST means in practice
HTTP in plain words
HTTP is the set of rules that browsers and web programs use to talk. The program that asks is the client. The program that answers is the server. The client sends one request, the server sends back one response, and that is the whole conversation.
Both are plain text. Here a client asks the bank's server for account 1, and the server answers.
| Term | Meaning |
|---|---|
| Client | The program that sends the request: a browser, a phone app, a Python script |
| Server | The program that waits for requests and answers them |
| Request | A method, a URL, some headers and sometimes a body |
| Response | A status code, some headers and usually a body |
| URL | The address of the thing you ask for, such as https://bank.example/accounts/1 |
| Endpoint | One method plus one path that the server answers, such as GET /accounts/{id} |
| API | Application programming interface: the full list of endpoints a server offers, and the rules for using them |
Concept: the server does not remember the client between requests. Each request must carry everything the server needs, including who is asking. HTTP is called stateless for this reason.
Methods
The method is the first word of the request. It says what kind of action the client wants.
| Method | Meaning | Bank example | Safe to repeat? |
|---|---|---|---|
GET | Read something. Changes nothing | GET /accounts/1 | Yes |
POST | Create something new, or run an action | POST /accounts, POST /transfers | No: twice means two accounts, two transfers |
PUT | Replace the whole thing with what is sent | PUT /accounts/1 | Yes: the result is the same |
PATCH | Change part of it | PATCH /accounts/1 with a new nickname | Depends on the change |
DELETE | Remove it | DELETE /accounts/1 | Yes: it stays deleted |
A request that gives the same result however many times it is sent is called idempotent. The last column matters when a network fails halfway and the client wants to try again. It comes back in Idempotency.
Status codes
Every response starts with a three-digit status code. The first digit gives the family: 2xx worked, 4xx the client made a mistake, 5xx the server failed.
| Code | Name | When the bank sends it |
|---|---|---|
| 200 | OK | The request worked, and here is the result |
| 201 | Created | A new account or transfer was created |
| 204 | No Content | It worked and there is nothing to send back, for example after a delete |
| 400 | Bad Request | The request makes no sense, for example broken JSON |
| 401 | Unauthorized | The server does not know who you are: no token, or a wrong one |
| 403 | Forbidden | The server knows who you are, and you may not do this: Hemkesh asking for Sandhya's account |
| 404 | Not Found | There is no account 99 |
| 409 | Conflict | The request is valid but clashes with the current state: not enough money for the transfer |
| 422 | Unprocessable Content | The JSON is readable but a value is wrong: an amount of -5 |
| 500 | Internal Server Error | A bug or crash in the server. Not the client's fault |
Rule: the status code is the first thing a client checks, so it must tell the truth. Never send 200 with {"error": "..."} in the body.
Headers and JSON
Headers are name and value pairs sent before the body. They describe the message, not the bank. The ones you meet first:
| Header | Says |
|---|---|
Content-Type: application/json | The body of this message is JSON |
Accept: application/json | The client would like JSON back |
Authorization: Bearer abc123 | Who is asking, as a secret token |
User-Agent: my-script/1.0 | Which program is the client |
JSON is the text format almost every API uses for the body. It looks like Python dicts and lists, with a few differences: strings always use double quotes, and True, False and None are written true, false and null. The standard json module converts both ways.
What REST means in practice
REST is a style for designing APIs. In practice it comes down to this: the URLs name things, called resources, and the HTTP method says what to do with them. The path is a noun. The method is the verb.
| Request | Meaning | Success code |
|---|---|---|
GET /accounts | List the accounts | 200 |
POST /accounts | Open a new account | 201 |
GET /accounts/1 | Read account 1 | 200 |
PATCH /accounts/1 | Change part of account 1 | 200 |
DELETE /accounts/1 | Close account 1 | 204 |
POST /accounts/1/deposit | Pay money into account 1 | 200 |
POST /transfers | Create a transfer between two accounts | 201 |
Use plural nouns, put the id in the path, and keep verbs out of URLs where you can: DELETE /accounts/1, not POST /deleteAccount?id=1. A transfer fits this by being a thing too. You do not "do" a transfer, you create one.
Honest note: not every action is a neat noun. POST /accounts/1/deposit is a small, common compromise. Being consistent matters more than being pure.
Calling an API
Part 2 of 4 · Python as the client
Python can call an API with the standard library alone, through urllib.request. Most people install requests or httpx, which do the same job with less code.
The two urllib chapters run on the standard library. The chapters from requests and httpx onwards need the install above.
About the practice server. These examples must run without the internet. So each one starts a tiny bank server inside the same program, using the standard http.server module in a background thread, and then calls it. Asking for port 0 lets the operating system choose any free port. The server lines are scaffolding. The part to study in each block comes after the comment # ---- the client ----. Against a real API you would write only that part, with the real URL.
- GET with urllib: query parameters, status, JSON
- POST with urllib: sending a JSON body
- requests and httpx: the same two calls, shorter
- Timeouts and errors
- Headers, tokens and API keys
- A Session or Client for reuse
- Retries
GET with urllib
Extra options for a GET go at the end of the URL as query parameters: /accounts?min_balance=4000. urlencode() builds that text from a dict and escapes awkward characters. urlopen() sends the request.
The response body arrives as bytes. json.loads() turns it into normal Python lists and dicts, and from there it is ordinary Python.
Rule: always pass timeout. Without it, a server that never answers makes your program wait for ever.
POST with urllib
A POST carries its data in the body. Convert the dict to JSON text, encode it to bytes, and say what it is with the Content-Type header. A Request object holds the URL, body, headers and method together.
Concept: with urllib, a 4xx or 5xx answer is raised as urllib.error.HTTPError. Catch it and read error.code to get the status.
requests and httpx
requests is the long-standing favourite. httpx is newer, has almost the same interface, and also supports async. Both build the query string from params=, send a dict as JSON with json=, and decode the answer with .json().
Pitfall: requests has no default timeout and will wait for ever. httpx defaults to 5 seconds. Set one yourself in both, so the next reader does not have to remember which is which.
Timeouts and errors
Three different things can go wrong. The server answers with an error status. The server is too slow. The server cannot be reached at all. requests does not raise for a 404 by itself: you get a normal response whose status_code is 404. Call raise_for_status() to turn any 4xx or 5xx into an exception.
All of these exceptions inherit from requests.RequestException, so one except requests.RequestException catches the lot when you do not need to tell them apart. httpx works the same way, with httpx.HTTPStatusError, httpx.TimeoutException and httpx.ConnectError, all under httpx.HTTPError.
Habit: the network is the least reliable part of any program. Every call to another service needs a timeout and a decision about what happens when it fails. Exceptions are covered in Python Errors and Files.
Headers, tokens and API keys
Most APIs want to know who is calling. The common way is a secret string, called an API key or token, sent in the Authorization header with the word Bearer in front. Extra headers are passed as a dict.
The token is a password. Do not write it in the code. Read it from an environment variable, a named value that the operating system hands to the program. The fallback value here exists only so that the example runs.
Pitfall: a key written in a source file gets committed, pushed and copied. Treat any key that reached version control as stolen, and replace it. Send tokens only over https:// URLs, where the traffic is encrypted.
A Session or Client for reuse
When you make several calls to the same API, create one requests.Session or httpx.Client and reuse it. It keeps your headers in one place, and it keeps the network connection open between calls, which is faster than opening a new one each time.
Difference: httpx.Client remembers a base URL and a timeout. A requests.Session does not remember a timeout, so you still pass it on every call.
Retries
Some failures are temporary: the server was restarting, or busy for a moment. Trying again a little later often works. Wait longer after each failure, which is called backoff, and stop after a few attempts. The practice server below answers 503 (Service Unavailable) twice, then works.
Retry only what is worth retrying: connection failures, timeouts and 5xx answers. A 400 or 404 will fail the same way every time. Real waits are seconds, not hundredths. Both libraries can do this for you: requests through urllib3's Retry class mounted on a session, and httpx through httpx.HTTPTransport(retries=3), which retries failed connections only.
Danger: retrying a GET is safe. Retrying a POST /transfers may pay twice, because you cannot tell whether the first try reached the server. The cure is in Idempotency.
Building an API with FastAPI
Part 3 of 4 · Python as the server
FastAPI is a framework for writing web APIs. You write normal functions with type hints, and it turns them into endpoints that check their input and produce JSON. It is not in the standard library. Everything in this part needs:
uvicorn is the server program that runs the app. httpx is needed by FastAPI's test client. The examples were checked with FastAPI 0.128 and Pydantic 2 on Python 3.9, so they write Optional[str] and List[...] from typing, which work on every version.
How these examples run. Each block builds an app and then calls it with TestClient, which sends requests straight to the app inside the same program. No server is started and no port is opened, yet the app handles each request exactly as it would for real.
A first app
Create a FastAPI object. Put a decorator on a function to say which method and path it answers. Whatever the function returns is sent back as JSON.
To run it as a real server, save the file as main.py (the last four lines are not needed then) and start it with uvicorn. main:app means "the object app in main.py". --reload restarts the server whenever you save the file, which suits development only.
Concept: decorators such as @app.get register the function with the app. They are explained in Python Decorators.
Path and query parameters
A name in braces in the path, such as {account_id}, is a path parameter. It is passed to the function argument of the same name. Any other argument is a query parameter, read from the part after ?. The type hints do real work: FastAPI converts the text from the URL to that type, and rejects the request if it cannot.
A query parameter with a default value is optional. Optional[str] = None means "text, or left out". The function body never sees "abc": FastAPI answered 422 before calling it.
Still missing: /accounts/99 would crash with a KeyError, and the client would get a 500. The proper 404 comes in Errors with HTTPException.
Request bodies with Pydantic
For POST, the data arrives as JSON in the body. You describe its shape with a Pydantic model: a class that inherits from BaseModel and lists the fields with their types. Field() adds rules such as "greater than zero". Use the model as the type of an argument, and FastAPI reads the body, checks it and hands you a ready object.
You wrote no checking code. The 422 response lists every problem, says where it is (loc) and what is wrong (msg), so the caller can fix the request. This is the real response body, with nothing left out.
Rule: never trust input. Anyone can send anything to a public API. A model with strict types and limits is your front door. Rules that depend on the data, such as "is there enough money", still have to be checked in your own code.
Response models and status codes
A response model describes what goes out. FastAPI keeps only the fields named in it, so nothing private slips into a response by accident. status_code= sets the code for a successful call: 201 for something created, 204 for "done, nothing to return".
Habit: use separate models for input and output. AccountCreate has no id, because the server chooses it. AccountOut has no pin, because nobody should read it back.
Errors with HTTPException
When a request cannot be carried out, raise HTTPException with the right status code and a short message. FastAPI stops the function there and sends that code, with the message under the key detail.
Which code for "not enough money"? The request is well formed, so it is not a 422. It clashes with the current state of the account, which is what 409 Conflict is for. Some APIs answer 400 here. Either is acceptable. Choose one, write it down and use it everywhere.
Pitfall: an exception you did not expect becomes a 500 with a plain "Internal Server Error" body. That is correct: the details belong in your logs, not in the response. See Python Debugging and Logging.
The bank API
Here is the whole small bank: open an account, read one, list them all, deposit and transfer. The accounts live in a dict in memory, so they vanish when the program stops. That is fine for learning. This block is longer than the others because it is a complete program.
data.model_dump() turns a Pydantic model into a dict (Pydantic 1 called it .dict()). The helper find() raises the 404 in one place, so every endpoint that needs an account gets the same behaviour. The inheritance used for Account is covered in Python Classes.
Limits: a dict in memory is lost on restart and is not shared between server processes. A real bank API keeps its data in a database and runs each transfer in a transaction. Connecting the API to a database sketches that.
Dependencies with Depends
The bank API above reaches for a global dict. That makes it hard to test and hard to change. Dependency injection turns it round: the endpoint declares what it needs as an argument, and FastAPI supplies it. Depends(get_bank) means "call get_bank() and pass me the result".
The endpoint does not know or care where the bank comes from. Today it is a dict in memory. Later it can be a database session, and only get_bank changes. Tests replace it through app.dependency_overrides without touching the endpoint code.
Design: this is the dependency inversion principle in action. The endpoint depends on "something that gives me a bank", not on one fixed object. See Design Principles, and Python Testing for fakes in tests.
Automatic docs
FastAPI reads your paths, type hints and models and builds a description of the whole API in a standard format called OpenAPI. From that it serves interactive documentation with no extra work.
| URL on the running server | What you get |
|---|---|
/docs | Swagger UI: every endpoint listed with its parameters, models and status codes, and a "Try it out" button that sends real requests from the browser |
/redoc | The same information as a reading-friendly reference page |
/openapi.json | The raw description as JSON, which other tools can use to generate client code |
Start the bank API with uvicorn main:app --reload and open http://127.0.0.1:8000/docs. It is the fastest way to try an endpoint by hand. The same description is available from Python.
Benefit: the docs are built from the code, so they cannot drift out of date the way a hand-written document does.
def or async def
FastAPI accepts both kinds of endpoint, and treats them differently. A plain def endpoint is run in a worker thread from a pool, so a call that blocks, such as a sqlite3 query, requests.get() or time.sleep(), holds up only that thread. An async def endpoint runs on the server's single event loop. It can serve a great many requests at once, but only if everything slow inside it is awaited through an async library, such as httpx.AsyncClient or an async database driver. One blocking call inside an async def endpoint freezes the event loop, and every other request waits behind it. So the rule is short: use async def when the body awaits async code, and plain def otherwise. If you are not sure, def is the safe choice. The event loop and await are explained in Python Async IO.
Testing the API
The TestClient used all through this part is the testing tool. A test sends a request and asserts on the status code and the JSON. This needs pip install pytest. In a real project the app lives in main.py and the test file starts with from main import app. Here both are in one file so that the example is complete.
Test the failures as carefully as the successes: the status code, and that nothing changed. Fixtures and assert are covered in Python Testing.
Beyond the basics
Part 4 of 4 · what a real API also needs
This part is a map, not a course. Each chapter names an idea you will meet as soon as an API has real users, says what it is for, and shows the smallest example that makes it concrete. The Flask example needs pip install flask. The others use FastAPI as before, or the standard library.
- Flask and Django, and when to pick which
- Authentication: API keys, tokens, password hashing
- CORS
- Pagination and filtering
- Idempotency: a retried payment must not pay twice
- Connecting the API to a database
- Deployment
- Quick reference
- Common mistakes
Flask and Django
FastAPI is one of three frameworks you will hear about most. Flask is the minimal one: routing, requests and responses, and little else. You add the rest yourself. Here is one endpoint in Flask, called through Flask's own test client.
The shape is the same: a decorator with a method and a path, a function, a return value. The difference is what you get for free. Flask does not check the body, so data["owner"] is your job to validate. Django sits at the other end: one large framework with its own ORM, migrations, user accounts, forms and an admin site, built for complete websites. APIs in Django are usually written with the add-on Django REST Framework.
| Framework | Style | Gives you | Pick it when |
|---|---|---|---|
| FastAPI | API first | Validation from type hints, automatic docs, async support | You are building a JSON API, new, today |
| Flask | Minimal | Routing and little else. You choose every other piece | A small service or a few pages, or you want full control |
| Django | Batteries included | ORM, migrations, admin site, users and logins, templates | A full website with pages, users and a database, built by a team |
Advice: the ideas on this page (methods, status codes, validation, errors, tests) are the same in all three. Learn one well and the others are a change of spelling.
Authentication
Authentication answers "who is calling?". Authorisation answers "may they do this?". A bank API needs both: know that the caller is Hemkesh (otherwise 401), then refuse to show him Sandhya's account (403).
| Idea | How it works | Typical use |
|---|---|---|
| API key | A long random string given to each client program, sent in a header with every request | One server calling another |
| Bearer token | The user logs in once with a password and gets back a token that expires. Later requests send Authorization: Bearer <token> | Apps with human users |
| JWT | A common token format: a small signed piece of JSON that names the user and an expiry time. The server checks the signature instead of looking the token up. It is signed, not secret, so never put private data in it | Bearer tokens |
An API key check fits naturally into a dependency. Any endpoint that lists it is protected.
Never store plain passwords
If the database is ever copied, every stored password is exposed. Store a hash instead: the output of a one-way function, from which the password cannot be worked back. Add a random salt per user, so that two people with the same password get different hashes, and use a deliberately slow function made for passwords. At login, hash what was typed and compare.
Advice: do not invent your own login system for anything real. Use a maintained library for password hashing (argon2 or bcrypt) and for tokens, and send everything over HTTPS. hmac.compare_digest() compares in a way that does not leak, through timing, how much of a secret was right.
CORS
A browser will not let JavaScript on one site read responses from an API on a different site unless that API says it is allowed. This rule is called CORS, cross-origin resource sharing, and the API gives permission through response headers that list the sites it trusts. In FastAPI you add CORSMiddleware with those sites in allow_origins. It applies only to browsers: a Python script calling your API is not affected.
Pagination and filtering
An endpoint that returns "all the rows" works until the table has a million of them. Pagination returns one page at a time. The simplest form uses two query parameters: limit, how many to return, and offset, how many to skip. Query() sets limits on them, so nobody can ask for a billion.
Returning total lets the client work out how many pages there are. Filters such as account_id are ordinary query parameters. With a database behind the API, these become WHERE, ORDER BY, LIMIT and OFFSET in the query, as in Selecting, filtering and ordering.
Idempotency
A client sends POST /transfers and the connection drops before the answer arrives. Did the transfer happen? The client cannot know. If it does not retry, the money may never have moved. If it retries, the money may move twice.
The standard cure is an idempotency key. The client invents a unique id for the payment and sends it in a header. The server remembers the result for each key. If the same key arrives again, the server does nothing new and returns the saved result.
The retry returned the first result and the balances moved once. A new key is a new payment. In real code the client generates the key with uuid.uuid4(), and the server stores it in the database inside the same transaction as the transfer, with a UNIQUE constraint, so that two copies of the request arriving together cannot both pass the check.
Kept short: this example leaves out the 404 and 409 checks from The bank API to keep the idea visible. A real endpoint needs both.
Connecting the API to a database
The two pages meet here. A dependency opens a SQLAlchemy session for each request and closes it afterwards. The endpoint runs the transfer in one transaction, using the single safe UPDATE from Two transfers at once. If an HTTPException is raised inside session.begin(), the transaction is rolled back before the error response is sent.
This block is an illustrative sketch and is not run with the page. It needs both fastapi and sqlalchemy, it creates a database file, and it has no calls at the end to show output. It uses a file because an in-memory SQLite database is private to one thread, and FastAPI runs def endpoints in worker threads.
Design: the endpoint asks for "a session" and does not build one. In tests, app.dependency_overrides[get_session] can supply a session on a throwaway database, exactly as in Dependencies with Depends.
Deployment
Deployment means running the API where its users can reach it. uvicorn main:app --reload is for your own machine only. In production you run a production server without --reload, with several worker processes so that all the CPU cores are used: uvicorn for FastAPI, or gunicorn for Flask and Django. In front of it sits a reverse proxy such as nginx, or a cloud load balancer, which receives traffic from the internet, handles HTTPS and passes requests on to your workers. Configuration, such as the database URL, API keys and the list of allowed CORS sites, comes from environment variables set on the server, never from the code, so the same code runs unchanged in development, testing and production.
Remember: several workers are several separate processes. Anything kept in a Python variable, like the dicts on this page, is not shared between them. Shared state belongs in the database.
Quick reference
Calling an API
| Task | requests (httpx is the same unless noted) |
|---|---|
| GET with query parameters | requests.get(url, params={"min_balance": 4000}, timeout=5) |
| POST a JSON body | requests.post(url, json={"owner": "..."}, timeout=5) |
| Status code | response.status_code |
| Body as Python data | response.json() |
| Raise on 4xx and 5xx | response.raise_for_status() |
| Send a token | headers={"Authorization": f"Bearer {token}"} |
| Reuse connection and headers | requests.Session(), or httpx.Client(base_url=..., timeout=5) |
| Catch everything | except requests.RequestException, or except httpx.HTTPError |
| Standard library only | urllib.request.urlopen(url, timeout=5) and json.loads(response.read()) |
Building an API with FastAPI
| Task | Code |
|---|---|
| Create the app | app = FastAPI() |
| Endpoint | @app.get("/accounts/{account_id}"), also post, put, patch, delete |
| Path parameter | def get_account(account_id: int) |
| Query parameter | def list_accounts(limit: int = 10, owner: Optional[str] = None) |
| Limits on a parameter | limit: int = Query(default=10, ge=1, le=100) |
| JSON body | def create(data: AccountCreate), a Pydantic BaseModel |
| Rules on a field | amount: int = Field(gt=0) |
| Shape of the response | @app.get(..., response_model=Account) |
| Success status | @app.post(..., status_code=201) |
| Error response | raise HTTPException(status_code=404, detail="Account not found") |
| Read a header | x_api_key: Optional[str] = Header(default=None) |
| Dependency | bank: Bank = Depends(get_bank) |
| Swap it in a test | app.dependency_overrides[get_bank] = lambda: test_bank |
| Call the app in a test | TestClient(app).get("/accounts/1") |
| Run for development | uvicorn main:app --reload, docs at /docs |
What needs installing
| Examples | Need |
|---|---|
JSON, urllib, the practice server, password hashing | Standard library only |
| requests and httpx, errors, tokens, sessions, retries | pip install requests httpx |
Everything with FastAPI and TestClient | pip install fastapi uvicorn httpx |
| The test file | pip install pytest as well |
| The Flask example | pip install flask |
| The database sketch | pip install sqlalchemy as well |
Common mistakes
| Mistake | What goes wrong | Do this |
|---|---|---|
| No timeout on a call | One slow server hangs your program for ever | Pass timeout= on every request |
| Returning 200 for errors | Clients think it worked, and retries, monitoring and caches all misbehave | Raise HTTPException with the true status code |
| Trusting input | Negative amounts, missing fields and oversized requests reach your logic | Pydantic models with types and limits, then your own business checks |
Blocking calls in async def endpoints | One slow request freezes the event loop and every user waits | Use plain def, or await an async library |
| Leaking stack traces | Error pages reveal file paths, SQL and library versions to attackers | Return a short message. Log the details on the server. Keep debug mode off in production |
| Secrets in the code | Keys and passwords end up in version control | Environment variables |
| Retrying a payment blindly | The customer pays twice | Idempotency keys |
| Returning every row | Responses grow until they time out | Pagination with a maximum limit |
| Verbs in URLs | An API nobody can guess their way around | Nouns for paths, methods for actions |
That completes the practice path. The bank now has data that survives, rules that protect it and an API in front of it. The next path is about how to shape code like this so that it stays easy to change, starting with Design Principles.