QA and test automation
Playwright: UI and API Test Automation
Playwright drives a real browser and calls HTTP APIs from code, so a program can check your application the way a tester would. This page starts from zero: what UI and API automation mean, how to install Playwright for Python, and a complete set of tests for a tiny bank app. Every test on the page was run with Playwright 1.60 and pytest.
What it is and why
Part 1 of 5 · the ideas, before any code
Manual testing vs automation
A manual test is a person following steps: open the site, type a username, click "Sign in", check that the dashboard appears. An automated test is a small program that performs the same steps and checks the same result, in seconds, as often as you like.
| Manual | Automated | |
|---|---|---|
| First run | Fast: just do it | Slower: someone has to write the test |
| Every run after that | Same effort again | Free, and identical each time |
| Best for | Exploring, new features, "does this feel right?" | Checks repeated on every release: login, payments, critical flows |
| Finds | Surprises nobody thought of | Things that used to work and broke (regressions) |
Regression: a feature that worked yesterday and is broken today because of an unrelated change. Catching regressions is the main job of an automated test suite.
UI testing vs API testing
Most applications have two doors. The UI (user interface) is the web page a person sees and clicks. The API is the set of URLs the page, the mobile app and other programs call behind the scenes to read and change data, usually exchanging JSON.
| UI automation | API automation | |
|---|---|---|
| What the test does | Opens a browser, clicks and types like a user | Sends HTTP requests, reads the replies |
| What it checks | What the user sees: text, buttons, pages, errors | Status codes, JSON data, rules, permissions |
| Example | "After signing in, the heading says Welcome" | "GET /api/accounts/2 returns balance 12000" |
| Speed | About a second per test | A few milliseconds per test |
| Breaks when | The page layout or wording changes | The data contract changes |
| How many to write | Few: the journeys that matter most | Many: every rule and every error case |
Rule of thumb: test the rules through the API, because it is fast and stable. Test through the UI only what a user must be able to do end to end. This shape, many fast tests below and few slow ones on top, is called the test pyramid.
What Playwright is
Playwright is a free, open-source automation library from Microsoft. One library does both jobs on this page:
- UI automation: it starts Chromium (the engine of Chrome and Edge), Firefox or WebKit (the engine of Safari) and controls it.
- API automation: it sends HTTP requests without any browser.
Three features make it friendly for beginners:
| Feature | What it saves you from |
|---|---|
| Auto-waiting | Writing sleep(3) and hoping the page has loaded. Playwright waits until the button is there and clickable. |
| Locators by role and label | Fragile addresses such as div > form > input:nth-child(2). You say "the field labelled Username". |
| Codegen and trace viewer | Guessing. One tool records your clicks as code, the other replays a failed test step by step. |
Playwright exists for JavaScript/TypeScript, Python, Java and .NET. This page uses Python with pytest. The ideas and method names are almost the same in the other languages.
How it works
Words you will meet in every Playwright test:
| Word | Meaning |
|---|---|
| Browser | One running Chromium, Firefox or WebKit |
| Context | A fresh, private browser profile: its own cookies and logins. Each test gets a new one, so tests cannot disturb each other. |
| Page | One tab. Almost everything you do is a method on page. |
| Locator | A description of how to find an element, such as "the button named Sign in" |
| Headless | The browser runs without a visible window. The default, and what servers use. Headed means you can watch. |
| Assertion | A check that must be true or the test fails: expect(...) for pages, assert for data |
| Fixture | A pytest helper that prepares something for a test, for example a ready page |
Playwright, Selenium and Cypress
You will hear these three names together. A fair short comparison:
| Playwright | Selenium | Cypress | |
|---|---|---|---|
| Languages | JS/TS, Python, Java, .NET | Most languages | JS/TS only |
| Browsers | Chromium, Firefox, WebKit | All major browsers | Chromium family, Firefox, WebKit (experimental) |
| Waiting | Automatic | Mostly written by you | Automatic |
| API testing built in | Yes | No, add another library | Yes |
| Known for | Speed, reliability, tooling | The oldest and most widely used | A pleasant interactive runner |
For API tests only, tools such as Postman, or pytest with the requests library, do the job too. The reason to use Playwright for both is one tool, one report, and tests that mix API and UI steps, as shown later.
Setup
Part 2 of 5 · install, get an app to test, run a first script
Install
You need Python 3.9 or newer. Make a project folder with its own virtual environment, then install two things: the Python package and the browsers it controls.
Two steps, both needed: pip install pytest-playwright is the only Python package to install. It brings in playwright (the library), pytest (the test runner) and pytest-base-url as dependencies. But pip does not download a browser. That is what playwright install chromium does. Skip it and the API tests still run, while every UI test fails with a message telling you to run playwright install.
playwright installwith no name downloads all three browsers. Start with Chromium.- Playwright uses its own browser copies, kept apart from the Chrome on your machine, so the browser version always matches the library.
- On Linux servers,
playwright install --with-deps chromiumalso installs the system libraries the browser needs.
The app under test
To learn testing you need something to test that will not change under you. Save the file below as bank_app.py. It is a tiny bank written with Python's standard library only, so there is nothing more to install.
| Door | Address | What it does |
|---|---|---|
| UI | / | Sign-in page. The only user is rishikesh with password secret123. |
/dashboard | Shows the balance and a Deposit button. Sends you back to / if you are not signed in. | |
| API | POST /api/login | Takes a username and password, returns a token |
GET /api/accounts | Lists the accounts. Needs the token. | |
GET /api/accounts/2 | One account. Needs the token. | |
POST /api/accounts/2/deposit | Adds money. Needs the token. |
Show bank_app.py (about 110 lines; you do not need to understand it to test it)
Start it in one terminal and leave it running. Open the address in your own browser and sign in once by hand, so you know what the tests will be doing.
A toy, on purpose: one user, a fixed token, data kept in memory. Never build real login this way. It exists so the tests have something predictable to check.
First UI script
In a second terminal (with the virtual environment active), save and run this. It opens the sign-in page, fills the form, clicks the button and takes a screenshot of what it finds.
- No window appeared: the browser ran headless. Change the launch line to
p.chromium.launch(headless=False, slow_mo=500)to watch it, half a second per step. dashboard.pngis now in the folder. Open it: that is what the browser saw.- There is no
sleepanywhere.click()waited for the button, and reading the heading waited for the new page.
First API script
Now the other door. This script never starts a browser. It sends three HTTP requests and prints each status code and JSON reply.
| Status code | Meaning | Seen here when |
|---|---|---|
200 | OK | The request worked |
400 | Bad request | The deposit amount is not valid |
401 | Not authenticated | The token is missing or wrong |
404 | Not found | There is no account with that id |
500 | Server error | The application crashed: always a bug |
These two scripts only print. A test must decide pass or fail by itself. That is what pytest adds next. You can stop bank_app.py now: from here on the tests start it themselves.
UI automation
Part 3 of 5 · tests that use the page like a person
Shared setup: conftest.py
pytest reads a file named conftest.py before the tests and makes its fixtures available to all of them. A test asks for a fixture by naming it as a parameter. The pytest-playwright package already provides page (a fresh tab in a fresh context for every test) and playwright. This file adds three more.
| Fixture | Gives the test | Used by |
|---|---|---|
page (built in) | A new browser tab | UI tests |
base_url | The address of the running bank, so tests can write page.goto("/") and not a full URL | Both |
api | An HTTP client pointed at the bank, with balances reset | API tests |
token | A login token, fetched through the API | API tests that need to be signed in |
In a real project the application is already running somewhere, so you delete the server lines and pass the address when you run: pytest --base-url https://staging.example.com.
Testing the sign-in page
A test is a function whose name starts with test_, in a file whose name starts with test_. Each one follows the same three steps: arrange (get to the starting point), act (do the thing), assert (check the result).
- One happy path, the rest are problems. Wrong password and "not signed in" are the cases a hurried manual tester skips.
- Each test starts clean. The last test gets a new context with no cookies, which is why it is not signed in even though the test before it was.
sign_inis an ordinary helper function. Repeated steps belong in one place.
Locators: finding things
A locator says how to find an element. Prefer the ones that describe the page the way a user would. They keep working when a developer restyles the page.
| Locator | Finds | Use it |
|---|---|---|
page.get_by_role("button", name="Sign in") | A button, link, heading, checkbox... by what it is and its visible name | First choice |
page.get_by_label("Username") | The form field with that label | Form fields |
page.get_by_placeholder("Search") | A field by its grey hint text | Fields with no label |
page.get_by_text("Welcome") | Any element showing that text | Messages, paragraphs |
page.get_by_test_id("balance") | An element with data-testid="balance" | When nothing visible identifies it. Ask developers to add the attribute. |
page.locator("#amount") | Anything by CSS selector or XPath | Last resort: it breaks when the HTML structure changes |
Narrow a search by chaining, and pick from several matches explicitly:
Strict by default: if a locator matches two elements and you try to click it, Playwright stops with an error that lists both. That is a feature: it never silently clicks the wrong one.
Inspecting elements with DevTools
Before you can locate an element you have to look at it. Every browser has built-in developer tools (DevTools) that show the HTML behind what is on screen. The steps below are for Chrome and Edge; Firefox is almost identical.
- Open the inspector on an element: right-click it on the page and choose Inspect. Or press
F12(Cmd+Option+Ion a Mac), click the arrow icon at the top left of DevTools, then click the element. - Read the highlighted line in the Elements panel. That is the element's HTML: its tag (
input,button), attributes (id,name,data-testid) and text. Hovering over a line highlights the matching area on the page. - Copy an address for it: right-click the highlighted line, open Copy, and pick one of the options in the table.
| Copy option | Gives, for the Amount field of the bank dashboard | Use in a test? |
|---|---|---|
| Copy selector | #amount | Fine when it is a short id like this. Long chains such as div > div:nth-child(3) are fragile. |
| Copy XPath | //*[@id="amount"] | Same: fine when it is based on an id |
| Copy full XPath | /html/body/label/input | Avoid. It is the path from the top of the page and breaks when anything above the element moves. |
| Copy JS path | document.querySelector("#amount") | For the Console, not for tests |
| Copy element | The HTML itself | Handy for a bug report |
Check a selector before you put it in a test. Two ways, both inside DevTools:
- In the Elements panel press
Ctrl+F(Cmd+F) and type a CSS selector or an XPath. It shows "1 of 1" when exactly one element matches, which is what you want. - In the Console panel,
$$("css selector")and$x("xpath")return the list of matches.
Find the role and name for get_by_role: select the element, then open the Accessibility tab beside Styles. It shows, for example, Role: button, Name: "Deposit", which is exactly what page.get_by_role("button", name="Deposit") looks for.
Easier still: Playwright has its own picker. Run playwright codegen <url>, press "Pick locator" and click any element: it prints the best locator, usually a role or label one, ready to paste. Use DevTools to understand the page and the picker to write the locator.
CSS selectors and XPath in Playwright
Older pages often have no labels or test ids, and many teams already hold a list of XPaths from Selenium days. page.locator() accepts both CSS and XPath. This test signs in to the bank using only those, and passes:
| I want | CSS | XPath |
|---|---|---|
| By id | #amount | //*[@id='amount'] |
| By attribute | input[name='username'] | //input[@name='username'] |
| By class | .error | //*[contains(@class, 'error')] |
| By exact text | Not possible in plain CSS | //button[text()='Sign in'] |
| By part of the text | Not possible in plain CSS | //button[contains(text(), 'Sign')] |
| Inside something | form input | //form//input |
| Parent of something | Not possible in plain CSS | //input[@name='username']/.. |
| The second match | li:nth-child(2) | (//li)[2] |
Trap with a full XPath: Playwright treats a string starting with // or .. as XPath. A copied full XPath starts with a single /, is read as CSS and fails with Unexpected token "/" while parsing css selector. Put xpath= in front, as in the test above.
- Good XPath is short and names a meaning: an id, a
name, adata-testid, a visible text. Bad XPath counts positions:/div[2]/div[1]/span[3]. - Order of preference: role or label locator, then test id, then a short CSS or XPath, and a copied full XPath never.
- Everything else still applies to these locators: auto-waiting,
expect, strictness.
Actions: doing things
| Action | What a user would do |
|---|---|
page.goto("/dashboard") | Type an address and press Enter |
locator.click() | Click. Also dblclick() and hover(). |
locator.fill("text") | Clear a field and type into it |
locator.press("Enter") | Press a key |
locator.check() / uncheck() | Tick or untick a checkbox or radio button |
locator.select_option("savings") | Choose from a dropdown |
locator.set_input_files("statement.pdf") | Pick a file to upload |
page.go_back(), page.reload() | Browser buttons |
page.screenshot(path="x.png") | Take a picture of the page |
Before each action Playwright checks that the element exists, is visible, is not moving, is enabled and is not covered by something else. It retries for up to 30 seconds, then fails with a message saying which check did not pass.
expect: checking things
expect() is Playwright's assertion for pages. It keeps trying for up to 5 seconds until the condition is true, so it copes with pages that update a moment after a click.
| Assertion | Passes when |
|---|---|
expect(page).to_have_title("Bank: Sign in") | The tab title matches |
expect(page).to_have_url("/dashboard") | The address matches |
expect(locator).to_be_visible() | The element is on screen. Also to_be_hidden(). |
expect(locator).to_have_text("5250") | Its text is exactly this. to_contain_text() for part of it. |
expect(locator).to_have_value("250") | A form field holds this value |
expect(locator).to_be_enabled() | A button can be clicked. Also to_be_disabled(), to_be_checked(). |
expect(locator).to_have_count(3) | The locator matches exactly 3 elements |
expect(locator).not_to_be_visible() | Any assertion can be reversed with not_ |
The beginner mistake: assert locator.inner_text() == "5250" reads the text once, right now, and fails if the page has not updated yet. expect(locator).to_have_text("5250") waits. Use expect for anything on a page, and plain assert only for data you already hold, such as an API reply.
A test that waits for the page
On the dashboard, clicking Deposit does not load a new page. JavaScript calls the API in the background and changes the balance on screen when the reply arrives. This is how most modern sites behave, and it is where tests written with fixed sleeps go wrong.
The last two checks run right after the click, before the reply has arrived. They pass because expect keeps looking until the number changes.
Never write time.sleep(2) in a test. Too short and the test fails on a slow day. Too long and a thousand tests waste half an hour. Wait for the thing you care about, which is what expect does.
Running tests
| Command | Effect |
|---|---|
pytest | Run everything, headless |
pytest tests/test_ui_login.py | One file |
pytest -k wrong_password | Only tests whose name contains that text |
pytest -v | One line per test with its name |
pytest -x | Stop at the first failure |
pytest --headed --slowmo 500 | Show the browser, half a second per step |
pytest --browser firefox --browser webkit | Run on other browsers (install them first) |
pytest --device "iPhone 13" | Emulate a phone: screen size, touch, user agent |
pytest --screenshot only-on-failure | Save a picture when a test fails |
pytest --video retain-on-failure | Keep a video of failed tests |
pytest --tracing retain-on-failure | Keep a full trace of failed tests |
pytest --base-url https://staging.example.com | Test another environment |
The whole suite built on this page, UI and API together:
Reading a failure
A failing test is the product doing its job. Here the deposit test was changed to expect 5300 when the right answer is 5250.
Read it from the top: the line that failed, what was expected, what was actually there, and a log showing Playwright saw 5000, then 5250, and kept checking for 5 seconds. The two options on the command line left evidence in test-results/:
The trace viewer opens a window with a timeline of every step. Click a step to see the page before and after it, the network requests it caused and the browser console. It is the fastest way to understand a test that fails on a server and passes on your machine.
Codegen: record a test
With the bank running, this opens a browser and a second window. Click through the site by hand and the second window writes the Python for each step, choosing good locators for you.
- Use it to find locators and to get a first draft. The "pick locator" button shows the best locator for any element you point at.
- Then clean up: recorded code has no assertions unless you add them with the toolbar, repeats itself, and has no helper functions.
To pause a test and inspect the live page, put page.pause() in it and run with --headed, or run PWDEBUG=1 pytest to step through from the start.
API automation
Part 4 of 5 · tests that talk to the server directly
Requests and replies
An API call is a request (a method, a URL, maybe headers and a body) and a response (a status code, headers and a body, usually JSON). The api fixture from conftest.py is Playwright's request client.
| Method | Means | With the api fixture |
|---|---|---|
GET | Read | api.get("/api/accounts", params={"kind": "savings"}) |
POST | Create, or perform an action | api.post("/api/login", data={"username": "..."}) |
PUT / PATCH | Replace / change part | api.put(...), api.patch(...) |
DELETE | Remove | api.delete("/api/accounts/2") |
| On the reply | Gives you |
|---|---|
reply.status | The status code, for example 200 |
reply.ok | True for any 2xx status |
reply.json() | The body as a Python dict or list |
reply.text() | The body as a string |
reply.headers | The response headers as a dict |
Passing a dict as data= sends it as JSON. Headers, such as the login token, go in headers=.
Testing the accounts API
Same arrange, act, assert shape as the UI tests, with plain assert because a reply is data you already hold: there is nothing to wait for.
@pytest.mark.parametrizeruns one test once per value. Four bad amounts make four tests, which is why seven functions count as ten.- Check the change, not just the reply. The deposit test reads the account again to prove the balance was really stored.
- Tests do not depend on each other. The
apifixture resets the balances first, so the tests pass in any order. - Ten API tests in under a second, including starting the app. The four sign-in UI tests alone took longer. That is the test pyramid in numbers.
What to check in an API test
| Check | Question it answers | Example on this page |
|---|---|---|
| Status code | Did the server say the right thing? | assert reply.status == 200 |
| Body values | Is the data correct? | reply.json()["balance"] == 12500 |
| Body shape | Are all the fields there, with the right types? | The list of owners |
| Side effect | Did it really change what it claims? | Read the account again after the deposit |
| Bad input | Does it refuse nonsense politely (4xx, not 500)? | Amounts 0, -50, "ten", 2.5 |
| Authentication | Is it locked without a login? | No token gives 401 |
| Missing things | What about an id that does not exist? | Account 99 gives 404 |
Negative tests are the ones that send wrong input on purpose. For an API, write more of them than positive tests: there is one way to be right and many ways to be wrong.
Putting it together
Part 5 of 5 · mixing both, keeping tests tidy, running them automatically
API for setup, UI for the check
The strongest reason to have both in one tool. To test that the dashboard shows a deposit, you do not need to click the deposit through the UI as well. Create the situation through the API in milliseconds, then use the browser only for the part you are testing.
The first test is the pattern to copy: set up by API, verify by UI. The second is explained next.
Faking a server reply
How do you test what the page shows when the server fails, without breaking the server? page.route() intercepts requests the browser makes and lets the test answer them itself. In test_mocked_api_failure_shows_error above, the deposit request never reaches the bank: Playwright replies with a 500 error, and the test checks that the page shows the message and leaves the balance alone.
| Inside the handler | Effect |
|---|---|
route.fulfill(status=500, json={...}) | Answer with a made-up reply |
route.abort() | Pretend the network is down. Also used to block ads or images. |
route.continue_() | Let the request through unchanged |
Use it sparingly. A test with a faked reply proves the page handles that reply. It proves nothing about the real server. Keep fakes for cases that are hard to produce for real: errors, slow replies, empty lists.
Sign in once, reuse everywhere
If 200 UI tests each fill the sign-in form, you spend minutes testing login 200 times. Sign in once, save the browser's cookies to a file (the storage state), and start every other test already signed in.
scope="session"means the fixture runs once for the whole run, not once per test.browser_context_argsis a fixture frompytest-playwright. Overriding it changes how every test's context is created, here by loading the saved cookies.- Put these two fixtures in
conftest.pyto apply them to a whole folder. Keep the login tests themselves in a folder without them.
Page objects
When the label "Username" changes to "Email", you do not want to edit 40 tests. A page object is a class that knows how to find and use one page. Tests call its methods and never mention locators. A change to the page is then a change in one file.
When to bother: not for the first ten tests. Start with helper functions like sign_in. Move to page objects when the same locators appear in several files.
Project layout
Naming files test_api_* and test_ui_* lets you run one kind: pytest -k api for a fast check on every change, everything before a release. Options you always want, such as --tracing retain-on-failure, can go in pytest.ini on a line starting addopts =.
A real case: Next.js frontend, FastAPI backend
A common setup: the web pages are a Next.js application, and they fetch their data from a Python FastAPI backend. A QA engineer is asked to automate one journey: log in, open a certain page, type a prompt, click Submit, and check that an answer appears.
The key idea: a UI test only needs the address of the frontend. The test drives the browser, and the browser calls FastAPI by itself, exactly as it does for a real user. The test does not care what the frontend or backend is written in.
- Get a place to test and a test account. A staging address where both parts are deployed, and a login made for automation. Never a real person's account.
- Install, as in Install.
- Do the journey once by hand with
playwright codegen https://staging.example.comand note the locators it suggests. Inspect anything it cannot name well. - Write the test below, replacing the labels and addresses with the real ones.
- Run it with
pytest --base-url https://staging.example.com --headedto watch, and without--headedin CI.
This file was run, and passes, against a small stand-in app with the same pages and a prompt endpoint that takes 1.5 seconds to answer. It was not run against a real Next.js and FastAPI application: the labels, addresses and the /api/prompts path are examples.
| Concern in this kind of app | What to do |
|---|---|
| Password in the test file | Never. Read it from environment variables, as here, and store it as a secret in CI. |
| Every test logs in again | Sign in once and reuse the saved state |
| Login through Google, SSO or a one-time code | Hard and fragile to automate through the UI. Ask the developers for a test-only login or a token you can set. |
| Next.js class names change on every build | Do not locate by class. Use role, label and placeholder, and ask the frontend developers to add data-testid to the prompt box and the answer area. |
| The answer takes many seconds, or streams in piece by piece | Wait for the response with page.expect_response or for a "finished" state on the page, with a longer timeout. Never a fixed sleep. |
| The answer comes from an AI model and differs every run | Check that an answer appears, is not empty and contains expected words or structure. Do not compare with an exact text. |
| You want to test the page without waiting for the model | Fake the reply with page.route, as the second test does |
| The backend's own rules: empty prompt, no login, very long input | API tests straight against FastAPI. Its /docs page lists every endpoint with the fields it expects. |
How to split the work: one or two UI tests for the journey, a handful of faked-reply tests for how the page behaves, and many API tests for the rules. That is the test pyramid applied to this application.
Flaky tests
A flaky test sometimes passes and sometimes fails with no change to the application. Flaky tests are worse than no tests: people stop believing red results. The usual causes and their cures:
| Cause | Cure |
|---|---|
| Fixed sleeps, or reading a value before the page updated | expect(...), which waits for the condition |
| Tests share data: one test's deposit changes another's balance | Reset or create fresh data in a fixture, as api does here |
| Tests must run in a certain order | Each test arranges everything it needs |
Locators tied to layout: div:nth-child(3) | Role, label and test-id locators |
| Depending on today's date, random values or a third-party site | Fixed test data, and faked replies for services you do not own |
| A slow shared test environment | Longer timeouts for that environment, never sleeps |
Running in CI
CI (continuous integration) is a server that runs your tests automatically on every code change, so a regression is caught minutes after it is written. The steps are the same ones you typed by hand. This is the shape of a GitHub Actions workflow for the project; it was not run for this page, so treat it as a starting point.
CI machines have no screen, which is why headless is the default. When a test fails only in CI, download the uploaded trace and open it with playwright show-trace.
Cheat sheet
| I want to | Write or run |
|---|---|
| Install | pip install pytest-playwright, then playwright install chromium |
| Open a page | page.goto("/") |
| Find a button | page.get_by_role("button", name="Sign in") |
| Find a form field | page.get_by_label("Username") |
| Type and click | .fill("text"), .click() |
| Check the page | expect(locator).to_have_text("...") |
| Call an API | api.get(url, headers=...), api.post(url, data={...}) |
| Check an API reply | assert reply.status == 200, reply.json() |
| Fake a server reply | page.route(pattern, lambda route: route.fulfill(...)) |
| Watch the browser | pytest --headed --slowmo 500 |
| Record a test | playwright codegen <url> |
| Debug a failure | pytest --tracing retain-on-failure, then playwright show-trace trace.zip |
| Run only API tests | pytest -k api |
For pytest itself (fixtures, parametrize, marks) see Python Testing. Back to the QA path.