Python basics, part 6 of 6
Python Errors and Files
Real programs meet bad input, missing files and dates. This page covers all three: how to catch and raise errors, how to read and write files, and how to work with dates and times. The first two parts are rewrites of the Pythofrank lessons on exception handling and file handling from February 2017, updated from Python 2.7 to Python 3. The part on dates is new.
Errors
Part 1 of 3 · when something goes wrong
When Python cannot carry out a statement, it creates an exception: an object that describes what went wrong. If nothing deals with it, the program stops. This part shows how to read the message, how to catch an exception and carry on, and how to create your own.
- Reading a traceback: what Python prints when it stops
- try and except: catch an error and carry on
- Catching specific exceptions: ZeroDivisionError, TypeError, ValueError
- Things that are not there: KeyError, IndexError, FileNotFoundError
- Several exceptions in one try: one handler each, or one for a group
- else and finally: code for "no error" and for "always"
- Raising an exception: report a problem yourself
- Your own exception classes: InsufficientFunds
- Common built-in exceptions: a table to look things up
Reading a traceback
This program asks for the balance of a person who has no account. It is marked as not runnable here because it stops with an error on purpose.
The text after the first line of output is a traceback. Python prints the full path of the file, which is shortened to balances.py here. Read the traceback from the bottom up.
- The last line is the answer: the kind of exception (
KeyError) and its message (the key that was not found). - The lines above it show the path Python took. The lowest pair is where the error happened: line 5, inside
show_balance. - The pair above that is who called it: line 9 of the file.
Notice: Done was never printed. An exception that nobody catches ends the program at that point.
try and except
Put the risky statements in a try block. If one of them raises an exception, Python jumps to the matching except block and then carries on after it. If nothing goes wrong, the except block is skipped.
Concept: the jump happens at the exact statement that fails. The rest of the try block is skipped.
Catching specific exceptions
Every kind of error has its own class. Name the one you expect after except. Add as error to get the exception object, which prints as its message.
Do not use a bare except
The 2017 lesson often wrote except: with no name. That catches everything, including typing mistakes in your own code and Ctrl+C from the keyboard, so real bugs stay hidden. Here a misspelled name is reported as bad input.
Rule: catch the most specific exception you can. With except ValueError the typo would have stopped the program with a clear NameError, which is what you want for a bug.
Things that are not there
Three exceptions mean "you asked for something that does not exist": a dictionary key, a list position, or a file.
Tip: you can often avoid the error. details.get("profession", "unknown") returns a default when the key is missing, and len(fruits) tells you the last valid position.
Several exceptions in one try
One try can have several except blocks. Python checks them from top to bottom and runs the first that matches. To handle a group the same way, put the classes in a tuple.
else and finally
Two more blocks can follow. else runs only when the try block finished with no exception. finally runs every time, error or not, which makes it the place for clean-up work.
Order: the blocks must be written as try, except, else, finally. Only try plus one of except or finally is required.
Raising an exception
Your own code can report a problem with raise. Create an exception object with a clear message and raise it. The function stops at that line, and the caller decides what to do.
Why raise and not print: a function that only prints "invalid amount" lets the program continue with wrong data. An exception cannot be ignored by accident.
Your own exception classes
An exception is a class, so you can write your own by inheriting from Exception. A custom class gives the error a name that says what it means in your program, and callers can catch exactly that.
Concept: the simplest custom exception is two lines: class InsufficientFunds(Exception): pass. Add an __init__ only when you want to carry extra data, as here.
Common built-in exceptions
| Exception | Raised when | Example |
|---|---|---|
ZeroDivisionError | Dividing by zero | 10 / 0 |
TypeError | A value has the wrong type for the operation | 10 + "Peter" |
ValueError | The type is right but the value cannot be used | int("1234y") |
KeyError | A dictionary has no such key | details["profession"] |
IndexError | A list or string has no such position | fruits[6] |
AttributeError | An object has no such attribute or method | "abc".push("d") |
NameError | A name was never defined, often a typo | print(amuont) |
UnboundLocalError | A function uses a local variable before giving it a value | n += 1 on an outer n inside a function |
FileNotFoundError | The file to open does not exist | open("no_such_file.txt") |
PermissionError | You are not allowed to read or write the file | Writing into a protected folder |
ImportError | A module or a name in it cannot be imported | from math import sqare |
KeyboardInterrupt | The user pressed Ctrl+C | Stopping a long loop |
Family tree: these classes inherit from each other. KeyError and IndexError are children of LookupError, and FileNotFoundError is a child of OSError. Catching a parent also catches its children. KeyboardInterrupt is not a child of Exception, so except Exception lets Ctrl+C through.
Files
Part 2 of 3 · keeping data after the program ends
Variables disappear when a program stops. A file keeps data on disk so that the next run, or another program, can use it. This part reads and writes plain text first, then two formats that every tool understands: CSV for tables and JSON for nested data.
- Opening a file:
open()and its modes - Writing and reading text:
with open(...) - Reading line by line: loop over the file
- Appending: add to the end without erasing
- Moving inside a file:
seek()andtell() - pathlib: paths as objects
- CSV files: rows and columns
- JSON files: dictionaries and lists
- A missing file: handle it without crashing
Opening a file
open(name, mode) returns a file object. The mode is a short string that says what you plan to do. The default is "r", read text.
| Mode | Meaning | If the file exists | If it does not exist |
|---|---|---|---|
"r" | Read | Starts at the beginning | FileNotFoundError |
"w" | Write | Erases the old content first | Creates it |
"a" | Append | Adds at the end | Creates it |
"x" | Create only | FileExistsError | Creates it |
"r+" | Read and write | Starts at the beginning, keeps the content | FileNotFoundError |
"rb", "wb" | The same in binary mode | Add b for files that are not text, such as images. You then get bytes, not str. | |
Watch out: "w" empties an existing file the moment it is opened, before you write anything.
Writing and reading text
Open files inside a with statement. The file is closed for you when the block ends, even if an error happens inside it. write() adds a string exactly as given, so you add \n yourself to end a line. read() returns the whole file as one string.
Python 2: the old lesson wrote f = open(...) and later f.close(). If an error happened in between, the file stayed open. with removes that risk. Passing encoding="utf-8" makes the file read the same on every computer.
Reading line by line
A file object can be used in a for loop. It hands over one line at a time, so even a very large file never has to fit in memory. Each line still ends with its \n, which strip() or rstrip("\n") removes.
Concept: a file object remembers how far it has read. readline() took the first line, readlines() took the rest as a list, and after that read() has nothing left and returns an empty string.
Appending
Mode "a" keeps what is in the file and adds new text at the end. It is the right mode for a log, where every run adds a line.
Moving inside a file
tell() gives the current position in the file and seek(n) moves to position n, counted from the start. With mode "r+" you can then overwrite part of the file in place. This is the second example of the 2017 lesson.
Tip: you rarely need this for text. Most programs read the whole file, change the data in memory and write the file again.
pathlib
The pathlib module treats a file path as an object. A Path can tell you about the file, build other paths with /, and read or write a whole file in one call.
Concept: open(path) accepts a Path as well as a string. as_posix() shows the path with forward slashes on every system. On Windows, printing the path directly shows backslashes.
CSV files
CSV means comma-separated values: one row per line, with commas between the columns. Spreadsheets and databases can all read and write it. The csv module handles the details, such as a comma inside a value.
csv.DictReader uses the first row as column names and gives each following row as a dictionary. Every value comes back as a string, so convert numbers yourself.
Rule: always pass newline="" when opening a file for the csv module. Without it, Windows adds an empty line after every row.
JSON files
JSON is a text format that looks very much like Python dictionaries and lists. It keeps nesting and basic types (text, numbers, true and false, nothing), which CSV cannot. json.dump() writes a value to a file and json.load() reads it back.
Remember: dump and load work with files. dumps and loads, with an s for string, work with text in memory. JSON has no tuples or sets: a tuple comes back as a list, and a set cannot be saved at all.
A missing file
The first time a program runs, its data file usually does not exist yet. There are two ways to deal with that. Ask first with Path.exists(), or just try to open the file and catch FileNotFoundError.
Tip: trying and catching is usually the safer of the two. A file can be deleted in the short moment between the check and the open().
Dates and times
Part 3 of 3 · the datetime module
Bank records are full of dates: when an account was opened, when a deposit was made, how many days until a payment is due. The datetime module in the standard library has a type for each need.
The examples use fixed dates so that they print the same result every day. In real code, date.today() and datetime.now() give the current date and time.
- date and datetime objects: make them and read their parts
- Dates to text and back:
strftimeandstrptime - Adding and subtracting time:
timedelta - The difference between two dates: days, weeks and age
date and datetime objects
A date holds a year, a month and a day. A datetime holds those plus hours, minutes and seconds. Both can be compared with < and >, where earlier is smaller.
The current moment is different on every run, so this block is marked as not runnable here. The comments show the kind of value you get.
Concept: dates are immutable, which means they cannot be changed in place. replace() returns a new date.
Dates to text and back
strftime() ("string format time") turns a date into text in the layout you choose. strptime() ("string parse time") does the reverse: it reads text into a datetime, given the same kind of layout. The layout is a string of format codes that each start with %.
| Code | Meaning | Example |
|---|---|---|
%Y | Year with four digits | 2017 |
%y | Year with two digits | 17 |
%m | Month as a number | 02 |
%B, %b | Month name, full and short | February, Feb |
%d | Day of the month | 13 |
%A, %a | Weekday name, full and short | Monday, Mon |
%H | Hour, 24-hour clock | 09, 21 |
%I | Hour, 12-hour clock | 09 |
%M | Minute | 30 |
%S | Second | 00 |
%p | AM or PM | AM |
Tip: for saving dates in files, use the ISO layout 2017-02-13. It has no day and month confusion and it sorts correctly as plain text.
Adding and subtracting time
A timedelta is a length of time: a number of days, hours, minutes and so on. Add it to a date to move forward and subtract it to move back. The calendar is handled for you, including month ends and leap years.
Watch out: timedelta has no months or years, because those are not a fixed length. For "the same day next year" use opened.replace(year=opened.year + 1).
The difference between two dates
Subtracting one date from another gives a timedelta. Its days attribute is the number of days between them. Two datetime values also give the seconds.
Concept: an age in years cannot be found by dividing days by 365, because of leap years. Comparing the month and day, as age_on does, is always right.
Quick reference
| You want to | Write |
|---|---|
| Catch one kind of error | try: ... except ValueError as error: ... |
| Catch several kinds the same way | except (ValueError, KeyError): |
| Run code only when nothing failed | else: after the except blocks |
| Run code no matter what | finally: |
| Report a problem | raise ValueError("Amount must be positive") |
| Name your own kind of error | class InsufficientFunds(Exception): pass |
| Write a text file | with open(name, "w", encoding="utf-8") as f: f.write(text) |
| Read a whole file | Path(name).read_text(encoding="utf-8") |
| Read line by line | for line in f: |
| Add to the end of a file | open(name, "a") |
| Read a table | csv.DictReader(f) |
| Save and load nested data | json.dump(data, f) and json.load(f) |
| Make a date | date(2017, 2, 13) |
| Date to text, text to date | d.strftime("%d %b %Y"), datetime.strptime(text, "%d %b %Y") |
| Move a date | d + timedelta(days=30) |
| Days between two dates | (later - earlier).days |
What changed since Python 2
The original lessons were written for Python 2.7. If you meet old code, these are the differences that matter for errors and files.
| Python 2.7 (2017 lessons) | Python 3 (this page) |
|---|---|
except IndexError, err: was allowed | Only except IndexError as err: |
print 10 / 4 gives 2 | print(10 / 4) gives 2.5, and 10 // 4 gives 2 |
raw_input() | input() |
A missing file raises IOError | FileNotFoundError, a child of OSError |
f = open(...) then f.close() | with open(...) as f: |
| Text files hold bytes | Text files hold Unicode text, so pass encoding="utf-8" |
Where to go next
The 2017 pages are still online as they were written, with their Python 2.7 code: Pythofrank: Exception handling and Pythofrank: File handling.
That completes the basics. The advanced path starts with Python Decorators.