Study material generated by AI from this openly licensed book (CC BY-NC 3.0). Not affiliated with or endorsed by the publisher.
Think Python, 2nd Edition, by Allen B. Downey, Green Tea Press, licensed under CC BY-NC 3.0. Read the original free
Chapter 14 · pp. 159–168
Most simple programs forget everything when they finish. This chapter shows how programs can keep data in permanent storage by using text files and databases. You will also learn how to format output, find files on disk, handle errors, and write code that can be reused as a module.
A transient program runs briefly, prints a result, and ends. Its data disappears, so running it again starts from a clean slate. A persistent program runs for a long time (or all the time) and keeps some data in permanent storage, such as a hard drive. If it shuts down and restarts, it picks up where it left off. Operating systems and web servers are good examples. (p. 159)
Two simple ways to make data persist are text files and databases. Both are covered below. (p. 159)
To write a file, follow three steps:
fout = open('output.txt', 'w').fout.write(line).fout.close().The file object remembers its position, so each new call to write adds text after what was already written. Each call also returns the number of characters it wrote. If you forget to close the file, Python closes it when the program ends, but closing it yourself is better practice. (pp. 159–160)
Common mistake: Many students think write mode adds to the old contents. It does not. Opening an existing file with 'w' wipes out its data and starts over. If the file does not exist, Python creates it. Opening an important file this way by accident destroys what was in it. (p. 160)
The argument to write must be a string. To store a number, convert it first. For example, fout.write(str(52)) puts the characters 52 in the file. Passing the integer 52 directly causes an error. (p. 160)
Another way to build strings is the format operator, %. With two integers, % means modulus. When the left operand is a string, it means formatting. The left string is the format string, and it holds format sequences that mark where values go:
%d formats an integer.%g formats a float.%s formats a string.Example: 'I have spotted %d camels.' % 42 gives 'I have spotted 42 camels.'. Note that the result is the string '42' inside the sentence, not the integer. (p. 160)
With more than one format sequence, the right operand must be a tuple. Values are matched to sequences in order. For example, 'In %d years I have spotted %g %s.' % (3, 0.1, 'camels') works. The tuple length and the types must match the sequences. Two failures show this:
'%d %d %d' % (1, 2) raises a TypeError because there are too few values.'%d' % 'dollars' raises a TypeError because %d needs a number, not a string.(pp. 160–161)
Files live in directories (folders). Every running program has a current directory, which is where Python looks when you open a file by simple name. A path is a string that identifies a file or directory. (p. 161)
/home/dinsdale, then memo.txt means /home/dinsdale/memo.txt./ at the top of the file system. It means the same thing wherever the program is running.os.path.abspath('memo.txt') converts a relative path to its absolute form. (p. 161)
The os and os.path modules offer many helpers:
os.getcwd() returns the current directory.os.path.exists(path) checks whether a path exists.os.path.isdir(path) and os.path.isfile(path) tell you which kind of thing it is.os.listdir(dir) returns the names inside a directory.os.path.join(dir, name) builds a full path from a directory and a name.(pp. 161–162)
Directories can contain other directories, so a program that lists all files needs recursion. The walk function loops over os.listdir(dirname). For each name it builds a full path with os.path.join. If the path is a file, it prints it. Otherwise it is a directory, so the function calls itself on that path. Because of the recursive call, folders nested at any depth get covered. (p. 162)
File work can fail in many ways. Opening a missing file raises FileNotFoundError. Writing to a protected file raises PermissionError. Opening a directory as a file raises IsADirectoryError. Checking for every problem ahead of time with os.path.exists and similar functions would take a lot of code. (p. 162)
It is easier to try the action and deal with problems if they happen. A try statement does this:
try:
fin = open('bad_file')
except:
print('Something went wrong.')
Python runs the try clause first. If no exception occurs, it skips the except clause. If one occurs, it jumps to the except clause. This is called catching an exception. The except clause can fix the problem, try again, or end the program gracefully. (pp. 162–163)
A database file is organized like a dictionary: it maps keys to values. The big difference is that it lives in permanent storage, so it still exists after the program ends. (p. 163)
With the dbm module:
db = dbm.open('captions', 'c') opens the database. Mode 'c' creates it if it does not exist.db['cleese.png'] = 'Photo of John Cleese.' stores an item.b), which acts much like a string.db.close() closes the database when you are done.Looping over db.keys() works, but some dictionary methods, such as items, do not. (p. 163)
A dbm database accepts only strings or bytes as keys and values. The pickle module gets around this. pickle.dumps(obj) turns almost any object into a string-like form, and pickle.loads(s) rebuilds the object. (p. 164)
Suppose t1 = [1, 2, 3], s = pickle.dumps(t1), and t2 = pickle.loads(s). Then t1 == t2 is True because the values are equal, but t1 is t2 is False because they are different objects. Pickling and unpickling therefore works like making a copy. The shelve module combines pickle and a database, so you can store non-string values directly. (p. 164)
Any Python file can be imported as a module. The problem is that importing runs all the top-level code in the file, including any test calls at the bottom. Usually you want an import to define functions without running them. (pp. 165–166)
The fix is this guard:
if __name__ == '__main__':
print(linecount('wc.py'))
Python sets the built-in variable __name__ when a program starts. If the file is run as a script, __name__ is '__main__', so the guarded code runs. If the file is imported, __name__ has a different value, so the guarded code is skipped. (p. 166)
notes.txt already holds your class notes. What happens to them if you run open('notes.txt', 'w')? Why?'%s scored %d points' % ('Ana', 12). Then explain why '%d points' % 'twelve' fails.try/except statement often better than checking a file with os.path.exists before opening it?