Chapter 20 · pp. 215–222
Debugging
Debugging means finding and fixing the mistakes in a program. Different mistakes need different tools, so the first job is to work out what kind of error you have. This chapter covers the three kinds of error and practical techniques for tracking each one down.
The three kinds of errors
- Syntax error: Python finds it while translating your code, because the structure is wrong. Example: leaving the colon off a
def line. (p. 215)
- Runtime error: Python finds it while the program is running. Example: a function that calls itself forever ends with a 'maximum recursion depth exceeded' error. (p. 215)
- Semantic error: The program runs with no error message but gives the wrong result. Example: an expression is evaluated in a different order than you expected. (p. 215)
The first debugging step is to decide which kind you have. If Python refuses to start, suspect syntax. If it crashes with a message, it is a runtime error. If it finishes with a wrong answer, it is semantic. Knowing this tells you where to look. Some tools, such as print statements, help in more than one case. (p. 215)
Syntax errors
Where the message points. A common wrong belief is that a syntax error message marks the exact faulty line. It actually shows where Python noticed the problem, and the real mistake is often earlier, frequently on the line before. For example, an unclosed bracket on line 3 may only cause an error on line 4, because Python treats line 4 as a continuation of line 3. If you add code a little at a time, check the last line you added first. (pp. 215–216)
Common slips. Most syntax errors come from a short list:
- Using a Python keyword as a variable name.
- Forgetting the colon at the end of a
for, while, if, or def header.
- Mismatched quotation marks, or curly quotes instead of straight quotes.
- An unterminated triple-quoted string. This may cause an error at the end of the program, or it may quietly turn the following code into part of the string with no error at all.
- An unclosed
(, {, or [.
- Writing
= where == is needed in a condition.
- Inconsistent indentation. Mixing tabs and spaces is a classic cause, and an editor that understands Python helps.
- Non-ASCII characters, which can sneak in when you paste text from a web page.
(p. 216)
When your changes seem to do nothing. Maybe you and Python are not looking at the same code. Test this by putting a deliberate syntax error at the top of the file and running it. If Python does not complain, it is running some other version. Likely causes:
- You did not save the file.
- You renamed the file but still run the old name.
- Your module has the same name as a standard module.
- You imported a module and did not restart or reload it, so importing again does nothing.
If you are still stuck, start from a working 'Hello, World!' program and add the original code back piece by piece. (pp. 216–217)
Runtime errors
A program that does nothing. This often happens when a file defines functions but never calls one. Add a call, and make sure execution actually reaches it. (p. 217)
A program that hangs. A hang usually means an infinite loop or infinite recursion. To check a suspect loop, print a message just before it ('entering the loop') and another just after ('exiting the loop'). If you see only the first message, the loop never ends. (pp. 217–218)
- Infinite loop: At the end of each pass, print the variables used in the loop condition and the value of the condition itself. For
while x > 0 and y < 0, you would print x, y, and the result of x > 0 and y < 0. On the last pass the condition should be False. If it never is, the printed values show why x or y is not changing as you expected. (p. 218)
- Infinite recursion: First check that the function has a base case, meaning a condition under which it returns without calling itself again. If there is none, the algorithm needs rethinking. If there is one but it is never reached, print the parameters at the start of each call. If the values are not moving toward the base case, you have a clue about why. (pp. 217–218)
Flow of execution. If you are unsure which functions run, or in what order, print 'entering function foo' at the start of each function. The output is a trace of the calls. It also shows when a function is defined but never called. (pp. 217–218)
Exceptions and tracebacks
When a runtime error occurs, Python prints the exception name, the line where it happened, and a traceback. The traceback lists the function running now, then the function that called it, then that function's caller, and so on, with a line number for each. Read it first, because it tells you where to look. (pp. 218–219)
Each common exception suggests a typical cause (p. 219):
- NameError: You used a name that does not exist. Check the spelling. Also remember that a local variable cannot be used outside its function.
- TypeError: A value is used improperly. Examples: indexing a list with something that is not an integer, a format string that does not match its values, or the wrong number of arguments in a function call.
- KeyError: The dictionary has no such key. With string keys, capitalization matters, so
'Name' and 'name' are different keys.
- AttributeError: The attribute or method does not exist. If the message mentions
NoneType, the object itself is None. Two common reasons are a function with no return statement (it returns None) and using the result of a method like sort, which returns None.
- IndexError: The index is greater than the length minus one. Print the index and the length just before the error to see which one is wrong.
The Python debugger, pdb, can also show the program's state just before an exception. (p. 219)
Too much print output
Print statements can bury you in output. You have two options (pp. 219–220):
- Simplify the output. Remove or comment out prints that do not help, combine them, or format them so they are easier to read.
- Simplify the program. Shrink the problem, for example by searching a small list or using the simplest input that still fails. Remove dead code. Split a large function into smaller ones and test each separately.
Finding the smallest failing case often reveals the bug, because a program that works in one situation but not another gives you a clue. (p. 220)
Semantic errors
Semantic errors are the hardest, because the interpreter gives no hints and only you know what the program should do. You need a hypothesis about what the program is really doing. Ask yourself three questions (p. 220):
- Is something supposed to happen that does not?
- Is something happening that should not?
- Is some code having an unexpected effect? If so, read the documentation for the functions you call and try them on simple test cases.
Often the problem is a wrong mental model of how the code works, not the code itself. The cure is to split the program into components, such as functions, and test each one independently until you find where reality differs from your expectation. If you build and test as you go, only a small amount of new, unverified code remains when something breaks. (pp. 220–221)
Taming complex expressions
A long expression is hard to debug. Break it into steps using temporary variables. The names document what each part means, and you can print or check the type of each intermediate value. (p. 221)
The order of evaluation can also surprise you. To write 2πx, you might type y = x / 2 * math.pi. But multiplication and division have the same precedence and run left to right, so this computes xπ/2. The fix is explicit parentheses: y = x / (2 * math.pi). Whenever you are unsure of the order, add parentheses. This makes the code correct and easier for others to read. (p. 221)
The same trick helps with return values. Instead of returning a complex expression directly, store it in a temporary variable such as count, print it, and then return it. (pp. 221–222)
Check yourself
- Which kind of error (syntax, runtime, or semantic) produces no error message but gives a wrong answer? Give an example.
- Line 4 of your program reports a syntax error, but line 4 looks fine. Where should you look, and why?
- Your edits seem to have no effect on the program's behavior. How can you use a deliberate syntax error to test whether you are running the right file, and what are two possible causes of the problem?
- A recursive function runs for a while and then reports 'maximum recursion depth exceeded'. What two things should you check, and what should you print?
- Why does
y = x / 2 * math.pi not compute 2πx, and how do you fix it?