Notes Tutorials

Debugging systematically

Debugging is not guessing until it works. It's a method: reproduce the bug, read what the program tells you, form a guess, test it, narrow it down. One real bug traced start to finish, plus print and logging, debuggers in four languages, and git bisect.

8 min readAlso available in Bahasa Indonesia →

Debugging is “the process of finding the root cause, workarounds, and possible fixes for bugs.” The important words are root cause. Making the symptom go away is easy: add an if, catch the exception, restart the server. Finding out why it happened is the actual job, and it’s the only thing that stops the bug from coming back in another form.

The good news is that debugging is a skill with a method, not a talent. This part walks through that method on one real bug, then covers the tools: printing and logging, debuggers, and git bisect.

The method

Almost every good debugging session follows the same loop:

  1. Reproduce the bug, reliably, ideally with the smallest input that shows it.
  2. Read what the program already tells you: the error, the stack trace, the logs.
  3. Guess a cause (a hypothesis) that would explain what you see.
  4. Test the guess with an experiment: print a value, set a breakpoint, change one thing.
  5. Narrow down. If the guess was wrong, you’ve still ruled something out. Repeat.
  6. Fix the cause, then add a test so it stays fixed (part 14).

It’s the scientific method applied to code. The most common mistake is skipping straight from “something’s wrong” to “change things until it works”. That sometimes makes the symptom disappear, but you don’t know why, so you don’t know what else you broke.

A real bug, start to finish

The monthly report from part 9 gets a complaint: “January revenue looks too low.” Here’s the file and the code:

date,product,qty,price,status
2026-01-03,book,2,85000,paid
2026-1-15,pen,10,4000,paid
2026-01-20,mug,3,35000,paid
2026-1-28,book,1,85000,paid
def month_of(row):
    return row["date"][:7]

def revenue_in_month(rows, month):
    return sum(int(r["qty"]) * int(r["price"]) for r in rows
               if month_of(r) == month and r["status"] == "paid")

Reproduce. Run it:

2026-01 revenue: 275000

By hand: 170,000 + 40,000 + 105,000 + 85,000 = 400,000. So the bug is real, and we can trigger it every time. That alone is half the battle. A bug you can reproduce on demand is almost always a bug you can fix.

Read. No error, no stack trace. The program is confidently wrong. That’s the silent kind of failure from part 13, so we have to make it talk.

Guess. 125,000 is missing, which is exactly the pen row (40,000) plus the second book row (85,000). What do those two rows have in common? A hypothesis: those rows aren’t being counted as January.

Test. Instead of staring at the code, ask the program which months it sees:

from collections import Counter
print(Counter(month_of(r) for r in read_orders("orders.csv")))
Counter({'2026-01': 2, '2026-1-': 2})

There it is. Two dates are written as 2026-1-15 and 2026-1-28, without the leading zero, and taking the first seven characters gives '2026-1-', which isn’t '2026-01'. The hypothesis was right, and now we know exactly why.

Fix the cause. The root problem isn’t “some rows are wrong”. It’s that the code treats a date as a string and assumes a fixed format. Parse it as a date instead:

from datetime import datetime

def month_of(row):
    d = datetime.strptime(row["date"], "%Y-%m-%d")
    return f"{d.year:04d}-{d.month:02d}"
2026-01 revenue: 400000

And a regression test so it stays fixed: month_of({"date": "2026-1-15"}) == "2026-01".

The same bug exists in any language that slices strings. In PHP, substr("2026-1-15", 0, 7) gives 2026-1-, while date("Y-m", strtotime("2026-1-15")) gives 2026-01.

Printing and logging

The Counter above was a print statement with a purpose, and printing is a perfectly good debugging tool. Two habits make it much more useful:

  • Print the question, not just the value. print(month_of(r)) shows a column of dates. Counter(...) answered “which months exist?” in one line.
  • Print with labels and repr. print(f"{qty=!r}") shows qty='3 ', with the quotes and the trailing space that a plain print(qty) would hide.

For anything that runs repeatedly or in production, use logging instead of print. Python’s logging HOWTO: “Logging is a means of tracking events that happen when some software runs.” Its own guidance is to use print() to “Display console output for ordinary usage of a command line script or program”, and a logger to “Report events that occur during normal operation of a program (e.g. for status monitoring or fault investigation)”.

import logging
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("import")

log.info("start file=%s rows=%d", "orders.csv", 4)
log.warning("skip line=%d reason=%s value=%r", 3, "bad date", "2026-13-01")
log.debug("this line is hidden at INFO level")
INFO import: start file=orders.csv rows=4
WARNING import: skip line=3 reason=bad date value='2026-13-01'

Levels let you keep detailed debug lines in the code and switch them on only when needed. And notice the log lines carry the same what/where/value context as the error messages in part 13. When a bug report comes in next month, good logs are often the only evidence you have.

Debuggers: stop the program and look around

A debugger lets you pause a running program at a line, look at every variable, and step forward one line at a time. Python’s pdb “supports setting (conditional) breakpoints and single stepping at the source line level, inspection of stack frames, source code listing, and evaluation of arbitrary Python code in the context of any stack frame.”

Here’s the same January bug, found with pdb instead of a print. Set a breakpoint in month_of, continue to it, and print the values:

(Pdb) b report.py:9
(Pdb) c
-> return row["date"][:7]
(Pdb) p row["date"]
'2026-01-03'
(Pdb) p row["date"][:7]
'2026-01'
(Pdb) c
-> return row["date"][:7]
(Pdb) p row["date"]
'2026-1-15'
(Pdb) p row["date"][:7]
'2026-1-'

The second stop shows the problem directly. You can also put breakpoint() in the code to stop exactly there, as mentioned in part 12.

Every language has the same tool:

  • PHP: “Xdebug’s step debugger allows you to interactively walk through your code to debug control flow and examine data structures.” It connects to VS Code, PhpStorm, and other editors.
  • Go: “Delve is a debugger for the Go programming language.” VS Code’s Go extension uses it under the hood.
  • JavaScript: the browser’s DevTools, or node --inspect for Node. The debugger statement “invokes any available debugging functionality, such as setting a breakpoint.”

Print debugging is quick when you already have a good guess. A debugger is better when you don’t know where to look yet, because you can explore without editing and re-running each time.

When the bug came from a change: git bisect

Sometimes you know something worked last week and doesn’t now, but there have been dozens of commits since. git bisect will “Use binary search to find the commit that introduced a bug”. It’s the binary search from part 10, applied to history.

You tell it one bad commit and one good one. If you also give it a script, it does the rest: the script “should exit with code 0 if the current source code is good/old, and exit with a code between 1 and 127 (inclusive), except 125, if the current source code is bad/new.”

In a small example repo with 12 commits, the 10-item discount from part 14 stopped working somewhere. The check:

import sys
sys.path.insert(0, ".")
from pricing import final_price
sys.exit(0 if final_price(100, 10) == 950 else 1)
$ git bisect start HEAD d72e099
$ git bisect run python3 check_discount.py
Bisecting: 5 revisions left to test after this (roughly 3 steps)
[91985178...] Tidy pricing comments (6)
Bisecting: 2 revisions left to test after this (roughly 1 step)
[e51a9244...] Tidy pricing comments (3)
Bisecting: 0 revisions left to test after this (roughly 1 step)
[52eac298...] Simplify discount condition
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[f6428db6...] Tidy pricing comments (4)
52eac29864676b636367ed3b96f25c590fa71a29 is the first bad commit

    Simplify discount condition

Four checks to search 12 commits. With 1,000 commits it would be about 10. Finish with git bisect reset to go back to where you were. Small, focused commits make this even more useful, which part 16 comes back to.

Habits that save hours

  • Check your assumptions, one by one. Is that the file you think it is? Is the code you’re editing the code that’s running? Is the cache cleared? A surprising number of “impossible” bugs are a stale build or the wrong environment.
  • Change one thing at a time. Change three things and it starts working, and you don’t know which one mattered, or whether the other two broke something.
  • Compare working and broken. Works for one customer, not another? Works locally, not on the server? List the differences. The bug is in one of them.
  • Make the input smaller. If a 40,000-row file fails, cut it in half and see which half still fails. Keep going until you have one row.
  • Read the whole error message, including the part below the first line. Stack traces show the path the program took; the most useful line is often your own code just above the library code.
  • Explain it to someone, or to a rubber duck (part 9). Saying “and then this value is always positive” out loud is often when you realize it isn’t.
  • Take a break. If you’ve been stuck for an hour, a walk often does more than another hour of staring.

Some bugs fight back. A heisenbug “seems to disappear or alter its behavior when one attempts to study it”, often because adding a print changes the timing. Those usually involve concurrency or timing, which part 21 covers.

Why this matters day to day

  • Most time on a bug is spent finding it, not fixing it. A method makes the finding part shorter and less stressful.
  • Root cause, not symptom. Wrapping the January bug in a try or adding a special case for “2026-1-” would have hidden it until February.
  • Good logs are a gift to your future self. Production bugs often can’t be reproduced locally; logs are the evidence.
  • AI tools help debug, but need the evidence. Pasting the actual error, the input, and what you’ve ruled out gets far better help than “it doesn’t work”. And a suggested fix still needs the same check: does it explain the cause?

Try it yourself

  1. Reproduce the January bug yourself, then find it once with print and once with breakpoint().
  2. Add a row dated 2026-01-5 to the CSV. Does the fixed month_of handle it? Add a test either way.
  3. Make a small repo, break something in one commit among ten, and find it with git bisect run.
  4. Next time you’re stuck, write down your hypothesis before you change any code. Was it right?

Summary

  • Debugging means finding the root cause, not hiding the symptom.
  • The method: reproduce, read, guess, test, narrow down, fix and add a test.
  • Print with purpose and with repr; use logging with levels and context for anything that runs repeatedly.
  • Debuggers (pdb, Xdebug, Delve, DevTools) let you pause and inspect without editing code.
  • git bisect binary-searches history for the commit that introduced a bug.
  • Check assumptions, change one thing at a time, compare working and broken, shrink the input.

References