Debugging Float Precision Issues

Bug report: "My test assert total == 1.0 fails but total prints as 1.0."

Python
import math

total = sum([0.1] * 10)
print(total)              # 1.0 (displayed, but actually 0.9999999999999999)
print(total == 1.0)       # False — the bug

# Fix: use fsum for accurate accumulation, isclose for comparison
total = math.fsum([0.1] * 10)
print(total == 1.0)       # True

Use this decision process when a numeric result is "wrong" but looks right:

Progress:

• Step 1: Reproduce — print repr(x) or f"{x:.20f}", not str(x), to see hidden precision error
• Step 2: Classify the bug (see decision tree below)
• Step 3: Apply the targeted fix
• Step 4: Re-verify with math.isclose() — never re-add a raw == check
• Step 5: Add a regression test using the same tolerance

Decision Tree

Symptom: equality comparison fails unexpectedly → Never compare floats with ==. Use math.isclose(a, b, rel_tol=1e-9).

• If comparing against zero, rel_tol is useless (relative to 0 is always 0) — pass abs_tol explicitly: math.isclose(x, 0, abs_tol=1e-9).

Symptom: summing many floats drifts from the mathematically exact answer → Built-in sum() accumulates rounding error one addition at a time. Replace with math.fsum(), which uses Shewchuk's algorithm for a correctly-rounded result.

Python
values = [1e16, 1.0, -1e16]
sum(values)        # 0.0  (wrong — 1.0 got lost)
math.fsum(values)  # 1.0  (correct)

Symptom: int(math.sqrt(n)) gives the wrong integer for large n → math.sqrt converts to float, which loses precision above ~2^53. Use math.isqrt(n) for exact integer results, no float roundtrip.

Python
n = 10**30
int(math.sqrt(n)) ** 2 == n   # False — precision lost
math.isqrt(n) ** 2 <= n       # True — isqrt is exact

Symptom: computing exp(x) - 1 or log(1 + x) for small x gives noisy/zero results → Catastrophic cancellation: subtracting two nearly-equal large-magnitude floats destroys precision. Use math.expm1(x) and math.log1p(x), which compute these directly without the cancellation step.

Python
x = 1e-15
math.exp(x) - 1     # 1.1102230246251565e-15 (noisy, wrong-ish)
math.expm1(x)       # 1.0000000000000007e-15 (accurate)

Symptom: combinatorics or factorial results overflow or lose precision → Don't compute n! / (k! * (n-k)!) manually with floats. Use math.comb(n, k) / math.perm(n, k), which stay in exact integer arithmetic.

Symptom: distance/norm calculation is unstable for very large or small coordinates → Manual sqrt(x**2 + y**2) can overflow/underflow for extreme values. Use math.hypot(x, y, ...), which scales internally to avoid overflow.

Example 1: Financial-style running total that never quite hits zero

Input:

Python
balance = 0.0
for amount in [10.10, -3.30, -6.80]:
    balance += amount
print(balance == 0.0)   # expected True, got False

Diagnosis: repeated += on floats accumulates rounding error. Output (fix):

Python
import math
balance = math.fsum([10.10, -3.30, -6.80])
print(math.isclose(balance, 0.0, abs_tol=1e-9))  # True

Example 2: Integer square root for cryptography-scale numbers

Input:

Python
n = 123456789123456789123456789123456789
root = int(n ** 0.5)
print(root * root <= n)   # False — overflowed float precision

Output (fix):

Python
import math
root = math.isqrt(n)
print(root * root <= n)          # True
print((root + 1) ** 2 > n)       # True — confirms floor sqrt is correct

Example 3: Probability computed as ratio of factorials blows up

Input:

Python
n, k = 1000, 500
p = math.factorial(n) // (math.factorial(k) * math.factorial(n - k))
# works but wastefully computes huge intermediate factorials

Output (fix):

Python
import math
p = math.comb(n, k)   # same result, computed more efficiently, no huge intermediates