Writing Python Code
Given a task, first identify the right standard library tool before writing custom code:
Python# Task: read a CSV, dedupe rows, write JSON output import csv import json from pathlib import Path def dedupe_csv_to_json(src: str, dst: str) -> None: seen = set() rows = [] with Path(src).open(newline="", encoding="utf-8") as f: for row in csv.DictReader(f): key = tuple(row.items()) if key not in seen: seen.add(key) rows.append(row) Path(dst).write_text(json.dumps(rows, indent=2), encoding="utf-8") if __name__ == "__main__": dedupe_csv_to_json("in.csv", "out.json")
Note: use pathlib over os.path, f-strings over % or .format(), and always specify encoding for file I/O.
Progress checklist for any non-trivial Python task:
- [ ] Clarify input/output shape and constraints
- [ ] Search stdlib for an existing module before writing custom logic
- [ ] Write a minimal working version
- [ ] Add type hints
- [ ] Add error handling for expected failure modes
- [ ] Write/verify with a quick test (doctest or assert)
- [ ] Review for idiomatic style (PEP 8, EAFP over LBYL)
Step 1 — Map the task to a stdlib domain. Common mappings:
| Need | Module |
|---|---|
| File paths | pathlib |
| CLI args | argparse |
| Structured data serialization | json, csv, pickle (trusted data only) |
| Config files | configparser, tomllib (read-only TOML) |
| Dates/times | datetime, zoneinfo |
| Regex | re |
| Concurrency (I/O-bound) | asyncio, concurrent.futures.ThreadPoolExecutor |
| Concurrency (CPU-bound) | concurrent.futures.ProcessPoolExecutor, multiprocessing |
| Data structures | collections (deque, Counter, defaultdict, namedtuple) |
| Iteration helpers | itertools, functools |
| Testing | unittest, doctest |
| Logging | logging (never print in libraries) |
| HTTP client | urllib.request for stdlib-only; note third-party requests/httpx preferred in practice |
| Type hints | typing, dataclasses |
| Enums/constants | enum |
| Context managers | contextlib |
| Temp files | tempfile |
| Hashing | hashlib |
| Randomness (non-crypto) | random; crypto-secure use secrets |
Step 2 — Write minimal code, then harden. Start with the happy path, then add:
- Type hints (
def f(x: int) -> str:) - Exception handling scoped to specific exceptions, not bare
except: - Resource cleanup via
withstatements
Step 3 — Validate. Add at least one runnable check (doctest, assert-based smoke test, or unittest case).
Example 1: CLI tool with argparse
Input: "Build a CLI that counts word frequency in a text file, prints top N."
Output:
Python#!/usr/bin/env python3 import argparse from collections import Counter from pathlib import Path def top_words(text: str, n: int) -> list[tuple[str, int]]: words = text.lower().split() return Counter(words).most_common(n) def main() -> None: parser = argparse.ArgumentParser(description="Word frequency counter") parser.add_argument("file", type=Path) parser.add_argument("-n", type=int, default=10, help="top N words") args = parser.parse_args() text = args.file.read_text(encoding="utf-8") for word, count in top_words(text, args.n): print(f"{word}\t{count}") if __name__ == "__main__": main()
Example 2: Concurrent I/O-bound fetches
Input: "Fetch 50 URLs concurrently and return status codes."
Output:
Pythonimport asyncio from urllib.request import urlopen async def fetch_status(url: str) -> tuple[str, int]: loop = asyncio.get_running_loop() def _blocking_get(): with urlopen(url, timeout=10) as resp: return resp.status status = await loop.run_in_executor(None, _blocking_get) return url, status async def main(urls: list[str]) -> dict[str, int]: results = await asyncio.gather(*(fetch_status(u) for u in urls)) return dict(results)
Note: for real HTTP workloads, asyncio + third-party async HTTP client is preferred over blocking urlopen in executor, but this shows the stdlib-only pattern.
Example 3: Choosing the right container
Input: "Track counts of items streaming in, need O(1) increment and easy sorting by frequency."
Output: collections.Counter — Counter(stream).most_common() gives sorted-by-frequency directly; avoid manual dict + manual sort.
- Prefer stdlib over hand-rolled logic. If a task looks common (grouping, caching, path handling), a module likely already solves it (
itertools.groupby,functools.lru_cache,pathlib). - EAFP over LBYL: use
try/exceptfor control flow (e.g., dict access) rather than pre-checking withif key in d, when failure is uncommon. - Use
dataclassesfor plain data containers instead of hand-written__init__/__repr__. - Use
enum.Enumfor fixed sets of related constants instead of bare strings/ints. - Always set
encoding="utf-8"explicitly when opening text files — platform defaults vary. - Use
loggingmodule, notprint, for anything beyond a throwaway script. - Use
contextlib.contextmanagerfor custom resource-management logic instead of manual try/finally when possible. - Type hints on public functions improve reviewability even without static enforcement.
- Use
pathlib.Pathuniversally instead of string path manipulation.
- Mutable default arguments:
def f(x, items=[])shares state across calls — useNonesentinel and initialize inside the function. - Bare
except:: swallowsKeyboardInterrupt/SystemExit. Catch specific exceptions. - Using
pickleon untrusted data: arbitrary code execution risk — usejsonfor untrusted/interop data. - Ignoring
withstatements: manualopen()/close()leaks file descriptors on exceptions. randomfor security-sensitive values: not cryptographically secure — usesecretsfor tokens/passwords.- Blocking calls inside
asynciocoroutines: freezes the event loop — offload viarun_in_executoror use async-native libraries. - String concatenation in loops: use
"".join(parts)instead of repeated+=for building large strings. - Comparing floats with
==: usemath.isclose()for floating-point comparisons. - Overusing
multiprocessingfor I/O-bound work: process overhead wastes resources; use threads orasyncioinstead for I/O, reservemultiprocessing/ProcessPoolExecutorfor CPU-bound work.