AI Skill Report Card
Computing Statistics
Computing Statistics with Python's statistics Module
Quick Start14 / 15
Pythonimport statistics as stats data = [2.5, 3.6, 4.1, 4.1, 5.8, 6.2, 7.0] stats.mean(data) # 4.757142857142857 stats.median(data) # 4.1 stats.mode(data) # 4.1 stats.stdev(data) # sample standard deviation stats.variance(data) # sample variance
Use this module for quick, dependency-free statistical calculations on small-to-medium datasets. For heavy numerical work (large arrays, matrix ops), prefer NumPy/SciPy instead.
Recommendation▾
Show actual expected output values for Examples 2 and 3 rather than approximate/descriptive text (e.g., compute exact q1/q3 and cdf value) to make them true concrete input/output pairs
Workflow13 / 15
Progress:
- Step 1: Identify whether data is a full population or a sample
- Step 2: Choose the right averaging function (mean, median, mode, or variants)
- Step 3: Choose variance/stdev function matching population vs sample
- Step 4: Handle multiple datasets (correlation, covariance, linear regression) if needed
- Step 5: Consider
NormalDistfor distribution-based calculations - Step 6: Validate results and handle edge cases (empty data, single value, non-numeric input)
Step 1: Population vs Sample
- Sample (subset of a larger population, most common case): use
stdev,variance - Population (complete dataset): use
pstdev,pvariance
Step 2: Averages
| Function | Use case |
|---|---|
mean(data) | Arithmetic mean |
fmean(data, weights=None) | Faster float-based mean, supports weights (3.11+) |
geometric_mean(data) | Mean of ratios/growth rates |
harmonic_mean(data, weights=None) | Rates (e.g. speed, price/earnings) |
median(data) | Middle value, robust to outliers |
median_low(data) / median_high(data) | Median with even-length tie-breaking |
mode(data) | Most common value |
multimode(data) | All values tied for most common |
Step 3: Spread
| Function | Use case |
|---|---|
variance(data, xbar=None) | Sample variance |
stdev(data, xbar=None) | Sample standard deviation |
pvariance(data, mu=None) | Population variance |
pstdev(data, mu=None) | Population standard deviation |
Pass a precomputed mean via xbar/mu to avoid recomputation across multiple calls.
Step 4: Relationships Between Datasets
Pythonx = [1, 2, 3, 4, 5] y = [2, 4, 5, 4, 5] stats.covariance(x, y) # joint variability stats.correlation(x, y) # Pearson correlation, -1 to 1 stats.correlation(x, y, method="ranked") # Spearman rank correlation (3.12+) stats.linear_regression(x, y) # LinearRegression(slope, intercept)
Step 5: Distributions with NormalDist
Pythonfrom statistics import NormalDist nd = NormalDist.from_samples(data) # fit from data nd = NormalDist(mu=100, sigma=15) # or specify directly nd.pdf(110) # probability density at x nd.cdf(110) # P(X <= 110) nd.inv_cdf(0.95) # value at 95th percentile nd.zscore(120) # standardized score # Combining/comparing distributions nd1 = NormalDist(90, 12) nd2 = NormalDist(100, 15) (nd1 + nd2) # sum of independent normals nd1.overlap(nd2) # overlap coefficient (0-1)
Also available: quantiles(data, n=4) for quartiles/percentiles without fitting a full distribution.
Step 6: Edge Cases
- Empty data → raises
StatisticsError mode/multimodework on any hashable data, not just numbers- Functions accept
DecimalandFractionfor exact arithmetic variance/stdevrequire at least 2 data points (sample stats need N-1 denominator)
Recommendation▾
Add an example demonstrating error handling for edge cases (empty list, single value) since these are mentioned but never demonstrated in code
Examples14 / 20
Example 1: Grade analysis (sample stats) Input:
Pythongrades = [88, 92, 79, 93, 85, 91, 76]
Output:
Pythonstats.mean(grades) # 86.28571428571429 stats.median(grades) # 88 stats.stdev(grades) # 6.626...
Example 2: Quartile-based outlier detection Input:
Pythondata = [5, 7, 8, 8, 9, 10, 12, 13, 100] q1, q2, q3 = stats.quantiles(data, n=4)
Output:
q1 ≈ 7.5, q2 (median) = 9, q3 ≈ 12.5
# Values far beyond [q1 - 1.5*IQR, q3 + 1.5*IQR] flagged as outliers (e.g., 100)
Example 3: Probability from a fitted normal distribution Input:
Pythonheights = [168, 172, 175, 180, 165, 170, 178] nd = stats.NormalDist.from_samples(heights) p = nd.cdf(175)
Output:
p ≈ probability that a random height in this distribution is <= 175 cm
Recommendation▾
Include a 'bad output' example showing a common mistake (e.g., using stdev instead of pstdev on population data) to reinforce the pitfalls section with concrete contrast
Best Practices
- Use
fmeaninstead ofmeanwhen data is all floats and speed matters. - Prefer
medianovermeanfor skewed data or when outliers are present. - Always match variance/stdev function to whether data is a sample or full population.
- Use
Fraction/Decimalinputs when exact (non-floating-point) results are required. - Reuse precomputed
xbar/muwhen calling variance/stdev repeatedly on the same dataset. - For anything beyond simple stats (large-scale numerical computing, hypothesis testing, ANOVA), switch to NumPy/SciPy/pandas.
Common Pitfalls
- Don't use
stdev/variance(sample versions) when data represents the entire population — usepstdev/pvarianceinstead. - Don't call statistics functions on empty lists — check length first or catch
StatisticsError. - Don't assume
modealways returns one value — ties return the first encountered; usemultimodefor all ties. - Don't confuse
correlation's default (Pearson, linear relationships) with ranked/Spearman correlation needed for monotonic but non-linear relationships. - Don't manually reimplement mean/stdev with floating-point loops — this module handles precision issues (e.g., using exact fractions internally) better than naive code.