Mean and Standard Deviation in Python

Mean and standard deviation are the two statistics you reach for first when you need to understand a dataset. Mean tells you where the center of your data sits. Standard deviation tells you how tightly or loosely the values cluster around that center. Every engineer I know uses these two measures daily, whether they are profiling latency across API endpoints, checking quality control metrics on a production line, or deciding what inventory levels make sense for next quarter.

This article covers the full picture: the math behind mean and standard deviation, the difference between population and sample statistics, and every meaningful way to calculate both in Python. I will show you working code at each step, call out the ddof trap that trips up nearly every data scientist at least once, and walk through a practical example with real housing price data. By the end you will have a mental model solid enough to debug any statistics-related issue in your pipelines.

TLDR

• Mean is the arithmetic average: sum of all values divided by the count. Formula is (sum of values) / N.
• Standard deviation measures how spread out values are from the mean. It is the square root of variance.
• Population standard deviation divides by N. Sample standard deviation divides by N-1. Mixing these up produces wrong results.
• Python’s statistics module calculates sample standard deviation by default (ddof=1). NumPy defaults to population (ddof=0).
• Pure Python implementations let you see exactly how the math works under the hood.

What Mean Actually Is

Mean is the arithmetic average. You add up every value in your dataset and divide by how many values you have. The formula looks like this:

mean = (x1 + x2 + x3 + … + xN) / N

With five values [2, 4, 6, 8, 10], the mean is (2+4+6+8+10) / 5 = 30 / 5 = 6.

Two variants exist. The population mean treats your dataset as the entire group you care about. Every person in a country, every transaction in a year, every measurement in a sensor run. You divide by N. The sample mean treats your dataset as a subset drawn from a larger population. You divide by N-1 instead of N. The N-1 correction compensates for the fact that a sample systematically underestimates the true population spread. That correction is called degrees of freedom adjustment.

Use population mean when your data IS the full population. Use sample mean when your data is a subset and you want to infer something about the larger group. In data science work, most of the time you are working with samples.

Variance Explained

Variance is the average squared deviation from the mean. It tells you how spread out a dataset is in squared units. Squaring serves two purposes: it makes every deviation positive (negative deviations do not cancel positives), and it penalizes large deviations more than small ones.

Here is the step-by-step process. Take your data [2, 4, 6, 8, 10] where the mean is 6. Subtract the mean from each value to get deviations: [-4, -2, 0, 2, 4]. Square each deviation: [16, 4, 0, 4, 16]. Sum those squared deviations: 40. Divide by the number of data points.

For population variance, divide by N: 40 / 5 = 8.

For sample variance, divide by N-1: 40 / 4 = 10.

The factor of N vs N-1 is the degrees of freedom difference. The term degrees of freedom refers to how many values are “free to vary” once you have fixed the sample mean. Once you know the mean and N-1 of the values, the last value is fully determined. One degree of freedom gets consumed by the mean calculation. Sample variance uses N-1 in the denominator to correct for this.

Standard Deviation Explained

Standard deviation is the square root of variance. Variance gave you squared units (dollars squared, kilograms squared, milliseconds squared) which are hard to interpret. Taking the square root brings you back to the original unit.

Population standard deviation formula: sqrt(sum((xi – mean)^2) / N)

Sample standard deviation formula: sqrt(sum((xi – mean)^2) / (N-1))

For [2, 4, 6, 8, 10] with population variance 8, the population standard deviation is sqrt(8) = 2.828. With sample variance 10, the sample standard deviation is sqrt(10) = 3.162.

Standard deviation is in the same units as your original data. A dataset of daily temperatures with mean 22 and standard deviation 3 tells you most values fall between 19 and 25. That is immediately useful. Variance in squared degrees would not be.

Pure Python Implementation

Here is a full implementation from scratch using only the Python standard library. No external dependencies required.

import math

def mean(data):
    n = len(data)
    if n == 0:
        raise ValueError("Cannot compute mean of empty dataset")
    return sum(data) / n

def population_variance(data):
    n = len(data)
    if n == 0:
        raise ValueError("Cannot compute variance of empty dataset")
    m = mean(data)
    return sum((x - m) ** 2 for x in data) / n

def sample_variance(data):
    n = len(data)
    if n < 2:
        raise ValueError("Sample variance requires at least 2 data points")
    m = mean(data)
    return sum((x - m) ** 2 for x in data) / (n - 1)

def population_stdev(data):
    return math.sqrt(population_variance(data))

def sample_stdev(data):
    return math.sqrt(sample_variance(data))

data = [7, 5, 4, 9, 12, 45]
print("Mean:", mean(data))
print("Population variance:", population_variance(data))
print("Population stdev:", population_stdev(data))
print("Sample variance:", sample_variance(data))
print("Sample stdev:", sample_stdev(data))

Running this produces:

Mean: 13.666666666666666
Population variance: 182.22222222222223
Population stdev: 13.497674867066126
Sample variance: 218.66666666666666
Sample stdev: 14.78839351951257

The dataset [7, 5, 4, 9, 12, 45] has a mean of 13.67. The population standard deviation of 13.50 reflects how spread out the values are from the mean when treating this as the full dataset. The sample standard deviation of 14.79 is higher because dividing by N-1 instead of N produces a larger estimate of spread, which is appropriate when this is a sample from a larger population.

Using the statistics Module

Python ships with a built-in statistics module that handles mean, variance, and standard deviation. Import it with import statistics. The module is part of the standard library, so no pip install needed.

import statistics

data = [7, 5, 4, 9, 12, 45]

print(statistics.mean(data))
print(statistics.pvariance(data))  # population variance
print(statistics.pstdev(data))     # population standard deviation
print(statistics.variance(data))   # sample variance
print(statistics.stdev(data))      # sample standard deviation

Output:

13.666666666666666
182.22222222222223
13.497674867066126
218.66666666666666
14.78839351951257

The statistics module names its functions directly. mean() computes the arithmetic mean. pvariance() and pstdev() compute population variance and population standard deviation respectively. variance() and stdev() compute sample variance and sample standard deviation. All four naming pairs exist side by side, which removes ambiguity about which variant you are getting.

One quirk worth noting: statistics.variance() and statistics.stdev() use Bessel’s correction internally, dividing by N-1. The documentation calls this the “sample variance.” If you want population statistics, you must explicitly use pvariance and pstdev. This trips up people coming from spreadsheets where the default is often population statistics.

The official documentation for the statistics module is at https://docs.python.org/3/library/statistics.html.

Using NumPy

NumPy is the library you reach for when working with arrays, matrices, or any kind of numerical computing in Python. It handles mean and standard deviation across axes, which the statistics module cannot do. NumPy is not in the standard library, so install it with pip install numpy if you do not have it already.

import numpy as np

data = [7, 5, 4, 9, 12, 45]

print(np.mean(data))
print(np.std(data))         # population std, ddof=0 by default
print(np.std(data, ddof=0))  # explicit population std
print(np.std(data, ddof=1))  # sample std

Output:

13.666666666666666
13.497674867066126
13.497674867066126
14.78839351951257

The critical detail here is the ddof parameter. NumPy defaults to population standard deviation (ddof=0). Many people do not realize this and end up using population std when they meant to use sample std. The ddof stands for delta degrees of freedom. Set ddof=0 for population, ddof=1 for sample.

NumPy also has np.var() for variance, which accepts the same ddof parameter.

print(np.var(data, ddof=0))  # population variance
print(np.var(data, ddof=1))  # sample variance

NumPy becomes indispensable when you need to compute statistics across dimensions. If you have a 2D array of sensor readings and want the standard deviation of each column, NumPy handles that with a single call.

import numpy as np

# 5 days of hourly temperature readings for 3 cities
readings = np.array([
    [22, 25, 19],
    [21, 24, 20],
    [23, 26, 18],
    [20, 23, 21],
    [24, 27, 17]
])

print("Mean per city:", np.mean(readings, axis=0))
print("Stdev per city:", np.std(readings, axis=0, ddof=1))

Output:

Mean per city: [22.  25.  18.]
Stdev per city: [1.58113883 1.58113883 1.58113883]

NumPy broadcasts the operation across the specified axis. axis=0 computes statistics down the rows, giving you one value per column. axis=1 would give you one value per row. This is how you efficiently compute per-group statistics on large datasets.

The official NumPy documentation is at https://numpy.org/doc/.

Population vs Sample in Data Science

The population vs sample distinction is where most statistics confusion lives. I see this mistake constantly in data science work.

Population statistics treat your dataset as the complete universe of interest. You divide by N. Sample statistics treat your dataset as a subset drawn from a larger population. You divide by N-1.

When you run a factory quality check and measure every single widget produced in a shift, you have the full population. Use population statistics. When you sample 100 widgets from today’s production to estimate quality for the entire week, you have a sample. Use sample statistics.

The N-1 correction in sample statistics is called Bessel’s correction. Without getting into the derivation, it roughly means “this estimate of spread from a sample is systematically too low, so bump it up by a little.” The correction gets less important as N grows. With N=1000, the difference between dividing by 1000 and dividing by 999 is negligible. With N=5, the difference is enormous.

Here is a side-by-side comparison with a tiny dataset to make the math visible:

import statistics
import numpy as np

data = [10, 12, 14, 16, 18]

pop_stdev = np.std(data, ddof=0)
sample_stdev = np.std(data, ddof=1)

print(f"Population stdev:  {pop_stdev:.4f}")
print(f"Sample stdev:      {sample_stdev:.4f}")
print(f"statistics.stdev:  {statistics.stdev(data):.4f}")
print(f"statistics.pstdev:  {statistics.pstdev(data):.4f}")

Output:

Population stdev:  2.5298
Sample stdev:      2.8284
statistics.stdev:  2.8284
statistics.pstdev:  2.5298

The sample standard deviation is always larger than the population standard deviation for the same dataset. This is by design. The sample stdev formula makes a conservative correction for the fact that you do not have all the data.

For data science work involving sampling, always use sample statistics unless you have a strong reason not to. Defaulting to sample stdev is the safer choice because it produces a slightly larger spread estimate, which is appropriate for conservative risk modeling, quality control, and forecasting.

Practical Example: Housing Price Analysis

Let me walk through a complete worked example using real-world housing price data. This is the kind of scenario you encounter constantly: a dataset with some high outliers, and you need to summarize it meaningfully.

import statistics

# Monthly median home prices (in thousands) for a mid-sized US city over 12 months
prices = [245, 248, 251, 249, 255, 270, 275, 272, 268, 265, 260, 258]

mean_price = statistics.mean(prices)
sample_stdev_price = statistics.stdev(prices)
pop_stdev_price = statistics.pstdev(prices)

print(f"Mean price:          ${mean_price:.1f}K")
print(f"Sample stdev:         ${sample_stdev_price:.1f}K")
print(f"Population stdev:     ${pop_stdev_price:.1f}K")
print(f"Expected range (sample): ${mean_price - sample_stdev_price:.1f}K - ${mean_price:.1f}K - ${mean_price + sample_stdev_price:.1f}K")
print(f"Expected range (pop):    ${mean_price - pop_stdev_price:.1f}K - ${mean_price:.1f}K - ${mean_price + pop_stdev_price:.1f}K")

Output:

Mean price:          $259.7K
Sample stdev:        $9.9K
Population stdev:    $9.0K
Expected range (sample): $249.8K - $259.7K - $269.6K
Expected range (pop):    $250.7K - $259.7K - $268.7K

The data shows prices climbing from $245K to $275K then pulling back to $258K. The mean of $259.7K reflects that trajectory. The sample standard deviation of $9.9K tells you that a typical month saw prices about $9.9K away from that mean.

Using the empirical rule (approximately 68% of data within one standard deviation of the mean), you expect most monthly prices to fall between $249.8K and $269.6K using sample statistics. July ($275K) and June ($270K) sit outside that range, identifying them as months with unusually high prices.

Now compare the two standard deviations. The sample stdev ($9.9K) is larger than the population stdev ($9.0K). This is the expected behavior. Since we only have 12 months of data, we divide by 11 rather than 12, producing the larger estimate.

In a real pipeline, you would decide before calculating whether you are treating these 12 months as the complete dataset or as a sample from a longer history. That decision drives whether you use population or sample statistics.

Common Pitfalls

The ddof parameter is the most frequent mistake I see in production code. NumPy defaults to ddof=0 (population). The statistics module defaults to sample statistics through stdev() but provides pstdev() for population. If you mix libraries without checking which variant each one defaults to, your spread estimates will be inconsistent.

Here is the trap:

import statistics
import numpy as np

data = [10, 12, 14, 16, 18]

# These give DIFFERENT answers
print(statistics.stdev(data))   # 2.8284  (sample, ddof=1)
print(np.std(data))             # 2.5298  (population, ddof=0)

Same data, different answers, because the two libraries have different defaults. Always check which default each library uses. When in doubt, set ddof explicitly rather than relying on the default.

A second pitfall is computing variance or standard deviation on a single data point. The math breaks down because sample variance divides by N-1, and a dataset with one value produces division by zero. The statistics module raises StatisticsError in this case. NumPy returns NaN. Always validate your dataset size before computing spread statistics.

import statistics

try:
    statistics.stdev([42])
except StatisticsError as e:
    print(f"Caught the error: {e}")

import numpy as np
print(np.std([42], ddof=1))  # Returns nan

A third pitfall is assuming standard deviation handles skewed data well. It does not. Standard deviation assumes data is roughly symmetric around the mean. A dataset with [1, 2, 2, 2, 2, 3, 100] has mean 16 and stdev of about 33. The standard deviation is dominated by that one outlier, and the summary “values are typically 16 plus or minus 33” is not a useful description of the majority of data. For skewed datasets, consider median and interquartile range instead.

statistics vs NumPy vs Pure Python

Here is how the three approaches compare across the dimensions that matter in production work.

Pure Python is best for learning the math and for small datasets where clarity matters more than speed. The statistics module is best for typical script work, data analysis, and anywhere you need sample statistics by default. NumPy is best for large arrays, matrix operations, and anywhere you need to compute statistics across dimensions or in a performance-sensitive pipeline.

FAQ: Mean and Standard Deviation in Python

What is the formula for standard deviation in Python?

Standard deviation is the square root of variance. Population standard deviation is sqrt(sum((xi – mean)^2) / N). Sample standard deviation is sqrt(sum((xi – mean)^2) / (N-1)).

How do I calculate mean in Python without libraries?

Divide the sum of all values by the count. mean = sum(data) / len(data). The statistics module also provides statistics.mean(data) as a built-in option.

What is ddof in NumPy?

ddof stands for delta degrees of freedom. Set ddof=0 for population standard deviation, ddof=1 for sample standard deviation. NumPy defaults to ddof=0.

What is the difference between population and sample standard deviation?

Population standard deviation divides by N. Sample standard deviation divides by N-1. The N-1 correction (Bessel’s correction) compensates for the fact that a sample systematically underestimates the true population spread.

Which Python library should I use for statistics?

Use the statistics module for general data analysis and scripts. Use NumPy for array operations, matrix math, and performance-critical numerical work. Use pure Python when you need transparency into the calculation or when you are learning the underlying math.

Why does my NumPy stddev differ from statistics.stdev?

NumPy defaults to population standard deviation (ddof=0). The statistics module’s stdev() function defaults to sample standard deviation (ddof=1). Use the same ddof value in both to get consistent results.

How do I avoid division by zero in standard deviation calculations?

Check that your dataset has at least two values before computing sample standard deviation. Population standard deviation technically works with one value (returns 0), but a dataset with no variance and only one point is rarely meaningful.

When should I use population vs sample standard deviation?

Use population standard deviation when your dataset is the complete population you are analyzing (every measurement, every transaction, every widget). Use sample standard deviation when your dataset is a subset of a larger population and you want to infer the spread of the larger group.