Descriptive Statistics

Descriptive statistics summarise a dataset in a handful of numbers. Before any chart, before any model, you need to know the central tendency, spread, and shape of your data. These numbers are the foundation of every analysis.

Learning Objectives

• Understand and compute measures of central tendency: mean, median, mode
• Compute and interpret spread: variance, standard deviation, IQR, range
• Understand shape: skewness and kurtosis
• Use df.describe() and go beyond it
• Know when to use mean vs median

---

Measures of Central Tendency

import pandas as pd
import numpy as np

df = pd.read_csv("orders_clean.csv")
df["total"] = df["quantity"] * df["unit_price"]

# Three ways to describe "the typical order"
mean   = df["total"].mean()
median = df["total"].median()
mode   = df["total"].mode()[0]

print(f"Mean:   £{mean:,.2f}")    # arithmetic average
print(f"Median: £{median:,.2f}")  # middle value when sorted
print(f"Mode:   £{mode:,.2f}")    # most frequent value

When to use which

Situation	Use
Symmetric distribution (bell curve)	Mean — uses all data
Skewed distribution (revenue, prices)	Median — robust to extreme values
Categorical or discrete data	Mode — "most common value"

Mean can mislead with skewed data

Average UK salary is ~£35,000 but median is ~£28,000. A few high earners pull the mean up. When reporting to stakeholders, always check if mean and median are far apart — if they are, report both.

---

Measures of Spread

std = df["total"].std()
var = df["total"].var()
rng = df["total"].max() - df["total"].min()
iqr = df["total"].quantile(0.75) - df["total"].quantile(0.25)

print(f"Std Dev:  £{std:,.2f}")      # average distance from the mean
print(f"Variance: £{var:,.2f}")      # std² — in squared units
print(f"Range:    £{rng:,.2f}")      # max - min
print(f"IQR:      £{iqr:,.2f}")      # spread of the middle 50%

Standard deviation — what it means

If mean = £250 and std = £80: - ~68% of orders are between £170 and £330 (mean ± 1 std) - ~95% of orders are between £90 and £410 (mean ± 2 std) - ~99.7% of orders are between £10 and £490 (mean ± 3 std)

This only holds if the distribution is approximately normal. If it's skewed, use IQR instead.

---

Percentiles and Quartiles

percentiles = df["total"].quantile([0.1, 0.25, 0.5, 0.75, 0.9, 0.95, 0.99])
print(percentiles)

Sample output:

0.10     29.99
0.25     89.97
0.50    149.00
0.75    298.00
0.90    389.00
0.95    520.00
0.99   1580.00

This tells you: - 50% of orders are under £149 (the median) - The top 10% of orders are over £389 - A handful of orders (1%) are over £1,580 — worth investigating

---

The describe() Extended

# Standard describe — numeric columns
print(df.describe())

# Include categorical columns
print(df.describe(include="all"))

# Custom percentiles
print(df["total"].describe(percentiles=[0.1, 0.25, 0.5, 0.75, 0.9, 0.95]))

# Describe a specific column with more detail
col = df["total"]
print(f"""
Count:    {col.count():,}
Mean:     £{col.mean():,.2f}
Median:   £{col.median():,.2f}
Std:      £{col.std():,.2f}
CV:       {col.std()/col.mean():.2%}   ← coefficient of variation (std/mean)
Skew:     {col.skew():.2f}             ← >0 = right-skewed, <0 = left-skewed
Kurtosis: {col.kurtosis():.2f}         ← >0 = heavy tails, <0 = light tails
""")

Skewness interpretation

Skewness	Interpretation
Close to 0	Roughly symmetric
> 0.5	Right-skewed (long right tail — most values are small, some very large)
< -0.5	Left-skewed (long left tail — most values are large, some very small)
> 1 or < -1	Highly skewed — consider log transform

Revenue, prices, and waiting times are almost always right-skewed.

---

Summary Statistics by Group

# Compare key stats across categories
print(
    df.groupby("category")["total"]
    .agg(["count", "mean", "median", "std"])
    .round(2)
    .sort_values("mean", ascending=False)
)

# Value counts with percentages for categorical columns
print(df["status"].value_counts())
print(df["status"].value_counts(normalize=True).mul(100).round(1).astype(str) + "%")

---

Practice Exercises

Warm-up 1. Compute mean, median, std, and IQR of the total column. 2. Print the 10th, 25th, 50th, 75th, and 90th percentiles of total. 3. Print df.describe() for the full DataFrame.

Main 4. Calculate summary stats (count, mean, median, std) for total grouped by status. 5. The mean and median of total differ by more than £100. Explain what this tells you about the distribution. 6. Calculate the coefficient of variation (std/mean) for unit_price. Which category has the most variable pricing?

Stretch 7. Write a function full_summary(series) that returns: count, mean, median, mode, std, IQR, skewness, kurtosis, and the 5 most common values. 8. Build a "profiling report" — for each numeric column, compute all the above stats. For each string column, compute: count, unique count, top 5 values and their frequencies.

---

Interview Questions

[Beginner] What is the difference between mean and median? When does each give you a better picture of the data?

[Mid-level] What does a highly skewed distribution tell you about your data? How does it affect which statistics you report?

[Senior] A new analyst reports average customer spend as £450 to the CEO. You know the median is £89. What happened, and how do you explain the discrepancy?

---

← Agenda · Next: Univariate Analysis →