Probability Basics¶
Probability is the mathematics of uncertainty. It's how we express "how likely" and "how confident" — and it underpins every statistical test, confidence interval, and machine learning model you'll encounter.
Learning Objectives¶
- Understand probability as a proportion between 0 and 1
- Apply the addition and multiplication rules
- Understand conditional probability
- Calculate Bayes' theorem for business scenarios
- Understand independence vs dependence
Probability Fundamentals¶
A probability is a number between 0 and 1: - 0 = impossible - 1 = certain - 0.5 = equally likely to happen or not
# Empirical probability: estimate from data
orders = pd.read_csv("orders.csv")
# Probability a random order is cancelled
p_cancelled = (orders["status"] == "cancelled").mean()
print(f"P(cancelled) = {p_cancelled:.3f}") # e.g., 0.085
# Probability an order is completed
p_completed = (orders["status"] == "completed").mean()
print(f"P(completed) = {p_completed:.3f}")
The Addition Rule — P(A or B)¶
If events are mutually exclusive (can't both happen):
# P(cancelled OR refunded) — mutually exclusive
p_cancelled = 0.085
p_refunded = 0.032
p_either = p_cancelled + p_refunded
print(f"P(cancelled or refunded) = {p_either:.3f}") # 0.117
If events can overlap (use inclusion-exclusion):
The Multiplication Rule — P(A and B)¶
If events are independent:
# P(two consecutive orders are both completed)
p_completed = 0.65
p_both = p_completed ** 2
print(f"P(two consecutive completed) = {p_both:.3f}") # 0.423
If events are dependent:
Conditional Probability — P(B|A)¶
"The probability of B, given that we know A has already happened."
# P(high order value | customer is VIP)
vip_orders = orders[orders["segment"] == "VIP"]
p_high_given_vip = (vip_orders["total"] > 300).mean()
print(f"P(total > £300 | VIP) = {p_high_given_vip:.3f}")
# Compare to overall probability
p_high_overall = (orders["total"] > 300).mean()
print(f"P(total > £300 | overall) = {p_high_overall:.3f}")
Bayes' Theorem — Updating Probability with Evidence¶
The most important formula in data science:
Business example: Your fraud detection model flags a transaction as suspicious. The flag is 90% accurate. But only 1% of transactions are actually fraud. What's the probability the transaction is really fraudulent?
# Bayes: P(fraud | flagged) = ?
p_fraud = 0.01 # prior: 1% of transactions are fraud
p_flagged_given_fraud = 0.90 # sensitivity: 90% of fraud gets flagged
p_flagged_given_legit = 0.05 # false positive rate: 5% of legit gets flagged
# P(flagged) = P(flagged|fraud)×P(fraud) + P(flagged|legit)×P(legit)
p_legit = 1 - p_fraud
p_flagged = (p_flagged_given_fraud * p_fraud) + (p_flagged_given_legit * p_legit)
# Bayes
p_fraud_given_flagged = (p_flagged_given_fraud * p_fraud) / p_flagged
print(f"P(fraud | flagged) = {p_fraud_given_flagged:.1%}") # ~15.5%
Even with a 90% accurate test, only ~15% of flagged transactions are actually fraudulent — because fraud is rare. This is why fraud review teams must still manually check alerts.
This is called the "base rate fallacy"
The rarer the event, the more false positives you get — even with a highly accurate test. This matters for fraud detection, medical testing, and churn prediction.
Practice Exercises¶
Warm-up 1. Calculate P(completed) from your orders data. 2. Calculate P(cancelled OR refunded) — assuming these are mutually exclusive. 3. Calculate P(two consecutive orders are both completed), assuming independence.
Main 4. Calculate the conditional probability P(total > £200 | Electronics) vs P(total > £200 | Books). Which category has higher-value orders? 5. A quality control test for defective products is 95% accurate (sensitivity = 95%), with a 3% false positive rate. If 2% of products are actually defective, what is P(defective | test positive)? 6. Using your orders data, calculate P(high value | VIP) and P(high value | Regular). Are segment and order value independent?
Stretch 7. Build a simple Naive Bayes classifier that predicts whether an order will be cancelled based on category and customer segment. 8. Explain the base rate fallacy using a real business example from your dataset.
Interview Questions¶
[Beginner] What does a probability of 0.05 mean?
[Mid-level] What is conditional probability? Give a business example.
[Senior] Your A/B test shows p = 0.04 for the B variant. The marketing team wants to roll it out immediately. What questions would you ask before recommending rollout?