NumPy and Pandas — Interview Questions¶

Questions for Python data analysis interviews. Focus: DataFrames, groupby, merge, filtering, apply, and performance.

NumPy¶

[Beginner] What is the difference between a Python list and a NumPy array?

Show answer

	Python list	NumPy array
Types	Mixed — any types in same list	Single dtype — all elements same type
Speed	Slow — Python overhead per element	Fast — C code, vectorised
Memory	More — Python objects	Less — compact typed array
Operations	Need explicit loops	Vectorised — `arr * 2` applies to all

import numpy as np

# List — need a loop or comprehension
revenues = [42000, 38500, 51200]
scaled = [r * 1.1 for r in revenues]

# NumPy — vectorised, no loop
arr = np.array(revenues)
scaled = arr * 1.1   # applies to every element

Use NumPy when you need fast mathematical operations on large arrays of numbers. Use Pandas when you need labelled columns, mixed types, or tabular data operations.

[Mid-level] What does the axis parameter control in NumPy functions like np.sum?

Show answer

axis specifies which dimension to operate across.

For a 2D array (matrix): - axis=0 → operate across rows → collapse into 1 row (column-wise result) - axis=1 → operate across columns → collapse into 1 column (row-wise result)

sales = np.array([
    [100, 150, 200],   # Q1: 3 regions
    [120, 130, 180],   # Q2: 3 regions
    [90,  160, 210],   # Q3: 3 regions
])

np.sum(sales)          # 1340  — total of everything
np.sum(sales, axis=0)  # [310 440 590]  — sum each column (region totals)
np.sum(sales, axis=1)  # [450 430 460]  — sum each row (quarterly totals)

Pandas — DataFrames¶

[Beginner] What does df.describe() show, and what does it NOT show?

Show answer

df.describe() shows summary statistics for numeric columns only: - count, mean, std, min, 25%, 50% (median), 75%, max

It does NOT show: - Statistics for string/categorical columns (unless you add include="all") - Missing value counts (use df.isnull().sum()) - Mode - Skewness or kurtosis

print(df.describe())                    # numeric columns only
print(df.describe(include="all"))       # all columns including strings
print(df["status"].value_counts())      # frequency for categorical

[Mid-level] What is the difference between df.loc and df.iloc?

Show answer

df.loc — label-based indexing (uses index labels and column names)
df.iloc — position-based indexing (uses integer positions, 0-based)

The key difference is the stop: .loc is inclusive at both ends. .iloc is exclusive at the stop.

df = pd.DataFrame({"a": [10, 20, 30]}, index=["x", "y", "z"])

df.loc["x":"y"]    # rows x AND y (2 rows) — inclusive
df.iloc[0:2]       # rows at positions 0 AND 1 (2 rows) — exclusive stop

df.loc["x", "a"]   # value at label x, column a
df.iloc[0, 0]      # value at position (0, 0)

Use .loc when you know the label. Use .iloc for positional access (first row, last 5 rows, etc.).

[Mid-level] Explain chained assignment in Pandas and why it's a problem.

Show answer

Chained assignment is when you assign to a slice of a DataFrame in two steps:

# WRONG — chained assignment
df[df["status"] == "pending"]["priority"] = "high"

# This may or may not modify df — it's undefined behaviour.
# Pandas raises a SettingWithCopyWarning.

The problem: df[df["status"] == "pending"] may return a copy or a view depending on the operation. Assigning to a copy has no effect on the original.

Fix: use .loc in a single step:

# CORRECT — single step with .loc
df.loc[df["status"] == "pending", "priority"] = "high"

GroupBy¶

[Beginner] What does .groupby() do, and how is it similar to SQL's GROUP BY?

Show answer

.groupby() splits a DataFrame into groups based on one or more columns, applies an aggregation function, and combines the results.

# Pandas
df.groupby("status")["total"].sum()

# SQL equivalent
# SELECT status, SUM(total) FROM orders GROUP BY status;

Both produce one row per unique group value. The difference is that Pandas returns the group keys as the index by default — use .reset_index() to convert them back to columns.

[Mid-level] What is the difference between .agg() and .transform() after groupby?

Show answer

.agg() collapses each group to one row — like SQL GROUP BY
.transform() returns a Series the same length as the original DataFrame — the group aggregate broadcast back to every row

# agg: 1 row per customer (4 customers → 4 rows)
customer_totals = df.groupby("customer_id")["total"].agg("sum")

# transform: 1 row per ORDER, with customer total added (N orders → N rows)
df["customer_total"] = df.groupby("customer_id")["total"].transform("sum")
df["pct_of_customer"] = df["total"] / df["customer_total"]

.transform() is the Pandas equivalent of SQL window functions with PARTITION BY.

[Mid-level] How do you calculate multiple aggregations in a single groupby? Write an example.

Show answer

summary = (
    df[df["status"] == "completed"]
    .groupby("customer_id")
    .agg(
        total_orders=("order_id", "count"),
        total_revenue=("total", "sum"),
        avg_order=("total", "mean"),
        max_order=("total", "max"),
        last_order=("order_date", "max"),
    )
    .reset_index()
    .sort_values("total_revenue", ascending=False)
)

This uses named aggregation — passing tuples of (column, function) with the output column name as the key. Available from Pandas 0.25+.

Merge¶

[Beginner] What is the difference between pd.merge() with how="inner" and how="left"?

Show answer

how="inner" (default): only rows where the join key exists in both DataFrames
how="left": all rows from the left DataFrame, plus matching rows from the right. Unmatched rows get NaN for right-side columns.

orders = pd.DataFrame({"order_id": [1, 2, 3], "customer_id": ["C1", "C2", "C99"]})
customers = pd.DataFrame({"customer_id": ["C1", "C2"], "name": ["Alice", "Bob"]})

inner = pd.merge(orders, customers, on="customer_id")
# Only rows with C1 and C2 (C99 dropped)

left = pd.merge(orders, customers, on="customer_id", how="left")
# All 3 orders; row with C99 gets NaN for "name"

[Mid-level] After a merge, your row count increased from 1,000 to 2,500. What happened and how do you fix it?

Show answer

A one-to-many relationship multiplied rows. For example, if customers has 2 rows for customer C001 (a data quality issue), every order from C001 gets duplicated.

Diagnosis:

# Check for duplicate keys in the right table
print(customers["customer_id"].duplicated().sum())

# Check row count before and after
before = len(orders)
merged = pd.merge(orders, customers, on="customer_id", how="left")
after = len(merged)
print(f"Rows: {before} → {after}")

Fix options: 1. Deduplicate the right table first: customers = customers.drop_duplicates(subset="customer_id") 2. If duplicates are expected, aggregate the right table before joining 3. Use validate="m:1" in pd.merge() to raise an error if the assumption is violated

[Senior] A Pandas groupby on a 50M-row DataFrame takes 4 minutes. What are your options?

Show answer

Use categorical dtypes for groupby columns — reduces comparison overhead significantly:
```
df["category"] = df["category"].astype("category")
```

Filter before groupby — reduce row count as early as possible:

df[df["status"] == "completed"].groupby(...)   # not df.groupby(...)

Use Polars — a Rust-based DataFrame library with multi-threaded groupby. 10-100x faster than Pandas on large datasets. Nearly identical API.
Use a database — push the aggregation to SQL/BigQuery/Redshift. Columnar stores are optimised for GROUP BY.
Partition the data — process month by month or region by region, then combine results with pd.concat.

Use Dask — distributed Pandas that processes chunks in parallel across cores:

import dask.dataframe as dd
ddf = dd.from_pandas(df, npartitions=8)
result = ddf.groupby("category")["total"].sum().compute()

← Mini Project · Next: Data Cleaning →