Python Data Analysis with Pandas

Use pandas to load tabular data, clean it, aggregate it, and produce a first chart-ready summary.

What you will learn

Install pandas and load CSV data
Inspect frames with head, info, describe
Filter, group, join, and export results
Avoid common performance and correctness traps

Prerequisites

Python 3.11+
pip or uv
Basic Python: lists, dicts, functions

Install

python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install pandas numpy matplotlib

Getting started: pandas getting started and 10 minutes to pandas.

Load and inspect

import pandas as pd

df = pd.read_csv("data.csv")
print(df.head())
print(df.info())
print(df.describe(include="all"))
print(df["country"].value_counts(dropna=False).head())

Always check dtypes and missing values before aggregating. A “number” stored as text will silently break sums.

Clean and aggregate

clean = df.dropna(subset=["country"]).copy()
clean["revenue"] = pd.to_numeric(clean["revenue"], errors="coerce").fillna(0)
clean["signed_up"] = pd.to_datetime(clean["signed_up"], errors="coerce")

summary = (
    clean.groupby("country", as_index=False)
    .agg(revenue=("revenue", "sum"), users=("user_id", "nunique"))
    .sort_values("revenue", ascending=False)
)
summary.to_csv("revenue-by-country.csv", index=False)
print(summary.head(10))

Filter and merge

eu = clean[clean["region"].eq("EU")]
users = pd.read_csv("users.csv")
merged = eu.merge(users, on="user_id", how="left", validate="m:1")

Use validate when you know the expected cardinality—it catches duplicate keys early.

Quick plot

summary.head(10).plot(x="country", y="revenue", kind="bar", title="Revenue by country")

Practice checklist

Load a CSV and list columns with missing values
Convert one numeric column with to_numeric(..., errors="coerce")
Produce a grouped summary and export it
Merge a second table and confirm row counts

Common pitfalls

Chained indexing (df[a][b] = ...) instead of .loc
Forgetting copy() after filters when you mutate later
Grouping on raw strings with inconsistent casing (US vs us)

Sources and further reading

Who this guide is for

Analysts replacing manual spreadsheet steps
Python developers preparing tabular data
Teams building reproducible exploratory workflows

Definition of done

A reproducible notebook or script that loads validated input, documents cleaning decisions, produces a checked aggregation, exports a versioned result, and records data-quality limitations.

Finishing the commands is not the same as finishing the guide. Keep the result only when another person can reproduce it, inspect the evidence, and understand the remaining risks.

Professional workflow

Phase 1 — Define the analytical question

Goal: Prevent exploration from becoming an unreviewable sequence of edits.

Write the decision or question in one sentence.
Define the expected row grain and required columns.
Record source, extraction time, units, and known limitations.

Verification: A reviewer can tell what one row means before running any code.

Phase 2 — Profile and clean explicitly

Goal: Make every data change observable.

Inspect types, missingness, duplicates, ranges, and category values.
Convert types with deliberate error handling.
Separate raw, cleaned, and derived dataframes.

Verification: Assertions fail when row counts, null rates, or ranges exceed documented expectations.

Phase 3 — Analyze and communicate

Goal: Produce results that can be reproduced and challenged.

Build aggregations from named intermediate steps.
Cross-check one result independently.
Export data and narrative with parameters and timestamp.

Verification: A clean environment reproduces the same result from the recorded input.

Review questions before you continue

What is the grain of the raw and final data?
Which missing values mean unknown versus zero?
Could a merge duplicate observations?
Which result would change the decision?

If an answer is unknown, record it as an explicit follow-up instead of hiding the uncertainty behind a successful demo.

Troubleshooting decision guide

A merge increases rows unexpectedly: validate key uniqueness and use the merge validation option.
Numbers become object dtype: inspect separators, currency symbols, and invalid values before conversion.
Memory use spikes: select columns early, use efficient dtypes, or process chunks.
Notebook output cannot be reproduced: restart the kernel and run all cells in order.

Change one variable at a time, preserve the first useful error, and write down the command or condition that fixed it. That turns a private debugging session into team knowledge.

Production-readiness checklist

Keep raw inputs immutable.
Version environment and parameters.
Validate schema and quality before analysis.
Avoid logging sensitive row-level data.
Move stable pipelines from notebooks into tested functions or scripts.

These checks are intentionally stricter than a beginner demo. Use them to decide whether to continue learning, run a controlled pilot, or request specialist review.

Your next 30 minutes

Add assertions for row count, uniqueness, and null thresholds.
Cross-check one metric with an independent calculation.
Restart and run the workflow from a clean environment.
Publish the output with a short limitations section.

CTA: Pick the first unchecked step, complete it now, and save the evidence in your project README or decision log. Then open the official documentation linked below before adopting the result in production.