Intro to Scientific Python: Cheatsheet

Where am I? What's here?

Moving around: cd

cd ~/workspace/gsk-intro-scientific-python-course

Absolute paths start with / or ~. Relative paths start from where you are, like data/gapminder.tsv.

Tab completion

Type the first few letters, then press Tab. The shell finishes the name for you.

• One match: it completes the whole name
• Several matches: press Tab twice to list them, type another letter, Tab again
• Works for folders, files, and git branches
• Same trick in ipython and notebooks: df. + Tab lists methods

cd ~/work[Tab]      # becomes ~/workspace/
cd gsk[Tab]        # gsk-intro-...

Faster, and no typos. ↑ recalls your previous commands.

Files & folders

conda & Jupyter

conda --version
conda env list
conda env create -f environment.yml
conda activate gsk
conda install gh
jupyter lab      # run from the repo root
ipython          # interactive Python

(gsk) at the start of your prompt means the environment is active. (base) is the default one.

Get a repo

git clone <url>      # copy a repo locally
git status          # what changed?
git remote -v       # list remotes + URLs

origin is your fork, upstream is the course repo.

Set up your fork

git remote add upstream <course-url>
git remote set-url origin <your-fork-url>
git remote -v

Fork on GitHub first (Fork button, top right).

Save your work

git add <file>            # stage
git commit -m "message"  # snapshot
git log                  # history

Stage, then commit. Press q to leave git log.

Branches

git branch               # list (* = current)
git switch -c new_branch # create, switch
git switch main          # switch

• main: only what's in the course repo
• personal: your notes and assignments
• yearbook_first_last: your yearbook PR

Stay in sync

git switch main
git pull upstream main   # get new modules
git switch personal
git rebase main          # replay yours on top

Rebase conflict? git status shows the files. Then git rebase --abort to undo, or fix the file, git add, and git rebase --continue.

See what changed

git diff                 # unstaged
git fetch upstream       # download only
git diff main upstream/main
git restore README.md    # discard edit

-arr = np.array([1, 2, 3, 4])
+arr = np.array([1, 2, 3, 4, 5])

- removed, + added, no prefix: unchanged.

Send it to GitHub

gh auth login --hostname github.com \
  --git-protocol https --web
gh auth status
git push origin personal
git push --force-with-lease \
  origin personal

After a rebase your commits have new hashes, so a plain push is rejected. --force-with-lease overwrites your fork only if no one else pushed.

Pull request workflow

• git switch main → git pull upstream main
• git switch -c my-branch
• edit, git add, git commit
• git push origin my-branch
• Open the PR on GitHub (base: course repo main, compare: your branch)
• After merge: git switch main → git pull upstream main

Moving work off main

git fetch upstream
git switch personal
git rebase main
git branch -f main upstream/main
git switch main

.gitignore

Lists files git should skip: build output, environments, caches, e.g. _build/, .ipynb_checkpoints, *.pdf.

Data types

x = 5                  # assignment
type(x)                # int
isinstance(x, int)     # True

Add ? to get help: print?

Strings

name = "Teon Brooks"
name.upper()          # 'TEON BROOKS'
name.lower()
name.split(" ")       # ['Teon', 'Brooks']
"-".join(["a", "b"])  # 'a-b'
f"Hello {name}!"      # f-string

Lists & slicing

nums = [1, 2, 3, 4, 5]
nums[0]        # 1 (zero-indexed)
nums[-1]       # 5 (last)
nums[2:4]      # [3, 4] (stop excluded)
nums[::2]      # [1, 3, 5]
nums.append(6)
nums.extend([7, 8])
nums.index(3)  # 2

list[start:stop:step]

Dictionaries

d = {"pet": "cat", "animals": ["cats", "dogs"]}
d["pet"]                    # 'cat'
d.get("missing", "n/a")     # 'n/a'
"pet" in d                  # True

d["missing"] raises a KeyError; .get() doesn't.

Built-in functions

Convert types: int() float() str() list() dict() set() tuple()

All of them: docs.python.org/3/library/functions.html

Truthiness & comparisons

Falsy: None, False, 0, 0.0, '', [], (), {}, set(). Everything else is truthy.

if & for

if x < 0:
    print("negative")
elif x == 0:
    print("zero")
else:
    print("positive")

if x % 2 == 0:
    print("even")
else:
    print("odd")

squares = []
for n in [1, 2, 3]:
    squares.append(n ** 2)

Indentation marks the block.

Functions

def greet(person):
    return f"Hello {person}!"

message = greet("Teon")

print() shows something; return hands it back so you can store it. Use snake_case names.

Classes

class Sample:
    def __init__(self, name):
        self.name = name       # attribute
    def shout(self):           # method
        return self.name.upper()

s = Sample("kenya")
s.name      # attribute, no ()
s.shout()   # method, with ()

Imports

import math
math.sqrt(16)

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns
from scipy import stats

Jupyter notebooks

Cells run in the order you run them. Out-of-order runs are a common source of bugs.

# Heading 1      ## Heading 2
**bold**  *italic*  `code`
- bullet
[a link](https://example.com)

Creating arrays

arr = np.array([1, 2, 3, 4])  # one sequence
np.zeros((3, 4))
np.ones(5)
np.arange(0, 21, 2)           # 0..20
np.linspace(0, 1, 5)          # stop included

Reshape & reduce

a = np.arange(24).reshape(4, 6)
a.ravel()           # flatten to 1-d
a.sum()             # everything
a.sum(axis=0)       # per column (6,)
a.sum(axis=1)       # per row (4,)
a.cumsum(axis=1)    # running total

Same axis idea for .min() .max() .mean(). Sizes must fit: 24 can't reshape to (5, 5).

Math & broadcasting

a = np.arange(24).reshape(4, 6)
a * a       # element-wise
a @ a.T     # matrix multiply
a * 10      # scalar broadcast
a * np.arange(1, 7)  # (4, 6) * (6,)

Broadcasting works when trailing dimensions match or one of them is 1. Here the 6 matches the 6 columns, so the row is applied to every row of a.

np.hstack((a, a))   # side by side
np.vstack((a, a))   # stacked

Random numbers

rng = np.random.default_rng(seed=42)
rng.random()                    # uniform 0-1
rng.integers(1, 7, size=10)     # dice
rng.choice(["H", "T"], size=10)
rng.normal(loc=0, scale=1, size=10)
rng.standard_normal((4, 100))
rng.shuffle(arr)                # in place

Same seed, same numbers. loc = mean, scale = std dev. size can be a tuple.

Simulation recipe

rolls = rng.integers(1, 7, size=10_000)
(rolls == 6).mean()   # P(roll a 6)

True counts as 1 and False as 0, so the mean of a Boolean array is a proportion.

The object-oriented way

import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(5, 2.7),
                       layout='constrained')
ax.plot(x, x, label='linear')
ax.plot(x, x**2, label='quadratic')
ax.set_title("Simple Plot")
ax.set_xlabel('x label')
ax.set_ylabel('y label')
ax.legend()

Figure = the canvas. Axes = one plot on it. One array passed to plot is treated as the y-values.

Styling & plot types

ax.plot(x, y, color='blue',
        linewidth=3, linestyle='--')
l, = ax.plot(x, y2)   # keep the line
l.set_linestyle(':')
ax.set_xscale('log')

Subplots & saving

fig, axes = plt.subplots(
    1, 2, figsize=(8, 3),
    layout='constrained')
axes[0].plot(data)          # 1-d: axes[i]
axes[1].hist(data, bins=20)
axes[0, 1]                  # 2-d: [row, col]

plt.subplots(2, 1, sharex=True)
fig.suptitle('Whole figure title')
fig.savefig('plot.png', dpi=150)

The extension picks the format: .png .pdf .svg.

Create & load

df = pd.DataFrame({'animal': ['panda', 'lion'],
                   'age': [1, 2.5]})
df = pd.read_csv('data/gapminder.tsv', sep='\t')

A DataFrame is a collection of Series. sep='\t' because the file is tab-separated. Paths are relative to the notebook.

Look at the data

Select columns & rows

df['age']                       # a Series
df[df['continent'] == 'Africa'] # mask
df.query("continent == 'Africa'")
df.query("continent in ['Europe', 'Africa']")

Strings inside query need different quotes than the outer string.

.iloc vs .loc

df.iloc[:100]            # 100 rows
df.iloc[:100, :2]        # by position
df.loc[:100]             # 101 rows!
df.loc[:100, ['continent', 'country']]
df.loc[df['year'] == 2007, ['country', 'lifeExp']]

Summary statistics

df['lifeExp'].mean()
df['lifeExp'].median()
df['age'].max()
df.describe()

Group-apply-combine

df.groupby('country')['lifeExp'].mean()
df.groupby(['continent', 'year'])['lifeExp'] \
  .mean().reset_index()
df.groupby('continent').agg({
    'lifeExp': ['mean', 'median', 'max']})

Group (split) → apply (compute) → combine. .reset_index() turns group labels back into columns.

Reshape: wide → long

long = counts.melt(id_vars='gene',
                   var_name='sample',
                   value_name='count')
long[['group', 'replicate']] = \
    long['sample'].str.split('_', expand=True)
long.groupby('group')['count'].mean()

To group by something, it has to be in a column, not stuck in the column names.

Rename columns

df = df.rename(columns={
    'lifeExp': 'life_expectancy',
    'gdpPercap': 'gdp_per_capita'})
df.columns = ['a', 'b', 'c', 'd', 'e', 'f']

.rename() returns a copy: save it back to df. It only changes the columns you name. Setting df.columns needs every name, in order.

Pandas cheat sheet

pandas.pydata.org/Pandas_Cheat_Sheet.pdf

Seaborn plots

import seaborn as sns
sns.histplot(x=data, kde=True)
sns.kdeplot(x=data)
sns.boxplot(data=df_2007, x='continent',
            y='life_expectancy')
sns.violinplot(data=groups)
sns.scatterplot(data=df_2007, x='gdp_per_capita',
                y='life_expectancy', hue='continent')
sns.lineplot(data=df, x='year',
             y='life_expectancy', hue='continent')

• Pass the DataFrame to data, name columns in x, y, hue
• sns.boxplot groups for you, like groupby
• lineplot draws the mean with a shaded interval

Seaborn + Matplotlib

fig, axes = plt.subplots(
    1, 2, figsize=(8, 3),
    layout='constrained')
sns.histplot(x=data, ax=axes[0])
sns.kdeplot(x=data, ax=axes[1])
axes[0].set_title('Histogram')

Pass ax= to place a seaborn plot, then use any Axes method.

Correlation

from scipy import stats
df['a'].corr(df['b'])          # pandas
stats.pearsonr(x, y)           # linear
stats.spearmanr(x, y)          # ranks
np.log10(df['gdp_per_capita']) # log

• r runs from -1 to 1; 0 means no linear relationship
• Curved but rising? Try Spearman or a log transform
• Correlation is not causation

T-tests

stats.ttest_ind(europe, africa)  # separate
stats.ttest_rel(y2007, y1952)    # paired

Each result has .statistic and .pvalue. For paired data, index both Series the same way (e.g. set_index('country')).

Which function?

Reading a p-value

• p is the chance of a result this extreme if there were truly no effect
• p < 0.05: unlikely to be chance
• p > 0.05: can't rule out chance. "Not significant" is not "no difference"
• A small p doesn't mean a big effect: look at r or the group means

"Life expectancy was higher in Europe
 (77.6 years) than in Africa (54.8 years),
 t = 12.6, p < 0.001."

Formulas & model_matrix

from formulaic import model_matrix

y, X = model_matrix(
    'life_expectancy ~ log_gdp', df_2007)
X = model_matrix(
    '~ log_gdp + C(continent)', df_2007)
X = X.drop(columns='Intercept')

• ~ reads "is modeled by". It returns the outcome y and the design matrix X
• Formulas add an Intercept column of 1s
• With no outcome, you get just X
• Drop Intercept before scikit-learn, which fits its own

Fit a regression (OLS)

import statsmodels.api as sm
from formulaic import model_matrix

y, X = model_matrix(
    'life_expectancy ~ log_gdp', df_2007)
model = sm.OLS(y, X)
results = model.fit()
print(results.summary())

• y comes first: sm.OLS(y, X)

Formula shortcut

import statsmodels.formula.api as smf

ols_model = smf.ols(
    'life_expectancy ~ log_gdp',
    data=df_2007)
results = ols_model.fit()
results.params

smf.ols takes the formula directly and adds the intercept for you.

Add predictors

'life_expectancy ~ log_gdp + log_pop'
'life_expectancy ~ log_gdp + C(continent)'

• + adds a predictor. C() marks a category, and is needed for numbers like year
• A category becomes 0/1 dummy columns. The first, alphabetically (Africa), is the baseline

Read the OLS summary

• coef: the slope. With log_gdp, each 10x in GDP per capita goes with 16.6 more years
• P>|t|: the p-value of each coefficient
• [0.025 0.975]: the 95% confidence interval
• R-squared: the share of variation explained

results.params
results.rsquared

Logistic regression

y, X = model_matrix(
    'high_life_exp ~ log_gdp', df_2007)
model = sm.Logit(y, X)
results = model.fit()
results.predict(X)

• For a yes/no outcome. coef is in log-odds
• np.exp(coef) is the odds ratio
• predict gives a probability. Above 0.5 is a yes

Check the predictions

actual = df_2007['high_life_exp']
probs = results.predict(X)
predicted = (probs > 0.5).astype(int)
(predicted == actual).mean()
pd.crosstab(actual, predicted)

Rows are the true class, columns the predicted class. The off-diagonal counts are the mistakes.

Fit & predict

from sklearn.linear_model import LinearRegression

X = df_2007[['log_gdp']]
y = df_2007['life_expectancy']
model = LinearRegression()
model.fit(X, y)
model.predict(X)
model.intercept_, model.coef_

X is a DataFrame (double brackets), y a Series. Fit on all the data, it gives the same line as statsmodels.

Train & test split

from sklearn.model_selection import train_test_split

splits = train_test_split(
    X, y, test_size=0.2, random_state=42)
X_train, X_test, y_train, y_test = splits
model.fit(X_train, y_train)
predictions = model.predict(X_test)

• Learn on the train set, judge on the test set
• random_state repeats the same split. stratify=y keeps class shares the same

Score a regression

from sklearn.metrics import r2_score, mean_squared_error

r2_score(y_test, predictions)
mean_squared_error(
    y_test, predictions) ** 0.5
model.score(X_test, y_test)

The square root of the mean squared error is the RMSE, in the units of y. With a small test set, R² changes with random_state.

Classification

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix

clf = LogisticRegression()
clf.fit(X_train, y_train)
predictions = clf.predict(X_test)
clf.predict_proba(X_test)
accuracy_score(y_test, predictions)
confusion_matrix(y_test, predictions)

• Rows are the true class, columns the predicted class
• Accuracy misleads when the classes are lopsided

statsmodels vs scikit-learn

y comes first in statsmodels, X first in scikit-learn. With scikit-learn, drop the Intercept column from model_matrix.

Working with an agent

cd gsk-intro-scientific-python-course
claude

• Start it in your project folder, and approve each edit or command
• Don't turn off the permission prompts
• Keep private data in a folder outside the project

Plan first

• Ask for a plan before any code. Read it, correct it, then say go
• Limit the libraries: "Use only the packages in environment.yml"
• Work in small steps. Put notes in a CLAUDE.md

Review what it wrote

• Is the data split into train and test?
• Is the score computed on the test set?
• Is anything fit on the test data?
• Is X a DataFrame and y a Series?
• Is there one intercept, not two?
• Do the numbers make sense? Run r2_score yourself