Session 3: Practice

Which model would survive a referee?

Thierry Warin, PhD

HEC Montréal

Outline · The workshop

  1. Ninety minutes
  2. Before you start
  3. Reproduce — 20 min
  4. Apply to your own angle — 35 min
  5. Break one assumption — 15 min
  6. Report it as one table
  7. Write it down — 20 min
  8. Submitting
  9. What loses marks
  10. Two-minute report
  11. After class, if you want it

What session 3 is for

The mathematics

  • Derive the least-squares slope, and read “controlling for” as a residual regression
  • Read residuals versus fitted and say what structure it reveals
  • Compute leverage from the hat matrix, and show it sums to \(p\)
  • Separate an outlier from an influential point with Cook’s distance
  • Compare models on AIC, and say why that is not model selection

The Python

  • Fit a simple and a multiple regression with smf.ols, and read .params
  • Pull diagnostics from a fitted model with get_influence()
  • Build the four diagnostic plots yourself rather than calling one function
  • Compare candidate specifications side by side on AIC and BIC
  • Report the comparison as a publication table with qmib.regtable

The international business

  • Quote a slope in units a manager can act on
  • Say why a slope is an association, and what more is needed to call it a cause
  • Say whether a published coefficient survives its own diagnostics
  • Decide what to do about the one country that moves the whole result
  • Refuse a model whose residuals show it has missed the structure

Your Python so far

Everything you need to type is on one sheet — how to start Python in VS Codium on your platform, how to make a file and run it, the DataFrame basics, and every name from sessions 1 to 3, each with a line showing it in use.

Python Cheatsheet · Session 03 (PDF) · in the repository at 03-…/PYTHON-CHEATSHEET.pdf

One sheet per package, functions explained with pictures: pandas · NumPy · matplotlib · statsmodels

Print it, or keep it open beside the editor. Nothing on it is examinable; all of it is assumed.

Practice

Which model would survive a referee?

Ninety minutes

00–20 Reproduce one result from the paper you read
20–55 Apply the method, to your own angle
55–70 Break one assumption, deliberately
70–90 Write, and report 250 words, then two minutes to the room

The presenter is drawn at random when your group is called. All three of you prepare.

Before you start

Everyone pushes at least once. Replace XX with your group number — group 07 uses group-07.

git checkout -b group-XX
mkdir -p groups/A2026/group-XX/session-03      # macOS and Linux
git checkout -b group-XX
mkdir groups/A2026/group-XX/session-03         # Windows — no -p

PowerShell’s mkdir creates the intermediate folders already, and rejects -p. The git log is the participation record. It is not a formality.

1 · Reproduce — 20 min

Take the result the paper rests on, and reproduce it. The package you downloaded is at 03-regression-adequacy-and-validity/data/replication/.

import qmib

data = qmib.load("core")     # the session's own data

Paper: Amsili, van Es & Schindelbeck (2024), Pedotransfer Functions for Field Capacity, Permanent Wilting Point, and Available Water Capacity · 10.7910/DVN/U5DAEP

A failed reproduction that you diagnose earns full marks. One you do not attempt earns none.

“It did not work” is not a diagnosis. Where did it stop, and what did you check?

2 · Apply to your own angle — 35 min

core = qmib.load("core")
mine = qmib.load("angle_c_country")     # your angle — see your dictionary
df   = core.merge(mine, on=["geo", "time"], how="inner")

Fit the session’s method on your data, and report what the lecture said to report — in the units of your own variables.

A coefficient without units is not a finding.

3 · Break one assumption — 15 min

Every method in this course rests on something.

Find the assumption this one leans on hardest. Break it deliberately. Record what happened to your answer.

This is the part that carries the marks. A group that shows how its result falls apart understands it; a group that only shows it holding does not yet know.

Report it as one table

Step 3 left you with two fits: the model as specified, and the model after you broke something. Put them in one table rather than two blocks of output — the comparison is the finding.

import qmib

print(qmib.regtable([before, after],
                    names=["as specified", "assumption broken"]))

Read across the row, not down the column: what happened to the coefficient you care about, in its own units, and what happened to its standard error.

Coefficients too small to read at one decimal — a logit’s usually are? digits=3. .as_latex() if you would rather paste it into the write-up than screenshot it. Pass stats= to change the rows at the foot.

4 · Write it down — 20 min

250 words in your submissions folder:

  • What you found, in units
  • Which assumption you broke, and what it did
  • What this result does not license you to claim

And the standing rule: one LLM output you identified as wrong, and how you established that.

Submitting

git add groups/A2026/group-XX/session-03
git commit -m "Session 03 practice — group XX"
git push -u origin group-XX

Deliverable. A diagnostic report on your group’s own regression: four plots, the observations you investigated, and a 250-word note on which conclusions survived the diagnostics and which did not.

What loses marks

  • Reporting \(R^2\) without a single diagnostic plot
  • Deleting an influential point without saying what it was
  • Reading a residual plot as “looks fine” with no statement of what you looked for
  • Choosing a model on AIC and reporting it as though the data selected it

And everywhere in this course: an association described in causal language.

Two-minute report

One number with its units and its uncertainty
One assumption the one you broke, and what happened
One limit what you cannot claim from this

Listen for traceable evidence, not confidence of delivery.

After class, if you want it

The QMIB Lab has a knowledge check on what you have just done: warin.ca/qmib-labs

Optional, never a prerequisite for the next lecture, and never marked for correctness. A completed report counts because attempting it is the engagement being measured — it is one of the three routes to the participation mark.

What is required before the next session: the article, its Dataverse package, and the self-check. That is the pre-session deck, not this.