Session 10: Practice

Can your project support a causal claim at all?

Thierry Warin, PhD

HEC Montréal

Outline · The workshop

  1. Ninety minutes
  2. Before you start
  3. Reproduce — 20 min
  4. Apply to your own angle — 35 min
  5. Break one assumption — 15 min
  6. Report it as one table
  7. Write it down — 20 min
  8. Submitting
  9. What loses marks
  10. Two-minute report
  11. After class, if you want it

What session 10 is for

The mathematics

  • State the fundamental problem of causal inference, and write the naive difference as effect plus selection bias
  • Explain why randomisation solves it, and what it costs
  • Estimate a propensity score and match treated to control units
  • Report an ATT, and state the assumption it rests on

The Python

  • Simulate both potential outcomes and watch a coin flip recover the effect
  • Fit a propensity score with smf.logit and match with NearestNeighbors
  • Check overlap and balance (standardised differences) before reading any estimate
  • Score the naive, regression and matched estimates against an experimental benchmark

The international business

  • Estimate the effect of a programme that firms chose rather than drew
  • Say why the firms that joined were already different
  • Report how far the naive comparison would have been wrong

Your Python so far

Everything you need to type is on one sheet — how to start Python in VS Codium on your platform, how to make a file and run it, the DataFrame basics, and every name from sessions 1 to 10, each with a line showing it in use.

Python Cheatsheet · Session 10 (PDF) · in the repository at 10-…/PYTHON-CHEATSHEET.pdf

One sheet per package, functions explained with pictures: NumPy · matplotlib · statsmodels · scikit-learn · SciPy

Print it, or keep it open beside the editor. Nothing on it is examinable; all of it is assumed.

Practice

Can your project support a causal claim at all?

Ninety minutes

00–20 Reproduce one result from the paper you read
20–55 Apply the method, to your own angle
55–70 Break one assumption, deliberately
70–90 Write, and report 250 words, then two minutes to the room

The presenter is drawn at random when your group is called. All three of you prepare.

Before you start

Everyone pushes at least once. Replace XX with your group number — group 07 uses group-07.

git checkout -b group-XX
mkdir -p groups/A2026/group-XX/session-10      # macOS and Linux
git checkout -b group-XX
mkdir groups/A2026/group-XX/session-10         # Windows — no -p

PowerShell’s mkdir creates the intermediate folders already, and rejects -p. The git log is the participation record. It is not a formality.

1 · Reproduce — 20 min

Take the result the paper rests on, and reproduce it. The package you downloaded is at 10-causal-inference-foundations/data/replication/.

import qmib

data = qmib.load("core")     # the session's own data

Paper: Atkin, Khandelwal & Osman (2017), Exporting and Firm Performance: Evidence from a Randomized Experiment, Quarterly Journal of Economics 132(2) · 10.7910/DVN/QOGMVI

A failed reproduction that you diagnose earns full marks. One you do not attempt earns none.

“It did not work” is not a diagnosis. Where did it stop, and what did you check?

2 · Apply to your own angle — 35 min

core = qmib.load("core")
mine = qmib.load("angle_c_country")     # your angle — see your dictionary
df   = core.merge(mine, on=["geo", "time"], how="inner")

Fit the session’s method on your data, and report what the lecture said to report — in the units of your own variables.

A coefficient without units is not a finding.

3 · Break one assumption — 15 min

Every method in this course rests on something.

Find the assumption this one leans on hardest. Break it deliberately. Record what happened to your answer.

This is the part that carries the marks. A group that shows how its result falls apart understands it; a group that only shows it holding does not yet know.

Report it as one table

Step 3 left you with two fits: the model as specified, and the model after you broke something. Put them in one table rather than two blocks of output — the comparison is the finding.

import qmib

print(qmib.regtable([before, after],
                    names=["as specified", "assumption broken"]))

Read across the row, not down the column: what happened to the coefficient you care about, in its own units, and what happened to its standard error.

Coefficients too small to read at one decimal — a logit’s usually are? digits=3. .as_latex() if you would rather paste it into the write-up than screenshot it. Pass stats= to change the rows at the foot.

4 · Write it down — 20 min

250 words in your submissions folder:

  • What you found, in units
  • Which assumption you broke, and what it did
  • What this result does not license you to claim

And the standing rule: one LLM output you identified as wrong, and how you established that.

Submitting

git add groups/A2026/group-XX/session-10
git commit -m "Session 10 practice — group XX"
git push -u origin group-XX

Deliverable. A matched comparison on your own data: the naive difference, the overlap check, the balance table, the matched estimate, and the paragraph defending conditional ignorability — that paragraph carries the marks.

What loses marks

  • Reporting a matched estimate with no overlap diagnostic
  • Matching on a variable affected by the treatment
  • Calling an association causal because you controlled for something
  • Omitting the balance table

And everywhere in this course: an association described in causal language.

Two-minute report

One number with its units and its uncertainty
One assumption the one you broke, and what happened
One limit what you cannot claim from this

Listen for traceable evidence, not confidence of delivery.

After class, if you want it

The QMIB Lab has a knowledge check on what you have just done: warin.ca/qmib-labs

Optional, never a prerequisite for the next lecture, and never marked for correctness. A completed report counts because attempting it is the engagement being measured — it is one of the three routes to the participation mark.

What is required before the next session: the article, its Dataverse package, and the self-check. That is the pre-session deck, not this.