Session 11: Practice

What is your counterfactual, and would anyone believe it?

Thierry Warin, PhD

HEC Montréal

Outline · The workshop

  1. Ninety minutes
  2. Before you start
  3. Reproduce — 20 min
  4. Apply to your own angle — 35 min
  5. Break one assumption — 15 min
  6. Report it as one table
  7. Write it down — 20 min
  8. Submitting
  9. What loses marks
  10. Two-minute report
  11. After class, if you want it

What session 11 is for

The mathematics

  • Set up a difference-in-differences design and identify the four cells
  • Read the interaction coefficient as the estimate
  • State the parallel-trends assumption and how you would probe it
  • Say what an instrument must satisfy, and why good ones are rare

The Python

  • Build the four cells, then get the same number from an interaction in smf.ols
  • Plot an event study and let the pre-period speak for the design
  • Handle staggered adoption without letting the cohorts blur

The international business

  • Evaluate a policy that arrived in different countries at different times
  • Defend parallel trends as an argument, not as a test that passed
  • Say plainly what the design cannot rescue

Your Python so far

Everything you need to type is on one sheet — how to start Python in VS Codium on your platform, how to make a file and run it, the DataFrame basics, and every name from sessions 1 to 11, each with a line showing it in use.

Python Cheatsheet · Session 11 (PDF) · in the repository at 11-…/PYTHON-CHEATSHEET.pdf

One sheet per package, functions explained with pictures: pandas · NumPy · matplotlib · statsmodels

Print it, or keep it open beside the editor. Nothing on it is examinable; all of it is assumed.

Practice

What is your counterfactual, and would anyone believe it?

Ninety minutes

00–20 Reproduce one result from the paper you read
20–55 Apply the method, to your own angle
55–70 Break one assumption, deliberately
70–90 Write, and report 250 words, then two minutes to the room

The presenter is drawn at random when your group is called. All three of you prepare.

Before you start

Everyone pushes at least once. Replace XX with your group number — group 07 uses group-07.

git checkout -b group-XX
mkdir -p groups/A2026/group-XX/session-11      # macOS and Linux
git checkout -b group-XX
mkdir groups/A2026/group-XX/session-11         # Windows — no -p

PowerShell’s mkdir creates the intermediate folders already, and rejects -p. The git log is the participation record. It is not a formality.

1 · Reproduce — 20 min

Take the result the paper rests on, and reproduce it. The package you downloaded is at 11-causal-inference-did/data/replication/.

import qmib

data = qmib.load("core")     # the session's own data

Paper: Cavallo, Gopinath, Neiman & Tang (2021), Tariff Passthrough at the Border and at the Store: Evidence from US Trade Policy, American Economic Review: Insights 3(1) · 10.7910/DVN/JV7FCH

A failed reproduction that you diagnose earns full marks. One you do not attempt earns none.

“It did not work” is not a diagnosis. Where did it stop, and what did you check?

2 · Apply to your own angle — 35 min

core = qmib.load("core")
mine = qmib.load("angle_c_country")     # your angle — see your dictionary
df   = core.merge(mine, on=["geo", "time"], how="inner")

Fit the session’s method on your data, and report what the lecture said to report — in the units of your own variables.

A coefficient without units is not a finding.

3 · Break one assumption — 15 min

Every method in this course rests on something.

Find the assumption this one leans on hardest. Break it deliberately. Record what happened to your answer.

This is the part that carries the marks. A group that shows how its result falls apart understands it; a group that only shows it holding does not yet know.

Report it as one table

Step 3 left you with two fits: the model as specified, and the model after you broke something. Put them in one table rather than two blocks of output — the comparison is the finding.

import qmib

print(qmib.regtable([before, after],
                    names=["as specified", "assumption broken"]))

Read across the row, not down the column: what happened to the coefficient you care about, in its own units, and what happened to its standard error.

Coefficients too small to read at one decimal — a logit’s usually are? digits=3. .as_latex() if you would rather paste it into the write-up than screenshot it. Pass stats= to change the rows at the foot.

4 · Write it down — 20 min

250 words in your submissions folder:

  • What you found, in units
  • Which assumption you broke, and what it did
  • What this result does not license you to claim

And the standing rule: one LLM output you identified as wrong, and how you established that.

Submitting

git add groups/A2026/group-XX/session-11
git commit -m "Session 11 practice — group XX"
git push -u origin group-XX

Deliverable. A difference-in-differences estimate on your angle, with the parallel-trends evidence shown rather than asserted, and a note on the threat you consider most serious.

What loses marks

  • A DiD with no evidence on parallel trends
  • Reading the post-treatment dummy as the effect
  • An instrument justified only by its first stage
  • Claiming a causal effect the design cannot deliver

And everywhere in this course: an association described in causal language.

Two-minute report

One number with its units and its uncertainty
One assumption the one you broke, and what happened
One limit what you cannot claim from this

Listen for traceable evidence, not confidence of delivery.

After class, if you want it

The QMIB Lab has a knowledge check on what you have just done: warin.ca/qmib-labs

Optional, never a prerequisite for the next lecture, and never marked for correctness. A completed report counts because attempting it is the engagement being measured — it is one of the three routes to the participation mark.

What is required before the next session: the article, its Dataverse package, and the self-check. That is the pre-session deck, not this.