Session 4: Practice

Can we predict a discrete outcome honestly?

Thierry Warin, PhD

HEC Montréal

Outline · The workshop

  1. Ninety minutes
  2. Before you start
  3. Reproduce — 20 min
  4. Apply to your own angle — 35 min
  5. Break one assumption — 15 min
  6. Report it as one table
  7. Write it down — 20 min
  8. Submitting
  9. What loses marks
  10. Two-minute report
  11. After class, if you want it

What session 4 is for

The mathematics

  • Say why the linear probability model fails, and where it fails worst
  • Read a coefficient as a log-odds and its exponential as an odds ratio
  • Compute a fitted probability by hand from \(x^\top\hat\beta\)
  • Separate a Wald test from a likelihood-ratio test, and report McFadden’s pseudo-\(R^2\) for what it is

The Python

  • Fit binary, ordinal and multinomial logits with smf.logit, OrderedModel, sm.MNLogit
  • Turn coefficients into odds ratios and predicted probabilities
  • Build the likelihood-ratio test from two models’ .llf

The international business

  • Model a categorical business outcome — enter or not, which mode, which grade
  • Report an odds ratio to someone who will not read the appendix
  • State what the proportional-odds assumption costs when it fails

Your Python so far

Everything you need to type is on one sheet — how to start Python in VS Codium on your platform, how to make a file and run it, the DataFrame basics, and every name from sessions 1 to 4, each with a line showing it in use.

Python Cheatsheet · Session 04 (PDF) · in the repository at 04-…/PYTHON-CHEATSHEET.pdf

One sheet per package, functions explained with pictures: pandas · NumPy · matplotlib · statsmodels · SciPy

Print it, or keep it open beside the editor. Nothing on it is examinable; all of it is assumed.

Practice

Can we predict a discrete outcome honestly?

Ninety minutes

00–20 Reproduce one result from the paper you read
20–55 Apply the method, to your own angle
55–70 Break one assumption, deliberately
70–90 Write, and report 250 words, then two minutes to the room

The presenter is drawn at random when your group is called. All three of you prepare.

Before you start

Everyone pushes at least once. Replace XX with your group number — group 07 uses group-07.

git checkout -b group-XX
mkdir -p groups/A2026/group-XX/session-04      # macOS and Linux
git checkout -b group-XX
mkdir groups/A2026/group-XX/session-04         # Windows — no -p

PowerShell’s mkdir creates the intermediate folders already, and rejects -p. The git log is the participation record. It is not a formality.

1 · Reproduce — 20 min

Take the result the paper rests on, and reproduce it. The package you downloaded is at 04-logistic-ordinal-multinomial/data/replication/.

import qmib

data = qmib.load("loans")     # the session's own data

Paper: Saganowski et al. (2019), Analysis of group evolution prediction in complex networks · 10.7910/DVN/ONOFS7

A failed reproduction that you diagnose earns full marks. One you do not attempt earns none.

“It did not work” is not a diagnosis. Where did it stop, and what did you check?

2 · Apply to your own angle — 35 min

core = qmib.load("core")
mine = qmib.load("angle_c_country")     # your angle — see your dictionary
df   = core.merge(mine, on=["geo", "time"], how="inner")

Fit the session’s method on your data, and report what the lecture said to report — in the units of your own variables.

A coefficient without units is not a finding.

3 · Break one assumption — 15 min

Every method in this course rests on something.

Find the assumption this one leans on hardest. Break it deliberately. Record what happened to your answer.

This is the part that carries the marks. A group that shows how its result falls apart understands it; a group that only shows it holding does not yet know.

Report it as one table

Step 3 left you with two fits: the model as specified, and the model after you broke something. Put them in one table rather than two blocks of output — the comparison is the finding.

import qmib

print(qmib.regtable([before, after],
                    names=["as specified", "assumption broken"]))

Read across the row, not down the column: what happened to the coefficient you care about, in its own units, and what happened to its standard error.

Coefficients too small to read at one decimal — a logit’s usually are? digits=3. .as_latex() if you would rather paste it into the write-up than screenshot it. Pass stats= to change the rows at the foot.

4 · Write it down — 20 min

250 words in your submissions folder:

  • What you found, in units
  • Which assumption you broke, and what it did
  • What this result does not license you to claim

And the standing rule: one LLM output you identified as wrong, and how you established that.

Submitting

git add groups/A2026/group-XX/session-04
git commit -m "Session 04 practice — group XX"
git push -u origin group-XX

Deliverable. A logistic model of a binary outcome in your own project data, with the odds ratios interpreted in words, and a note on what the model does not license you to say.

What loses marks

  • Reporting log-odds as though they were probabilities
  • Interpreting an odds ratio as a relative risk
  • Reporting accuracy on an imbalanced outcome with no base rate

And everywhere in this course: an association described in causal language.

Two-minute report

One number with its units and its uncertainty
One assumption the one you broke, and what happened
One limit what you cannot claim from this

Listen for traceable evidence, not confidence of delivery.

After class, if you want it

The QMIB Lab has a knowledge check on what you have just done: warin.ca/qmib-labs

Optional, never a prerequisite for the next lecture, and never marked for correctness. A completed report counts because attempting it is the engagement being measured — it is one of the three routes to the participation mark.

What is required before the next session: the article, its Dataverse package, and the self-check. That is the pre-session deck, not this.