What we could check, and what no course can check for you
Why this matters in private organisations
Why this matters in public organisations
What the course did not cover
Six habits to carry
Europe 2031, one more time
What session 12 is for
The mathematics
Say which of the term’s methods your question actually required
State a result with its uncertainty, and name the kind of uncertainty
Name the assumption that identifies your estimate, and whether it is testable
The Python
Hand over work that runs on someone else’s machine
Show the specification alongside the coefficient
Make the figure that carries the argument without the code
The international business
Present to a decision-maker who will not read your code
State a recommendation, its confidence, and its falsifier, in twelve minutes
Defend an identification choice under questioning
Your Python so far
Everything you need to type is on one sheet — how to start Python in VS Codium on your platform, how to make a file and run it, the DataFrame basics, and every name from sessions 1 to 12, each with a line showing it in use.
Print it, or keep it open beside the editor. Nothing on it is examinable; all of it is assumed.
1 · The question we started with
Twelve weeks later, it is still the question
Session 1 asked one thing
When can a number about the past be used to make a claim about the future?
Every method since has been an answer to some version of it, and every answer came with conditions attached.
The methods will be looked up again. The conditions will not. That is why the conditions are the thing worth carrying out of this room.
The sentence the course has been making
A quantitative result is an argument, and the arithmetic is only its smallest part.
What makes a number worth acting on is the case that can be made for the conditions under which it means what it appears to mean.
That case is constructed by the analyst. It is not computed by the software.
2 · Five things that kept happening
Not five techniques — five recurring facts
One · The arithmetic never refuses
The normal equations return coefficients for any numbers you feed them
Logistic regression on separated data runs its coefficients to infinity — and reports a large number, not an error
Nearest neighbours predicts confidently outside the range of its training data
A structural equation model fits the covariance matrix implied by an entirely wrong causal diagram
A misspecified model does not raise an exception. It returns four decimal places in the same typeface as a correct one.
Which is why diagnostics are a discipline
Not a formality, and not a box to tick after the interesting work is done.
“The model says” is never a complete sentence.
Two · The same data answer differently
Session 6, on one dataset and one pair of variables:
Specification
Estimate
Pooled
1,111
Country effects
275
Country and year effects
1,149
Each was correct for the question its identifying variation could answer. A reader shown any one alone would draw a different conclusion.
And it was not an isolated case
Session 2 — the slope fell 26% when two controls were added, because “controlling for” is literally a residual regression
Session 7 — one soil sample out of 7,232 moved an interaction from p = 0.72 to p = 0.04, depending only on whether it was kept
Session 10 — the estimate ran from +6.07 to +2.68, depending on whether selection was addressed
A coefficient is meaningless without the specification that produced it. Reporting one without the other is reporting half a result.
Three · Fit does not identify structure
Session 7 — the feature a paper ranked first for predicting a group’s fate had no effect at all in a logit on the same data (p = 0.91): a ranking is not a structure
Session 9 — two structural models with opposite causal arrows returned identical \(\chi^2\), identical degrees of freedom, identical CFI, identical AIC
Session 3 — four datasets shared every summary statistic and differed completely
The data were silent between the alternatives
The choice was made by the analyst, on grounds outside the data.
That is not a defect to be engineered away. It is the permanent condition of empirical work.
The honest response is to name the alternative you rejected, and say why — which is a different activity from reporting a fit index.
Four · Flexibility is a purchase, not a virtue
Session 5 introduced a deliberately biased estimator and watched it beat the unbiased one, because unbiasedness is worth nothing if the variance is large enough.
Session 8 made the trade explicit and then measured it: squared bias, variance and irreducible error summing to test error at every \(k\), with a minimum somewhere in the middle.
Then session 8 sent itself the bill
On real data, nearest neighbours scored an AUC of 0.51 — a coin flip — where logistic regression, the rigid method with the strong assumption, reached 0.68.
On accuracy alone, the comparison would have shown all methods identical and concluded that nothing worked.
Complexity has to be justified against a simpler alternative that was actually fitted. Not against an imagined one.
Five · Selection is the problem
In session 10 the naive comparison overstated a known effect of +2.4 percentage points by more than 150% — because the countries that adopted the policy were already different.
Nothing in the fit statistics of that naive comparison would have suggested a problem.
That gap is the reason for matching, balance diagnostics, parallel trends and event studies — none of which removes the need for an argument about why treatment arrived where it did.
3 · What we could check
And what no course can check for you
Some things are not opinions
Because the course data are fixtures generated from a known structure, several claims could be verified rather than asserted.
identity discrepancy
TSS - (ESS + RSS) -3.052e-05
multiple regression - FWL slope 4.840e-09
sum of leverages - p 2.265e-14
These are algebraic identities. They hold for every regression, on every dataset, forever.
The failures were more instructive
The lasso retained three pure-noise columns out of ten after cross-validated selection
Regression adjustment on proxy covariates left 0.7 percentage points of bias
A measurement model taken from an exploratory factor solution failed every fit index
A post-treatment control moved an estimate closer to the truth while being methodologically wrong
That last one is the course’s most useful result
Being closer to the right answer is not the same as using the right method.
Outside a textbook, you never learn which one you had.
A standing caveat
The course data in data/spine/ are teaching fixtures, generated from a known structure with a fixed seed. They are not observations of Europe.
Every number quoted in these twelve weeks is a fact about a simulation, not a fact about the world. Do not cite one in a paper. The reasoning transfers; the numbers do not.
4 · Why this matters in private organisations
Where the estimate meets a decision with money on it
The firm’s questions are this course’s questions
The decision
The session it belongs to
Should we enter this market, and what happens if we do?
10, 11 — effects, not correlations
Which of forty indicators actually predicts default?
5, 8 — selection and the bias–variance trade
Did the pricing change work, or did the market move?
11 — difference-in-differences
What is “brand strength”, and can we measure it?
7, 9 — latent constructs
Which countries behave alike for planning purposes?
6, 7 — structure across a panel
What a manager is actually buying from you
Not a model. A defensible answer, and an honest account of its limits.
A point estimate with no interval cannot be planned against
A coefficient with no units cannot be checked against what the business already knows
A result with no named assumption cannot be revisited when conditions change
The most expensive analytical failures in business are not arithmetic errors. They are confident answers to questions the data could not address.
The asymmetry that should govern your caution
A model that is wrong and uncertain gets challenged
A model that is wrong and confident gets implemented
Your job includes making sure the second case is not you.
5 · Why this matters in public organisations
Where the stakes are different in kind, not degree
Public decisions raise the bar
A firm that misprices a product loses money and adjusts. A ministry that misreads an evaluation sets a policy that affects people who did not choose it, and often cannot adjust for years.
The counterfactual is rarely available — programmes are not randomly assigned
The people affected are not the people who chose
The decision is public, so the reasoning must survive being read by opponents
Which makes session 10 and 11 the centre of gravity
Did the subsidy raise employment, or did healthier regions take up the subsidy?
Did the regulation change behaviour, or was behaviour changing already?
Would the trend have been parallel? Say how you would probe it, not that you assumed it.
In public work, “we could not identify this effect from these data” is not a failure to deliver. It is often the single most valuable thing you can report.
Both sectors, one standard
Whether the reader is a CFO or a director general, the same three questions decide whether your number survives contact with them:
What exactly is being claimed, in units, with an interval?
What has to be true for that claim to hold?
What would change your mind, and would you notice if it happened?
6 · What this course did not cover
Knowing the shape of the gap
Five absences worth naming
Time series proper — autocorrelation, unit roots, cointegration, forecasting. Our panels are short and wide; session 6 only gestures at it
Bayesian inference — everything here is frequentist, and the Bayesian framing is often the more natural one for a decision problem
Machine learning at scale — gradient boosting, random forests, neural networks. Session 8’s framework is their foundation; the methods are a course of their own
Text, networks, unstructured data — where much of the growth in this field now is
Causal machine learning — double ML and causal forests sit exactly at the junction of sessions 5, 8 and 10, and are the natural next thing to read
7 · What to carry
Six habits that outlast every technique here
The six
Look at the data before modelling it. Nearly every analysis that goes wrong went wrong before the model was fitted.
Say the units. Checking against what you already know is the fastest way to catch an implausible result.
Report the uncertainty, and say what kind. Classical, robust, clustered — they differed by more than a factor of two on one coefficient in session 6.
Name the assumption that identifies the estimate, and say whether it is testable. Most of the important ones are not — which is exactly why they must be argued.
Fit the simpler thing too. Complexity should have to beat something.
Say what the result does not license.
And the one that contains them
A well-argued refusal is a complete result.
“This cannot be claimed from these data, and here is what would be needed.”
That is often the most valuable output of a serious analysis — and the answer that takes the most expertise to produce.
8 · Europe 2031, one more time
The course opened with it. It closes with it.
In session 1 you had no tools
You read Europe 2031 and were asked to separate four kinds of statement that the text moves between without announcing it.
Evidence
A claim tied to a source and an observed period
Scenario
A coherent imagined future — not a forecast by default
Prediction
A dated, testable statement about an outcome
AI
A computational system producing outputs from inputs and learned parameters
Read it again this week. You will find seams you could not see in August.
It is an optimistic document
And that is a feature of it, not a flaw — a scenario is meant to make a future feel thinkable.
But the optimism is carried by the assumptions, not by the evidence. Several of them are inflated, and a diagnosis built on inflated assumptions is sometimes simply wrong.
Take the compute assumption
The scenario assigns the United States roughly 12 times Europe’s AI compute stock in 2031, and treats compute as the dominant measure of geopolitical capability.
Where does the ratio come from — measured, or assumed?
Is compute the right measure of capability, or the available one?
What observation would count as evidence against it?
You now have the training to ask that in the form of a testable statement. In August you did not. That is the distance you have travelled.
Actual AI is not the AI of the discourse
The buzz
The thing itself
A system that “understands”
A function fitted to data, producing outputs from learned parameters
Reasoning, in the human sense
Very good conditional prediction over patterns it has seen
Improvement without limit
Improvement bounded by data, compute, and an irreducible error term
Ground truth
Confident output, whether or not the question was answerable
Session 8’s bias–variance decomposition applies to every one of these systems. The irreducible error does not go away because the model got larger.
Which is the same lesson as week one
The model on your laptop does not refuse either. It answers fluently when it should not, exactly as the normal equations do.
The skill this course built — knowing which question a given number can answer, and saying so when it cannot — is precisely the skill that the arrival of these systems has made scarcer and more valuable, not less.
Where the work goes now
Language models have made the arithmetic and the code effectively free. They have not made the judgement free, and there is no sign that they will.
That is the argument for learning to derive a method before using it. Not because the derivation will be needed at the keyboard — but because it is what lets you notice that the answer on the screen cannot be right.
Thank you
You arrived able to run a regression. You leave able to say what one is entitled to claim.
That is a much rarer thing, and it is the whole point.
Europe 2031 · europe2031.ai — read it again, and mark the seams.
Now, the last five groups
That was the last teaching of the course. What follows is not an epilogue to it — it is the argument being made by you instead of me.
Same three questions, from the floor:
What exactly is being claimed, in units, with an interval?
What has to be true for that claim to hold?
What would change your mind, and would you notice if it happened?