Session 8: Before the session

Flexible, or just unstable?

Thierry Warin, PhD

HEC Montréal

Outline · Before class

  1. What to do, and how long it takes
  2. Read the paper
  3. Get BOTH datasets
  4. The course data
  5. Self-check, and what the session will ask of you
  6. Before you run anything

What session 8 is for

The mathematics

  • Explain how KNN classifies, by the vote of the \(K\) nearest neighbours
  • Find nearest neighbours by hand from a Euclidean distance
  • Decompose test error into squared bias, variance and irreducible error
  • Say why cross-validation estimates out-of-sample error, and what classical in-sample fit cannot tell you

The Python

  • Standardise, split, and choose \(k\) by cross-validation
  • Measure a classifier with AUC rather than accuracy alone
  • Fit the simple alternative too, and let it win if it does

The international business

  • Classify firms or countries when you have no theory of the boundary
  • Say why a flexible method that scores 0.51 has told you something
  • Justify complexity against a simpler model that was actually fitted

Before the session

Flexible, or just unstable?

What to do, and how long it takes

1 Read the paper 45–60 min
2 Get both datasets — the paper’s, and your own angle 10 min
3 Attempt the self-check 15 min

The lecture assumes all three are done. The practice assumes you have already read both datasets once on your own machine.

1 · Read the paper

Bluwstein, Buckmann, Joseph, Kapadia & Şimşek (2023), Credit growth, the yield curve and financial crisis prediction: evidence from a machine learning approach, Journal of International Economics 145

Read the article

Read for the argument, not for coverage. Annotate four things:

1 The research question, in one sentence
2 The method or design, and why they chose it
3 The strongest single piece of evidence
4 One limitation you would raise as a referee

2 · Get BOTH datasets

The practice uses two: the paper’s data, to reproduce one of its results, and your own angle, to apply the method. Read both once before you arrive.

a) The paper’s data — the Jordà–Schularick–Taylor Macrohistory Database, on the course site

18 advanced economies, 1870–2020, 88 banking crises, with the paper’s predictors already derived (credit growth, the yield-curve slope, …). Copy this into a file in a folder of your own, and run it:

import pandas as pd

SITE = "https://warint.github.io/quantitative-methods/data/"

# One row per country-year. crisis_next2 = 1 if a banking crisis starts within two years.
jst = pd.read_csv(SITE + "session-08/jst_crises.csv")
print(jst.shape)                              # (2718, 12)
print(jst["crisis_next2"].value_counts())     # how many country-years have a crisis ahead

Jordà, Schularick & Taylor (2017), Macrofinancial History and the New Business Cycle Facts, NBER Macroeconomics Annual 2016 · shared under CC BY-NC-SA 4.0, as the publisher’s licence requires.

2b · The course data

import pandas as pd

SITE = "https://warint.github.io/quantitative-methods/data/"

core = pd.read_csv(SITE + "core.csv")              # shared by every group
mine = pd.read_csv(SITE + "angle_c_country.csv")   # REPLACE with your own angle

print(core.shape, mine.shape)

angle_c_country is an example, not your angle. Yours is named in RESEARCH-MANDATES.md.

Both blocks printed a shape? Then download the practice script, which does the rest: session08_practice.py — save it in the same folder. You will run it in class; you may read it now.

3 · Self-check, and what the session will ask of you

Answer these on paper. If you cannot, that is what the lecture is for.

  1. In one sentence: what does this session’s method let you claim that the previous one did not?
  2. What must be true of your data for it to apply?
  3. Which claim in the paper rests on this method — and how hard does it lean on it?

By the end of the session you should be able to:

  • Explain how KNN classifies, by the vote of the \(K\) nearest neighbours
  • Compute a Euclidean distance and find nearest neighbours by hand
  • Explain why KNN requires standardised predictors
  • Choose \(k\) by cross-validation, and read the trade-off the curve shows
  • Say why cross-validation estimates out-of-sample error, and what classical in-sample fit cannot tell you
  • Say what happens to KNN as the number of predictors grows

Arrive with: the paper read and annotated · both datasets read once on your machine · session08_practice.py in your folder · your self-check answers on paper.

Before you run anything

Session 08 runs in a folder of your own — Desktop/session-08, say — not in the course repository. Open it in VS Codium (File → Open Folder…), then, once, in its terminal:

open /Applications/Python\ 3*/Install\ Certificates.command   # python.org Python only, once
python3 -m venv .venv
source .venv/bin/activate
pip install pandas numpy matplotlib scikit-learn
python3 -m venv .venv
source .venv/bin/activate
pip install pandas numpy matplotlib scikit-learn
py -m venv .venv
.venv\Scripts\Activate.ps1
pip install pandas numpy matplotlib scikit-learn

Your prompt begins with (.venv), and in VS Codium this folder’s .venv is the interpreter (Ctrl/Cmd + Shift + P → Python: Select Interpreter). Every new terminal needs the activate line again.

CERTIFICATE_VERIFY_FAILED on a Mac: run the first macOS line. ModuleNotFoundError: the prompt has no (.venv) — activate, then pip install again.

Session 08: README.md · the lecture: 01-lecture/