PyStatlas
Gallery/Machine learning

Machine learning

How well does a model predict on new data, and why? Out-of-fold evaluation, learning curves and model-agnostic explanations — each with its uncertainty.

Confusion matrix with metrics
Machine learning

Confusion matrix with metrics

How well a logistic regression predicts who voted Dole in 1996 — out-of-fold confusion matrix with counts and row shares, and bootstrap intervals for the headline metrics.

yardstick::conf_mat() + autoplot()
Precision–recall curves
Machine learning

Precision–recall curves

Out-of-fold precision–recall curves for two classifiers predicting which married women report an affair, with average precision and the no-skill baseline.

yardstick::pr_curve() + autoplot()
Learning curve
Machine learning

Learning curve

Training and cross-validated accuracy as the training set grows, for two models predicting which married women report an affair.

caret::learning_curve_dat() + ggplot()
Cross-validated model comparison
Machine learning

Cross-validated model comparison

Fold-by-fold R² of six models predicting California school districts' maths results, with means and corrected 95% intervals.

caret::resamples() + bwplot() / tune::autoplot()
Permutation importance
Machine learning

Permutation importance

Which district characteristics a random forest relies on to predict maths results — drop in test-set R² when each one is shuffled.

vip::vip(method = "permute")
Partial dependence with ICE curves
Machine learning

Partial dependence with ICE curves

How predicted maths results change with poverty and with spending per pupil — the average (PDP) over one curve per school district (ICE).

pdp::partial(ice = TRUE) + autoplot() / iml::FeatureEffect