置换重要性
随机森林预测数学成绩时依赖哪些学区特征——逐个打乱每项特征时测试集 R² 的下降。
置换重要性提出一个简单且与模型无关的问题：打乱某个特征、切断它与结果的联系后，模型在 留出 数据上的预测会变差多少？下降越大，说明模型越依赖该特征。打乱是随机的，因此每个特征重复 50 次；箱体显示离散程度，圆点为均值。箱体跨越零的特征对该模型没有可用的信号。
与树模型内置的基于不纯度的重要性不同，置换重要性在测试数据上计算，可在不同类型的模型之间比较，且不会偏向取值较多的特征。它衡量的是 模型 用到了什么，而非因果效应；相关的特征会分摊（甚至相互掩盖）各自的重要性。数据为 statsmodels 自带的 1998 年 STAR 考试中加利福尼亚州 303 个学区（sm.datasets.star98）；结果变量为数学成绩高于全国中位数的学生占比，由学区的十项特征预测（贫困、族裔构成、教师、支出、班级规模、大学预备课程）。模型为随机森林；重要性来自 sklearn.inspection.permutation_importance。这里低收入学生占比占据主导，一旦知道它，教师薪酬、支出和班级规模就不再提供任何信息——这是关于模型预测的陈述，并不证明资源无关紧要。
代码及图中文字均为英文。
import matplotlib.pyplot as plt
import numpy as np
import statsmodels.api as sm
from sklearn.ensemble import RandomForestRegressor
from sklearn.inspection import permutation_importance
from sklearn.model_selection import train_test_split
# Data: 303 California school districts; outcome = % of students above the national median in maths
d = sm.datasets.star98.load_pandas().data
y = 100 * d["NABOVE"] / (d["NABOVE"] + d["NBELOW"])
names = {"LOWINC": "% low-income students", "PERASIAN": "% Asian students", "PERBLACK": "% Black students",
"PERHISP": "% Hispanic students", "AVYRSEXP": "Teacher experience (years)",
"AVSALK": "Teacher salary ($000)", "PERSPENK": "Spending per pupil ($000)",
"PTRATIO": "Pupils per teacher", "PCTAF": "% in college-prep courses",
"PCTYRRND": "% year-round schools"}
X = d[list(names)]
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0)
model = RandomForestRegressor(n_estimators=500, min_samples_leaf=3, random_state=0).fit(X_tr, y_tr)
r2 = model.score(X_te, y_te)
imp = permutation_importance(model, X_te, y_te, n_repeats=50, random_state=0, scoring="r2")
order = np.argsort(imp.importances_mean)
fig, ax = plt.subplots(figsize=(8, 5.2))
for i, j in enumerate(order):
v = imp.importances[j]
c = "#007AFF" if np.percentile(v, 2.5) > 0 else "#8E8E93"
ax.boxplot(v, positions=[i], widths=0.5, vert=False, showfliers=False, patch_artist=True,
boxprops=dict(facecolor=c, alpha=0.25, edgecolor=c), medianprops=dict(color=c, lw=1.5),
whiskerprops=dict(color=c), capprops=dict(visible=False))
ax.scatter(v.mean(), i, s=40, color=c, zorder=3, edgecolor="white", lw=1)
ax.axvline(0, color="#3A3A3C", lw=0.8)
ax.set_yticks(range(len(order)), [names[X.columns[j]] for j in order])
ax.set_xlabel("Drop in test-set R² when the feature is shuffled")
ax.grid(axis="x"); ax.grid(axis="y", visible=False)
drop = imp.importances_mean[list(X.columns).index("LOWINC")]
ax.set_title(f"Poverty dominates: shuffling it costs {drop:.2f} of R², school resources add nothing", pad=26)
ax.text(0, 1.02, f"Random forest · test R² = {r2:.2f} · n = {len(X_te)} held-out districts · "
"50 shuffles per feature · grey = interval includes 0", transform=ax.transAxes,
fontsize=9.5, color="#6E6E73")
plt.show()
图表使用了可选的 PyStatlas 样式：添加 plt.style.use("pystatlas.mplstyle") 即可完全还原其外观。
数据来源
- California STAR test results, 1998 (303 school districts). Gill, J. (2001). Generalized Linear Models: A Unified Approach (Quantitative Applications in the Social Sciences, No. 134). Sage. https://www.statsmodels.org/stable/datasets/generated/star98.html
Original data from the California Department of Education's Standardized Testing and Reporting (STAR) program, 1998. Subset distributed with statsmodels (sm.datasets.star98) with the author's permission.
这些示例用于说明统计方法。图表标题描述的是该图在这些特定数据中呈现的内容——这些数据往往样本较小、年代较早或并不完整——而不是作者对实质问题的立场。关于解释、背景和数据质量，请以此处引用的原始来源为准。
如何引用
PyStatlas 由 Thierry Warin 创建并维护。如果它对你的工作有所帮助，敬请引用。
Warin, T. (2026). Permutation importance. In PyStatlas: A Python Gallery of Statistical Graphics for the Social Sciences. https://warint.github.io/pystatlas/machine-learning/permutation-importance/
@misc{warin2026statlas,
author = {Warin, Thierry},
title = {PyStatlas: A Python Gallery of Statistical Graphics for the Social Sciences},
year = {2026},
howpublished = {\url{https://warint.github.io/pystatlas/}},
note = {Source code: https://github.com/warint/pystatlas}
}
