Lesson 3 of 4 9 min · 150 words
Interpretation
Feature importance, partial dependence and explaining predictions.
Interpretation answers *why* a model makes its predictions — essential for trust, debugging, fairness and regulation.
- Intrinsically interpretable models: linear models (weights), small trees (rules).
- Permutation importance: shuffle one feature's values and measure how much the score drops — model-agnostic and simple.
- Partial dependence plots: how the average prediction changes as one feature varies.
- SHAP values: split each individual prediction into contributions from each feature, based on game theory.
- LIME: fit a simple local model around one prediction to explain it.
- For images: saliency maps and Grad-CAM highlight the pixels that drove the decision.
Interpretation often uncovers problems: a pneumonia model that relied on the hospital's watermark in the X-ray, or a loan model that used a proxy for a protected attribute.
Quick check · +5 XP