Pipelines preserve the fitted procedure
A pipeline keeps preprocessing and prediction steps together so the same fitted rules are used on later records.
Safe order: define target and cutoff; separate training/test; fit preprocessing on training; transform each set with the saved rules; fit the prediction rule on training inputs and labels; evaluate the fixed procedure on held-out data. Use validation for choosing settings. Keep the final test for the final assessment rather than repeatedly repairing the model after seeing its score.
Numerical columns may take imputation then scaling. Categorical columns take encoding. Dates may produce elapsed-time features. IDs serve matching and tracking. A ColumnTransformer is a software mechanism for sending different column types through different routes; a Pipeline stores an ordered procedure.
If a training numeric column has missing values, learn its replacement from observed training values, fill training inputs, then learn the scaler on those completed training columns. Repeat the fitted steps without relearning on test/future rows. In cross-validation, the same principle applies inside each training fold; this is a useful extension of the boundary rule, not a new licence to use the validation fold for fitting.
Sources: W4 PDF pp. 2-42; L4 Tasks 1-7. All fixed worked values here were recalculated from the supplied examples. These toy rules are teaching devices, not deployable student-support recommendations.
The training rows supply every replacement and scaling value. Filling a gap lets the model run, but does not recover the true measurement.
The code should carry out the exact stated rules, including missing-value handling, feature order and tie-breaking. Explain these decisions rather than memorising lines.
Practice problems
Supplementary study questions on the lecture concepts — not verbatim exam questions. Each answer counts toward spaced review.
1. Conceptual understanding
Definitions and distinctions.
Which sequence protects evaluation?
Where should a missing-value replacement be learned?
2. Application
Apply the concept to a case or calculation.
What happens first in the safe workflow?
Where should the missing-value replacement be learned?
3. Interpretation and limitations
Evaluate assumptions and justify conclusions.
Which set should help choose model settings?
Why might numeric and category columns follow different pipeline steps?
Explain it in writing
Self-checked, not marked. Write a few sentences, then compare with the example answer. Your writing is saved as you type.
Show an example answer
The scaler is defined on the completed training input column. Learn the replacement from observed training values, fill training gaps, then compute the scaling parameters from that completed column.
Check: did you answer the question, use evidence or working, and avoid claiming more than the evidence supports?
Show an example answer
Validation helps choose or tune the procedure. The final test estimates performance after those choices are fixed. Repeatedly changing the model based on final-test results weakens that independence.
Check: did you answer the question, use evidence or working, and avoid claiming more than the evidence supports?