Missing-value choices and sensitivity
Q7a–Q7b · 15 points
The lecture separates recovered observations from assumed replacements. The mean-fill demonstration and sensitivity table show different consequences of assumptions.
Week 3: missing-data responses and sensitivity, PDF pp. 23–39
Your task
Complete the four-response table. Compare mean, SD and observed count before/after filling, then interpret the target decision under alternative missing means.
Evidence to include
Quote two sensitivity rows with different decisions, the smallest/largest all-student means and the unchanged number of genuinely observed answers.
Notebook questions and code
Read here; run in your downloaded notebook. Start with its setup cells and run in order. Use your own ID and outputs where requested.
Download Lab 3 notebook4 · Four responses to missing values
Decide what each situation permits. Recovering a verified value is different from estimating it.
| Situation | Your choice: recover / keep missing / exclude for stated analysis / impute as a model assumption | Reason and what to report |
|---|---|---|
| A verified exam sheet for the exact student and exam contains the missing score | … | … |
| Programme is known; phone is blank; the question is programme counts | … | … |
| You explicitly report scores only for students with recorded scores | … | … |
| A separate demonstration fills blanks with the observed mean | … | … |
Q7a (5 points): complete this table. Explain why the fourth row does not create newly observed evidence. Keep your final registry unfilled.
Python · cell 19 — read here, run in your notebook
# Separate demonstration only: neither clean nor survey is overwritten.
example = pd.Series([2.0,4.0,np.nan,np.nan,np.nan])
filled_demo = example.fillna(example.mean())
display(pd.DataFrame({
"treatment":["Observed only", "Mean-filled demonstration"],
"mean":[example.mean(), filled_demo.mean()],
"sample_SD":[example.std(), filled_demo.std()],
"numeric_cells":[example.count(), filled_demo.count()],
"genuinely_observed_answers":[2,2]
}).round(3))
# Hypothetical values for missing responses: sensitivity, NOT a cleaned dataset.
observed_total = survey.rating.sum()
missing_n = int(survey.rating.isna().sum())
sensitivity = pd.DataFrame({"assumed_mean_of_missing":[1.,2.,3.,4.,5.]})
sensitivity["hypothetical_all_student_mean"] = (observed_total + missing_n*sensitivity.assumed_mean_of_missing)/len(survey)
sensitivity["meets_target_3_5"] = sensitivity.hypothetical_all_student_mean >= 3.5
display(sensitivity.round(3))
print("Target: all-student satisfaction mean at least 3.5")Your explanation
Q7b · What can we claim? (10 points).
- In the imputation demonstration, what changes and what stays the same? Why is “we now have five observed answers” false?
- Cite two rows from your sensitivity table that lead to different target decisions. Are these recovered answers, or assumptions?
- State the smallest and largest possible all-student mean on the 1–5 scale using the table. Is “the course definitely meets the target” defensible?
- Write a cautious alternative conclusion and suggest what information would strengthen it. No formula derivation is needed.
Write here: …
Saved only in this browser. This note doesn’t fill in the downloaded notebook; “reviewed” is a personal checklist, not a grade.