Who answered the survey?
Q5–Q6 · 20 points
The lecture connects nonresponse with representation. The survey separates group invitation shares from reply shares and asks what MCAR/MAR/MNAR mean.
Week 3: missingness and response mechanisms, PDF pp. 23–39
Your task
Predict representation, calculate group missing fractions, report the observed mean and match the three hypothetical mechanisms.
Evidence to include
Use your actual reply counts and denominators. For MAR given group, explain independence from satisfaction within each group.
Notebook questions and code
Read here; run in your downloaded notebook. Start with its setup cells and run in order. Use your own ID and outputs where requested.
Download Lab 3 notebook3 · Missingness changes who we see
This is a separate fictional survey, not the entry-score registry. Everyone in the roster was invited. Satisfaction ranges from 1 to 5. Group is known for everyone. No answer means unknown, not zero.
Before running: if Evening students answer less often, will the respondent mix necessarily match the invited mix? Write a prediction in Q5 first.
Python · cell 14 — read here, run in your notebook
survey_rng = np.random.default_rng(seed + 1)
n_day, n_evening = 20, 10
reply_day = int(survey_rng.integers(15,19))
reply_evening = int(survey_rng.integers(3,6))
survey = pd.DataFrame({"group": ["Day"]*n_day+["Evening"]*n_evening,
"rating": [4.0]*reply_day+[np.nan]*(n_day-reply_day)+[2.0]*reply_evening+[np.nan]*(n_evening-reply_evening)})
summary = survey.groupby("group").rating.agg(invited="size", replies="count", observed_mean="mean")
summary["missing"] = summary.invited-summary.replies
summary["response_percent"] = (100*summary.replies/summary.invited).round(1)
summary["share_of_invited_percent"] = (100*summary.invited/len(survey)).round(1)
summary["share_of_replies_percent"] = (100*summary.replies/summary.replies.sum()).round(1)
display(summary)
print("Pooled observed mean:", round(survey.rating.mean(),3))
summary.response_percent.plot.bar(color=["#238b8d", "#dd9050"], ylim=(0,100), ylabel="Responses / invited (%)", rot=0)
plt.title("Who answered in your version?")
plt.show()Your explanation
Q5 · Denominators and representation (10 points).
- Prediction before running: …
- Report the missing fraction for each group using counts and denominators.
- Compare Evening's share of invitations with its share of replies. Who becomes less represented after dropping missing ratings?
- Write one accurate sentence reporting the pooled observed mean and valid n. Explain why it does not establish the all-student mean.
- Explain the difference between 0 hours of lab use, unknown hours, and not enrolled in the lab module. Which students belong in the target population when summarizing hours for enrolled students?
Write here: …
Three possible stories for the same survey
Story A: after everyone answers, a technical process independently loses each answer with equal probability, unrelated to group or rating.
Story B: Evening students have less time to answer. Within each group, reply probability does not depend on satisfaction.
Story C: within the Evening group, dissatisfied students are less likely to answer than satisfied students, even after accounting for known group.
These are hypothetical mechanisms; the observed response counts do not tell us which story actually occurred.
Your explanation
Q6 · Explain mechanisms, not just labels (10 points).
- Match A, B, C to MCAR, MAR, MNAR and justify each in an ordinary sentence.
- Does the group difference in your table prove MAR? State exactly what must be true within each group for Story B.
- Propose one practical follow-up that could help investigate nonresponse. Explain what would remain uncertain.
- If both groups had identical response rates, would this guarantee an unbiased comparison? Give a counterexample.
Write here: …
Saved only in this browser. This note doesn’t fill in the downloaded notebook; “reviewed” is a personal checklist, not a grade.