The claim ladder
These are different kinds of questions. A pattern in past data does not automatically tell us what will happen if we change something.
| Claim | Example | Evidence required and limit |
|---|---|---|
| Descriptive | What withdrawal rate was recorded last year? | Correct count, eligible denominator, relevant population |
| Comparative | Do recorded pass rates differ by programme? | Comparable groups, counts, units and uncertainty; no automatic causal claim |
| Predictive | Can early signals predict later withdrawal? | Information available at prediction time and honest held-out evaluation |
| Causal | Would providing support reduce withdrawal? | A suitable design and identification assumptions, such as a defensible randomised comparison |
A correlation is evidence of association. It can motivate a predictive model, but prediction performance still needs testing on unseen cases. It does not establish that forcing the input to change would change the outcome.
In W1, logins and mean grades are reported as correlated at about 0.68 for 1,200 students. Entry score is associated with both. Possible explanations include a causal effect, reverse direction and common causes such as preparation. Those correlations alone do not prove which explanation is correct or how much of the association each explains.
Confounder: in the causal question being considered, a pre-existing common cause can influence both the exposure and outcome. A variable merely correlated with both is a candidate to investigate, not automatic proof of a causal structure.
Sources: W1 PDF pp. 3, 13-17, 22-30, 32-53, 57-60; L1 Parts 1-5.
Changing groupby changes the comparison, not its causal strength. Course mix and preparation can still differ between programmes.
The lecture's claim ladder becomes a written answer: report the comparison, explain its limits and identify a useful next check.
The lecture's audit trail becomes a final report. Structural checks verify stated rules, while source accuracy and broader claims still need separate evidence.
Practice problems
Supplementary study questions on the lecture concepts — not verbatim exam questions. Each answer counts toward spaced review.
1. Conceptual understanding
Definitions and distinctions.
“Forcing more logins will improve grades” is what kind of claim?
Grades and logins correlate. What follows?
2. Application
Apply the concept to a case or calculation.
What evidence is needed to claim a rule predicts new students well?
Preparation affects both logins and grades. What role might it play?
3. Interpretation and limitations
Evaluate assumptions and justify conclusions.
“Programme A's recorded pass rate exceeds B's” is what claim?
Which claim requires stronger causal evidence?
Explain it in writing
Self-checked, not marked. Write a few sentences, then compare with the example answer. Your writing is saved as you type.
Show an example answer
Logins may help learning, or better preparation may increase both logins and grades. Reverse direction and other factors may also matter. The observed pattern does not choose among these explanations.
Check: did you answer the question, use evidence or working, and avoid claiming more than the evidence supports?
Show an example answer
A well-designed randomised support comparison could help, along with clear outcomes and follow-up. An observational comparison needs strong assumptions about selection and other causes.
Check: did you answer the question, use evidence or working, and avoid claiming more than the evidence supports?