← Data Science learning path0%
05
Week 5 · 53 slides
Exploratory Data Analysis
Confounding, stratification, correlation and Simpson’s paradox.
Not started yet
Start Quiz mastery0%
What you’ll take away
- Question → check → explore → explain.
- Count tells how many; rate tells what share.
- Subtract for points; divide for relative change.
- Report what is known and what could still change.
- Match the plot to the measurement and question.
- Zero linear association can still hide a strong curve.
- Different weights can reverse the headline.
- Compare the mix before comparing the headline.
- A fairer comparison is not automatically a causal answer.
- Show the denominator and the unknowns.
Lecture topics
Structured exploration; Counts, rates and denominators; Percentage points versus relative percentage; Unknown outcomes produce an honest range; Match plots to variables; Correlation captures a linear pattern; Simpson's paradox: calculate all three views; The course-registration example; Why stratification does not prove causation; A compact EDA implementation pattern
Lessons
- Structured explorationExploratory data analysis, or EDA, is a way to learn what is in the data and check possible explanations. It is more than producing many charts.6 min
- Counts, rates and denominatorsA count says how many successes happened. A rate says what share of eligible cases succeeded. The denominator chooses the question.6 min
- Percentage points versus relative percentageBoth compare percentages, but they use different calculations. Always say which one you mean.6 min
- Unknown outcomes produce an honest rangeIf some outcomes are unknown, you can report the rate among known outcomes and a possible range for everyone. These are different statements.6 min
- Match plots to variablesThe chart should help answer the question and preserve the meaning of the variables. Always label units and counts.7 min
- Correlation captures a linear patternPearson correlation measures the direction and strength of a linear relationship. A value near zero does not rule out a curved relationship.6 min
- Simpson's paradox: calculate all three viewsA group can do better within every subgroup but worse overall if the groups contain different mixes of easier and harder cases.7 min
- The course-registration exampleThe lecture's part-time group has a higher pooled pass rate even though full-time leads in both broad course groups. Different course mixes help explain this.7 min
- Why stratification does not prove causationComparing people within the same measured group can address one composition difference. Other important differences can still remain.6 min
- A compact EDA implementation patternA short summary table should make the row count, known outcomes and missing outcomes visible. It should not disguise unknown rates as zero.6 min
Key terms
- EDA
- Checking and exploring data before making strong claims.
- Pattern
- A visible feature that still needs interpretation.
- Rate
- Successes divided by eligible cases.
- Weighted mean
- An average that uses the relevant group sizes.
- Percentage-point difference
- Subtract two percentages.
- Relative percentage change
- Divide the difference by the starting percentage.
- Known-outcome rate
- Successes among recorded outcomes.
- Bounds
- Extremes obtained by assigning all unknown outcomes one way or the other.
- Histogram
- Numeric values grouped into intervals.
- Bar chart
- Category counts or rates.
- Scatter plot
- Pairs of numeric values.
- Linear
- A straight-line pattern.
- Pearson r
- A unitless measure between −1 and 1 when both variables vary.
- Simpson's paradox
- Reversal between pooled and within-group comparisons.
- Weight
- The share of cases in a subgroup.
- Registration
- One student's course enrolment.
- Small cell
- A subgroup with few recorded cases.
- Stratify
- Compare within groups.
- Baseline
- Measured before an action.
- Post-intervention
- Measured after an action and possibly affected by it.
- size
- All group rows. count: nonmissing outcomes. sum: recorded successes for a 0/1 outcome.
Practice this topic60 questions · earn XP and build your streak