aNet.ai
Data Science/Learning path
0day streak0XP · Lv 1
← 3. Data Cleaning and Quality
Lesson 4.10 6 min · 138 words

Outliers, invalid values and the population

A value can be statistically unusual and still correct. A value inside normal-looking limits can still be a recording error.

An IQR flag means unusual relative to the observed distribution. The dictionary determines validity; source evidence determines accuracy. Age 220 is unresolved invalid data. A verified 47-year-old student is an unusual valid member of the cohort. Age 21 could still be a typo for 31.

Flag invalid age as missing while preserving the student's usable programme and score. Correct a value only with evidence. Retain valid influential values, choose robust summaries when appropriate and report sensitivity as a comparison, not as a replacement for the intended-population result.

In W3 p. 43, all 100 students include 10 withdrawals. Five verified older students include four withdrawals. Removing them leaves 6/95=6.32%, versus 10/100=10%. Nobody's outcome improved; numerator, denominator and represented population changed. Lead with 10% if the target is all enrolled students, and show subgroup rates with their small sample sizes.

Source: W3 PDF pp. 4-55. The 14-row registry example below comes from the Week 3 lab code you previously pasted in this conversation; it is separate from the lecture's smaller examples.

Practice problems

Supplementary study questions on the lecture concepts — not verbatim exam questions. Each answer counts toward spaced review.

1. Conceptual understanding

Definitions and distinctions.

Question 1 of 6 · +5 XP

How should a verified 47-year-old and age 220 be treated?

Question 2 of 6 · +5 XP

Does an age inside IQR review limits guarantee accuracy?

2. Application

Apply the concept to a case or calculation.

Question 3 of 6 · Invalid age · +5 XP

The dictionary allows ages 16–100, but a row says 220. What is supported without a verified replacement?

Question 4 of 6 · Verified unusual age · +5 XP

A source verifies age 52 for an enrolled student outside the IQR fences. What should happen?

3. Interpretation and limitations

Evaluate assumptions and justify conclusions.

Question 5 of 6 · Population change · +5 XP

Deleting valid older students lowers a withdrawal percentage. What changed directly?

Question 6 of 6 · Plausible error · +5 XP

Which mistake could pass a reasonable age range and an IQR check?

Explain it in writing

Self-checked, not marked. Write a few sentences, then compare with the example answer. Your writing is saved as you type.

Show an example answer

It is a plausible guess, not verified evidence. The intended age could be something else. Flag it and consult the source while keeping the raw value.

Check: did you answer the question, use evidence or working, and avoid claiming more than the evidence supports?

Show an example answer

The rate for all enrolled students. You can additionally show age-group rates with counts, but excluding verified older students changes the target population.

Check: did you answer the question, use evidence or working, and avoid claiming more than the evidence supports?

Practice this topic
Saved in this browser as you type.
aNet.ai · Data Science Fundamentals · Weeks 1–5 · practice material, not an official examProgress saved in this browser