Thinking Like a Data Scientist

Lessons

1. 2.1Data science, AI, machine learning and deep learningData science means using data to answer a useful question. A good answer explains what the data show and what they cannot show.6 min
2. 2.2Frame a question with four componentsBefore calculating anything, say who you mean, what you will measure, when you will measure it, and why the answer matters.6 min
3. 2.3The claim ladderThese are different kinds of questions. A pattern in past data does not automatically tell us what will happen if we change something.7 min
4. 2.4Tables, rows, columns and keysOne row might be one student, one course registration, or one student in one week. Count rows only after you know what a row represents.7 min
5. 2.5Measurement meaning determines valid arithmeticA number can be a measurement, a label, or an ordered response. Its meaning decides which calculations make sense.7 min
6. 2.6Population, sampling frame and observed sampleThe people you want to describe may differ from the people you could contact and the people who answered.6 min
7. 2.7Operational definitions, proxies and ethicsWe often measure something easy to record as a clue about something harder to observe. Check how good that clue is and whether using it could harm people.6 min
8. 2.8Joins: predict the result before running itA join adds information from another table by matching keys. Extra or missing matches can change the number or meaning of the rows.7 min
9. 2.9Lab 1: what you should be able to explainThe lab joins student information to course registrations, then compares recorded pass rates. Each pass result still belongs to a registration.6 min
10. 2.10Reproducibility and provenanceSomeone should be able to understand where your data came from and run your analysis again from the beginning.6 min

Key terms

Data

Recorded information.

Model

A rule that finds patterns or makes predictions.

Population

The people or things you want to describe.

Variable

A recorded feature, such as score or logins.

Association

Two measurements vary together.

Causal effect

Changing one thing changes an outcome.

Confounder

A common cause that can affect both.

Row unit, or grain

What one row represents.

Key

The value or set of values that identifies one row.

Nominal

Categories with no order.

Ordinal

Ordered categories without guaranteed equal gaps.

Numeric

A measured or counted amount.

Target population

Everyone the conclusion concerns.

Sampling frame

The list used to reach people.

Sample

The people actually observed.

Proxy

A stand-in measurement.

Reliability

Consistent measurement.

Measurement validity

Whether it measures the intended idea.

Left join

Keep all left-side records and add matches.

Many-to-one

Many left records can match one right record.

Binary outcome

A value such as 1 for pass and 0 for fail.

Denominator

The count you divide by.

Provenance

Where data came from and what happened to it.

Data dictionary

Definitions and rules for each field.