Math

Evaluating Statistical Claims: Observational Studies and Experiments

2 min readEasy5-question drill

A few of these questions show up on every test, and they're often the fastest points in the whole Math section — no algebra required. You just need to know the rules about when a study can prove cause and when it can only show a connection.

Where this lesson fits

You are here

→Inference from Sample Statistics and Margin of Error →Two-Variable Data: Models and Scatterplots

These questions test whether you understand how data was collected and what conclusions that lets you draw. There's no calculation — just logic. Master a few definitions and you'll get them right in seconds.

Two ways to gather data:

An observational study watches subjects without changing anything. The researcher just records what's already happening (e.g., surveying people about how much they sleep and how stressed they feel).
An experiment actively imposes a treatment on subjects and compares groups. To do this well, researchers use random assignment — randomly deciding who gets the treatment and who's in the control group (the group that gets no treatment or a fake one).

Why this matters: Only a well-designed experiment with random assignment can show cause and effect. An observational study can only show an association (a relationship) — never causation. This is the single most-tested idea here.

Loading…

Were subjects randomly ASSIGNED to treatment groups?

Yes ↓

→ Causal conclusion allowed (experiment)

No ↓

→ Association only — no causation

Random assignment decides whether you can claim cause and effect.

Generalizing to a population: Whether you can apply results to a larger group depends on the sample — the people actually studied. To generalize, the sample must be chosen using random selection (every member of the population has an equal chance of being picked). A simple random sample does this and avoids bias — a systematic tendency to over- or under-represent part of the population.

Watch for biased sampling methods:

Convenience sample — surveying whoever's easy to reach (e.g., people at one mall). Not representative.
Voluntary response — only people who choose to respond (e.g., online poll). The strongly-opinionated answer.
Surveying one subgroup (one department, early arrivals) instead of the whole population.

Loading…

Sampling methods

Method	Representative?	Why
Simple random sample	Yes	Everyone has equal chance
Convenience sample	No	Only easy-to-reach people
Voluntary response	No	Only strong opinions reply
One subgroup only	No	Excludes the rest of population

Only random selection produces a representative, unbiased sample.

Put it together with two questions:

Were subjects randomly selected? → If yes, results generalize to the population. If no, they don't.
Were subjects randomly assigned to treatment groups? → If yes, you can claim a cause. If no, you can only claim an association.

A classic correct answer reads: "There is an association, but no causal conclusion can be drawn because subjects were not randomly assigned to groups." When you see a study where people chose their own behavior, causation is off the table.

Don't overreach. The test loves answer choices that claim a study "proves" something it can't. If random assignment is missing, reject any cause-and-effect language.

Loading…

Quick check

Check your understanding with a question from this topic:

A researcher wants to estimate the average commute time of employees at a company with 500 workers. Which sampling method would best avoid bias?

Survey the first 50 employees who arrive at work each morning

Assign each employee a number and use a random number generator to select 50 employees

Survey all employees in the marketing department

Survey employees who volunteer to participate

Worked examples

Example 1

A researcher wants to estimate the average number of hours per week that students at a university of 8,000 students spend exercising. Which sampling method would best allow the results to be generalized to all students at the university?

Example 2

Researchers studied 300 adults who volunteered to track their diets. They found that adults who drank more green tea had lower blood pressure on average. Which conclusion is best supported?

Example 3

In a study, 200 patients with chronic headaches were randomly assigned to either a new medication or a placebo. The group taking the medication reported significantly fewer headaches. The patients were all recruited from a single clinic and volunteered for the study. Which conclusion is most appropriate?

Common pitfalls

Confusing random selection with random assignment

Random selection (from the population) is what lets you generalize; random assignment (to groups) is what lets you claim cause. A study can have one without the other — read carefully which is present.

Treating correlation as causation

In an observational study, even a strong association never proves cause. Reject any answer with 'causes,' 'leads to,' or 'because of' unless there was random assignment to treatment groups.

Overgeneralizing from a biased sample

If subjects volunteered, came from one location, or self-selected, results don't apply to the whole population. Be suspicious of answers that say 'all people' when the sample was narrow.

Picking the boldest answer

The test rewards cautious, hedged wording ('there is an association' or 'may not generalize'). Sweeping claims that something is 'proven' for 'everyone' are usually the trap.

Key takeaways

Observational studies show association only; experiments with random assignment can show causation.
Random selection from the population → results generalize; without it, they don't.
Random assignment to groups → causal conclusions allowed; without it, only association.
Convenience samples and voluntary-response samples are biased and not representative.
Choose the cautious, properly-limited answer over any sweeping 'proves it for everyone' claim.

Watch & learn

Curated Khan Academy walkthroughs on Evaluating Statistical Claims: Observational Studies and Experiments. They're complementary to this lesson — watch one if a written explanation isn't clicking, or after to reinforce.

Tracks your progress across lessons.

Try it yourself

5 practice questions on Evaluating Statistical Claims: Observational Studies and Experiments, drawn from the question bank. The tutor is one click away if you get stuck.

Where this lesson fits

Evaluating Statistical Claims: Observational Studies and Experiments

You are here

→Inference from Sample Statistics and Margin of Error →Two-Variable Data: Models and Scatterplots