Sampling Methods and Errors | Introduction to Statistics

Opening

You're running a phone survey to find out how many people in your city exercise weekly. You have a phone book with 20,000 listings but only time to call 400. Who do you call? Pick the wrong 400 and your results could be wildly off — so how do you choose, and what can go wrong even when you choose carefully?

The idea

How you pick a sample and how you collect its data determine whether your results reflect the whole population or quietly mislead you.

Choosing a sample

Recall the phone survey example: you number 20,000 listings and take every 50th name after a random start. That is systematic sampling, a simple method. But not all sampling is random — convenience sampling uses results that are readily available, like interviewing customers who happen to be in a store. It can be very good in some cases and highly biased in others.

Definition

convenience sampling

A non-random sampling method that uses results that are readily available, such as interviewing customers who happen to be in a store.

You grab whoever is easy to reach, not a carefully chosen cross-section of the population.

Convenience sampling can be very good in some cases and highly biased (favor certain outcomes) in others. For example, a computer software store might interview potential customers who happen to be browsing in the store — easy, but those shoppers aren't necessarily like all software buyers.

Vocabulary· 2 terms

In most populations, simple random sampling without replacement is done, because the sample is usually small compared to the population, so the chance of picking the same individual twice with replacement is very low. The difference only becomes a mathematical issue when the population is small.

Vocabulary· 2 terms

When you analyze data, it's important to be aware of both kinds of errors. In reality, a sample will never be exactly representative of the population, so there will always be some sampling error. As a rule, the larger the sample, the smaller the sampling error.

Definition

sampling bias

A bias created when a sample is collected from a population and some members of the population are not as likely to be chosen as others, even though each member should have an equally likely chance of being chosen.

When your sampling method makes some people more likely to be picked than others, your sample is skewed.

When sampling bias happens, incorrect conclusions can be drawn about the population being studied. Sampling data should be done very carefully — collecting data carelessly can have devastating results. For example, surveys mailed to households and then returned may be very biased (they may favor a certain group). It's better for the person conducting the survey to select the sample respondents.

Definition

self-selected samples

Responses only by people who choose to respond, such as call-in surveys, which are often unreliable.

When only the people who feel strongly bother to respond, you don't hear from everyone.

Critical evaluation

When evaluating statistical studies critically, common problems to watch for include: samples that aren't representative (biased samples give results that are inaccurate and not valid), self-selected samples, sample size issues (too small may be unreliable; larger is better, though small samples are sometimes unavoidable, as in crash testing cars or medical testing for rare conditions), undue influence (collecting data or asking questions in a way that influences the response), and non-response or refusal to participate (the collected responses may no longer be representative, since people with strong opinions often answer).

Sampling errors

Recap

Say each point to yourself, then tap it to check.