Who Got Asked

A sample is a small group that stands in for a big group. It only works if everyone in the big group had a fair chance to be picked. If you ask only people who are easy to reach, or who volunteer, your answer leans their way. Drawing names at random gives everyone the same chance, and that is what keeps the sample honest — not how many people you ask.

A population is the entire group you want to describe; a sample is the part you actually measure. A sample is biased when the way it was selected systematically favors some members over others, so it leans away from the truth in a predictable direction. Bias is a flaw in the method, not bad luck — a larger biased sample is just as wrong. The dependable defense is random selection: giving every member a known, equal chance of being chosen. Randomness has no agenda, so it cannot systematically favor loud, nearby, or willing people the way convenience and volunteer samples do.

Bias is systematic error in a sampling method — a persistent lean that does not shrink as the sample grows. Its common sources have names: selection bias, when the frame excludes part of the population; voluntary-response bias, when subjects opt in and strong opinions dominate; convenience sampling, when the easy-to-reach stand in for the whole; and nonresponse bias, when those who decline differ from those who answer. Random sampling is the corrective because it makes selection independent of any trait a respondent holds. In a simple random sample, every group of the chosen size is equally likely — which both removes the systematic tilt and lets you measure the random error that remains.

You want to know the class's favorite fruit. You ask only your friends at the apple table, and they all say apples. But the whole class never got asked. To be fair, draw names from a hat instead.

Distinguish the target population, the sampling frame (the list actually drawn from), and the achieved sample; bias can enter at each gap. Probability sampling — simple random, stratified, cluster, systematic — gives every unit a known nonzero inclusion probability, which is what licenses design-unbiased estimation and honest margins of error; nonprobability samples (convenience, quota, voluntary response) forfeit this. Random selection controls selection bias but not measurement bias — leading wording, interviewer effects, and especially nonresponse can distort a perfectly drawn sample. The 1936 Literary Digest poll, wrong despite 2.4 million responses, is the standing lesson: size narrows random error but does nothing whatever to bias.

Under a probability design, the inclusion probabilities πᵢ define the Horvitz–Thompson estimator, whose unbiasedness and variance follow from the sampling scheme rather than any model of the population. Nonprobability samples forfeit this footing: without known πᵢ, unbiasedness and interval coverage can be recovered only through untestable modeling assumptions — quasi-randomization, calibration, propensity weighting. The stakes are current, since most large datasets are nonprobability samples: web panels, administrative traces, opt-in apps. Meng's identity shows estimation error factoring into a data-quality correlation, the problem's difficulty, and √((N − n)/n) — so at fixed data-defect correlation, error grows with population size. Big data can be more confidently wrong than a small random sample.