Inferring and Explaining

98 Sample Size (people) accuracy 100 ±10 percent 500 ±5 percent 2,000 ±2 percent 10,000 ±1 percent InferrIng and exPlaInIng You might note a couple of things about this little chart. One is how nicely the frst digit in the sample size correlates with the accuracy measurement, thus making it pretty darn easy to remember. Te other is what economists call “the law of diminishing returns.” Increasing the sample from one hundred to fve hundred buys you a lot of increased accuracy; increas- ing it from two thousand to ten thousand buys you hardly any increased accuracy. You will fnd, I predict, that almost all the polls you read about in the newspapers will have sample sizes around fve hundred. Tis is because an accu- racy of about ±5 percent is all that is needed for most purposes, and it would be very expensive and time consuming to improve that accuracy signifcantly. Couldn’t the Sample Be Biased? Te notion of bias in colloquial speech ofen con- veys a lack of openness or even prejudice, which counts as a kind of character defect—for exam- ple, “he’s really biased in his grading against stu- dent athletes.” I’m biased toward folk and rock music because it’s what I grew up with. Some of you, God forbid, are biased toward hip-hop for the same reason. All the notion really means is that people are not equally open—to giving good grades, appreciating a song as a good one, or noticing that the dishes need to be washed. We need to make sure that our samples are not biased but equally open to everyone or every- thing in the population. Statisticians desire randomly selected sam- ples. Tis is technical jargon that means every single individual in the population has an equal probability of being selected as a member of the sample. My computer can approximate random selection, so it would be relatively easy for me to feed in all my class rosters for the past fve years, randomly select three students from each course, and then query this sample to discover things about my teaching, grading, and so on. Not a bad idea, actually. In the real world, however, technical ran- domness is often impossible. We only have a couple of days to find out voter sentiment in the upcoming election, and so we phone a sample of six hundred likely voters. Obvi- ously, this is not a true random sample, since every likely voter did not have an equal chance of being selected—some didn’t have phones, some were away on vacation, and some screen their calls. But for practical purposes, if the phone numbers are randomly selected from a master list of likely voters who answer their phones, the information we gather approxi- mates what could be gathered from a techni- cally random sample, and our sample might be characterized as practically random . Techni- cally random samples are the exception, while what we hope are practically random samples are the rule. Consider a very famous poll that went spec- tacularly wrong. Te Literary Digest had been conducting polls on presidential elections since 1920 and had gotten the winner right in four straight elections; indeed, in the 1932 election, they got the popular vote right within 1 percent.