Here’s yet another example of an article that confuses correlation with causation. The Associated Press released a story last week on a study that says that in locales with bans on smoking restaurants there are fewer teens who smoke. Most headlines said something to the effect of “Restaurant tobacco ban influences teen smoking.” [Emphasis added] Does it really? Not according to what I read in the article.
All the article shows is that two facts show up in many towns: smoking ban and fewer smoking teens. But nothing in the study, at least in what was printed, claimed that there was any evidence to show they were connected. That’s like saying that because my town has both a high church attendance rate and a high auto theft rate, that it proves that high church attendance causes more auto theft.
In fact, why couldn’t the reverse the be true? Couldn’t it be true that towns where there are fewer teens who smoke, which perhaps is evidence that parents and town leaders are doing a good job educating kids about the evils of smoking, are also towns more likely to pass smoking bans? Thus if no smoking ban was in place, the teens would still be getting vigilant oversight from authority.
After all, are kids hanging out in restaurants and bars so much that what happens in them influences their decision-making? I think not.
And in the end, this study doesn’t even tell us as much as it claims:
Siegel and his colleagues tracked 2,791 children between ages 12 and 17 who lived throughout Massachusetts. … Overall, about 9 percent became smokers — defined as smoking more than 100 cigarettes. In towns without bans or where smoking was restricted to a designated area, that rate was nearly 10 percent. But in places with tough bans prohibiting smoking in restaurants, just under 8 percent of the teens became smokers.
So the difference—based on a study of just 2,700 teens in Massachusetts—is just 2 percent. What’s the margin of error here?
If you wish to delve into the statistics, you can get the original study here. But in my layman’s reading of it, I don’t see any attempt to address why these two facts are connected other than wishing to construct ever more reasons to ban smoking.
The sample size is really not an issue. I found this counter-intuitive when I studied statistics as a math major 30 years ago, but there is rigorous, provable math behind sample-size technique and the sample sizes you need are surprisingly modest.
Once you get a couple of thousand samples from a large mass of data, you can draw information from it that is “statistically significant”, i.e., 95% likely to be valid for the whole population.