# Statistics: Math or Myth?

## Answering a Question  with the Wrong Statistics

by Kent Johnson

Simon and Cathay are chefs at two restaurants in town. They recently discovered the recipe for a delicate pastry that is made just before serving. If the pastry is not made correctly, it falls flat and cannot be served. Simon and Cathay decided to have a friendly competition to see who is more skilled at making these pastries. Last Tuesday and Wednesday, they both offered a special that contains the pastry; whoever had the largest percentage of pastries turn out well would win a bottle of single-malt scotch. On Tuesday night, Simon and Cathay met after work to compare their evening’s records; that night, 20% of Simon’s pastries were successful, but only 10% of Cathay’s were. They both thought they could do better, so they practiced during the night. On Wednesday night, 95% of Simon’s pastries turned out well, but only 80% of Cathay’s did. Upon seeing this, Simon victoriously grabbed the scotch and poured a drink. But before he could toast his victory, Cathay quietly showed him something she’d written on some paper. They studied the figures. Eventually, Simon let out a big laugh, graciously handed Cathay the scotch, and declared her to be the contest true winner! On average, Simon was better at pastry-making on Tuesday night, and he was again on Wednesday night. Yet Cathay was better on average on Tuesday and Wednesday nights taken together. How could this happen? If this strikes you as bizarre, you are not alone. However, cases such as this one, which are known as a “Simpson’s Paradox”, can occur quite easily. To see what happened, we must look at the absolute numbers that produced the percentages, not the percentages themselves. Since the contest concerned the best overall average, Cathay tallied up all her trials and all of her successes over both days. Thus, these two facts were what mattered most for their overall scores in the contest. When we consider a statistic, we are viewing a summary that will hopefully allow us to draw relevant and appropriate conclusions. But summaries always involve some loss of information. That is, a statistic provides an accurate report of one aspect of what may be a very complex situation. Asking a statistic to say something about some other aspects of the situation can be a risky business.