Despite misuses, statistics still has solid foundation
In many realms of science today, “statistical wisdom” seems to be in short supply. Misuse of statistics in scientific research has contributed substantially to the widespread “reproducibility crisis” afflicting many fields (SN: 4/2/16, p. 8; SN: 1/24/15, p. 20). Recently the American Statistical Association produced a list of principles warning against multiple misbeliefs about drawing conclusions from statistical tests. Statistician Stephen Stigler has now issued a reminder that there is some wisdom in the science of statistics. He identifes seven “pillars” that collectively provide a foundation for understanding the scope and depth of statistical reasoning.
Stigler’s pillars include methods for measuring or representing aggregation (measures, such as averages, that represent a collection of data); information (quantifying it and assessing how it changes); likelihood (coping with probabilities); intercomparison (involving measures of variation within datasets); regression (analyzing data to draw inferences); design (of experiments, emphasizing randomization); and residual (identifying the unexplained “leftovers” and comparing scientific models).
His approach is to identify the historical origins of these seven key pillars, providing some idea of what they are and how they can assist in making sense of numerical data. His explanations are engaging but not thorough (it’s not a textbook), and while mostly accessible, his writing often assumes a nontrivial level of mathematical knowledge. You’ll have to cope with expressions such as L(Θ)=L(Θ)|Χ and Cov(L,W)=E{Cov(L,W|S)}+Cov(E{L|S}, E{W|S}) every now and then.
While Stigler defends statistics from some of the criticisms against it — noting, for instance, that specific misuses should not be grounds for condemning the generic enterprise — he acknowledges that some issues are still a source of concern, especially in the new era of “big data” (SN: 2/7/15, p. 22). Using common statistical tests when many comparisons are made at once, or applying tests at multiple stages of an experimental process, introduces problems that the seven pillars do not accommodate. Stigler notes that there is room, therefore, for an eighth pillar. “The pillar may well exist,” he writes, “but no overall structure has yet attracted the general assent needed for recognition.”