The Most Difference in Means: A Statistic for Null and Near-Zero Results

Abstract

Two-sample p-values test for statistical significance. Yet p-values cannot determine if a result has a negligible (near-zero) effect size, nor compare evidence for negligibility among independent studies. We propose the most difference in means ({\delta}M) statistic to assess the practical insignificance of results by measuring the evidence for a negligible effect size. Both {\delta}M and the relative form of {\delta}M allow hypothesis testing for negligibility and outperform other candidate statistics in identifying results with stronger evidence of negligible effect. We compile results from broadly related experiments and use the relative {\delta}M to compare practical insignificance across different measurement methods and experiment models. Reporting the relative {\delta}M builds consensus for negligible effect size by making near-zero results more quantitative and publishable.

Next
Previous