# Reply to Held: When is a harmonic mean *p*-value a Bayes factor?

^{a}Big Data Institute, Nuffield Department of Population Health, Li Ka Shing Centre for Health Information and Discovery, University of Oxford, Oxford OX3 7LF, United Kingdom

See allHide authors and affiliations

### This article has a Letter. Please see:

- Relationship between Letter and Reply - March 19, 2019

I welcome this opportunity (1) to acknowledge Good’s papers (2⇓⇓⇓–6), which I had missed. Good proposed the harmonic mean p-value (HMP) as a classical analog to a model-averaged Bayes factor (BF) which “should be regarded as an approximate tail-area probability [p-value]” (2). His presentation was amusingly apologetic; for example, “an approximate rule of thumb is tentatively proposed in the hope of provoking discussion” and “this rule of thumb should not be used if the statistician can think of anything better to do” (2). I believe my paper dispels these misgivings by formalizing Good’s intuitive argument that the HMP is approximately well-calibrated when small (equation 5 of ref. 7) and deriving an asymptotically exact test for general use (equation 4 of ref. 7). Further, I showed the HMP is a multilevel test procedure (equation 6 of ref. 7), demonstrating with examples that it consequently provides a powerful alternative to Bonferroni and Benjamini–Hochberg (8) correction for large-scale multiple testing problems.

Held (1) considers the Bayesian properties of the HMP, which are relevant to its interpretation and power. Applications of the HMP are not limited to model selection problems, however, as it provides a general alternative to Fisher’s method (9) for combining tests that are not independent (2, 7).

Good claimed that the HMP is inversely proportional to a model-averaged BF based on his empirical observations that

Good’s claim depends on the density of p-values under the alternative,

Held (1) considers whether the class of alternatives **2**), in which

## Acknowledgments

D.J.W. is a Sir Henry Dale Fellow jointly funded by the Wellcome Trust and Royal Society Grant 101237/Z/13/Z and is supported by a Big Data Institute Robertson Fellowship.

## Footnotes

- ↵
^{1}Email: daniel.wilson{at}bdi.ox.ac.uk.

Author contributions: D.J.W. designed research, performed research, analyzed data, and wrote the paper.

The author declares no conflict of interest.

Data deposition: The R code used to generate Fig. 1 has been deposited on figshare (https://doi.org/10.6084/m9.figshare.7699955).

Published under the PNAS license.

## References

- ↵
- Held L

- ↵
- ↵
- Good IJ

- ↵
- Good IJ

- ↵
- Good IJ

- ↵
- Good IJ

- ↵
- Wilson DJ

*p*-value for combining dependent tests. Proc Natl Acad Sci USA 116:1195–1200. - ↵
- ↵
- Fisher RA

- ↵
- Mikosch T

- ↵

*p*-value a Bayes factor?

## Citation Manager Formats

## Article Classifications

- Physical Sciences
- Statistics

- Biological Sciences
- Genetics

### See related content:

- The harmonic mean
*p*-value for combining dependent tests- Jan 04, 2019