TY - JOUR

T1 - Replacing P-values with frequentist posterior probabilities of replication

T2 - When possible parameter values must have uniform marginal prior probabilities

AU - Llewelyn, Huw

N1 - Publisher Copyright:
© 2019 Huw Llewelyn. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

PY - 2019/2/27

Y1 - 2019/2/27

N2 - The prior probabilities of true outcomes for scientific replication have to be uniform by definition. This is because for replication, a study’s observations are regarded as samples taken from the set of possible outcomes of an ideally large continuation of that study. (The sampling is not done directly from some source population.) Therefore, each possible outcome is based on the same ideally large number of observations so that all possible outcomes for that study have the same prior probability. The calculation methods were demonstrated on a spreadsheet with simulated data on the distribution of people with an imaginary genetic marker. Binomial distributions are used to illustrate the concepts to avoid the effects of potentially misleading assumptions. Uniform prior probabilities allow a frequentist posterior probability distribution of a study result’s replication to be calculated conditional solely on the study’s observations. However, they can be combined with prior data or Bayesian prior distributions. If the probability distributions are symmetrical then the frequentist posterior probability of a true result that is equal to or more extreme than a null hypothesis will be the same as the one-sided P-value. This is an idealistic probability of replication within a specified range based on an assumption of perfect study method reproducibility. It can be used to estimate a realistic probability of replication by taking into account the probability of non-reproducible methods or subjects. A probability of replication will be lower if the subsequent outcome is a narrower range corresponding to a specified statistical significance, this being a more severe test. The frequentist posterior probability of replication may be easier than the P-value for non-statisticians to understand and to interpret

AB - The prior probabilities of true outcomes for scientific replication have to be uniform by definition. This is because for replication, a study’s observations are regarded as samples taken from the set of possible outcomes of an ideally large continuation of that study. (The sampling is not done directly from some source population.) Therefore, each possible outcome is based on the same ideally large number of observations so that all possible outcomes for that study have the same prior probability. The calculation methods were demonstrated on a spreadsheet with simulated data on the distribution of people with an imaginary genetic marker. Binomial distributions are used to illustrate the concepts to avoid the effects of potentially misleading assumptions. Uniform prior probabilities allow a frequentist posterior probability distribution of a study result’s replication to be calculated conditional solely on the study’s observations. However, they can be combined with prior data or Bayesian prior distributions. If the probability distributions are symmetrical then the frequentist posterior probability of a true result that is equal to or more extreme than a null hypothesis will be the same as the one-sided P-value. This is an idealistic probability of replication within a specified range based on an assumption of perfect study method reproducibility. It can be used to estimate a realistic probability of replication by taking into account the probability of non-reproducible methods or subjects. A probability of replication will be lower if the subsequent outcome is a narrower range corresponding to a specified statistical significance, this being a more severe test. The frequentist posterior probability of replication may be easier than the P-value for non-statisticians to understand and to interpret

KW - Bayes Theorem

KW - Binomial Distribution

KW - Clinical Trials as Topic/statistics & numerical data

KW - Data Interpretation, Statistical

KW - Humans

KW - Models, Statistical

KW - Probability

UR - http://www.scopus.com/inward/record.url?scp=85062168898&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0212302

DO - 10.1371/journal.pone.0212302

M3 - Article

C2 - 30811495

SN - 1932-6203

VL - 14

JO - PLoS One

JF - PLoS One

IS - 2

M1 - e0212302

ER -