TY - JOUR
T1 - Replacing P-values with frequentist posterior probabilities of replication
T2 - When possible parameter values must have uniform marginal prior probabilities
AU - Llewelyn, Huw
N1 - Publisher Copyright:
© 2019 Huw Llewelyn. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
PY - 2019/2/27
Y1 - 2019/2/27
N2 - The prior probabilities of true outcomes for scientific replication have to be uniform by definition. This is because for replication, a study’s observations are regarded as samples taken from the set of possible outcomes of an ideally large continuation of that study. (The sampling is not done directly from some source population.) Therefore, each possible outcome is based on the same ideally large number of observations so that all possible outcomes for that study have the same prior probability. The calculation methods were demonstrated on a spreadsheet with simulated data on the distribution of people with an imaginary genetic marker. Binomial distributions are used to illustrate the concepts to avoid the effects of potentially misleading assumptions. Uniform prior probabilities allow a frequentist posterior probability distribution of a study result’s replication to be calculated conditional solely on the study’s observations. However, they can be combined with prior data or Bayesian prior distributions. If the probability distributions are symmetrical then the frequentist posterior probability of a true result that is equal to or more extreme than a null hypothesis will be the same as the one-sided P-value. This is an idealistic probability of replication within a specified range based on an assumption of perfect study method reproducibility. It can be used to estimate a realistic probability of replication by taking into account the probability of non-reproducible methods or subjects. A probability of replication will be lower if the subsequent outcome is a narrower range corresponding to a specified statistical significance, this being a more severe test. The frequentist posterior probability of replication may be easier than the P-value for non-statisticians to understand and to interpret
AB - The prior probabilities of true outcomes for scientific replication have to be uniform by definition. This is because for replication, a study’s observations are regarded as samples taken from the set of possible outcomes of an ideally large continuation of that study. (The sampling is not done directly from some source population.) Therefore, each possible outcome is based on the same ideally large number of observations so that all possible outcomes for that study have the same prior probability. The calculation methods were demonstrated on a spreadsheet with simulated data on the distribution of people with an imaginary genetic marker. Binomial distributions are used to illustrate the concepts to avoid the effects of potentially misleading assumptions. Uniform prior probabilities allow a frequentist posterior probability distribution of a study result’s replication to be calculated conditional solely on the study’s observations. However, they can be combined with prior data or Bayesian prior distributions. If the probability distributions are symmetrical then the frequentist posterior probability of a true result that is equal to or more extreme than a null hypothesis will be the same as the one-sided P-value. This is an idealistic probability of replication within a specified range based on an assumption of perfect study method reproducibility. It can be used to estimate a realistic probability of replication by taking into account the probability of non-reproducible methods or subjects. A probability of replication will be lower if the subsequent outcome is a narrower range corresponding to a specified statistical significance, this being a more severe test. The frequentist posterior probability of replication may be easier than the P-value for non-statisticians to understand and to interpret
KW - Bayes Theorem
KW - Binomial Distribution
KW - Clinical Trials as Topic/statistics & numerical data
KW - Data Interpretation, Statistical
KW - Humans
KW - Models, Statistical
KW - Probability
UR - http://www.scopus.com/inward/record.url?scp=85062168898&partnerID=8YFLogxK
U2 - 10.1371/journal.pone.0212302
DO - 10.1371/journal.pone.0212302
M3 - Article
C2 - 30811495
SN - 1932-6203
VL - 14
JO - PLoS ONE
JF - PLoS ONE
IS - 2
M1 - e0212302
ER -