Where do the Bayesian priors come from

The Bayesian Approach - what is it?

This is an alternative to the so-called frequentist approach that continues to dominate statistics, as I said. The Bayesian formula plays a central role here.

The Bayesian approach is often interpreted in such a way that the unknown parameter of interest is assumed not to be fixed, but itself to be random. So it has a probability distribution. In the discrete case, this means that every possible expression of this parameter has a probability. The American statistician Sander Greenland protests against the interpretation of the parameter as random and instead emphasizes the other, "subjective" concept of probability [4]: ​​The probability of a certain parameter value corresponds to the degree to which someone (for example the study director or the statistician) believes in this value - like when someone says "Tomorrow it will probably rain (90%)".

The Bayesian approach consists of four steps:

1. First, as in the conventional approach, a statistical model is set up for the distribution of the data that contains one or more parameters.

2. A probability is now assigned to all possible parameter characteristics (in the example all possible therapeutic effects D). This probability distribution is the so-called "prior", the distribution of the probabilities before knowledge of the data (a priori). In the formula, this corresponds to a definition of P (D) for all D. Often the probability is distributed as evenly as possible (“non-informative prior”).

3. Now a study is carried out. The result is data x, the probability of which P (x | D) is known for each possible parameter value on the basis of the model. Furthermore, one can calculate P (x) by summing (general integration) of P (x | D) ⋅P (D) over all D.

4. Using the Bayesian formula, one finally obtains the so-called “posterior” P (D | x), that is, the probability distribution for the possible parameter characteristics after observing the data (a posteriori). So you have corrected your prior a priori knowledge P (D) by observing the empirical data x.

It would go too far to explain here why this method, which is considered flexible but also requires a lot of calculation, is controversial, as the author of the article "Acute Respiratory Distress Syndrome: Treatment Success with Glucocorticoids in ARDS Not Convincing" in this booklet of drug therapy (see pages 426 to 427 [3]) correctly. In his very readable article [4], Sander Greenland accuses the “frequentists” of mistakenly believing that the Bayesian method involves putting questionable assumptions into the analysis with the prior. He stressed that the prior must be chosen carefully and that it is often possible and desirable to use empirical data for this purpose. The assumption on which the frequentist approach is based that every experiment can theoretically be repeated as often as desired under exactly the same circumstances, on the other hand, appears unrealistic to the Bayesian, especially in connection with observational studies.

Although this article sparked a debate [5], the excitement with which “Bayesians” and “frequentists” initially fought each other bitterly has long since given way to a matter-of-fact, sober discussion. It has been recognized that both approaches exist side by side, complement one another and can mutually stimulate one another. In the Anglo-Saxon-speaking area in particular, statisticians often compare both approaches and routinely use the method appropriate to the respective problem for applications. This also applies to the authors of the BMJ article [6], who refer to the work of one of the protagonists of Bayesian analysis (and inventor of the Bayesian software WinBUGS), David Spiegelhalter [7].

The difference in the “philosophy” of both approaches is reflected in the formation of the term. While the frequentist estimates an effect that is assumed to be fixed, the Bayesian states its posterior, for example in the form of “Bayesian outcome probabilities” P (D | x). In the BMJ article, for example, probabilities P [odds ratio ≥ 1] [%] are tabulated. What is meant here is the probability that the respective therapy will lead to increased mortality, for example. There is no such thing with the frequentist approach: the classic statistician carries out a test that shows that the probability is increased, or not. But be careful: the probability with which this result is correct cannot be stated for reasons of principle; it can only be estimated [8]. The p-value doesn't help here either. It indicates the probability with which the observed result is to be expected under the null hypothesis (here: the therapies do not differ in their effectiveness / tolerability). Under no circumstances should p-values ​​be compared directly with “Bayesian outcome probabilities”. If there is a large difference in effectiveness, we expect a small p-value, but a large Bayesian “probability of effectiveness”. Other statements correspond to a certain extent: for example, the confidence interval of the frequentist is offset by the credible interval (credible parameter range) of the Bayesian, and so on. For many situations there is a corresponding frequentist method for a special Bayesian method and vice versa.

Such a comparison allows the results of both methods to be numerically compared in application examples. However, one should never lose sight of the different philosophy of the two approaches when interpreting them.


1. Trampisch HJ and Windeler J, Medical Statistics, Springer-Verlag, 2nd edition, 2000.

2. Weiss C. Basic knowledge of medical statistics. Springer-Verlag, 3rd edition, 2005.

3. Neubeck M. Acute Respiratory Distress Syndrome. Treatment success with glucocorticoids in ARDS not convincing. Drug Therapy 2008; 26: 426-7.

4. Greenland S. Bayesian perspectives for epidemiological research: I. Foundations and basic methods. Intern J of Epidemiol 2006; 35: 765-75.

5. Carpenter JR. Commentary: on Bayesian perspectives for epidemiological research. Intern J of Epidemiol 2006; 35: 775-7; author 777-8.

6. Peter JV, John P, Graham PL, Moran JL, George IA, Bersten A. Corticosteroids in the prevention and treatment of acute respiratory distress syndrome (ARDS) in adults: meta-analysis. BMJ 2008; 336: 1006-9.

7. Warn DE, Thompson SG, Spiegelhalter DJ. Bayesian random effects meta-analysis of trials with binary outcomes: methods for the absolute risk difference and relative risk scales. Stat Med 2002; 21: 1601-23.

8. Ioannidis JP. Why most published research findings are false. PLoS Med 2005; 2: e124, epub (08/30/2005).

Dipl.-Math. Gerta Rücker, Institute for Medical Biometry and Statistics (IMBI), Freiburg University Medical Center, Stefan-Meier-Str. 26, 79104 Freiburg, email: [email protected]