There is a debate among some scientists, philosophers of science and statisticians about which of frequentist statistics and Bayesian statistics is correct. Here is a simple case for Bayesian statistics.
1. Everyone agrees that Bayes theorem is true
Bayes theorem is stated mathematically with the following theorem:
As far as I know, everyone accepts that Bayes’ theorem is true. It is a theorem with a mathematical proof.
2. The probability of the hypothesis given the data is what we should care about
When we are developing credences in a hypothesis, H, what we should ultimately care about is the probability of H given the data, D, that we actually have. This is what is on the left hand side of the equation above. Here ends the defence of Bayesian statistics; no further argument is needed. Either you deny a mathematical proof or you deny that we should form beliefs on the basis of the evidence you have. Neither is acceptable, so Bayesian statistics is correct. This argument is straightforward and there should no longer be any debate about it.
Appendix. Contrast to frequentism
(Note again that the argument for Bayesian statistics is over, this is just to make the contrast to frequentism clear.) In contrast to Bayesianism, frequentist statistics using p-values asks us:
Assuming that the hypothesis H is false, what is the probability of obtaining a result equal to or more extreme that the one you in fact observed?
What you actually care about is how likely the hypothesis is, given the data, rather than the above question. So, you should not form beliefs on the basis of p-values. Whether a Bayesian prior is ‘subjective’ or not, it is necessary to form rational beliefs given the evidence we have.