# Bartlett's paradox in Bayesian evidence

I've come across Bartlett's "paradox" (not to be confused with Lindley's paradox, also known as the Lindley-Bartlett paradox) in Bayesian statistics. The paradox originates from Bartlett's 1957 paper, A Comment on D. V. Lindley's Statistical Paradox .

Bartlett's paradox is quite trivial. Suppose that with a Bayes-factor, one compares two models, $a$ and $b$, each with adjustable parameters. Model $b$ has an adjustable parameter $p$ that a priori could take on broad range of values $N$, for which a uniform distribution is an appropriate prior (suppose that it's a location parameter).

The Bayes-factor is $$\frac{p(D\mid M_a)}{p(D\mid M_b)} \propto N$$ If a priori we know very little about the parameter $p$, it might be that we want to consider an improper prior $N\to\infty$ or at least very large $N$. In this case, the we ought to favor model $a$, almost regardless of the data (though here we must take care with the limit - if each $p(D\mid M)$ is normalized, $p(D\mid M_b) \ge p(D\mid M_a)$ for some $D$).

What are we to make of this? Is it paradox? How should we deal with this situation in which we have insufficient prior information to constrain a model parameter but want to make a model comparison? Maybe nothing has gone wrong, and we really should favour model $a$ in this situation?

So suppose $P(D|M_a)$ indeed gives the probability or probability distribution when the null hypothesis is true. When sample size $n$ is very large, you can imagine the standard error to be very small (proportional to $1/\sqrt{n}$). In this case, it's very likely that the frequentist approach will reject $M_a$. However, in the Bayesian approach, we favor $M_a$ due to the Bayes factor getting large.
One way to fix this is to also impose the precision (or the parameter that gives the effective sample size) some prior distribution, so that the joint prior distribution will have heavier tails, say, the Normal prior for $\mu \sim \mathcal{N}(\nu, 1/n_0)$ will become Cauchy, after imposing $n_0\sim \text{Gamma}$.