1
$\begingroup$

The answer appears to be no because the distribution of X is defined conditionally by θ which is also assumed to have a distribution as opposed to be a constant. Essentially, the formulation of the probability distribution function of X is explicitly dependent on θ so how could X and θ be independent of each other.

However, I am confused by an equation in the proof of Bayes’ Theorem for random variables:

Theorem:

Suppose that the n random variables X1, . . . , Xn form a random sample from a distribution for which the p.d.f. or the p.f. is f (x | θ). Suppose also that the value of the parameter θ is unknown and the prior p.d.f. or p.f. of θ is ξ(θ). Then the posterior p.d.f. or p.f. of θ is

ξ(θ|x) =[f(x1|θ) . . . f(xn|θ)ξ(θ)]/gn(x)

for θ ∈ Ω,

where gn is the marginal joint p.d.f. or p.f. of X1, . . . , Xn.

In the proof of the theorem it is stated that the joint distribution of x (vector of observed X's) and θ--f(x,θ)--is equal to the product of the conditional joint distribution of x given θ--fn(x|θ)--and the marginal distribution of θ--ξ(θ).

f(x,θ)=fn(x|θ)ξ(θ)

I know that if X and θ are independent random variables, then the joint distribution of X and θ, f(x,θ)=g(x)ξ(θ). The above looks similar except for, of course, the p.d.f. of X is conditional upon θ. So, I wanted to make sure I was not missing the implication that X and θ are actually independent of each other.

Thanks in advance for your help. It seems straightforward, but I couldn't find an explicit answer on the website or in my readings...perhaps because it is so obvious.

$\endgroup$
1
$\begingroup$

Short answer is no - you are not missing anything. Although the condition of indepenence and the formula for the bayesian joint posterior look similar, they are very different, since $\theta$ is present in both equations in the Bayes formulation. If $\theta$ and x are truly indepenent, then fn(x|$\theta$)=fn(x), returning the indpendence condition. But that is not a given.

Qualitatively, you can think about it this way: One uses Bayes theorem to update one's knowledge of one random quantity given knowledge about another random quantity. If x were always independent of $\theta$ then Bayesian inference would be a non-starter.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.