If X and θ are both random variables and θ is the parameter of the distribution of X, are X and θ independent?

Question

The answer appears to be no because the distribution of X is defined conditionally by θ which is also assumed to have a distribution as opposed to be a constant. Essentially, the formulation of the probability distribution function of X is explicitly dependent on θ so how could X and θ be independent of each other.

However, I am confused by an equation in the proof of Bayes’ Theorem for random variables:

Theorem:

Suppose that the n random variables X1, . . . , Xn form a random sample from a distribution for which the p.d.f. or the p.f. is f (x | θ). Suppose also that the value of the parameter θ is unknown and the prior p.d.f. or p.f. of θ is ξ(θ). Then the posterior p.d.f. or p.f. of θ is

ξ(θ|x) =[f(x1|θ) . . . f(xn|θ)ξ(θ)]/gn(x)

for θ ∈ Ω,

where gn is the marginal joint p.d.f. or p.f. of X1, . . . , Xn.

In the proof of the theorem it is stated that the joint distribution of x (vector of observed X's) and θ--f(x,θ)--is equal to the product of the conditional joint distribution of x given θ--fn(x|θ)--and the marginal distribution of θ--ξ(θ).

f(x,θ)=fn(x|θ)ξ(θ)

I know that if X and θ are independent random variables, then the joint distribution of X and θ, f(x,θ)=g(x)ξ(θ). The above looks similar except for, of course, the p.d.f. of X is conditional upon θ. So, I wanted to make sure I was not missing the implication that X and θ are actually independent of each other.

Thanks in advance for your help. It seems straightforward, but I couldn't find an explicit answer on the website or in my readings...perhaps because it is so obvious.

Answer 1

Short answer is no - you are not missing anything. Although the condition of indepenence and the formula for the bayesian joint posterior look similar, they are very different, since $\theta$ is present in both equations in the Bayes formulation. If $\theta$ and x are truly indepenent, then fn(x|$\theta$)=fn(x), returning the indpendence condition. But that is not a given.

Qualitatively, you can think about it this way: One uses Bayes theorem to update one's knowledge of one random quantity given knowledge about another random quantity. If x were always independent of $\theta$ then Bayesian inference would be a non-starter.