# Are $X_1$ and $X_2$ independent? Is there a more efficient way to solve this?

I am currently doing an exercise:

Suppose that $$X_1$$ and $$X_2$$ are two random variables, where each can take one of three states. Here is a probability table $$P$$ describing their joint distribution, $$P_{ij}=p(X_1=i,X_2=j)$$: $$\begin{pmatrix} 0.06 & 0.37 & 0.12 \\ 0.01 & 0.07 & 0.02 \\ 0.03 & 0.26 & 0.06 \end{pmatrix}$$ Are $$X_1$$ and $$X_2$$ independent? Support your answer and if they are not independent, what is the minimal number of entries in the table that you need to change to make them independent?

So first I'll find the marginals $$P(X_1 = i)$$ and $$P(X_2 = j)$$ for $$i,j \in \{1,2,3\}$$: $$P(X_1 = 1) = 0.55, P(X_1 = 2) = 0.1, P(X_1 = 3) = 0.35 \\ P(X_2=1)= 0.1, P(X_2 = 2) = 0.7, P(X_3 = 3) = 0.2$$

So, now I check whether each entry is the product of the respective marginals: $$P(X_1 = 1)P(X_2 = 1) = 0.55*0.1 = 0.055 \approx 0.6 = P_{11} \\ P(X_1 = 1)P(X_2 = 2) = 0.55*0.7 = 0.385 \approx 0.39 \neq 0.37 = P_{12} \\ P(X_1 = 1)P(X_2 = 3) = 0.55*0.02 = 0.011 \neq 0.12 = P_{13} \\ P(X_1 = 2)P(X_2 = 1) = 0.1*0.1 = 0.01 = P_{21} \\ P(X_1 = 2)P(X_2= 2) = 0.1*0.7 = 0.07 = P_{22} \\ P(X_1 = 2)P(X_2 = 3) = 0.1*0.2 = 0.02 = P_{23} \\ P(X_1 = 3)P(X_2 = 1) = 0.35*0.1 = 0.035 \approx 0.04 \neq 0.03 = P_{31} \\ P(X_1 = 3)P(X_2 = 2) = 0.35*0.7 = 0.245 \approx 0.25 \neq 0.26 = P_{32} \\ P(X_1 = 3)P(X_2 = 3) = 0.35*0.2 = 0.07 \neq 0.06 = P_{33}$$

Since 5 entries don't satisfy $$P(X_1= i,X_2 = j) = P(X_1 = i)P( X_2 = j)$$, the variables are not independent. In order for them to become independent, 5 entries at minimal should be changed to make them independent.

Is my answer correct? Is there a more efficient way to solve this?

• I see several mistakes in computation starting with $P(X_1=1)$. Jul 25, 2022 at 11:43
• @geetha290krm edited matrix ($P_{12}$) and $P(X_2 = 1)$
– user
Jul 25, 2022 at 11:47
• Not sure if there is a nicer way to solve it. But there is definitely a nicer way to write down the calculations. Namely as vector product of $(0.55,0.1,0.35)^T$ and $(0.1,0.7,0.2)$. Jul 25, 2022 at 12:00

If $$X_1$$ and $$X_2$$ are independent, then the matrix would be equal to $$\begin{pmatrix} p(x_1=1) \\ p(x_1=2) \\ p(x_1=3) \end{pmatrix} \cdot\begin{pmatrix} p(x_2=1) & p(x_2=2) & p(x_3=3)\end{pmatrix}$$

which must have rank $$1$$, but your given matrix has rank $$2$$. For example, you can clearly see that the second row is not a scalar multiple of the first row, hence the rank is at least $$2$$.

This is a problem that is set up to have "easy" numbers, so that if students notice some things they can do it fairly simply, but the method used here won't really work with real world data. With that said, ....

If $$X_1$$ and $$X_2$$ were independent, then the columns would be multiples of each other. We notice (hopefully) that the first and third columns are in fact multiples of each other, and they are in the same proportion as their marginals are, i.e. $$0.1 :0.2$$, or $$1:2$$, (as independence would require).

Next, noticing (hopefully) that the marginals for the first and second column are in ratio $$1:7$$, we see that we can make the columns multiples of each other by making the second column equal to 7 times the first, i.e. $$(0.42, 0.7,0.21)$$, which incidentally leaves the middle entry unchanged.

So we establish that, as given, they are not independent by just using the values in the upper-left 2x2 block, and noting that $$0.37:0.06$$ is not the same ratio as $$0.07:0.01$$. And we see that we can make them independent by changing just 2 values, $$0.37\rightarrow 0.42$$ and $$0.26\rightarrow 0.21$$. Notice that you can't change just one value in the matrix (the values are constrained to add up to $$1.0$$), so changing two values is the minimal number of entries to change, and we acheived that.

I do think that your approach of computing the marginals and using them to refill the matrix is a good one. I'd guess that that approach ends up changing the values less overall, which seems like a better standard than changing the fewest number of entries. Notice that the solution I presented does on fact change the marginals.

(I found this question a bit irritating. It rests on noticing very special numbers, and like I said, doesn't use a general method. Though that is just my personal opinion.)

• So would it be correct to say that 2 entries at minimum should be changed or 5 entries?
– user
Jul 25, 2022 at 12:40
• Yep, we can make them independent by changing just 2 entries, the top and bottom in the middle column. Jul 25, 2022 at 12:43

All you need to do is to work out whether the rows and columns of that matrix are linearly dependent. For the variables to be independent, you're looking for the probabilities to be multiples of the form

$$\begin{pmatrix} ap & aq & ar \\ bp & bq & br \\ cp & cq & cr \end{pmatrix}$$ under the conditions $$a+b+c = 1$$, $$p+q+r=1$$. The culprits are staring you in the face. $$\begin{pmatrix} 0.06 & \color{red}{0.37} & 0.12 \\ 0.01 & 0.07 & 0.02 \\ 0.03 & \color{red}{0.26} & 0.06 \end{pmatrix}$$