# How to work with an expression involving matrix products and Hadamard products in least squares?

With the system $$\boldsymbol{A} \boldsymbol{x} = \boldsymbol{b}$$, where $$\boldsymbol{A}$$ is a known matrix, $$\boldsymbol{b}$$ is a measured vector and $$\boldsymbol{x}$$ the vector we want to estimate, the ordinary least squares solves our problem: $$\hat{\boldsymbol{x}} = (\boldsymbol{A}^\mathrm{H} \boldsymbol{A})^{-1} \boldsymbol{A}^\mathrm{H} \boldsymbol{b} = (\boldsymbol{A}^\mathrm{H} \boldsymbol{A})^{-1} \boldsymbol{A}^\mathrm{H} \boldsymbol{A} \boldsymbol{x} = \boldsymbol{x}.$$

Now, I have a system slightly more complicated: $$(\boldsymbol{A} \odot \boldsymbol{C}) \boldsymbol{x} = \boldsymbol{b}$$, where $$\odot$$ denotes the Hadamard product, and $$\boldsymbol{C}$$ contains random values that I don't know. In this case, if I try to solve it like the ordinary least squares, I have:

$$(\boldsymbol{A}^\mathrm{H} \boldsymbol{A})^{-1} \boldsymbol{A}^\mathrm{H} \boldsymbol{b} = (\boldsymbol{A}^\mathrm{H} \boldsymbol{A})^{-1} \boldsymbol{A}^\mathrm{H} (\boldsymbol{A} \odot \boldsymbol{C}) \boldsymbol{x} = \:?$$

Then, I am stuck here, because from what I know and what I read, we cannot simplify an expression where matrix products and Hadamard products are mixed (there are some things possible when vector or diagonal matrices are involved, but this is not the case here).

Can I go any further with this equation, and try to have an expression for the error introduced by the presence of $$\boldsymbol{C}$$ ? Or can I do something else than the ordinary least squares ?

I have another similar problem where I am also stuck. I have $$((\boldsymbol{A} \boldsymbol{C}) \odot \boldsymbol{D}) \boldsymbol{x} = \boldsymbol{b}$$, where I know $$\boldsymbol{D}$$ but not $$\boldsymbol{C}$$, and if I try to solve it like the ordinary least squares, I have: $$((\boldsymbol{A} \odot \boldsymbol{D})^\mathrm{H} (\boldsymbol{A} \odot \boldsymbol{D}))^{-1} (\boldsymbol{A} \odot \boldsymbol{D})^\mathrm{H} \boldsymbol{b} = ((\boldsymbol{A} \odot \boldsymbol{D})^\mathrm{H} (\boldsymbol{A} \odot \boldsymbol{D}))^{-1} (\boldsymbol{A} \odot \boldsymbol{D})^\mathrm{H} ((\boldsymbol{A} \boldsymbol{C}) \odot \boldsymbol{D}) \boldsymbol{x} = \:?$$

• You can try to replace the Hadamard product with the Kronecker product since the first is a block of the latter. $(A \odot C)x=P(A \otimes C)Qx=b$, where $P$ and $Q$ are selection matrices. Commented Dec 2, 2020 at 21:22
• Thanks for your proposition. However, I dont think it will allow me to extract the expression of the error introduced. I guess what I wanted is just not feasable. We cannot get a sum when only products are involved. Commented Dec 15, 2020 at 20:55

The best way to write the least squares solution of $$Ax=b$$ is $$\hat x = A^+b$$ where $$A^+$$ denotes the Moore-Penrose pseudoinverse, because when you introduce a Hadamard factor and replace $$A\to(A\odot C)$$ it can destroy the full-rank property such that $$(A^HA)$$ becomes singular.
With this in mind, the solution to your first problem is $$\hat x = (A\odot C)^+b$$ and the solution to the second one is $$\hat x = (AC\odot D)^+b$$
• I agree that this is what you should compute to properly estimate $x$. However in my case, I don't know $C$, so I cannot compute these. This is why I want to compute $\hat{x} = A^{+} b$, and would like to have an expression to determine the error I will cause by doing this. Commented Dec 15, 2020 at 20:37