# Conditions for which two matrices multiplied together can be separated using PCA

Suppose that I have two real-valued matrices $\bf{A}$ and $\bf{B}$. Both matrices are exactly the same size. I multiply both matrices together in a point-by-point fashion similar to the Matlab A .* B operation.

Under what conditions can I approximately separate $\bf{A}$ and $\bf{B}$ using Principle Components Analysis (PCA)? Would it be possible to remove some components of the product A .* B to get an approximation of $\bf{A}$ or $\bf{B}$?

What algorithm might be best suited for this operation?

I am not looking for an exact separation of the matrices, but a separation using some sort of (statistical or numerical?) constraints. How would I set this problem up, and is there a good example of how to do this?

-

It seems to me that you can't separate the matrices from their pointwise multiplication (Hadamard/Schur product) without additional constraints.

Consider some matrix C. Any number in C is decomposable into an infinite number of products of two real numbers... which would give you an infinite number of "perfect" decompositions.

For example, you can always decompose C into 1 (a matrix of ones) and C. In fact, for any choice of A you can find a B such that their Schur product will result in any C that is specified (ignoring some problems when zero appears)...

-
Thanks, bitwise. But which constraints could I use? And could NMF be used with the proper constraints? – Nicholas Kinar Nov 10 '12 at 18:52
Is there a reference that documents how to set up the constraints? Ideally I would like to find a numerical procedure (and perhaps a sample problem) that will be able to separate the two matrices in a non-exact way. – Nicholas Kinar Nov 10 '12 at 19:10
I will explain it more clearly: You cannot find A and B from their Schur product in general. You can do this only if you assume something about what A and B should look like. For example, if you specify A exactly, then you can find B (that is simple division). – Bitwise Nov 10 '12 at 19:21
Thanks, Bitwise. Is there some numerical technique that can be used to set up the assumption? How do I apply constraints if I know "something" about A and B? Is there maybe some published example of the technique? – Nicholas Kinar Nov 10 '12 at 19:36
@NicholasKinar this is highly dependent on what kind of assumptions you want to make. What kind of knowledge do you have about A and B? For example, do you know what kind of process generated these matrices? Another example is to assume that A is some kind of signal and B is a noise matrix (both with some assumptions about what kind of noise and signal). – Bitwise Nov 10 '12 at 19:57

If the entries are non-negative then you could use NMF (non-negative matrix factorization). Or let $\textbf{C} = \textbf{A} \textbf{B}$. Then you could use singular value decomposition on $\textbf{C}$.

-
Thank you so much, lerije! Yes, both matrices are indeed non-negative. Is this for point-by-point multiplication of the matrices? Could you suggest a good reference with some example applications on how to set up this class of methods and apply constraints? – Nicholas Kinar Nov 10 '12 at 18:40
I think this type of factorization isn't for pointwise multiplication (Hadamard/Schur product). – Bitwise Nov 10 '12 at 18:45
This report on non-negative matrix factorization (linked from cs.virginia.edu/~jdl/nmf) talks about a component product (cs.kuleuven.ac.be/publicaties/rapporten/cw/CW440.pdf). Is this the same as the Hadamard/Schur product? – Nicholas Kinar Nov 10 '12 at 18:47
It is, but the document addresses a different problem that factorizes a combination of the Schur product with normal multiplication. Also, see my answer. – Bitwise Nov 10 '12 at 18:56