# Loss of information when multiplying by singular matrix

The group axioms demand that one has an inverse element. When it comes to multiplication zero has no inverse element because one cannot reconstruct the original number after it was multiplied by zero:

There is an irreversible loss of information!

Therefore all numbers cannot form a group under multiplication as long as zero is included.

When it comes to matrices some matrices have an inverse, some don't have one. Singular matrices have determinant zero and don't have one. When the column vectors are linear dependent the determinant is zero and the matrix is singular.

My question
Where does the loss of information happen here? How is 'more' information lost when a vector is multiplied by a singular matrix (compared to multiplying it by an invertible matrix)?

-

Let $A$ and $x$ be given, and set $b=Ax$. If $A$ is invertible, $x$ can be uniquely recovered from $b$ as $x=A^{-1}b$ (no information is lost, provided that we know $A$). But if $A$ is singular, someone who knows $A$ and $b$ cannot say for sure what $x$ was, since there are infinitely many solutions of the linear system $Ax=b$.

-
@Thank you, Hans: Could you give an example? – vonjd Nov 14 '10 at 10:14
@vonjd: OK, take A=[1,0; 0,0]. I'm thinking of a secret vector x=[x1; x2] and when I compute b=Ax I get b=[17;0]. There is no way you can tell from that information what number x2 I was thinking of. – Hans Lundmark Nov 14 '10 at 10:17

The loss of information is measured by the nullity (co-rank) of the matrix. A matrix with nullity $k$ is similar to a block matrix with regular first block and $0_{k\times k}$ second block. So there's a basis where $k$ of the basis vectors are "thrown away".

-

When multiplying by an invertible matrix $A$, you might say that no information is lost because you can recover $B$ from $AB$ by multiplying by $A^{-1}$. Another way to put it is that the map $B\mapsto AB$ is injective.

If $A$ is singular, then information is lost because the map $B\mapsto AB$ is no longer injective. One explanation is that $A$ has nontrivial nullspace, and $AB=0$ for each matrix $B$ whose column space lies in the null space of $A$. It follows that $AB=AC$ when the column space of $B-C$ lies in the null space of $A$.

-
Oops, I didn't read the question very carefully, and my answer concerned multiplying a singular matrix with other matrices rather than vectors. – Jonas Meyer Nov 14 '10 at 10:19
Well, that's quite a natural thing to think of, since the question starts out talking about group axioms! – Hans Lundmark Nov 14 '10 at 10:23
@Hans: True, but in the question itself it clearly indicates multiplication by a vector, and the misreading was all mine. Well, this answer is still valid when considering the column spaces of $n\times 1$ matrices... (Incidentally, to whomever, I don't plan to bother editing, because there are already 2 other answers that address the actual question.) – Jonas Meyer Nov 14 '10 at 10:36