Loss of information when multiplying by singular matrix The group axioms demand that one has an inverse element. When it comes to multiplication zero has no inverse element because one cannot reconstruct the original number after it was multiplied by zero: 
There is an irreversible loss of information!
Therefore all numbers cannot form a group under multiplication as long as zero is included.
When it comes to matrices some matrices have an inverse, some don't have one. Singular matrices have determinant zero and don't have one. When the column vectors are linear dependent the determinant is zero and the matrix is singular.
My question
Where does the loss of information happen here? How is 'more' information lost when a vector is multiplied by a singular matrix (compared to multiplying it by an invertible matrix)?
 A: Let $A$ and $x$ be given, and set $b=Ax$. If $A$ is invertible, $x$ can be uniquely recovered from $b$ as $x=A^{-1}b$ (no information is lost, provided that we know $A$). But if $A$ is singular, someone who knows $A$ and $b$ cannot say for sure what $x$ was, since there are infinitely many solutions of the linear system $Ax=b$.
A: The loss of information is measured by the nullity (co-rank) of the matrix. A matrix with nullity $k$ is similar to a block matrix with regular first block and $0_{k\times k}$ second block. So there's a basis where $k$ of the basis vectors are "thrown away".
A: When multiplying by an invertible matrix $A$, you might say that no information is lost because you can recover $B$ from $AB$ by multiplying by $A^{-1}$.  Another way to put it is that the map $B\mapsto AB$ is injective.  
If $A$ is singular, then information is lost because the map $B\mapsto AB$ is no longer injective.  One explanation is that $A$ has nontrivial nullspace, and $AB=0$ for each matrix $B$ whose column space lies in the null space of $A$.  It follows that $AB=AC$ when the column space of $B-C$ lies in the null space of $A$.
