Suppose $AB=I$, where A has full rank. Is $BA=I$? We know that the title statement is correct for square matrices $A,B$.
What about non-square matrices with full rank? 
 A: $$(1\;0)\binom 10=1\;,\;\;\;\text{yet}\;\;\;\binom10(1\;0)=\begin{pmatrix}1&0\\0&0\end{pmatrix}\neq I$$
A: If $A$ is full rank and it is, say, $m\times n$, then $m< n$, (because it isn't a square matrix it can't be the case that $m=n$). Furthermore if $B$ is $n\times p$, for some natural number $p$, for $BA$ to be defined, necessarily $p=m$.
Summarizing: 


*

*$A$ is $m\times n$

*$B$ is $n\times m$

*$AB=I_m$

*Thesis is $BA=I_n$


Assume $BA=I_n$.
A known property is that the $\text{rank}(XY)\leq \min \left(\left\{\text{rank}(X), \text{rank}(Y)\right\}\right)$.
So $n=\text{rank}(I_n)=\text{rank}(BA)\leq \min(\left\{m,n\right\})=m<n$.
This shows that not only does it fail, it always fails just as long as the product $BA$ is defined.
A: It's not true, even when multiplication is defined.  Look at $A=(1\quad 1)$ and $B=(2 \quad -1)^T$ (where $T$ indicates transpose). 
A: It is not true. You can however use it to create a right null space for $A$ (it will have square dimension):
$$A\left( BA - I\right) = 0$$
Because
$$A\left( BA - I\right) = ABA - A = IA - A = 0$$
