What does that fact that $AB = I_3$ tell us about $A$ and $B$? If we have to two matrices $A$ and $B$ such that, $A \in \mathbb{R^{3 \times 4}}$ and $B \in \mathbb{R^{4 \times 3}}$. 

What does that fact that $AB = I_3$ tell us about $A$ and $B$?

 A: The question is a bit vague, it could mean various things, for example:
Note 1: $AB=I_3$ tells you exactly what the rank of $A$ is. There is only ONE value the rank of $A$ can take, and conversely, given a matrix of that rank you can always find such a $B$.
Note 2: $AB=I_3$ is equivalent to $T_A \circ T_B =Id$, and for functions this is equivalent to $T_A$ being onto and $T_B$ being one-to-one.
P.S. Hopefully, this makes the relation more clear:
Let $e_1,e_2,e_3$ be the cannonical basis in $\mathbb R^3$ and $v_i =Ae_j$.
Then $v_1,v_2,v_3$ must be linearly independent in $\mathbb R^3$.
Next, fix a vector $w$ such that $\{ v_1, v_2, v_3, w \}$ are linearly independent.
Then, $AB=I_3$ if and only if there exists some $v \in \mathbb R^3$ such that $B$ is the only matrix with 
$$Bv_1=e_1 \\Bv_2=e_2 \\ Bv_3=e_3 \\
Bw=v$$
$B$ exists and is unique because of the above linear independence.
Geometrically $T_A$ is a bijection between $\mathbb R^3$ and a subspace $V \subset \mathbb R^4$. Then $T_B$ must be the inverse of this mapping from $V \to \mathbb R^3$, extended arbitrarily from $V \subset \mathbb R^4$ to the entire $\mathbb R^4$.
