Here's the "math" way of looking at this. In three dimensions there are various "isometries", mappings that preserve distances between points. Some of these are linear transformations, and these can be represented in the usual way as multiplication by an orthogonal matrix, real 3x3 matrix $P$ such that $P^{-1} = P^T$, the inverse is the transpose. If $P$ has determinant 1, then the mapping realized as multiplication by $P$ is a rotation. Otherwise the determinant can be -1, and the mapping is termed a reflection.
Now linear transformations always fix the origin. If you want an isometry that maps the origin to some other point, you need a translation mapping.
Combining linear transformations with translations gives the larger class of mappings we call affine transformations. The theorem here is that every isometry in Euclidean space (of whatever dimension) is an affine transformation, but not every affine transformation is necessarily an isometry (distance preserving).
The real 4x4 matrices being discussed here are a clever way of representing 3D affine transformations generally (and isometries in particular) with matrix multiplication in one dimension higher. The special structure of these is:
$$\left( \begin{array}{cc} P & v \\ 0 & 1 \end{array} \right)$$
where $P$ is a 3x3 matrix representing the linear transformation part of the affine mapping, $v$ is a 3x1 column representing offset of the origin, "0" denotes a 1x3 row of zeroes, and 1 is just scalar one.
Consider what happens if we take a 3D vector $u^T = (x,y,z)$, tack a synthetic component 1 onto the end, transpose it to a column and multiply the result by the special matrix above:
$$\left( \begin{array}{cc} P & v \\ 0 & 1 \end{array} \right)
\times
\left( \begin{array}{c} x \\ y \\ z \\ 1 \end{array} \right) =
\left( \begin{array}{c} {Pu + v} \\ 1 \end{array} \right) $$
Thus a matrix multiplication can be used to compute $Pu + v$, which amounts to applying the linear transformation to source point $u$, then adding the offset $v$ to get the destination point. If $P$ is orthogonal and has determinant 1, then we might speak of rotating the source point and adding the offset. Such an operation is often required in computer graphics to "pan" and "dolly" a virtual camera.
While not "space efficient" in terms of the extra dimension, this representation is convenient for programmming because a general matrix multiplication routine can be used instead of coding separate steps of adding the offset after a matrix multiplication.
Note also that "undoing" the represented mapping will amount to multiplying by a matrix inverse. It is left as an exercise for the reader to work out what the inverse of the 4x4 matrix is in the special (isometry) case of orthogonal $P$.