In case anybody is looking for an actual closed form (that is, not forward substitution or a Neumann expansion), because this came up in my research at one point:
You can also work out a closed form for each element of the inverse of a triangular matrix. From Cramer's rule, we know $[A^{-1}]_{ij} = \frac{1}{det(A)}C_{ji}$, where $C_{ji}$ is the matrix of cofactors of $A$. The $ji$-cofactor is given by $(-1)^{i+j}$ times the $ji$-minor of $A$, which is defined as the determinant of the matrix obtained by eliminating the $j$th row and $i$th column of $A$. In the case of a (say, lower) triangular matrix, eliminating one row and column results in a block lower triangular matrix, where
$|C_{ji}|= \left|\begin{matrix}T_0 & \mathbf{0} & \mathbf{0} \\ A_0 & H & \mathbf{0} \\ A_1 & A_2 & T_1\end{matrix}\right|$.
Here, $T_0$ and $T_1$ are lower triangular and $H$ is lower Hessenberg. By nature of block determinants, $|C_{ji}| = |T_0||T_1||H|$. The relative size of these matrices depends on $i$ and $j$. For $i=j$, $C_{ji}$ is triangular. For $i=1$ and $j=n$, the entire matrix is Hessenberg. The determinant of triangular matrices $T_0$ and $T_1$ is just the product of the diagonals, and there is a nice form for the determinant of Hessenberg matrices given here:
https://www.math.uni-bielefeld.de/ahlswede/pub/tamm/hessen.ps
There they assume a unit upper diagonal, so for the general case it will need to be modified, but doing so is fairly straightforward.