Remember the implicit function theorem First, I know the implicit function theorem, but unfortunately I always have to look it up again and again. If $F(x,y)=0$ then I always forget whether I have to invert the first matrix of the Jacobian or the second one. The remark that only the second one is in general a square matrix is not really helpful to me, as I use the invertibility (that I cannot remember) to keep in mind which one of the two submatrices of the Jacobian has to be square. I mean what we want is to write $F(x,y)=0$ as $F(x,f(x))=0$ in a small neighbourhood of $x$, but where is the conceptual link between this and the invertibility of $D_2f(x,y)$? Somehow I cannot see the bridge between these two things.
 A: $\newcommand{\Reals}{\mathbf{R}}$In case it's a helpful mnemonic, the implicit function theorem (in this setting) is essentially a non-linear version of Gaussian elimination from linear algebra.
Specifically, let $m$ and $k$ be positive integers, and put $n = m + k$. Suppose $A$ is an $m \times n$ matrix, and let $F:\Reals^{n} \to \Reals^{m}$ be multiplication by $A$.
If $A$ has rank $m$, then (by reordering coordinates in $\Reals^{n}$ if necessary) we may as well assume the last $m$ columns of $A$ are linearly independent (i.e., that $\det(D_{y}F) \neq 0$). Writing $x$ and $y$ for general elements of $\Reals^{k}$ and $\Reals^{m}$ respectively, the kernel of $F$ (a.k.a., the level set of $F$ through the origin of $\Reals^{m}$) is
$$
\ker(F) = \{(x, y) \text{ in } \Reals^{k} \times \Reals^{m} \simeq \Reals^{n} : F(x, y) = 0\}.
$$
By Gaussian elimination on the last $m$ columns of $A$, we may treat the $y$ variables as basic and the $x$ variables as free. In other words, there exists a function $f:\Reals^{k} \to \Reals^{m}$ (in this case, a linear function) such that $\ker(F)$ is the graph of $f$.
A: I have written an answer to explain what is going on, but a little mistake made me loose the LaTeX original file. Fortunately, I have it on PDF ; I include it as a picture :
Take care that your $F$ is transformed in $f$ from the $7$-th line!
I add a picture which summarizes what I wrote :
Hope it will be helpful!
A: I always had trouble remembering the implicit function theorem as well.  It helped me to formulate it this way:¹

If for Banach spaces $E$, $F$ a function $f \colon E \oplus F → F$ is $C^k$ such that $f(0) = 0$ and the tangential of $f∘i_F \colon F → F $ is invertible at $0$ for the inclusion $i_F \colon F → E \oplus F,~x ↦ 0 \oplus x$, …
then there are neighbourhoods $U ⊂ E$ and $V ⊂ F$ of the origins in $E$ and $F$ and a $C^k$ function $φ \colon U → V$ such that
$$\mathrm{graph}~φ = f^{-1}(0) ∩ (U × V).$$

You can tell that $φ$ has to be $U → V$ and not the other way around, because for $E = 0$, the local inversion theorem assures the existence of a neighbourhood $U × V$ in which there is only one zero of $f$, so the graph of $φ$ then has to be a singleton (true for $φ \colon 0 → V$, very false for $φ \colon U → 0$).

¹: But, to be honest, I’m not even sure if this formulation of the implicit function theorem is correct and I don’t use the theorem in my daily life – I can’t even remember ever using it once.
A: A quick-and-dirty mnemonic is that "by cancellation of differentials", you "should" have
$$\frac{dy}{dx} = \frac{\frac{\partial G}{\partial x}}{\frac{\partial G}{\partial y}}$$
You have this, but with an additional minus sign, and with division replaced by a matrix inverse. This mnemonic was part of how I was taught this rule in thermodynamics, where it is often called the "triple product rule", since it can be rearranged to
$$\frac{\partial z}{\partial y} \frac{\partial y}{\partial x} \frac{\partial x}{\partial z} = -1.$$
