Let $f(x,y)=x^2+y^2$ and it is easy to observe that $\operatorname{grad} f(a,b)$ is always perpendicular to the tangent line at point $(a,b)$ and I just can't prove why it would be like that.

-
I am a bit puzzled. Are you looking at the surface $\{(x,y,z)\in\mathbb{R}^3: x^2 + y^2 = z\}$, or are you looking at the circle $y = \pm\sqrt{r^2 - x^2}$ for some fixed $r > 0$ and $x\in[-r, r]$? In either case, it doesn't quite matter; in the former case you have a tangent plane, in the latter - a tangent line. For the line case, compute the gradient and take the inner product of the gradient with the tangent vector, and observe that it is zero. – William Jan 7 '12 at 6:53

So, you are essentially trying to show the following

$(\nabla f) \perp (f=c)$, which is that the gradient of the function is perpendicular to the level surface (f = constant).

Consider in general a curve $r=r(t)$ that stays on the level surface $f=c$. By definition $v = \frac{dr}{dt}$ is tangent to the level curve $f=c$. By the chain rule, the total differential $\frac{df}{dt}$ gives, $$\frac{df}{dt} =\frac{\partial f}{\partial x} \cdot \frac{dx}{dt} + \frac{\partial f}{\partial y} \cdot \frac{dy}{dt} + \frac{\partial f}{\partial z} \cdot \frac{dz}{dt} = \nabla f \cdot \frac{dr}{dt} = \nabla f \cdot v = 0$$ since $f(t)=c$. So, the gradient vector $\nabla f$ is perpendicular to $v$. This is true for any motion on $f=c$, and $v$ can be any vector tangent to $f=c$.

So, given any vector $v$ tangent to the level curve, $\nabla f \perp v$, so $\nabla f \perp$ tangent plane.

Let me know if you need clarification about anything.

-

Here, the assumption is that $f(x,y)$ can be parametrized by a continuously differentiable function in a neighborhood $I$ of a point $(x_0,y_0)$ in the domain wrt to a parameter, say $t$ as

$r(t)=x(t)i+jy(t)$ such that it has a non-zero tangent vector, say $r'(t)$ and that If $t\epsilon I,$ $f(r(t))=c$, where c is a constant.

Hence, $\displaystyle\frac{d}{dt} f(r(t))=\nabla{f(r(t)).r'(t)}=0.$

Thus, gradient of $f(r(t))$ i.e. of $f(x,y)$ is perpendicular to the tangent at the level curve at any given point $(x_0,y_0)$.

Take a look at this link

http://poncelet.math.nthu.edu.tw/disk3/cal01/tg.pdf

-