Question about gradients? I am trying to solve a minimization problem of $\min_I \|I-I_0\|_2^2 + \alpha \|\nabla I\|_2^2$. This is already convex so I can just take gradient to find the minimum. The gradient of the first term is $2(I-I_0)$ However for the second term with the gradient, how do I take the gradient of the norm exactly? If $I = (I_1, I2, \ldots , I_n)$, then $\nabla \|\nabla I\|_2^2 = \nabla \left[ \left(\frac{\partial I}{\partial I_1} \right)^2 + \cdots+ \left(\frac{\partial I}{\partial I_n} \right)^2 \right] = \left(2 \frac{\partial I}{\partial I_n}  \frac{\partial^2 I}{\partial I_n^2}, \ldots ,  2 \frac{\partial I}{\partial I_n}  \frac{\partial^2 I}{\partial I_n^2} \right)$ Is there a simpler way to write this? Also, this doesnt make sense since first term is $2(I-I_0)$, which is a vector... what does it mean to be a minimum vector?
 A: Supposing that $I:\Bbb{R}^n\rightarrow\Bbb{R}$, which is the only way this makes sense, we take each derivative separately to see what's happening: 
$$
\frac{\partial}{\partial x_j}\|\nabla I\|_2^2=2\frac{\partial f}{\partial x_1}\frac{\partial^2 I}{\partial x_1\partial x_j}+\ldots+2\frac{\partial f}{\partial x_n}\frac{\partial^2 I}{\partial x_n\partial x_j}
$$ so the gradient will end up being: 
$$
\nabla\|\nabla I\|_2^2=2H\nabla I
$$ where $H$ is the Hessian matrix: 
$$
(H)_{ij}=\frac{\partial^2 I}{\partial x_i\partial x_j}
$$
A: Suppose that $g(x) = f(Ax)$.  Then it follows from the chain rule that
\begin{equation}
\nabla g(x) = A^T \nabla f(Ax).
\end{equation}
(Here $f:V \to \mathbb R$ and $g:U \to \mathbb R$, $U$ and $V$ are finite dimensional inner product spaces over $\mathbb R$, $A:U \to V$ is a linear transformation, and $f$ is differentiable at $Ax$.  $\nabla g(x)$ is a vector in $U$.)
In your case, the gradient of $g(I) = \| \nabla I \|_2^2$ is
\begin{equation}
\nabla g(I) = 2 \nabla^T \nabla I.
\end{equation}
(Here the linear transformation $\nabla^T$ is being applied to the vector $\nabla I$.)
To solve your optimization problem, you can set the gradient equal to $0$ and solve for $I$.
