Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am trying to follow this paper. In it they define a functional

$$J(f) = \sum_{x \in \Omega} \psi (f(x) - u(x)) + \beta \sum_{x \in N_x} \phi(f(x) - f(y)), $$

where, for my purposes, $f$ and $u$ are matrices of size $192 \times 192$ (images), and $\psi, \phi \in C^2(\mathbb{R})$. The goal is to find the matrix $f$ that minimizes $J$. I've been trying to minimize $J$ for the longest time and am really stuck. I'm trying to do newton's method, but am getting stuck on finding the gradient and hessian for $J$. Everywhere I look online has the gradient defined for vector functions. At first I had

$$\nabla J(f) = \sum_{x \in \Omega} \psi' (f(x) - u(x)) + \beta \sum_{x \in N_x} \phi'(f(x) - f(y)) $$ and $$\nabla^2 J(f) = \sum_{x \in \Omega} \psi'' (f(x) - u(x)) + \beta \sum_{x \in N_x} \phi''(f(x) - f(y)),$$

but then of course I can't iterate $f_{k+1} = f_k - \frac{\nabla J(f)}{\nabla^2 J(f)}$. Any help or pointers to resources online would be greatly appreciated.

share|cite|improve this question

Instead of looking at $f$ and $u$ as $192\times192$ matrices, just think of them as $192^2$-dimensional vectors, for example by concatenating the columns. The expression you wrote is simply the usual gradient with respect to a vector, except they've put it back into the shape of a $192\times192$ matrix. But thinking of these things as matrices is just going to be confusing once you try to take the second derivative of the functional.

share|cite|improve this answer
up vote 1 down vote accepted

I believe this is the answer to my question, and I wanted to post it just in case anyone else is looking for it in the future. After more searching I found this useful appendix which outlines matrix calculus. In particular, if we have a matrix $X$ and a scalar function $y = f(X)$ (just like I did with $J$ above), then the "gradient matrix" is defined by

$$\frac{\partial y}{\partial X} = \sum_{i, j} E_{ij} \frac{\partial y}{\partial x_{ij}},$$

where $E_{ij}$ is the typical elementary matrix and $x_{ij}$ is the $ij^{th}$ entry of $X$. This makes sense to me, as it looks like they are just taking the vector gradient of each column of the matrix. I'm going to stick with this for now, but would very much appreciate a confirmation that I have found the correct expression and can now try to use Newton's method ...

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.