Minimizing a functional

Question

I am trying to follow this paper. In it they define a functional

$$J(f) = \sum_{x \in \Omega} \psi (f(x) - u(x)) + \beta \sum_{x \in N_x} \phi(f(x) - f(y)), $$

where, for my purposes, $f$ and $u$ are matrices of size $192 \times 192$ (images), and $\psi, \phi \in C^2(\mathbb{R})$. The goal is to find the matrix $f$ that minimizes $J$. I've been trying to minimize $J$ for the longest time and am really stuck. I'm trying to do newton's method, but am getting stuck on finding the gradient and hessian for $J$. Everywhere I look online has the gradient defined for vector functions. At first I had

$$\nabla J(f) = \sum_{x \in \Omega} \psi' (f(x) - u(x)) + \beta \sum_{x \in N_x} \phi'(f(x) - f(y)) $$ and $$\nabla^2 J(f) = \sum_{x \in \Omega} \psi'' (f(x) - u(x)) + \beta \sum_{x \in N_x} \phi''(f(x) - f(y)),$$

but then of course I can't iterate $f_{k+1} = f_k - \frac{\nabla J(f)}{\nabla^2 J(f)}$. Any help or pointers to resources online would be greatly appreciated.

Answer 1

Instead of looking at $f$ and $u$ as $192\times192$ matrices, just think of them as $192^2$-dimensional vectors, for example by concatenating the columns. The expression you wrote is simply the usual gradient with respect to a vector, except they've put it back into the shape of a $192\times192$ matrix. But thinking of these things as matrices is just going to be confusing once you try to take the second derivative of the functional.

Answer 2

I believe this is the answer to my question, and I wanted to post it just in case anyone else is looking for it in the future. After more searching I found this useful appendix which outlines matrix calculus. In particular, if we have a matrix $X$ and a scalar function $y = f(X)$ (just like I did with $J$ above), then the "gradient matrix" is defined by

$$\frac{\partial y}{\partial X} = \sum_{i, j} E_{ij} \frac{\partial y}{\partial x_{ij}},$$

where $E_{ij}$ is the typical elementary matrix and $x_{ij}$ is the $ij^{th}$ entry of $X$. This makes sense to me, as it looks like they are just taking the vector gradient of each column of the matrix. I'm going to stick with this for now, but would very much appreciate a confirmation that I have found the correct expression and can now try to use Newton's method ...