How to prove a matrix function is convex or nonconvex? I have a function of three matrix variables. But now, the authors fix two of them, then update one, and I cannot understand how this function is convex in each iteration in the paper.
This formula is :
$$f(W,V,B) =\|XW-V\|^2_F +\|Y-VB\|^2_F +\operatorname{tr}(V'LV) +2\operatorname{tr}(W'DW),$$
where $X$, $Y$ are constant matrices and $L$ is constant laplace matrix. Suppose $D$ is a constant diagonal matrix.
Now, we fix two variables $W$ and $V$, then update $B$. How to solve?
If we do not fix any variables, how to explain that the objective function is non-convex?
In general, we use the Hessian matrix , but what should I do when the variable is a matrix? 
 A: I want to give an answer to the first question, expanding what was already hinted. The goal is to prove that the function $B \mapsto f(V,W,B) $ is convex for every $V,W$ fixed (and lying in some space which is not specified by the OP). Since $B$ appears in the second term only, we investigate it alone: $$  \| Y - VB \|_F ^2 = \langle Y - VB , Y - VB \rangle_F = \|Y\|_F ^2 + \|V B \|_F^2 - 2 \langle Y , VB \rangle_F. $$ For $B_1$ and $B_2$ matrices we get $$ \begin{split} \| Y - V (tB_1 + (1-t)B_2)  \|_F ^2 & =\|Y\|_F ^2 + \|V (tB_1 + (1-t)B_2) \|_F^2 - 2 \langle Y , V(tB_1 + (1-t)B_2) \rangle_F \\ & \le \|Y\|_F ^2 + t \| VB_1 \|_F ^2 + (1-t) \|VB_2 \|_F ^2 \\ & - 2( t \langle Y ,VB_1\rangle_F + (1-t)\langle Y ,VB_2\rangle _F) \end{split} $$where I have used the (bi)linearity of the inner product and the elementary fact that if $f,g $ are two convex functions with $g$ nondecreasing then $g \circ f$ is a convex function; in our specific case $ g(x) = x^2$ and $f (X) = \|X\|_F$ (norms are convex). 
