What is the first order Taylor expansion of a matrix to scalar function?

wikipedia says that if the real function f has the Taylor expansion:

$$f(x) = f(0) + f'(0) \cdot x + f''(0) \cdot \frac{x^2}{2!} + \dots$$

then a matrix function can be defined by substituting x by a matrix: the powers become matrix powers, the additions become matrix sums and the multiplications become scaling operations.

I am not sure what is scaling operations here?

For function $$f : \mathbf{R}^{n \times n} \rightarrow \mathbf{R}$$, such as $$f(\mathbf{X}) = trace(\mathbf{S} \mathbf{X})$$, where $$X$$ is symmetric, what is the first order taylor expansion?

I got:

$$\hat{f}(\mathbf{X} + \Delta) \approx trace(\mathbf{S} \mathbf{X}) + \mathbf{S} \Delta$$

but $$trace(\mathbf{S} \mathbf{X})$$ is a scalar, while $$\mathbf{S} \Delta$$ is not, I must made some mistake here, any idea?

From the related literature, I can infer that someone got:

$$\hat{f}(\mathbf{X} + \Delta) \approx trace(\mathbf{S} \mathbf{X}) + trace(\mathbf{S} \Delta)$$

but I don't know how they got there, any help will be appreciated.

I just realized that matrix function refers to function which maps a matrix to another matrix. Sorry about that.

Here I just want to figure out the taylor expansion of a function maps a matrix to a scalar.

Let $f : \mathbb R^{n \times n} \to \mathbb R$ be defined by $f (\mathrm X) = \operatorname{tr} (\mathrm S \mathrm X)$. As the trace is a linear operator,
$$f (\mathrm X + h \mathrm V) = \operatorname{tr} (\mathrm S (\mathrm X + h \mathrm V)) = \operatorname{tr} (\mathrm S \mathrm X + h \mathrm S \mathrm V) = \operatorname{tr} (\mathrm S \mathrm X) + h \operatorname{tr} (\mathrm S \mathrm V) = f (\mathrm X) + h \operatorname{tr} (\mathrm S \mathrm V)$$
Hence, the directional derivative of $f$ in the direction of $\mathrm V$ is $D_{\mathrm V} f (\mathrm X) = \operatorname{tr} (\mathrm S \mathrm V)$.
• Why can't we simply follow the taylor expansion rule: $f(x+\Delta) \approx f(x)+f′(x) \dot \Delta$? For this problem, $f'(x) = S$, so we got $f(x+\Delta) \approx tr(SX) +S \Delta$ ? What's wrong with this approach? Jun 3 '16 at 2:27
• @cinvro The trace is a scalar. $S \Delta$ is a matrix. You cannot add scalars and matrices. Jun 3 '16 at 12:33