I'm reading the Pattern Recognition and Machine Learning book by Christopher Bishop and I'm having trouble understanding one thing.

Basically, he says:

Given a transformation from $\boldsymbol{x}$ to $\boldsymbol{y}$ defined as $\boldsymbol{y} = (y_{1}, ..., y_{D})$ where $y_{i}=\boldsymbol{u}_{i}^T(\boldsymbol{x} - \boldsymbol{\mu})$ and $\{\boldsymbol{u}_i\}$ is an orthonormal basis, we have a Jacobian matrix $\boldsymbol{J}$ with elements defined as

$$J_{ij} = \frac{\partial x_i}{\partial y_j} = U_{ji} \qquad (1)$$

where $U_{ji}$ are the elements of the matrix $\boldsymbol{U}^T$, which is a matrix of the orthonormal basis vectors $\boldsymbol{u}_i$ as columns.

I don't understand why (1) is true. How come the elements of the Jacobian are equal to the elements of $\boldsymbol{U}^T$?


Let's use the notation $\mathbf{u}_i = (U_{i1}, \dots, U_{iD})^T$. Then if $U$ has $\mathbf{u}_i$ as columns, then $U^T$ has entries $U_{ij}$.

Writing the expressions for $y_i$ in coordinates, we get

$$ y_i = \mathbf{u}_i^T (\mathbf{x} - \boldsymbol{\mu}) = \sum_j U_{ij}(x_j-\mu_j) = \sum_j U_{ij} x_j - \sum_j U_{ij} \mu_j $$

I assume $\boldsymbol{\mu}$ is a vector of constants, so the second sum above doesn't contribute to any derivatives. The first sum is just a linear combination of $x_j$, so the coefficients in the Jacobian are

$$ \frac{\partial y_i}{\partial x_j} = U_{ij} $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.