How to interpret the (possible) relationship between Jacobian and Covariance matrix I'm researching a problem which suggests that progress could be achieved if the Jacobian of a vector function might be in some way considered in the manner of a covariance matrix.
Specifically, and to take the case in $\mathbb{R}^2$, let $f_1 := f_1(x,y)$ and $f_2 := f_2(x,y)$ for $x,y \in \mathbb{R}$ and $f_1,f_2: \mathbb{R}^2 \rightarrow \mathbb{R}$, the quadratic form of the Jacobian is
$$
\mathbf{J}^{\intercal}\mathbf{J} = 
\left[
\begin{array}{cc}
\left(\frac{df_1}{dx}\right)^2+\left(\frac{df_2}{dx}\right)^2 & \frac{df_1}{dx}\frac{df_1}{dy}+\frac{df_2}{dx}\frac{df_2}{dy} \\
\frac{df_1}{dx}\frac{df_1}{dy}+\frac{df_2}{dx}\frac{df_2}{dy} & \left(\frac{df_1}{dy}\right)^2+\left(\frac{df_2}{dy}\right)^2
\end{array}
\right].
$$
Suppose we were to compare with the form of covariance matrix $\mathbb{\Sigma}$ expressed as
$$
\mathbb{\Sigma} = 
\left[
\begin{array}{cc}
\sigma_x^2 & \rho \sigma_x \sigma_y \\
\rho\sigma_x \sigma_y & \sigma_y^2
\end{array}
\right],
$$
it would impose that the correlation coefficient is given by
$$
\rho = \frac{\frac{df_1}{dx}\frac{df_1}{dy}+\frac{df_2}{dx}\frac{df_2}{dy}}
{\sqrt{\left(\frac{df_1}{dx}\right)^2+\left(\frac{df_2}{dx}\right)^2}
\sqrt{\left(\frac{df_1}{dy}\right)^2+\left(\frac{df_2}{dy}\right)^2}
}.
$$
My questions are therefore:

*

*Is $\rho$ admissible as a correlation coefficient in the sense that $|\rho| \leq 1$?

*Are there conditions on $f_1,f_2$ under which (1) is satisfied if not satisfied always?

*What is the geometric/functional interpretation of the two terms in the denominator?  Clearly they represent norms or 'distances' but is there an intuitive characterisation in terms of $f_1,f_2$?

*Are there any immediate implications for finding the eigenvalues of $\mathbf{J}^{\intercal}\mathbf{J}$?

*What branch of mathematics have I happened upon?

*Any suggestions for reading?

 A: Here are some facts which might make this less mysterious.

*

*The covariance matrix of $n$ random variables can be any $n \times n$ positive semidefinite matrix.


*If $M$ is any matrix (not necessarily square), then $M^T M$ is a positive semidefinite matrix, and every positive semidefinite matrix can be written in this form.
Now some more specific answers to your questions. Write $f = (f_1, f_2)$.

*

*Yes. This follows from the Cauchy-Schwarz inequality. This "correlation coefficient" is the cosine of the angle between the tangent vectors $\frac{\partial f}{\partial x}$ and $\frac{\partial f}{\partial y}$.


*It's always satisfied, as above.


*They describe the lengths of the tangent vectors $\frac{\partial f}{\partial x}$ resp. $\frac{\partial f}{\partial y}$, which describe the infinitesimal distance that the vector $f$ travels if $x$ resp. $y$ moves infinitesimally.


*The eigenvalues of $J^T J$ are the squares of the singular values of $J$ (this is true for any matrix, not necessarily square). They have the same interpretation singular values always do. For example, the largest singular value is the operator norm. The operator norm of a Jacobian tells us approximately the maximum amount by which a function locally distorts distances.


*I don't know where this matrix is studied. Typically people are more interested in the eigenvalues of the Jacobian, at least as far as I've seen. The google search for "singular values of the Jacobian" might be helpful.
