8
$\begingroup$

Let $\Bbb{S}_{++}^n$ denote the space of symmetric positive-definite $n\times n$ real matrices. I am looking for hints concerning the visualization of such spaces for $n=1,2,\ldots$. I know that $\Bbb{S}_{++}^n$ is a convex cone, but I am not sure how it does "look like". How could I compute the equation of the cone analytically? My idea is to define $\Sigma\in\Bbb{S}_{++}^2$, as $$ \Sigma=\pmatrix{\sigma_{11} &\sigma_{12} \\ \sigma_{12} &\sigma_{22}}, $$ and demand $\lvert\Sigma\rvert>0\implies\sigma_{11}\sigma_{22}-\sigma_{12}^2>0$, but then what?

Additionally, is we consider the space $\Bbb{R}^n\times\Bbb{S}_{++}^n$, what would it look like for $n\geq2$? For $n=1$, $\Bbb{R}\times\Bbb{S}_{++}$ should be the half-space $\{(x,\sigma)\in\Bbb{R}^2\colon\sigma>0\}$.

Any advice on how might I visualize spaces such as the above would be much appreciated! The visualization methods do not need to be strictly rigor, any intuitive solution would be appreciated too. Thanks in advance!

EDIT

Let me add another question associated with my original one. Let $f\colon\Bbb{R}^n\times\Bbb{S}_{++}^n\to\Bbb{R}$ given by $$ f(\mathbf{x},\Sigma) = \sum_{i=1}^{l} \alpha_i \exp \Bigg( - (\mathbf{x}-\mathbf{x}_i)^\top\Bigg(\frac{\Sigma+\Sigma_i}{2}\Bigg)^{-1}(\mathbf{x}-\mathbf{x}_i) - \ln \Bigg( \frac{\rvert\frac{\Sigma+\Sigma_i}{2}\lvert}{\sqrt{\rvert\Sigma\lvert\rvert\Sigma_i\lvert}} \Bigg) \Bigg) , $$ where $\alpha_i\in\Bbb{R}$, $\mathbf{x}_i\in\Bbb{R}^n$ and $\Sigma_i\in\Bbb{S}_{++}^n$ is the mean vector and the covariance matrix, respectively, of the $i$-th Gaussian distribution from a set of $l$ distributions $\{\mathcal{N}(\mathbf{x}_i,\Sigma_i)\}_{i=1}^{l}$.

I am interested in visualizing $f(\mathbf{x},\Sigma)=0$. I know that the space $\Bbb{R}^n\times\Bbb{S}_{++}^n$ has dimensionality equal to $n+\frac{n(n+1)}{2}=\frac{n^2+3n}{2}$. I am particularly interested in visualizing $f(\mathbf{x},\Sigma)=0$ in the case of $n=2$. Is that possible?

Yet Another EDIT

Concerning my second question (the first "EDIT" above), this is what I have come up with. I am thinking of drawing the $2$D curve that arises when one sets $\Sigma$ to be the zero matrix of order $2$. Then, I guess that I have the projection of the original space ($\Bbb{R}^n\times\Bbb{S}_{++}^n$) onto the Euclidean space ($\Bbb{R}^n$) for $n=2$. Is this correct? How could I denote that? For instance, could I state that I will give the visualization of $f(\mathbf{x},\Sigma)=0$ in the space $\Bbb{R}^n\times\{\mathbf{0}\}$, where $\mathbf{0}$ denotes the zero matrix of order $n$?

$\endgroup$
3
+50
$\begingroup$

The space of all symmetric, positive definite $2\times 2$ matrices consists of all matrices of the form $$ \begin{bmatrix}x & z \\ z & y\end{bmatrix} $$ satisfying the inequalities $$ xy - z^2 > 0 \qquad\text{and}\qquad x>0. $$ This comes from the fact that a matrix is positive definite if and only if its leading principal minors are all positive.

Geometrically, this is the open cone in $\mathbb{R}^3$ obtained by rotating the first quadrant $x>0$, $y>0$ in the $xy$-plane around the line $y=x$. The vertex of the cone is the origin $(0,0,0)$, and the axis is the line $y=x$ in the $xy$-plane. One way to see this is to note that the first inequality above can be written $$ \left(\frac{x-y}{\sqrt{2}}\right)^2 + z^2 < \left(\frac{x+y}{\sqrt{2}}\right)^2 $$ where $(x-y)/\sqrt{2}$, $(x+y)/\sqrt{2}$, and $z$ are an orthonormal system of coordinates on $\mathbb{R}^3$.

The intersection of this cone with the $xy$-plane is the open first quadrant. This represents the matrices for which $z=0$, i.e. the diagonal matrices with positive entries. Note that the eigenspaces of such a matrix are precisely the $x_1$ and $x_2$ axes in the $x_1x_2$-plane. Conceptually, rotating this quadrant around the line $y=x$ corresponds to rotating the eigenspaces in the $x_1x_2$-plane. However, note that a rotation by an angle of $\theta$ in $xyz$ space corresponds to a rotation of the eigenspaces by an angle of $\theta/2$ in the $x_1x_2$-plane.

In general, if we think of the space of all symmetric $n\times n$ matrices as $\mathbb{R}^{n(n+1)/2}$, then the matrices with determinant $0$ form some sort of hypersurface in this space, and the positive definite matrices will be one component of the complement of this hypersurface (which can be defined analytically using the leading principal minors). The positive definite matrices can always be obtained by "rotating" the space of positive diagonal matrices, although in general the rotation is more complicated than a simple rotation around an axis. This rotation corresponds geometrically to rotating the perpendicular frame of eigenspaces in $\mathbb{R}^n$.

$\endgroup$
  • $\begingroup$ Thank you @Jim for your detailed answer! That makes the understanding of the symmetric (positive-definite) cone much more convenient. Would you mind please taking a look at my edits above, as well? Thanks again! $\endgroup$ – nullgeppetto May 30 '15 at 15:36
  • $\begingroup$ @nullgeppetto Your second question puzzles me. Assuming each covariance matrix $\Sigma_i$ is positive definite, all the terms in the summation are non-negative, so the only way for $f(\textbf{x},\Sigma)$ to be zero is if $\textbf{x} = \textbf{x}_1 = \cdots = \textbf{x}_l$. $\endgroup$ – Jim Belk May 30 '15 at 15:48
  • $\begingroup$ Well, yes, you are right! I missed something. I have edited my question, now I think that it should work (be zero not only trivially). $\endgroup$ – nullgeppetto May 30 '15 at 16:02
  • 1
    $\begingroup$ @nullgeppetto This book amazon.com/dp/1482259508 I need to understand conformal geometric algebra for some meaningless project... $\endgroup$ – Troy Woo May 31 '15 at 17:01
  • 1
    $\begingroup$ @nullgeppetto Thanks. I will take a look at your question after dinner. $\endgroup$ – Troy Woo May 31 '15 at 17:03
1
$\begingroup$

Not sure what you intend with the second question. This is what I think: we don't really need to consider all the terms do we...$f(\mathbf x,\Sigma)=0$ is equivalent to one of the terms blowing up to infinity:

$$ (\mathbf x-\mathbf x_i)^T\left(\frac{\Sigma+\Sigma_i}{2}\right)^{-1}(\mathbf x -\mathbf x_i)=\infty $$

One naturally resorts to homogenization $\mathbf x\mapsto\mathbf x/\mathrm x_0$: $$ (\mathbf x/\mathrm x_0 -\mathbf x_i)^T\left(\frac{\Sigma+\Sigma_i}{2}\right)^{-1}(\mathbf x/\mathrm x_0 -\mathbf x_i)=\infty $$ being also equivalent to $$ \begin{split} (\mathbf x-\mathbf x_i\mathrm x_0)^T\left(\frac{\Sigma+\Sigma_i}{2}\right)^{-1}(\mathbf x-\mathbf x_i\mathrm x_0)=\mathbf x^T\left(\frac{\Sigma+\Sigma_i}{2}\right)^{-1}\mathbf x&=0\\ \mathrm x_0&=0 \end{split} $$ This does have the geometric meaning of a quadric induced in a hyperplane if you fix $\Sigma$. When you change $\Sigma$, you end up with a family of quadrics. But of course if $\Sigma+\Sigma_i$ is positive definite, the quadric is an empty set so long as $\mathbb R^n$ is considered. I'm not sure how that might help.

$\endgroup$
  • $\begingroup$ Well, @Troy, I truly thank you for your time, but I need to make just another change to my original post. And this because of your answer... Let me do the edit and check again, if -of course- you still want to! $\endgroup$ – nullgeppetto May 31 '15 at 19:19
  • $\begingroup$ Now, after editing that, it's clear that $f(\mathbf{x},\Sigma)$ can be equal to zero, not just in the case you mention above. That is, the coefficients $\alpha_i$ can be positive or negative, so the function may be zero without the need of a quadratic to tend to $\infty$. Well, I know that changes your approach a lot, but what about now? Many thanks! $\endgroup$ – nullgeppetto May 31 '15 at 19:23
  • $\begingroup$ @nullgeppetto Now thats really something I am not familiar with. If you take this linear combination of exponential, it should be a gamma distribution. I really don't know how to visualize a gamma with so many varying parameters. $\endgroup$ – Troy Woo May 31 '15 at 19:47
  • $\begingroup$ Yes, that's indeed something else, but let's forget about what $f$ is. What would you call the equation $f(\mathbf{x},\Sigma_0)=0$, where $\mathbf{x}\in\Bbb{R}^n$ and $\Sigma_0$ is a fixed $n\times n$ matrix in $\Bbb{S}^n_{++}$? Assume $n=2$ such that the curve lives in the plane. After all, is this a projection on $\Bbb{R}^n$ or something totally different? I am looking for the right terminology right here... $\endgroup$ – nullgeppetto May 31 '15 at 19:53
  • $\begingroup$ @nullgeppetto I would call it a family of quadrics. $\endgroup$ – Troy Woo May 31 '15 at 20:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.