$X$ normally distributed, then $X^T \Sigma ^{-1} X$ follows chi square distribution. Suppose $X\sim \mathcal{N} _p (0, \Sigma )$. I am not sure why $X^{\top} \Sigma ^{-1} X$ follows a $\chi ^2$-distribution with $p$ degrees of freedom.
I think it has something to do with the square root of $\Sigma ^{-1}$, since $$\Sigma ^{-1/2} X \sim \mathcal{N} _p (0, \Sigma ^{-1/2} \Sigma {\Sigma ^{-1/2} }^{\top}) = \mathcal{N} _p (0, \Sigma ^{-1/2} \Sigma ^{1/2} \Sigma ^{1/2}  \Sigma ^{-1/2} ) = \mathcal{N} _p (0, I_p),$$ but how do I know that $\Sigma$ and $\Sigma ^{-1}$ have square roots and how do I know that $\Sigma ^{-1/2}$ is symmetric?
 A: Suppose $X\sim \mathcal{N} _p (0, \Sigma )$ with $\Sigma$ an invertible covariance matrix. 
To prove: $X^{\top} \Sigma ^{-1} X$ follows a $\chi ^2$-distribution with $p$ degrees of freedom.
Proof:
Since $\Sigma$ is a symmetric real matrix, there are eigenvalues $\lambda _1, \lambda _2, \dots, \lambda _p$ and corresponding orthonormal eigenvectors $u_1, u_2, \dots, u_p$ such that $$\Sigma = Q \Lambda Q^{\top}$$ with $Q$ the $p\times p$ matrix with the orthonormal eigenvectors as columns, and $\Lambda$ the diagonal matrix with the eigenvalues on the diagonal. Note that $$Q^{\top} Q = I_p$$ since $u_1, u_2, \dots, u_p$ are orthonormal. The eigenvalues of $\Sigma$ are non-zero because $\Sigma$ is of full rank. Note that,
$$\text{var}(Q^{\top}X) = Q^{\top} \Sigma Q = Q^{\top} Q\Lambda Q ^{\top} Q = \Lambda,$$
so the eigenvalues $\lambda _1, \lambda _2, \dots, \lambda _p$ are positive. This also shows that $\Sigma$ is positive definite.
Let $$\Lambda ^{1/2} = \left [\lambda _i ^{1/2} \delta _{ij} \right ] _{1\leq i,j \leq p} \ \ \text{ and } \ \ \Sigma ^{1/2} = Q \Lambda ^{1/2} Q^{\top}.$$ We immediately see that $\Sigma ^{1/2}$ is a symmetric matrix. The matrix $\Sigma ^{1/2}$ is positive definite, since $\Lambda ^{1/2}$ is positive definite, 
$$v^{\top} \Sigma ^{1/2} v= v^{\top} Q \Lambda ^{1/2} Q ^{\top} v = (Q^{\top} v)^{\top} \Lambda ^{1/2} (Q ^{\top} v) \geq 0 \ \ \ \  \text{ for all $p\times 1$ vectors $v$,}$$
where the inequality is strict for $v\neq 0$.
We have, 
$$ \Sigma ^{1/2} \Sigma ^{1/2}
= Q \Lambda ^{1/2} Q^{\top} Q \Lambda ^{1/2} Q^{\top}  
= Q \Lambda ^{1/2} \Lambda ^{1/2} Q^{\top} 
= Q \Lambda Q^{\top} 
= \Sigma .
$$  Thus, $\Sigma ^{1/2}$ is a symmetric positive-definite square root of $\Sigma$.
Since matrix inverses are unique and since $\Sigma$ is symmetric, it is easy to show that $(\Sigma ^{-1} ) ^{\top} = \Sigma ^{-1} $. The inverse $\Sigma ^{-1}$ has the same eigenvectors as $\Sigma$ and the eigenvalues of $\Sigma ^{-1}$ are the reciprocals of the eigenvalues of $\Sigma$. Since these reciprocals are positive, $\Sigma ^{-1}$ is positive definite. Let $$\Lambda ^{-1} = \left [\delta _{ij} / \lambda _i \right ] _{1\leq i, j \leq p}, \ \Lambda ^{-1/2} = \left [\delta _{ij} / \left (\lambda _i ^{1/2} \right ) \right ] _{1\leq i,j \leq p} \ \ \text{ and } \ \ \Sigma ^{-1/2} = Q \Lambda ^{-1/2} Q^{\top}.$$ The matrix $\Sigma ^{-1/2}$ is obviously symmetric and it is positive definite since $\Lambda ^{-1/2}$ is positive definite. 
Now, we have
$$ \Sigma ^{-1/2} \Sigma ^{1/2} = Q \Lambda ^{-1/2} Q^{\top} Q \Lambda ^{-1/2} Q^{\top} = Q \Lambda ^{-1} Q ^{\top} = \Sigma ^{-1}.$$
Thus, $\Sigma ^{-1/2}$ is a symmetric positive-definite square root of $\Sigma ^{-1}$.
It follows that
\begin{align*} \Sigma ^{-1/2} X \sim \mathcal{N} _p (0, \Sigma ^{-1/2} \Sigma \Sigma ^{-1/2} ) &  = \mathcal{N} _p (0, Q \Lambda ^{-1/2} Q^{\top} Q \Lambda Q^{\top}Q \Lambda ^{-1/2} Q^{\top} )
\\
& = \mathcal{N} _p (0, Q \Lambda ^{-1/2} \Lambda \Lambda ^{-1/2} Q^{\top} ) \\
%& = \mathcal{N} _p (0, Q \Lambda ^{-1/2} \Lambda ^{1/2} \Lambda ^{1/2} \Lambda ^{-1/2} Q^{\top} ) \\
& = \mathcal{N} _p (0, Q Q^{\top} ) \\
& = \mathcal{N} _p (0, I_p ) 
\end{align*} 
where we used that $Q^{\top} Q = I_p$.
Now, we have
$$X^{\top} \Sigma ^{-1} X = (\Sigma ^{-1/2} X) ^{\top} (\Sigma ^{-1/2} X) \sim \chi ^2 _p .$$
A: Since $\Sigma$ is a covariance matrix, it's symmetric with positive eigenvalues.
A: In the case where $\Sigma$ is singular, the number of degrees of freedom in the chi-square distribution is smaller than $p;$ in any case it's the rank of $\Sigma.$
You have $\Sigma=\operatorname E((X-\mu)(X-\mu)^\top) = \operatorname E(XX^\top)$ where $\mu=0$ is the $p\times 1$ column vector $\operatorname E(X).$
From that it is obvious that $\Sigma$ is symmetric. It is easy to show $\Sigma$ is positive definite:
$$
a^\top \Sigma a = \operatorname{var}(a^\top X) \ge 0
$$
if $a$ is any $p\times 1$ constant (i.e. non-random) vector. The random variable $a^\top X$ is scalar-valued so its variance is a non-negative scalar, strictly positive if $a\ne0$ (and that last uses the assumption that $\Sigma$ is non-singular).
A theorem of linear algebra says that since $\Sigma$ is symmetric and all of its entries are real, there is some orthogonal matrix $G$ (i.e. a matrix $G$ for which $G^\top G = GG^\top = I_p$) and some diagonal matrix $\Lambda$ such that $\Sigma = G\Lambda G^\top.$
The diagonal entries in $\Lambda$ must be positive since they are variances of components of $G^\top X.$
So now replace the positive numbers that are the diagonal entries of $\Lambda$ with their square roots and call that  $\Lambda^{1/2}$ and try to show that $G\Lambda G^\top$ is a symmetric positive-definite square root of $\Sigma.$
