Is the orthogonality of eigenvectors preserved in a rank deficient Hermitian matrix? If $\boldsymbol{A}$ is a rank deficient Hermitian matrix are the following true?
1) Is $<\boldsymbol{x}_j,\boldsymbol{x_k}>=0$ when $\lambda_j=\lambda_k$?
2) Is $<\boldsymbol{x}_j,\boldsymbol{x_k}>=0$ when $\lambda_j\neq\lambda_k$?
In the above two points $\lambda_i$ is an eigenvalue of $\boldsymbol{A}$ and
$\boldsymbol{x}_i$ is its associated eigenvector.  
 A: Is in not the case that, whether $A$ is rank-deficient or not, we have
$\langle x, Ay \rangle = \langle A^{\dagger} x, y \rangle = \langle Ax, y \rangle? \tag{1}$
And by (1), 
$\langle x_k, \lambda_j x_j \rangle = \langle x_k, Ax_j \rangle = \langle Ax_k, x_j \rangle = \langle \lambda_k  x_k, x_j \rangle, \tag{2}$
which immediately leads to
$\lambda_j \langle x_k, x_j \rangle = \lambda_k \langle x_k, x_j \rangle, \tag{3}$
or
$(\lambda_k - \lambda_j) \langle x_k, x_j \rangle, \tag{4}$
and with $\lambda_k \ne \lambda_j$ we have
$\langle x_k, x_j \rangle = 0. \tag{5}$
That's half the story; I'll try to say more tomorrow, after I get some shut-eye.
Monday 2 December 2013 12:35 PM PST:  Good Morning!  I'm Baaaaack!!!
And now for the second part of our story, which was in fact the first question posed:
Again, the possible rank-deficiency of $A$ does not really bear on this issue.
If we have $\lambda_j = \lambda_k = \lambda$ for two linearly independent eigenvectors $x_j$ and $x_k$, they don't necessarily have to be orthogonal.  Indeed, if they were orthogonal, $\langle x_j, x_k \rangle = 0$, we could always construct two linearly independent, non-orthogonal eigenvectors $y_j$, $y_k$ such that $\text{span}(y_j, y_k) = \text{span}(x_j, x_k)$; simply set $y_j = x_j$ and set $y_k = \alpha x_j + \beta x_k$ with the scalars $\alpha, \beta$ satisfying $\alpha \ne 0 \ne \beta$; then $\langle y_j, y_k \rangle = \langle x_j, \alpha x_j + \beta x_k \rangle = \alpha \langle x_j, x_j \rangle \ne 0$ and $\text{span}(y_j, y_k) = \text{span}(x_j, x_k)$.  Furthermore, if $x_j$ and $x_k$ weren't orthogonal, $\langle x_j, x_k \rangle \ne 0$ then taking $y_j = x_j$ and 
$y_k = x_k - (\langle x_j, x_k \rangle /  \langle x_j, x_j \rangle) x_j$ then $\langle y_j, y_k \rangle = 0$; I leave it to you to show $Ay_k = \lambda y_k$ and so forth, and that  $\text{span}(y_j, y_k) = \text{span}(x_j, x_k)$ here as well.  The point is that
in the event of $\lambda_j = \lambda_k$, taking the eigenvectors orthogonal is an option, not a mathematical necessity.
Hope this helps.  Cheerio,
and as always,
Fiat Lux!!!
