How to calculate a vector that is rotated by 90 when multiplied by a random matrix Suppose I've a symmetric matrix A in $R^{3\times3}$. I need to know the vector x in $R^3$ which is rotated by 90 degree when multiplied by A. i.e.
if y=Ax, then x should be such that $x^Ty$ = 0; Can anybody please help me out? Thanks in advance.
 A: The equation to be solved in $x$ is
$$
x^TAx=0,\qquad A\in \mathbb R^{n\times n}_{sym}
$$
Since $A$ is symmetric, it is diagonalizable: $A=Q^T\Lambda Q $, with $Q$ a rotation matrix and 
$$
\Lambda=\textrm{diag}(\lambda_1,\ldots,\lambda_n)
$$
Therefore, putting $y=y(x):=Qx$
$$
x^TAx=x^TQ^T\Lambda Qx=(Qx)^T\Lambda (Qx)=y^T\Lambda y=\lambda_1 y_1^2+\ldots+ \lambda_n y_n^2
$$
If $A$ is positive semidefinite, then
$ \sum_i\lambda_iy^2_i=0\Leftrightarrow\lambda_iy_i^2=0\,\forall i\Leftrightarrow\lambda_iy_i=0\,\forall i$. In that case, from the last equality it follows that $y^T\Lambda y=0\Leftrightarrow\Lambda y=0$ . 
Therefore
$$ 
x^T Ax=0\Leftrightarrow\Lambda y=0 \Leftrightarrow \Lambda Qx=0\Leftrightarrow 0=Q^T(\Lambda Qx)=Ax
$$
Note that I have used the fact that, since $Q$ is a rotation matrix, $Q^Tz=0\Leftrightarrow z=0$.
So the vectors $x$ which solve your problem are exactly the elements of the kernel of $A$:
$$
\forall A\textrm{ positive semi-definite: }Ax \perp x \Leftrightarrow x\in \mathrm{Ker}(A).
$$
In the general case, you have to rely on the equation
$$
\lambda_1 y_1^2+\ldots+ \lambda_n y_n^2=0,
$$
i.e. you have to find all the vectors $v=[ y_1^2,\ldots,y_n^2] $ with positive components orthogonal to the vector $ [ \lambda_1,\ldots,\lambda_n]$
A: Before doing any calculations for a concrete case we should obtain a general overview over the problem. Well known theorems say that a symmetric real matrix $A$ has real eigenvalues $\lambda_i$ , and that there is an orthonormal basis of ${\mathbb R}^3$ diagonalizing $A$. With respect to this basis 
$$A(x_1,x_2,x_3)=(\lambda_1 x_1,\lambda_2 x_2,\lambda_3 x_3)\ ,$$
so that $Ax\perp x$ amounts to
$$Ax\cdot x=\lambda_1 x_1^2+\lambda _2 x_2^2+\lambda_3 x_3^2=0\ .\tag{1}$$
I leave the discussion of the cases where some $\lambda_i=0$ to you. If all $\lambda_i$ are $>0$, or all of them are $<0$, then there are no nontrivial solutions of $(1)$. The only interesting case is when $\lambda_1\geq\lambda_2>0$ and $\lambda_3<0$. In this case the vectors $x$ we are after have to satisfy
$$x_3^2={\lambda_1\over|\lambda_3|} x_1^2+{\lambda_2\over|\lambda_3|} x_2^2\ .$$
These vectors $x$ form a $2$-dimensional conical surface with elliptical sections $x_3=c$. It follows that in this case there is not only one direction with the special property mentioned in the question, but there is an infinity of such directions.
