$A = \left[\begin{matrix} 1 & 0 \\ 3 & -1 \\ \end{matrix} \right]$
How to find the maximum of $\|Ax\|$ for all unit vectors $x \in\mathbb R^2$?
My professor told me the answer is: $\sqrt{\frac{11+ \sqrt{117} }{2}}$
I know this has something to do with the characteristic polynomial of $A$, which is $X^2-11x+1$.
Since $\Vert Ax \Vert \ge 0$ and the function $y \to y^2$ is monotonic for $y \ge 0$, $\Vert Ax \Vert$ and $\Vert Ax \Vert^2$ take their respective maxima at a common vector $x$, so we can without loss of generality work with $\Vert Ax \Vert^2$.
Note that
$\Vert Ax \Vert^2 = \langle Ax, Ax \rangle = \langle x, A^TA x \rangle, \tag{1}$
and further observe that unit vectors $x \in \Bbb R^2$ may be written as
$x(\theta) = (\cos \theta, \sin \theta)^T, \tag{2}$
where $0 \le \theta < 2\pi$. We have
$\Vert x(\theta) \Vert^2 = \langle x(\theta), x(\theta) \rangle = \cos^2 \theta + \sin^2 \theta = 1. \tag{3}$
We see that the family of unit vectors $x(\theta)$ depends only on the single parameter $\theta$ and that
$x'(\theta) = \dfrac{d}{d\theta}x(\theta) = (-\sin \theta, \cos \theta)^T \ne 0 \tag{4}$
for all $\theta \in [0, 2\pi)$. (4) also shows us that
$\langle x'(\theta), x(\theta) \rangle = -(\sin \theta)(\cos \theta) + (\cos \theta)(\sin \theta) = 0, \tag{5}$
which may also be seen by direct differentiation of the equation $\langle x(\theta), x(\theta) \rangle = 1$:
$0 = 1' = \langle x(\theta), x(\theta) \rangle' = \langle x'(\theta), x(\theta) \rangle + \langle x(\theta), x'(\theta) \rangle = 2\langle x'(\theta), x(\theta) \rangle. \tag{6}$
Bearing these facts in mind, we next compute $\langle x, A^TA x \rangle'$ as follows:
$\langle x(\theta), A^TA x(\theta) \rangle' = \langle x'(\theta), A^TA x(\theta) \rangle + \langle x(\theta), A^TA x'(\theta) \rangle$ $= \langle x'(\theta), A^TA x(\theta) \rangle + \langle (A^TA)^T x(\theta), x'(\theta) \rangle = 2 \langle x'(\theta), A^TA x(\theta) \rangle \tag{7}$
since $(A^TA)^T = A^TA$. $\Vert Ax(\theta) \Vert^2$ thus takes its maximum at some $x(\theta)$ such that
$\langle x'(\theta), A^TA x(\theta) \rangle = 0, \tag{8}$
and since we are working in $\Bbb R^2$ and $\langle x'(\theta), x(\theta) \rangle = 0$, (8) implies that $A^TA x(\theta)$ is collinear with $x(\theta)$; that is, there is a $\lambda \in \Bbb R$ with
$A^TA x(\theta) = \lambda x(\theta), \tag{9}$
i.e., $\lambda$ is an eigenvalue of $A^TA$ with eigenvector $x(\theta)$; since $x(\theta)$ is a unit vector, (9) implies
$\langle x(\theta), A^TA x(\theta) \rangle = \langle x(\theta), \lambda x(\theta) \rangle = \lambda \tag{10}$
at eigenvectors $x(\theta)$, and this further implies that the maximum value of $\Vert Ax(\theta) \Vert^2 = \langle x(\theta), A^TA x(\theta) \rangle$ is equal to the largest eigenvalue of $A^TA$; thus we easily compute $A^TA$:
$A^T A = \begin{bmatrix} 1 & 3 \\ 0 & -1 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 3 & -1 \end{bmatrix} = \begin{bmatrix} 10 & -3 \\ -3 & 1 \end{bmatrix}; \tag{11}$
the characteristic polynomial of $A^TA$ is thus easily seen to be
$p_A(\lambda) = \lambda^2 - \text{Tr} (A^TA) \lambda + \det (A^TA) = \lambda^2 - 11 \lambda + 1, \tag{12}$
and the roots of $p_A(\lambda)$ are
$\lambda = \dfrac{1}{2}(11 \pm \sqrt{117}) >0, \tag{13}$
the greater manifestly being $\dfrac{1}{2}(11 + \sqrt{117})$; thus this is the maximum value of $\Vert Ax \Vert^2 = \langle x, A^TA x \rangle$; hence the maximum of $\Vert Ax \Vert$ for $\Vert x \Vert = 1$ is $\sqrt{\dfrac{1}{2}(11 + \sqrt{117})}$.
The eigenvector of $A^TA$ corresponding to $\dfrac{1}{2}(11 + \sqrt{117})$ may readily be computed, and from it the value of $\theta$ may be had, but the question does not explicitly ask for this information, so I will leave this computation to my audience; it is not difficult.
Hope this helps. Cheerio,
and as always,
Fiat Lux!!!
The norm of a matrix $A$ can be seen to be the square root of the largest eigenvalue of the matrix $AA^T$ (you can see a proof of this in these notes written by Keith Conrad). Since $$AA^T=\begin{pmatrix} 1 & 3 \\ 3 & 10\end{pmatrix}$$ and its characteristic polynomial is $\lambda^2-11\lambda+1$, the result follows.
Let $x = (a,b)$, then $||Ax|| = ||(a, 3a - b)|| = \sqrt{a^2 + (3a - b)^2}$.
Consider: $f(a,b) = a^2 + (3a - b)^2 = a^2 + 9a^2 - 6ab + b^2 = 10a^2 - 6ab + b^2$ subject to: $g(a,b) = a^2 + b^2 = 1$. Using Lagrange Multiplier method:
$\nabla f = \lambda\cdot \nabla g$:
$(20a - 6b, 2b - 6a) = (2a\lambda, 2b\lambda)$. So:
$20a - 6b = 2a\lambda$
$2b - 6a = 2b\lambda$.
If $\lambda = 0$, then: $b = 3a$, and $10a = 3b \to a = b = 0$, this can't happen.
So: $\lambda \neq 0$, then if $a = 0$ or $b = 0$, then both $a = b = 0$, and this can't happen. So $a \neq 0$, and also $b \neq 0$. Then:
$\dfrac{10a - 3b}{b - 3a} = \dfrac{a}{b}$
So: $10ab - 3b^2 = ab - 3a^2 \to -a^2 + b^2 = 3ab \to 1 - t^2 = 3t$, with $t = \dfrac{a}{b}$. So:
$t^2 + 3t - 1 = 0 \to t = \dfrac{-3 \pm \sqrt{13}}{2}$. Together with $a^2 + b^2 = 1$, you can solve for $a$, and $b$, and then find $\sqrt{f_{max}}$
