8
$\begingroup$

Suppose

$$ A = \left( \begin{array}{cc} 1 & 4 \\ 5 & 6 \end{array}\right) $$

How do I calculate $\|A\|_{\text{OP}}$?

I know the definition of operator norm, but I am clueless on how to calculate it for real example like this. Can somebody please give me a step-by-step instruction on how to do this?

$\endgroup$
3
  • $\begingroup$ Find the eigenvalue with the biggest magnitude. $\endgroup$ Feb 28, 2018 at 9:48
  • $\begingroup$ @TonyS.F. No. For example if $A=\begin{bmatrix}0&1\\0&0\end{bmatrix}$ then the norm of $A$ is certainly non-zero, since $A\ne0$, but $0$ is the only eigenvalue. What you say is correct if $A$ is a symmetric matrix... $\endgroup$ Feb 28, 2018 at 13:28
  • $\begingroup$ sorry should have said singular value $\endgroup$ Feb 28, 2018 at 14:02

3 Answers 3

16
$\begingroup$

For a matrix $A$, $||A||_{OP}$ is the square root of the largest eigenvalue of $A^TA$, where $A^T$ is $A$'s transpose.

The transpose of $\left( \begin{array}{cc} 1 & 4 \\ 5 & 6 \end{array}\right)$ is $\left( \begin{array}{cc} 1 & 5 \\ 4 & 6 \end{array}\right)$, and hence:

$$A^TA=\left( \begin{array}{cc} 1 & 5 \\ 4 & 6 \end{array}\right)\left( \begin{array}{cc} 1 & 4 \\ 5 & 6 \end{array}\right)=\left( \begin{array}{cc} 26 & 34 \\ 34 & 52 \end{array}\right)$$

The eigenvalues of this matrix are $\{39 + 5\sqrt{53};\space 39-5\sqrt{53}\}$. Therefore, $$||A||_{OP}=\sqrt{39 + 5\sqrt{53}}$$

$\endgroup$
4
  • $\begingroup$ Is the operator norm always defined as the square root of the largest eigen value? $\endgroup$
    – abina shr
    Feb 28, 2018 at 10:12
  • $\begingroup$ And one more stupid question, in matlab is norm(A,2), the operator norm? $\endgroup$
    – abina shr
    Feb 28, 2018 at 10:13
  • $\begingroup$ @abinashr 1) Yes, for matrices. More about the operator norm can be found at Wolfram MathWorld – Citing this article, The operator norm of a linear operator $T:V\rightarrow W$ is the largest value by which $T$ stretches an element of $V$ [...] When $T$ is given by a matrix, say $T(v)=Av$, then $||T||$ is the square root of the largest eigenvalue of the symmetric matrix $A^TA$, all of whose eigenvalues are nonnegative. 2) Sorry, I don't know MATLAB. But I am pretty sure that Mathematica's Norm gives you the answer too. $\endgroup$
    – Mr. Xcoder
    Feb 28, 2018 at 10:17
  • $\begingroup$ It is better to phrase the spectral norm as the largest singular value of $A$, as opposed to largest eigenvalue of $A^TA$ $\endgroup$ Apr 6, 2020 at 2:13
2
$\begingroup$

You need the square root of the largest eigenvalue of $A^TA $.

Or, if you want to do it by definition, it becomes a Lagrange multiplier problem. In fact, in this $2$-dimensional case, it can be reduced to a one-variable optimization.

Concretely, using a bit of first year calculus at the end, you have that \begin{align} \|A\|^2&=\max\{\|Ax\|^2:\ \|x\|^2=1\} =\max\{(x+4y)^2+(5x+6y)^2:\ x^2+y^2=1\}\\ \ \\ &=\max\{26x^2+52y^2+68xy:\ x^2+y^2=1\}\\ \ \\ &=\max\{52-26x^2+68x\sqrt{1-x^2}:\ 0\leq x\leq1\}\\ \ \\ &=39+5\sqrt{53}. \end{align} So $\|A\|=\sqrt{39+5\sqrt{53}}$.

$\endgroup$
1
  • $\begingroup$ I don't quite understand the second part. Could you please elaborate, or even better, if you show it by example. Sorry I am new to this and all the things I am reading are really confusing me. I hope an example will help me understand better. $\endgroup$
    – abina shr
    Feb 28, 2018 at 9:59
2
$\begingroup$

The $2$-norm of matrix $\mathrm A$ can also be computed numerically, say, by solving the following convex optimization problem in $t > 0$

$$\begin{array}{ll} \text{minimize} & t\\ \text{subject to} & \| \mathrm A \|_2 \leq t\end{array}$$

or, using the Schur complement, by solving the following semidefinite program (SDP)

$$\begin{array}{ll} \text{minimize} & t\\ \text{subject to} & \begin{bmatrix} t \, \mathrm I_2 & \mathrm A\\ \mathrm A^\top & t \,\mathrm I_2\end{bmatrix} \succeq \mathrm O_4\end{array}$$

Using CVXPY (with NumPy),

from cvxpy import *
import numpy as np

A = np.array([[1, 4],
              [5, 6]])

# create 2 x 2 identity matrix
I2 = np.identity(2)

# create optimization variable
t = Variable()

# create constraints
constraint1 = [ norm(A,2) <= t ]
constraint2 = [ bmat([[ t*I2,   A],
                      [  A.T,t*I2]]) >> 0 ]

# create optimization problem
optprob = Problem( Minimize(t), constraint1 )

# solve optimization problem
optprob.solve()
print t.value

Using

  • constraint1, the minimum is 8.68334897643.

  • constraint2, the minimum is 8.68262817347.

From the other answers, the exact value of the minimum is

$$\sqrt{39 + 5 \sqrt{53}} \approx 8.68334897642624$$

and, thus, using constraint1 produces more accurate results.

$\endgroup$
2
  • $\begingroup$ The optimization problem is trivial: it is minimized when $t = ||A||_{OP}=||A||_2$. Hence, in your code, the numerical computation of the norm is performed by the Python function numpy.linalg.norm (numpy.org/doc/stable/reference/generated/numpy.linalg.norm.html). The answer gives no clue about the implementation of this function. $\endgroup$
    – jpvigneaux
    Nov 28, 2023 at 22:01
  • $\begingroup$ @jpvigneaux I suspect you missed the constraint2 part. $\endgroup$ Nov 29, 2023 at 7:26

Not the answer you're looking for? Browse other questions tagged .