Questions tagged [spectral-norm]
The spectral norm of a matrix is its maximum singular value.
199
questions
0
votes
0
answers
14
views
What is the second derivative of the spectral norm of a symmetric matrix?
It is well known that the derivative of a matrix $A$'s 2-norm with respect to $A$ is $$\frac{\partial \sigma_{\max}(A)}{\partial A}=u_1v_1^T,$$ where $u_1,v_1$ is the are the first column/row in the ...
- 392
0
votes
0
answers
24
views
How to prove the inequality for the spectral radius and spectral norm of an arbitrary matrix?
How to prove $\rho(A)\leq \min_D||D^{-1} A D||_2$, where $D$
is any invertible matrix, $\rho(\cdot)$ is the spectral radius, $\||\cdot\||$ is the spectral norm for the matrix? Are there any other ...
- 1
0
votes
2
answers
70
views
2-norm of transpose proof
I don't understand the proof of ‖x‖2=‖xT‖2.
I know you can use the Cauchy-Schwartz inequality and that you can prove it by first showing ‖xT‖2 ⩽ ‖x‖2 and then ‖x‖2 ⩽ ‖xT‖2. I get proving both ...
- 11
2
votes
1
answer
51
views
Find the matrix maximizing a summation of bilinear forms
Given $d, n \in \mathbb{N}$ and $x_k, y_k \in \mathbb{R}^d$, for $1 \leq k \leq n$,
we want to find
$\arg \max_{A: ||A||_2 = 1} \sum_k \langle x_k, A y_k \rangle$,
where $||\cdot||_2$ denotes spectral ...
- 1,373
1
vote
2
answers
64
views
Operator norm $4$ different definitions how to prove that $\sup$ is $\max$ and $\inf$ is $\min$ for the last two?
From what I have understood, all the $\sup$'s and $\inf$'s in the $4$ different definitions of the operator norm can be taken as $\max$'s and $\min$'s.
For a linear map $A\in \mathscr L (V,W)$ between ...
- 1,081
1
vote
0
answers
75
views
How to minimize $\| x {\bf I} - {\bf A} \|_2$?
Given the matrix ${\bf A} \in {\Bbb R}^{n \times n}$,
$$ \begin{array}{ll} \underset {x \in {\Bbb R}} {\text{minimize}} & \left\| x {\bf I}_n - {\bf A} \right\|_2 \end{array} $$
where $\| \cdot \|...
- 21.5k
1
vote
1
answer
86
views
Case of equality in spectral norm matrix triangle inequality
Let $(A,B)\in M_n(\mathbb{R})\times M_n(\mathbb{R})$ be two matrices. We denote by $\|\cdot\|_2$ the spectral norm.
Without any additional assumptions on $A$ and $B$, can we characterize the case of ...
- 49
1
vote
0
answers
77
views
Spectral norm of product and spectral radius
I have been thinking about the following problem on the upper bound of the spectral norm of the product: Consider $||\cdot||$ as the spectral norm, by the definition of matrix norm we have
$$||AB||\...
- 163
0
votes
1
answer
83
views
Maximal spectral norm of balanced $\pm 1$ matrix [closed]
Suppose that we have a square matrix $A = [a_{ij}] \in {\Bbb R}^{n \times n}$ whose entries are $\pm 1$ and whose columns are balanced, i.e., $\sum_{i=1}^n a_{ij}=0$. How large can the spectral norm ...
- 341
0
votes
0
answers
98
views
Subgradient of the spectral norm
I am working on developing a numerical algorithm that needs to use a subgradient of $\|\cdot\|_2$ (matrix norm) at each iteration. According to Characterization of the Subdifferential of Some Matrix ...
- 1
0
votes
2
answers
80
views
Prove $\| A\|_2=\|A^T\|_2$
Let $A \in \mathbb{R}^{n \times n}, $ Prove $\| A\|_2=\|A^T\|_2$
Similarly as the 1-norm I could show that $\| A\|_2=\max_{i} \sqrt{\sum |a_{ij}|^2}$. But this result does not help me to prove this ...
- 349
2
votes
1
answer
86
views
Can a convex combination of matrices in the unit spectral norm ball be unitary? [closed]
This is a generalization of "Unit sum" of unitary matrices equal to another unitary matrix?.
Does there exist a finite set of at least two distinct matrices $M_i$ with spectral norm $\leq 1$ ...
- 39
1
vote
1
answer
60
views
Spectral norm of modified matrix
Let $K$ be a symmetric, positive definite matrix and $S$ be a diagonal matrix such that all entries on its diagonal are greater than $1$. I am trying to prove the following relation.
$$ \| S K S - K \|...
- 2,262
2
votes
1
answer
88
views
Is there a notion of the spectrum of $L^1$ functions and does it correspond with the support of a random variable?
Is it possible to talk about the spectrum of a function? I know that $$A:=(L^1(\mathbb R^n,\mathbb C),+,*)$$ is a Banach algebra (the product operation ‘$*$’ is the usual convolution product). ...
- 2,445
0
votes
0
answers
23
views
$2$-norm of a matrix obtained through discretization of the eigenvalue problem
Define $$A = \frac{1}{2h} \begin{pmatrix}
0 & 1 & & & & -1\\
-1 & 0 & 1 & & \\
&\ddots & \ddots & \ddots \\
& & & -1 & 0 & 1 \\
1 &...
- 1,396
1
vote
0
answers
73
views
Bounding spectral norm of the matrix $f(\Sigma)$
Suppose that we have a sequence of matrices $\{\Sigma_n\in\mathbb{R}^{n\times n}\}_{n=1}^\infty$ with uniformly bounded spectral norm, that is to say, the sequence $\{\|\Sigma_n\|\}_{n=1}^\infty$ is ...
- 147
1
vote
0
answers
40
views
Can we express the spectral norm of a matrix $F$ in terms of the singular values of the sub-matrix $K$?
Let $\bf K$ be a generic $n \times n$ matrix, and let
$$ {\bf F} := \begin{bmatrix} 0 & {\bf 1}_n^\top \\ {\bf 1}_n & {\bf K} \end{bmatrix} $$
Can we express the spectral norm of $\bf F$ in ...
- 33
0
votes
0
answers
124
views
Frobenius and spectral norms of rank-$1$ matrices
How to prove that $$\left\lVert x y^{\ast}\right\rVert_F = \left\lVert x y^{\ast}\right\rVert_2 = \left\lVert x\right\rVert_2 \left\lVert y\right\rVert_2$$ where $\forall x, y \in \mathbb{C}^n$?
I ...
- 41
2
votes
1
answer
60
views
Does there exist an upper bound on the $\frac{\lVert F \rVert_F}{\lVert F \rVert}$ ratio knowing that F is the sum of two matrices?
Let
$$
J =
\begin{bmatrix}
0 & 1 & \dots & 1\\
1 & 0 & \dots & 0\\
\vdots & \vdots & \ddots & \vdots\\
1 & 0 & \dots & 0
\end{bmatrix},
\ \ \ \ \ K=
\...
- 33
0
votes
0
answers
40
views
Does $\sup_{\Vert x \rVert = 1} \langle A x, x \rangle$ equal the largest eigenvalue of $A$?
If $A$ is a $n \times n$ matrix, taking $x$ to be an eigenvector associated with the largest eigenvalue $\lambda_\max$ yields $\sup_{\Vert x \rVert = 1} \langle A x, x \rangle \ge \lambda_\max$.
It is ...
- 64
0
votes
0
answers
32
views
Distance between eigenvalues vectors based on $\|A^{-1}B-I_n\|\le \epsilon$
Given $n\times n$ positive definite matrices $A,B \succ 0$, with some desirably tight bound on $\|A^{-1}B - I_n\|_F\le \epsilon$ and $\|B^{-1}A - I_n\|_F\le \epsilon$ can we bound their eigenvalue ...
- 639
0
votes
1
answer
88
views
spectral norm of PSD matrices and inner product
Is the operator norm on the space of $n \times n$ real PSD matrices derived from an inner product?
In particular, spectral norm for matrix $A \in \mathbb{R}^{n \times n}$ defined as
\begin{equation}
\|...
- 638
1
vote
1
answer
80
views
Spectral norm of difference of two similar matrices
Suppose we have two similar matrices $R$ and $L$.
Is there a bound on the spectral norm of $R-L$ using the singular values of $R$ and $L$?
Suppose $R^TR$ and $L^TL$ are similar matrices:
Can we ...
- 165
3
votes
2
answers
62
views
Show that there exists a $v$ s.t. $A^TAv=\|A\|_2^2v$
Given $A \in \mathbb{R}^{m \times n}$, show that there exists a $v$, s.t. $\|v\| = 1$ and $A^T A v = \|A\|_2^2 v$, where $$\|A\|_2 :=\max_{\|v\|=1}\|Av\|$$ is the spectral norm.
My attempt:
By the ...
- 1,189
1
vote
0
answers
61
views
How to bound the spectral norm of this kind of matrix?
I have asked the same question before but the contents were not complete, and I propose it again.
Assume that A, B are both symmetrical positive definite matrices, and
$$ 0 \prec aI \preceq A \preceq ...
- 21
1
vote
3
answers
202
views
The spectral norm of the difference of two rank-$1$ matrices is equal to the sine of their angle [closed]
Can someone help me understand either geometric intuition or mathematical proof that how for two unit vectors $a$ and $b$, the following holds:
$$\left\lVert aa^T - bb^T \right\rVert_2 = \sin \theta$$
...
- 29
2
votes
1
answer
114
views
How to prove ${\lambda _n}\left( {{X^T}AX} \right) \le {\lambda _n}\left( A \right){\rho ^2}\left( X \right)$? [closed]
I am trying to prove the following inequality
$${\lambda _n}\left( {{X^T}AX} \right) \le {\lambda _n}\left( A \right){\rho ^2}\left( X \right)$$
where $\rho$ is the spectral radius of a matrix and $\...
- 57
0
votes
0
answers
58
views
Prove a linear oparator is surjective
Let $H$ be a Hilbert space, $T\in L(X)$ with $\|T\|\le 1$ and $A=Id-T^*T$. Suppose that: $\ker(\sqrt{A})=\{x:\|x\|=\|Tx\|\}$ and $\|T\| < 1$. Prove that $A$ is invertible. I try to solve:
$Ax=0 \...
0
votes
0
answers
41
views
How to find the eigenvalues of a matrix with largest complex component?
Is there an algorithm (cheaper than solving the whole eigenspectrum) that determines the eigenvalue(s) of a (non-hermitian) matrix with the largest (magnitude of) complex component?
I have not found ...
- 158
1
vote
0
answers
55
views
Inequality in Conjugate gradient method for solving linear system
Given the system of linear equations
$$Ax =b,$$
where $A \in \mathbb{R}^m\times \mathbb{R}^n$ and $b \in R^m$. Let $\widehat{x}$ be the approximate solution solved by Conjugate Gradient Method and ...
- 859
1
vote
1
answer
108
views
how can I prove ${\lambda _n}\left( {{X^T}AX} \right) \leqslant \left\| X \right\|_F^2{\lambda _n}\left( A \right)$ where $A$ is a PD matrix
The following inequality intuitively holds in my opinion, however I am facing hard time proving it
${\lambda _n}\left( {{X^T}AX} \right) \leqslant \left\| X \right\|_F^2{\lambda _n}\left( A \right)$
...
- 57
1
vote
1
answer
93
views
Spectral radius of a specific matrix multiplied by a diagonal matrix $D=diag(\mathrm{e}^{i\alpha_1},...\mathrm{e}^{i\alpha_n})$
Let $A=(a_{ij})\in M_n(\mathbb C)$, $n\geq 3$ be a matrix satisfying :
$\sum_{j=1}^n\lvert a_{ij}\rvert^2=1,$ for all $i=1,\ldots n$.
$\sum_{i=1}^n\lvert a_{ij}\rvert^2\leq1,$ for all $j=1,\ldots n$.
...
- 123
1
vote
0
answers
43
views
Sufficient condition for $\langle Ax, Ay\rangle\leq \lambda^2_{max}\langle x, y\rangle$
Question. Is there a ''non-trivial'' sufficient condition on $x, y\in \mathbb R^n$ such that
$$\langle Ax, Ay\rangle\leq \lambda^2_{\max}\langle x, y\rangle$$
where $A\in \mathbb R^{m\times n}$ and $\...
- 1,344
1
vote
0
answers
167
views
Spectral norm of a random matrix whose entries are product of two standard gaussians
While solving a research problem related to approximations in neural networks, I've faced the following problem which I have not been able to solve after trying different approaches for a while.
Let's ...
- 55
4
votes
1
answer
138
views
Given any symmetric matrix $A$, how to find a diagonal matrix $B$ such that $B \succeq A$?
Given any symmetric (not necessarily PSD) matrix $A$, how can we find a good diagonal matrix $B$ such that $A \preceq B$? We want each of the diagonal elements of $B$ to be as small as possible.
One ...
- 41
0
votes
1
answer
205
views
On the minimization of the spectral norm
Given a matrix $\mathbf Z$ and $\mathbf G$, how to solve the following spectral minimization problem?
$$\min_{\mathbf m \in \mathbb C^n} \|\mathbf Z-\hat{\mathbf Z}\|_2 \quad \text{such that} \quad \...
1
vote
1
answer
131
views
Inequality on spectral norm of sum of tensor products [closed]
I am looking for a lower bound on
$$\left\|\sum\nolimits_i M_i \otimes M_i\right\|_{\infty}$$
when the $M_i$'s are positive semi-definite matrices and $\|.\|_{\infty}$ denote the spectral norm. I ...
- 223
1
vote
1
answer
110
views
Why is the numerical radius at least half the norm?
Let $A$ be a bounded operator on a Hilbert space,
$$r(A) \:=\:\sup_{\|v\|\,=\,1}{v^*Av}$$
be the numerical radius and $\|A\|_2$ the standard operator norm. Wikipedia (see #13) claims: $$r(A)\leq\|A\|...
- 9,777
0
votes
1
answer
199
views
Is there a relation between the mutual coherence and largest eigenvalue?
Let $A=[a_1,\dots,a_n]\in\mathbb R^{m\times n}$ be a matrix with columns $a_1,\dots,a_n$ having unit norm, i.e., $\|a_i\|=1$ for all $i=1,\dots,n$. Let
$$ L := \lambda_{\max} \left( A^T A \right) $$
...
- 1,344
1
vote
1
answer
325
views
Upper Bound for the Induced 2-Norm $\|(A+B)^{-1}A\|$
Assume $A$ is a positive definite matrix, and $B$ is a positive semi-definite matrix. I am interested in the problem of whether there exists a constant upper bound for the induced 2-norm (spectral ...
- 163
0
votes
1
answer
200
views
A closed form relation to compute the spectral norm of a 2x2 real matrix
I am reading this book (page 215) and I found in the bottom of this page an interesting relation that I want to use. The relation is
$$||A||_2 = \sqrt{\frac{||A||_F^2 + \sqrt{||A||_F^4 - 4 (\mathrm{...
- 783
0
votes
0
answers
21
views
Error incurred upon truncating a diagonal matrix to only the k largest eigenvalues.
Say I have a diagonal matrix (A) of dim n x n. What is the error incurred if I approximate it with only its k-largest eigenvalues? I am using the 1-norm. I am trying to quantify the error in the ...
0
votes
1
answer
76
views
If $A = R R^T$, prove that $||R_1 R_1^T||_2 \le ||A||_2$ where $R_1$ is first column of $R$.
I am trying to solve the following problem:
For $A$ positive definite, let the Cholesky factorization be $A = R R^T$. Prove that $$\|R\|_2 \le \sqrt{\|A\|_2}$$ and that the vector $2$-norm of $R_1 ...
- 145
2
votes
1
answer
422
views
What's a sphere in the space of matrices?
Given a normed vector space $V$ over a field $k$ it's sphere is the set of points of unit norm.
$$ S(V) = \{ v \in V \mid 1 = \lVert v \rVert^2 \}$$
Over the reals the n-spheres are just spheres of ...
- 1,276
1
vote
0
answers
28
views
A matrix whose spectral norm is logarithmic in its size?
I'm trying to find a kernel function $f(x,x')$ such that the N by N kernel matrix $A_{i,j} = f(x_i,x_j)$, where $x_i=i/N$, has a spectral norm (largest singular value) of $O(log N)$.
Could anyone find ...
- 11
0
votes
1
answer
1k
views
On the gradient of spectral norm of matrix
Let $\| A \|_2 := \sqrt{\lambda_{\max}(A^TA)}$. As part of the gradient of a regularized loss function (for machine learning), I need the gradient $\nabla_A \| A \|_2^2$, which, using the chain rule, ...
- 43
0
votes
0
answers
980
views
Bounding the spectral norm of a Gaussian random matrix
For a random matrix $A\in\mathbb{R}^{n\times m}$ with entries $A_{ij}\sim \mathcal{N}(0,1)$ i.i.d., the spectral norm of it can be bounded by $\|A\| \leq C(\sqrt{m} + \sqrt{n}+t)$ with probability $1-...
- 149
1
vote
1
answer
76
views
Why are we able to define the Frobenius norm as so?
I have seen that, in order to prove
$$ ||A||_{2} \leq ||A||_{F}$$
this inequality holds, proofs will use the following definition for a Frobenius norm.
$$||A||_{F}^{2} = \sum_{j=1}^{n} ||Ae_{j}||_{2}^{...
-1
votes
1
answer
112
views
Can we deduce this property for 2 norm with respect to sub matrix of Cholesky factorization? [closed]
We have $A \in \mathbb{R}^{n \times n}$ which is symmetric and positive-definite. Also, $A$ is a block matrix:
$$A = \begin{pmatrix} A_{11} & A_{21}^{\top} \\ A_{21} & A_{22} \\ \end{pmatrix}.$...
- 527
0
votes
1
answer
79
views
How to prove this inequality consisting of the minimum singular value? [closed]
How does one proceed in solving this inequality?
X is of full column rank
$$\left\|\left(\boldsymbol{X}^{T} \boldsymbol{X}\right)^{-1} \boldsymbol{X}^{T} \boldsymbol{v}\right\|_{2} \leq \frac{1}{\...
- 137