Why is $\max_{1≤i≤m,1≤j≤n}|a_{i,j}|≤||A||_{op}$ Why is
$$\max_{1≤i≤m,1≤j≤n}|a_{i,j}|≤||A||_{op}$$
where $$||A||_{op}=\sup\{||Ax||\space |\space x\in\mathbb{R^n}, ||x||=1\}\space\space\text{(operator norm)}$$
?
 A: As you have not defined the norm on $\mathbb{R}^n$ and $\mathbb{R}^m$ and all norms are equivalent, I will go with the supremum norm, thus $\| \left( \begin{matrix} b_1 \\ \cdots \\ b_m \end{matrix} \right) \| = \max_{1 \le j \le m} | b_j |$.
Then for $x = e_i \in \mathbb{R}^n$ we have $$\| Ae_i \| =\| \left( \begin{matrix} a_{1,i} \\ \vdots \\ a_{m,i}\end{matrix} \right)\| = \max_{j=1 \dots m} | a_{j,i}| \le \| A \|_{op}$$
Since this holds for all $i = 1 \dots n$, we find $$\max_{1 \le i \le n, 1 \le j \le m} |a_{j,i}| \le \| A \|_{op}$$
A: Assuming that the norm in $\mathbb R^n$ is the Euclidean norm we have (using Cauchy-Schwarz), for any $i,j$,
$$
|a_{ij}|=|\langle Ae_j,e_i\rangle|\leq \|Ae_j\|\,\|e_i\|=\|Ae_j\|\leq\|A\|_{\rm op}\,\|e_j\|=\|A\|_{\rm op}.
$$
A: Let $X=\begin{bmatrix}0\\\vdots\\0\\1\\0\\\vdots\\0\end{bmatrix}$ where $1$ is in the $j$th position, then $AX=\begin{bmatrix} a_{1,j}\\\vdots\\a_{n,j}\end{bmatrix}$ thus $||AX||^2=a_{i,j}^2+\sum_{k\neq i}a_{k,j}^2\ge a_{i,j}^2$
A: It is not true in general.
Let $S = \begin{bmatrix} 1 & 100 \\ 1 & 0 \end{bmatrix}$ and define
$\|x\|_* = \|Sx\|_2$. Since $S$ is invertible it is easy to check that $\|\cdot \|$ is a norm.
Let $A=\begin{bmatrix} 1 & 100 \\ 0 & 1 \end{bmatrix}$, then we have
\begin{eqnarray}
\|A\|_* &=& \sup_{\|x\|_* \le 1} \|Ax\|_* \\
&=& \sup_{\|x\|_* \le 1} \|SAx\|_2 \\
&=& \sup_{\|Sx\|_2 \le 1} \|SAx\|_2 \\
&=& \sup_{\|x\|_2 \le 1} \|SAS^{-1}x\|_2 \\
&=& \|SAS^{-1}\|_2
\end{eqnarray}
Since $\| SAS^{-1}\|_2 = \| \begin{bmatrix} 2 & -1 \\ 1 & 0 \end{bmatrix} \|_2 = \sqrt{2}$, we see that
$|[A]_{12}| = 100 > \sqrt{2} = \|A\|_*$.
Addendum:
If the vector norm is the Euclidean norm, then we have
$\|A\|_2 = \sup_{\|u\|_2 \le 1, \|v\|_2 \le 1 } | u^T A v |$, hence
choosing $u=e_i, v=e_j$ gives the required result.
