# Tag Info

4

We can use the Schur complement of $A$ to get the formula $$B^{-1} = \pmatrix{A^{-1} + A^{-1}\mathbf 1 (B/A)^{-1}\mathbf 1' A^{-1} & -A^{-1}\mathbf 1(B/A)^{-1}\\ -(B/A)^{-1}\mathbf 1' A^{-1} & (B/A)^{-1}},$$ where $$B/A = 0 - \mathbf 1' A^{-1} \mathbf 1 = -\mathbf 1' A^{-1} \mathbf 1 \in \Bbb R.$$ We can rewrite the above in the following ...

3

You can't. Define $$v_1 = \left[ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right]$$ and $$v_2 = \left[ \begin{array}{c} 0 \\ 1 \\ 0 \end{array} \right]$$ By your definition: $$M v_1 = \left[ \begin{array}{c} 1 \\ 4 \\ 7 \end{array} \right]$$ and $$M v_2 = \left[ \begin{array}{c} 2 \\ 5 \\ 8 \end{array} \right]$$ However, M (v_1 + v_2) = \left[ \begin{array}... 3 Because A is symetric psd matrix, we can diagonalize A as: A = UDU^T where U is a orthogonal matrix (U^T = U^{-1}) and D is a diagonal matrix. Hence, we have \begin{align} (A+xI)^{-1} &= (UDU^T+xUU^T)^{-1} \\ &= (U(D+xI)U^{-1})^{-1} \\ &= U(D+xI)^{-1}U^{-1} \\ \end{align} Then \begin{align} a & =\theta^TA(A+xI)^{-1}(A+xI)^{-1}A\... 2 Using matrix units, you can write X(t)=\sum_{k,j} x_{kj}(t)\,E_{kj}, $$where x_{kj}:[a,\infty)\to\mathbb R are scalar functions. An integral is a limit of sums of the form "value of the function times size of the region". In particular, you expect an integral to be linear. So you want$$ \int_a^\infty X(t)\,dt=\sum_{kj}\bigg(\int_a^\infty x_{...

2

Let's call this infimum $I$. For all $x$ we have $||Ax||\leq ||A||\cdot ||x||$, and so $I\leq ||A||$. Conversely, if $||Ax||\leq C||x||$ for all $x$, then for all $x\ne 0$ we have $\frac{||Ax||}{||x||}\leq C$. By taking the supremum on $x\ne 0$ we obtain $||A||\leq C$. Now we take the infimum on all such values of $C$ to get $||A||\leq I$.

2

This is a pretty straightforward application of the chain rule. Given \begin{aligned} 0 &= f(\lambda, w(\lambda), s(\lambda)) = (1-\lambda s) w - 1 \\[2ex] \implies 0 &= \frac{{\rm d\,} f(\lambda, w(\lambda), s(\lambda))}{{\rm d\,} \lambda} = \frac{\partial f}{\partial \lambda} + \frac{\partial f}{\partial w}\frac{\partial w}{\partial \lambda} + \... 2 The answer is no. Consider the case A,B are diagonal positive definite matrices. It is easy to make d arbitrarily small/arbitrarily close/equal to p without the matrices being close/equal. To make this clear, let x_1,..., x_p be arbitrary positive numbers with x_1+...+x_p=p. Set A=I, B= \mbox{diag}(x_1,...,x_p). Then d(A,B)=p. 2 One standard approach to computing matrix functions times a vector f(M)x or quadratic forms x^Tf(M)x when M is symmetric is via the Lanczos algorithm. Lanczos computes an orthonormal basis Q_k = [q_1, \ldots, q_k] for Krylov subspace \operatorname{span}(x,Ax,\ldots, A^{k-1}x) by a Gram-Schmidt like procedure. This results in a factorization AQ_k =... 2 It's just the matrix multiplication of the two Jacobians. In fact, this can be seen as the reason why matrix multiplication is defined like that. If you see a m\times n matrix as a linear function from \mathbb R^n \to \mathbb R^m, then the Jacobian of the matrix is the matrix itself. The matrix multiplication is just the composition of the two linear ... 2 \def\B{\Big}\def\L{\left}\def\R{\right}\def\o{{\tt1}}\def\p#1#2{\frac{\partial #1}{\partial #2}}Define the all-ones vector \o and a vector v such that\eqalign{ v\odot v &= Ax\odot Ax + \delta^2\o \\ 2\,v\odot dv &= 2\,Ax\odot A\,dx \\ dv &= Ax\odot A\,dx\oslash v \\ }$$where \odot denotes the elementwise/Hadamard product and \... 2 Rearranging a bit and renaming \lambda=1/v^T u we obtain$$Av=\lambda u\iff v=\lambda A^{-1}u$$Dotting with u yields the equation that determines \lambda:$$\lambda^2=\frac{1}{u^T A^{-1}u}$$which if A is positive definite yields two solutions:$$v=\pm\frac{A^{-1}u}{\sqrt{u^TA^{-1}u}}$$both of which are minima of the function. EDIT: To reflect the ... 1 The answer is yes. In particular, see corollary 5 of this document. 1 Ok, lets look at the general definition of a differential df on normed spaces, for f:\mathbb{S}\to\mathbb{P}$$f(x+h)=f(x)+df(x,h)+o(||h||_S)$$where the differential df(x,h) is linear and continuous in h. Lets look at \mathbb{S}=\mathbb{R}^m,\mathbb{P}=\mathbb{R}^n. Then any linear operator \mathbb{R}^m\to\mathbb{R}^n can be represented by some ... 1 Your work suggests the problem is equivalent to maximizing |v^\top u| subject to the constraint v^\top A v = 1. Making the change of variables z=A^{1/2} v and y=A^{-1/2} u, this is equivalent to maximizing |z^\top y| subject to z^\top z = 1. By Cauchy-Schwarz, this is attained when z = \pm y/\|y\|. Working backwards, this leads to v = A^{-1/2} ... 1 \def\m#1{\left[\begin{array}{c}#1\end{array}\right]}\def\p#1#2{\frac{d #1}{d #2}}Given a scalar function, like f(x)=\exp(x), and the matrices$$\eqalign{ M = M(a)&= aXt + Yt,\qquad &\dot M = \dot M(a) = \p{M}{a} = Xt \\ F = F(a) &= f(M), &\dot F = \dot F(a) = \p{F}{a} \\ }$$Evaluating the function with a block-triangular argument yields ... 1 \def\R#1{{\mathbb R}^{#1}}\def\v{{\rm vec}}\def\M{{\rm Reshape}}\def\m#1{\left[\begin{array}{r}#1\end{array}\right]}\def\p#1#2{\frac{\partial #1}{\partial #2}}For ease of typing, replace the subscripted variables with single-letter names$$\eqalign{ R = H_R \qquad Q = H_I \qquad P=P_d \\ }$$and define the matrices$$\eqalign{ I_2 &= \m{1&0\\0&...

1

$\def\m#1{\left[\begin{array}{c}#1\end{array}\right]}$Multiplying two block-wise centrosymmetric matrices yields \eqalign{ \m{A&B\\B&A}\cdot\m{X&Y\\Y&X} &= \m{(AX+BY)&(AY+BX)\\(BX+AY)&(BY+AX)} \\ } Since matrix addition commutes $\big({\rm i.e.}\;(BX+AY) = (AY+BX)\big)$ the product is also block-wise centrosymmetric. Therefore ...

1

This is not even true for a $2 \times 2$ matrix, take $$A = \begin{pmatrix} 1 & 2 \\ 2 & 3 \end{pmatrix}, \quad A^{-1} = \begin{pmatrix} -3 & 2 \\ 2 & -1 \end{pmatrix}$$

1

First, note that your $W_{n+1}$ should be a vector to get 1d output. So we need to treat the cases where an index $=n+1$ anywhere differently from cases where all indices $<n+1$. Using differentials we have $$dz_i=f'(W_iz_{i-1})\circ dW_iz_{i-1}+f'(W_iz_{i-1})\circ W_idz_{i-1}$$ Using the chain rule for differentials we have $$dMSE[z_{i+1}(z_i),dz_i]=dMSE[... 1 First, it is important to see how matrices act on vectors. In our case, we are transforming vectors from \mathbb{R}^4 into vectors in \mathbb{R}^5 (i.e. T will transform a vector x = (x_1,x_2,x_3,x_4) into b =(b_1,b_2,b_3,b_4,b_5). If we write a general transformation T: \begin{bmatrix} t_{11} & t_{12} & t_{13} & t_{14}\\ ... 1 \def\T{\operatorname{Tr}}\def\p#1#2{\frac{\partial #1}{\partial #2}}For typing convenience, let's define the matrix variable$$B=S\circ S\circ A$$and use a colon to denote the trace/Frobenius product$$\eqalign{ A:B &= \T(A^TB) \\ A:A &= \big\|A\big\|_F^2 \\ }$$The properties of the trace allow the terms in such a product to be rearranged$$\...

1

These rules pertain to differentials not to gradients. Let's use them properly, starting with your second example function. \eqalign{ f_2 &= x^TAx \\ df_2 &= dx^TAx+x^TA\,dx \\ &= (Ax)^Tdx+(A^Tx)^Tdx \\ &= (Ax+A^Tx)^Tdx \\ \frac{\partial f_2}{\partial x} &= (Ax+A^Tx) \\ } Setting $A=I$ turns this into your first function. ...

1

Let's use the numerator-layout notation. First note that $\frac{dx}{dx}=I$ but $\frac{dx^T}{dx}=\begin{bmatrix}\begin{pmatrix}1&0&...&0\end{pmatrix},\begin{pmatrix}0&1&0&...&0\end{pmatrix},...,\begin{pmatrix}0&0&...&0&1\end{pmatrix}\end{bmatrix}$, a tensor, technically 1 x n x n. In denominator layout fashion, $\... 1 According to my calculations, $$\begin{matrix}\frac{dy_1}{dW_{11}}=b_1&\frac{dy_1}{dW_{12}}=b_2&\frac{dy_1}{dW_{21}}=0&\frac{dy_1}{dW_{22}}=0\\ \frac{dy_2}{dW_{11}}=0&\frac{dy_2}{dW_{12}}=0&\frac{dy_1}{dW_{21}}=b_1&\frac{dy_2}{dW_{22}}=b_2\end{matrix}$$ So it looks like$b^T\bigotimes I_{2\times 2}would give you $$\begin{pmatrix}b_1&... 1 \def\T#1{\operatorname{tr}(#1)}I would suggest using Newton's Method, i.e.$$\eqalign{ f(x) &= 0, \quad f_k &= f(x_k), \quad f'_k &= f'(x_k) \quad\implies\quad x_{k+1} &= x_k - \frac{f_k}{f'_k} \\ }$$For typing convenience, define the matrices$$\eqalign{ B &= A+Ix \quad\implies\quad \frac{dB^n}{dx} = nB^{n-1} \\ M &= A\,\theta\... 1 By definition, the Fenchel conjugate off(X)$is given by: $$\label{eq:f}\tag{1} f^{*}(Y) = \sup_{X} X\circ Y - f(X).$$ It can be shown that$g(X) \overset{\text{def}}{=} X\circ Y - f(X)$is a concave function in terms of$X$. (Actually, you can use the second derivative of$f(X)$with respect to$X$to show that$f(X)$is a ... 1 On way to define the operator-valued integral is as follow: Define$\sigma:\mathbb{R}^{n}\times\mathbb{R}^{n}\rightarrow\mathbb{R}$by$\sigma(x,y)=\int_{a}^{\infty}\langle X(t)x,y\rangle dt.$(Assuming that$t\mapsto\langle X(t)x,y\rangle$is continuous if you are working with Riemann integral. However, this assumption can be weaken and only require that$t\...

1

For ease of typing, define \eqalign{ G &= \Gamma = (I+wC) \quad\qquad F=(BGB^T)^{-1} \quad\qquad p=(B^Th-a) } and note that $F$ and $G$ are symmetric since $C^T=C.$ Next, eliminate the variable $s$ in favor of $w$ \eqalign{ s &= p^TCp \\ w &= (1-\lambda s)^{-1} \;\doteq\; (1-\lambda p^TCp)^{-1} \\ \tfrac{w-1}{w\lambda} &= p^TCp \\ } ...

1

$\def\p#1#2{\frac{\partial #1}{\partial #2}}$There is a problem with your first derivative \eqalign{ y_{ij} &= a_{ik}b_{kj} \\ \p{y_{ij}}{a_{mn}} &= \left(\p{a_{ik}}{a_{mn}}\right)b_{kj} \\ &= \delta_{im}\delta_{kn}\;b_{kj} \\ &= \delta_{im}\,b_{nj} \\ } If you switch the indices to $\Big((m,n)=(i,k)\Big)$, then you obtain \eqalign{ \... 1 Let's use a convention where a greek letter denotes a scalar, lowercase latin a vector, and uppercase latin a matrix. So replace your original variable names with\eqalign{ \alpha_k = h_k ,\qquad \phi(X) = f(X) ,\qquad x = z }$$and define two new matrices which will be used later$$\eqalign{ P &= \sum_{k=1}^n \alpha_kX^k \\ M &= 2z(Pz-y)^T\\ }...

Only top voted, non community-wiki answers of a minimum length are eligible