Questions on the various algorithms used in linear algebra computations (matrix computations).

learn more… | top users | synonyms

2
votes
3answers
62 views

What is the significance of reversing the polarity of the negative eigenvalues of a symmetric matrix?

Consider a full rank $n\times n$ symmetric matrix $A$ (coming from a set of physical measurements). I do an eigendecomposition of this matrix as $$A = E V E^T$$ Most of the eigenvalues are positive, ...
5
votes
1answer
41 views

Numerical Calculation of Eigenvalues of a large real Symmetric tridiagonal matrix

If I have an $N \times N$ matrix where every entry is zero except for along the super-diagonal and sub-diagonal, where the each entry is the conjugate of the last, like the following $5 \times 5$ ...
2
votes
1answer
860 views

How to find the Householder transformation?

Assume $x=(1,0,4,6,3,4)^T$. Find a Householder transformation and a positive number $\alpha$ such that $Hx=(1,\alpha,4,6,0,0)^T$. I'm sorry that I don't know how to start with this problem. A ...
0
votes
1answer
31 views

Household reflector or transformation

Let $A\in\mathbb{R}^{n\times k}$, $n\geq k$, and $rank(A) = k$. Consider the use of Household reflectors, $H_i$, $1\leq i\leq k$, to transform $A$ to upper trapezoidal form, i.e., ...
0
votes
0answers
24 views

Question on SVD Uniqueness Proof

I have problem understanding the proof of uniqueness for SVD by by Trefethen & Bau. If the lengths of the semi-axes of the hyper-ellipse are distinct, then semi-axes themselves are ...
0
votes
0answers
40 views

Linear system equations

I need to get, preferably by a numerical method, a solutions of: $$\left\{\begin{array}{lll} 2\sum_{i=1}^n b_{ik}x_i+x_{n+1}=0&\text{for}& k=1,2\ldots,n\\ \sum_{i=1}^n x_i=0 ...
0
votes
1answer
15 views

Computing $PAQ = LU$ using Gaussian elimination with complete pivoting

Suppose $PAQ = LU$ is computed via Gaussian elimination with complete pivoting. Show that there is no element in $e_i^{T}U$ i.e., row $i$ of $U$, whose magnitude is larger than $|\mu_{ii}| = ...
0
votes
1answer
24 views

[Part of a system of linear equations!]: Find $B$ such that $A = B\times C$, but $C\times C'$ is non-invertable

I have the following Equation: $A = B\times C$ $A$ is a $(N\times 1)$ Known Matrix $B$ is a $(N\times M)$ Unknown Matrix, where $N>M$ $C$ is a $(M\times 1)$ Known Matrix $C\times C'$ is a ...
0
votes
1answer
25 views

Vector norm lemma and proof

I have a question from Numerical linear algebra book by Trefethen & Bau : Let $\|\cdot\|$ denote any norm on $C^m$. The corresponding dual norm $\|\cdot\|'$ is defined by the formula ...
0
votes
1answer
18 views

Elementary reflector $Q$ is orthogonal iff

Recall that an elementary reflector has the form $Q = I + \alpha xx^T\in\mathbb{R}^{n\times n}$ with $\|x\|_{2}\neq 0$. Show that $Q$ is orthogonal iff $$\alpha = \frac{-2}{x^Tx} \ \ \text{or} \ \ ...
0
votes
0answers
20 views

Query about the Moore Penrose pseudoinverse method

I have recently discovered the Moore-Penrose psuedoinverse method, and I am currently testing the waters with it. I noticed if I have a system, say $$a_1x_1=0$$ $$a_2x_1+a_3x_2=0$$ $$\vdots$$ ...
2
votes
4answers
805 views

Non-monotonic decrease of residuals in Conjugate Gradients:

In some of my numerical programming using conjugate gradient solvers, I noticed an alarming problem: The residuals were not monotonically decreasing to zero, but were sometimes increasing. In this ...
1
vote
0answers
45 views

Dual Norm proof

Let $\|.\|$ denote any norm on $C^m$. The corresponding dual norm $\|.\|'$ is defined by the formula $\|x\|' = sup_{\|y\|=1}|y^*x|$. (a)Prove that $\|.\|'$ is a norm? (b) Let $x, y \in C^m $ with ...
1
vote
1answer
45 views

If the determinant of a matrix goes to infinity, does it means it has no inverse?

Context I have a linear time-invariant (single-input, single-output) system in state space representation (https://en.wikipedia.org/wiki/State-space_representation#Linear_systems): $$ \mathbf{x'}(t) ...
0
votes
0answers
21 views

Schur complement of a matrix $A$

Let $A\in\mathbb{R}^{n\times n}$ and its inverse be partitioned $$A = \begin{pmatrix} A_{11} & A_{12}\\ A_{21} & A_{22}\\ \end{pmatrix},\:\: A^{-1} = \begin{pmatrix} \tilde{A_{22}} & ...
0
votes
1answer
45 views

$\kappa(B^{-1}A)=\kappa (AB^{-1}) = \frac{\lambda_{\max}(B^{-1}A)}{\lambda_{\min}(B^{-1}A)}$

Prove or disprove: if $A,B$ are symmetric positive definite (s.p.d.) matrices then operator 2-norm condition number $\kappa(B^{-1}A)=\kappa (AB^{-1}) = ...
0
votes
1answer
18 views

Matlab algorithm for non-orthogonal diagonalization of symmetric matrices

I need to find a basis in which the symmetric bilinear form given by the n x n symmetric matrix which has 2's along the diagonal and 1's everywhere else becomes the identity. That is, if S denotes ...
0
votes
0answers
40 views

Numerical method for solving equation with $u \frac{\mathrm{d}u}{\mathrm{d}x} + u$

I'm looking for a finite difference method to solve $$a(x) u \frac{\mathrm{d}u}{\mathrm{d}x} + u = b(x)$$ where $u(0) = c$. I tried to do a lagging convergence on the $u$ ie $$a(x) u^{(n)} ...
0
votes
0answers
17 views

Gauss Seidel - Finite Element Method

I am solving an equation using finite element method, and for that I have to use Gauss Seidel to invert a matrix. In Gauss Seidel I am using a "while" which breaks if the absolute error reaches the ...
0
votes
1answer
15 views

Gauss transforms to factor $A = LU$

Consider a symmetric matrix $A$, i.e., $A = A^{T}$. Consider the use of Gauss transforms to factor $A = LU$ where $L$ is unit lower triangular and $U$ is upper triangular. You may assume that the ...
3
votes
1answer
57 views

$LU$ Factorization

Suppose the $A\in\mathbb{R}^{n\times n}$ is nonsingular and that $A = LU$ is its $LU$ factorization. Give an algorithm that can compute, $e_i^TA^{-1}e_j$,i.e., the $(i,j)$ element of $A^{-1}$ in ...
0
votes
1answer
23 views

Write column form elementary matrix in terms of element form elementary matrices

Recall that any unit lower triangular matrix $L\in\mathbb{R}^{n\times n}$ can be written in factored form as \begin{equation} L = M_1 M_2\ldots M_{n-1} \end{equation} where $M_i = I + l_i ...
1
vote
1answer
31 views

Numerical Range of A and A transpose.

I was playing around with the numerical range [NR] (or field of value) of a matrix $A \in \mathbb{C}^{n\times n}$ lately. And was actually looking for a proof to show: \begin{equation} A=A^H : F(A) = ...
2
votes
2answers
508 views

Space spanned by matrices

I have a set of $5$ by $5 $matrices, $M_1,M_2,...,M_{19} ,M_{20}$. I want to try to find a basis from this set and also to find relationships between these matrices. This is how I think I should ...
0
votes
1answer
27 views

Condition number for each variable

Condition number of a matrix tells us how viable it is to solve $Ax=b$ $$A= \begin{bmatrix} 1.001&1\\ 1&1 \end{bmatrix} $$ Is a matrix that would be difficult to solve numerically. However ...
0
votes
0answers
10 views

Determine an algorithm for $LU$ factorization and determine the number of operations [duplicate]

Suppose that $A\in\mathbb{R}^{n\times n}$ is a nonsingular matrix and that $A = LU$ is its $LU$ factorization. Give an algorithm that can compute, $e_i^{T}A^{-1}e_j$, i.e., the $(i,j)$ elements of ...
0
votes
0answers
7 views

Determine an efficient algorithm and describe the computational/storage complexity

Recall that a unit lower triangular matrix $L\in\mathbb{R}^{n\times n}$ is a lower triangular matrix with diagonal elements $e_i^{T}L e_i = \lambda_{ii} = 1$. An elementary unit lower triangular ...
1
vote
1answer
24 views

Numerical solution of heat equation on periodic domain

Consider the steady heat equation $\nabla\cdot(k(x) \nabla u)=f$ in two dimensions on a periodic domain, say $[0,1]\times[0,1]$. My goal is to solve it numerically with standard central 5-points ...
1
vote
1answer
26 views

Gaussin Elimination preserves S.P.D.

Let $A \in \mathbb{R}^{n \times n} $ be symmetric positive definite with positive diagonal entries. I'm trying to show that at each step $m$ of gaussian elimination $$ a^{(m+1)}_{ij} = a^{(m)}_{i,j} ...
0
votes
0answers
16 views

Different ways to leave linearly dependent vectors of a set of vectors

Let a set $S=\left\{ {{\mathbf{v}}_{i}}:i\in \mathbb{Z}_{n}^{+} \right\}$, where $\mathbb{Z}_{n}^{+}=\left\{ 1,2,...,n \right\}$ and ${{\mathbf{v}}_{i}}\in {{\mathbb{R}}^{m}}$ for each $i\in ...
0
votes
0answers
23 views

absolute value matrix and derivation of A^1 b

I have a question who could I solve the following sentence? Given is the vector $\vec{b} \in \mathbb{R^n}$ and the function $f : GL(n,\mathbb{R}) \to \mathbb{R}^n$ with $f(A) = A^{-1}b$. Then ...
4
votes
1answer
111 views

Numerically stable method for angle between 3D vectors

I'm looking for a numerically stable method for computing the angle between two 3D vectors. Which of the following methods ought to be preferred? Method 1: $$ u\times v = ||u||~||v|| \sin(\theta) ...
1
vote
0answers
14 views

why does matrix balancing improve the linear systems condition number of the eigenvector matrix?

Matrix balancing or diagonal scaling, where at each iteration we choose a diagonal matrix so that the row and column norms are approximately equal (Osborne, 1960, Parlett and Reinsch, 1969, many ...
0
votes
1answer
33 views

Matrix-free conjugate gradient

In the conjugate gradient method for solving $Ax = b$, to update the search direction $p$ you would need to evaluate the matrix-vector product $Ap$, i.e. making sure that each search direction are ...
2
votes
2answers
48 views

If $A\in \mathbb{R}^n$ is symmetric and satisfies [the following] then $A$ is positive definite.

The following being: $$A(i,i) >\sum_{j\ne i} |A(i,j)| \quad \text{for} \quad i=1,2,...,n $$ How can I prove this?
2
votes
1answer
1k views

Gauss-Seidel method convergence algorithm

From Wikipedia: The convergence properties of the Gauss–Seidel method are dependent on the matrix A. Namely, the procedure is known to converge if either: ...
1
vote
1answer
52 views

Please explain the last step of this newton method for system of equations

The step of working out x$^1$. I know the above is the formula but do they actually work out the inverse of the derivative matrix, is there a quicker way to do this?
0
votes
1answer
20 views

Prove These Properties of the Pseudoinverse

Prove these properties of the pseudo inverse: 1) $(AA^*)^{\dagger}={A^{\dagger}}^*A^{\dagger}$; 2) $A^{\dagger}=A^*(AA^*)^{\dagger}$. I'm quite sure I need to use the four properties of the pseudo ...
1
vote
0answers
5 views

Fast square of a row-stochastic matrix

I would like to implement the square $M^2$ of a row-stochastic matrix $M$. Running time is critical. Are there any known algorithms that exploit the special nature of $M$ and are faster than the usual ...
2
votes
0answers
97 views

Kalman Filter Predict Update of LDL Decomposition of a Covariance Matrix

From the state predict equation: http://en.wikipedia.org/wiki/Kalman_filter# $$P_{n+1}=AP_nA^T + Q$$ Suppose the $LDL^{T}$ ( http://en.wikipedia.org/wiki/Cholesky_decomposition#LDL_decomposition_2 ) ...
0
votes
1answer
407 views

How do I find transformation matrix with respect to given basis in the domain and/or the codomain, given the transformation in the standard basis?

I´m being given a linear transformation, for which I can find the standard basis in the domain and codomain; but then, the book ask to find the associated matrix related to a new basis for the ...
2
votes
1answer
40 views

Prove that if $\|A\|<1$, then $\|(I-A)^{-1}\|\geq {1\over1+\|A\|}$.

Prove that if $\|A\|<1$, then $\|(I-A)^{-1}\|\geq {1\over1+\|A\|}$. I'm not sure how to prove this result. I see feel like a geometric series is involved though. Any solutions or hints are ...
0
votes
5answers
85 views

Why, if a matrix $Q$ is orthogonal, then $Q^T Q = I$?

I was looking at the definition of an orthogonal matrix, which is as follows: Square matrix $Q$ is orthogonal if its columns are pairwise orthonormal, i.e., $$Q^TQ = I$$ Hence also ...
0
votes
1answer
17 views

Bounds on a quadratic form

I am currently in the middle of a proof where it would be nice to have some estimates on the size of a quadratic form. In particular, I am looking at $$x^TAx$$ where $A$ is "small" (in the analyst's ...
0
votes
1answer
21 views

Columns of a matrix linearly independent and spans

Let $\mathbf v_1, \mathbf v_2, ...,\mathbf v_n \in \Bbb R^n$ and let $P$ be the $n\times n$ matrix whose columns are $\mathbf v_1, \mathbf v_2, ...,\mathbf v_n$ I'm wondering why the followings are ...
0
votes
1answer
15 views

Condition number of preconditioned system

Suppose we are solving an ill-conditioned system $Ax = b$, and we are trying to solve it using preconditioned technique. Given $\kappa (T)\approx \kappa(A)$, where $\kappa(A)$ is condition number of ...
3
votes
2answers
66 views

Efficiently solving many sets of linear equations without inversion or factorization

Suppose I have the normal set of linear equations $Ax = b$. If I can store and manipulate $A$ I have a variety of techniques available to me such as inversion, factorization, or an iterative method. ...
0
votes
1answer
14 views

Reference that Explains Preconditioning

I would like to understand Preconditioning techniques and why they work. Could someone provide a good reference for this type of information?
0
votes
0answers
24 views

Compute the condition number of the matrix and show for what $\Delta x$ it is singular

Given the laplacian $N \times N$ matrix \begin{align*} A=\frac{1}{(\Delta x)^2}\begin{pmatrix} 2&-1& & &\\ -1&2&-1& &\\ &\ddots&\ddots&\ddots&\\ ...
3
votes
1answer
56 views

Detecting singular system during Cholesky resolution

I am solving small linear systems with a symmetric positive matrix by the method of Cholesky, without pivoting. "Bad" matrices are detected when you take the square root of a diagonal element, which ...