Questions on the various algorithms used in linear algebra computations (matrix computations).

learn more… | top users | synonyms

10
votes
0answers
384 views

Computing the SVD factorization on C++ (using the proof of the existence of the SVD factorization)

I am doing a C++ program that computes the SVD factorization of a real matrix A without using any known library of algebra that contains the implementation. In addition, QR descomposition is not ...
9
votes
0answers
117 views

How to solve linear system of form $(A \otimes B + C^{T}C)x = b$ when $A \otimes B$ is too large to compute?

For the given linear system: $$(A \otimes B + C^{T}C)x = b$$ where $\otimes$ is the Kronecker product, $A$ and $B$ are dense and symmetric positive-definite, and $C^{T}C$ is a sparse symmetric block ...
6
votes
0answers
475 views

Simulating from a Multivariate Gaussian without Cholesky

I'd like to draw a sample from a multivariate Gaussian distribution $\mathcal{N}(\mu, \Sigma)$, where $\mu$ is the mean vector (can assume it to be $\boldsymbol{0}$), and $\Sigma$ is a sparse positive ...
5
votes
0answers
49 views

Maximizing the pairwise Frobenuis distance between M othrogonal matrices

I want to maximize the pairwise Frobenius distance between $M$ orthogonal matrices. That is, I'm looking for $Q_{i}, i = 1, 2, ... M$ such that \begin{equation*} \begin{aligned} & \underset{ 1 ...
5
votes
0answers
108 views

IEEE 754 as a mathematical space

Integer operations in computers (i.e. 32-bit integers) probably can be represented best by modular arithmetic (because of integer overflows/underflows). What about IEEE 754 floating point arithmetic? ...
5
votes
0answers
239 views

numerical linear algebra tricks for repeated sums and inversions with symmetric positive-definite matrices

I'm doing the following procedure to get the max-likelihood estimate of a matrix-variate normal distribution from $r$ samples of matrices in $\mathbb{R}^{n \times p}$ (algorithm from Dutilleul ...
5
votes
0answers
94 views

Is there a way to exploit the fact that the covariance matrix has a blocked structure to more easily compute the multivariate normal density?

I'm trying to minimize the (negative) multivariate normal log likelihood (dropping constants): $$ \log |\boldsymbol\Sigma|\,+(\mathbf{x}-\boldsymbol\mu)^{\rm ...
4
votes
0answers
110 views

Solving linear equations with block structure and weak coupling

I have a standard set of linear equations $Ax=b$ where the Hessian matrix $A$ has the special block structure as shown: $A= \begin{pmatrix} T & U\\ U^T & V \end{pmatrix}$, $x= ...
4
votes
0answers
113 views

What does affine invariance mean in the context of the Newton's method?

The textbook Numerical Solution of Boundary Value Problems for Ordinary Differential Equations (by Ascher, Mattheij, and Russell) states on page 329: [W]e observe that Newton's method is affine ...
4
votes
0answers
313 views

Optimized way to compute L1 distance matrix

I'm computing distances between two groups of multi-dimensional points giving a matrix of distances pairwise between points. For the L2 (euclidean) distance I can use optimized matrix multiplication ...
4
votes
0answers
68 views

Conditions of a Monotonic Process?

$f$ is the output of a discrete time process described by $f(k)=\sum_{i=1}^{k-1}w_{ki}f(i)$ where $f(1)\geq0$ is a known initial condition and $w_{ki}\geq0$ are weights of previous states on the ...
4
votes
0answers
822 views

General properties of eigenvalues of a Jacobian matrix when premultiplied by a symmetric, positive definite matrix?

For a particular engineering problem that I'm working on, I have computed a Jacobian matrix $J$ and there is another matrix $M$ associated with the problem. $M$ is known to be symmetric, real-valued, ...
3
votes
0answers
62 views

Invertibility of infinite order matrix

how the matrix $[e^{-(x_j-x_k)^2}]$ is invertible where $\{x_j\}$ be any real sequence such that $(x_{j+1}-x_j) >0$ for all $j \in Z$ where $Z$ denotes the set of integers.
3
votes
0answers
82 views

How do I apply this PDE as an image filter?

I'm trying to preprocess a height map image with a helmholtz-type equation as described in this paper. The equation is: $$ddx(h') + ddy(h') + y(h'-h) = 0$$ I solved for h and got: ...
3
votes
0answers
120 views

Express Lagrange polynomial in term of Cauchy matrix

Given 2n distinct real numers $s_1,s_2, \dots, s_n$ and $t_1, t_2, \dots,t_n$ define the $n \times n$ Cauchy matrix $C = C(t,s)$ by $C_{ij} = \frac{1}{t_i - s_j}$. Express the Lagrange interpolation ...
3
votes
0answers
87 views

Numerically stable method for angle between 3D vectors

I'm looking for a numerically stable method for computing the angle between two 3D vectors. Which of the following methods ought to be preferred? Method 1: $$ u\times v = ||u|| ||v|| \sin(\theta) ...
3
votes
0answers
52 views

Lagrange multiplier for more than one constraints.

How to minimize $x^TAx$ over the set $D=(x\geq 0, x^TBx=1$ and $(I-A^\dagger A)x=0$), where $A$ is copositive matrix of order $n-1$ and $B$ is strictly copositive matrix of order $n$. If I drop the ...
3
votes
0answers
64 views

Does anyone know any reference for this matrix?

For $n \geq 4$, $A$ is $(n-1) \times (n-1)$ tridiagonal block matrix $$A = n^2 \begin{bmatrix}B & -I & 0 & \cdots & \\-I & B & -I & 0 & \\ 0 & -I & B & -I ...
3
votes
0answers
220 views

Inverse of the sum of a symmetric and diagonal matrices

I have two square matrices, $A$ and $B$. $A$ is a block symmetric matrix with 1's along the diagonal (and therefore 1's in whole blocks along the diagonal). $B$ is diagonal, with the same block ...
3
votes
0answers
170 views

Difference between Householder Reflections and Gram-Schmidt?

In numerical QR decomposition, when we calculate the orthonormal factor Q of a matrix, what is the difference in results if we use Householder Reflections to normalize the matrix or use Gram-Schmidt ...
3
votes
0answers
185 views

Inverse of Sum of Matrix Inverses

Given $N$ positive-definite matrices $\Lambda_i$, I need to efficiently compute $\Gamma_N$, where $$ \Gamma_n = \left(\sum_{i=1}^n \Lambda_i^{-1}\right)^{-1}. $$ Applying the Woodbury matrix identity ...
3
votes
0answers
857 views

What is the Moore-Penrose pseudoinverse for scaled linear regression?

The matrix equation for linear regression is: $$ \vec{y} = X\vec{\beta}+\vec{\epsilon} $$ The Least Square Error solution of this forms the normal equations: $$ ({\bf{X}}^T \bf{X}) \vec{\beta}= ...
3
votes
0answers
95 views

Solver for sparse linearly-constrained non-linear least-squares

Reposted from stackoverflow on the advice of Nick Rosencrantz: Are there any algorithms or solvers for solving non-linear least-squares problems where the jacobian is known to always be sparse, and ...
3
votes
0answers
887 views

Why is Cholesky factorization numerically stable

It's often stated (eg: in Numerical Recipes in C) that Cholesky factorization is numerically stable even without column pivoting, unlike LU decomposition, which usually need pivoting schemes. But ...
3
votes
0answers
84 views

What's the state of the art for computing the largest singular value of a matrix

My matrix is not sparse, and is sized 30k by 30k. Most importantly, the gap between the largest and the second largest singular values is small or even 0. ARPACK, SLEPc, Matlab, PROPACK? Which ...
3
votes
0answers
553 views

Numerical methods to find eigenvectors with 0 eigenvalue

I'm curious if there's any numerical way of directly finding the eigenvectors with eigenvalue 0. If I didn't have to do it directly, I would probably do it like this in pseudocode: ...
2
votes
0answers
19 views

Is it possible to design a strongly stable linear multistep method of order 7 which has stiff decay

I'm studying for a test and I'd like to know is it possible to design a strongly stable linear multistep method of order 7 which has stiff decay. I have no clue to verify the claim. Can anyone give me ...
2
votes
0answers
30 views

LU Decomposition vs. QR Decomposition for similar problems

Suppose I want to solve the 2D Poisson equation with Neumann boundary conditions. The solution is non-unique up to an additive constant. I have previously asked a related question here for the 1D ...
2
votes
0answers
57 views

Eigenvalues of a sum of matrices given eigenvalues of different sum

Firstly, what I want are the eigenvalues of a sum of matrices $(A + C)$. I am not asking how to express them in terms of the eigenvalues of the summands*. What I am hoping for is that there may be ...
2
votes
0answers
23 views

Why use the logarithm of the relative error?

In my numerical analysis course, we had an assignment to use MATLAB to numerically solve the Poisson Equation $-\nabla\cdot\nabla u = 0$ in one dimension. We computed the numerical solution, plotted ...
2
votes
0answers
54 views

Faster Cholesky factorization of $diag(\mathbf{x}_{i}) \otimes \mathbf{A} + \mathbf{B}$ for $i=1,\ldots,n$?

I have many positive integer vectors $\mathbf{x}_{i} \in \mathcal{N}^{d_1}$, $i=1,\ldots,n$, a p.s.d. matrix $\mathbf{A} \in \Re^{d_2\times d_2}$ and a p.s.d. matrix $\mathbf{B} \in \Re^{d_1d_2\times ...
2
votes
0answers
25 views

Iterative methods: What happens when the spectral radius of a matrix is exactly 1?

I know that an iterative method (I'm using Jacobi and Gauss-Seidel in this case) will converge iff the spectral radius (max absolute value of eigenvalues) of its iterative matrix is strictly less than ...
2
votes
0answers
33 views

Computing Cholesky Factorisation by Hand

It is a common exam problem to compute the Cholesky factorisation of a small (typically 4x4) matrix. I know that this can be done by first finding the matrix $U$ in the $LU$-decomposition (e.g. by the ...
2
votes
0answers
81 views

Inverse of generalized arrow matrix $A = M^T * M + I$

If we have the following linear system: Ax=b And matrix A is created by multiplying a rectangular matrix with it's transpose: $A = M^T * M + I$ What is the best method to solve for x for different b ...
2
votes
0answers
35 views

skyline storage integration to Cholesky decomposition

I'm trying to develop direct solver for FEM application, solver uses Cholesky decomposition(with following code) but without skyline storage technique, so my question is 2 fold: 1)Comprising skyline ...
2
votes
0answers
28 views

Fast computation of component-wise $\exp(-XY^T)G$ for random $G$

I have the following question: Suppose I have two matrices $X,Y$ both of size $m\times p$ and a random i.i.d Gaussian matrix $G$ of size $m \times k$, $m\gg p>k$. Is there a fast way to compute ...
2
votes
0answers
26 views

Looking for matrices such that $\kappa(A) =1$

Looking clues for this problem. Find all the matrices such that $\kappa(A) = 1$ We define $\kappa(A) = \|A\|\,\|A^{-1}\|$. If I'm looking matrices such that $\kappa(A) = 1$, I was thinking in ...
2
votes
0answers
603 views

Number of Arithmetic Operations in Gaussian-elimination/Gauss-Jordan Hybrid Method for Solving Linear Systems

I am stucked at this problem from the book Numerical Analysis 8-th Edition (Burden) (Exercise 6.1.16) : Consider the following Gaussian-elimination/Gauss-Jordan hybrid method for solving linear ...
2
votes
0answers
76 views

Fastest way to find linearly independent columns of a matrix

Given a rectangular matrix $X$ of size $n\times m$ with $m>n$, what is the fastest way to find the linearly independent coloums. Robust methods like SVD or RRQR decompostion have complexity of ...
2
votes
0answers
30 views

Reference request for finite difference method

I am trying to use finite difference method to solve the minimizing problem $$ J[u]=\min_{u\in BV(Q)}\{\|u-f\|_{L^1(Q)}+|u|_{BV(Q)}\} $$ where $Q=(0,1)\times (0,1)$ is a uint square and ...
2
votes
0answers
45 views

Nearest non-negative solution for $Av=b$

Let $A$ be a $n\times m$ matrix. Let us define the system $$Av=b$$ $$v\geq 0$$ I want to find a solution $v$ of this system that is the closest (euclidean norm) to $v_0$, a given $n$-dimensional ...
2
votes
0answers
23 views

Dense Rank Deficient Linear System

What are some of the best methods for solving a Dense Rank Deficient Linear System $Ax = b$, where $A$ is Dense, Symmetric but possibly Rank Deficient. I know SVD can solve it pretty nicely while ...
2
votes
0answers
140 views

Solving a structured partitioned linear system

I am trying to solve the following partitioned linear system, where each letter represents a block $\begin{pmatrix}-H & A^T & I_n \\ A & 0_1 & 0_2 \\ z_D & 0_2^T & ...
2
votes
0answers
45 views

Wiedemann for solving sparse linear equation

I am new member. I am researching in Wiedemann algorithm to find solution $x$ of $$Ax=b$$ Firstly, I will show a Wiedemann's deterministic algorithm (Algorithm 2 in paper Compute $A^ib$ for ...
2
votes
0answers
56 views

Find the solution of linear equation using Wiedemann/ Krylov method

I am using Wiedemann (some literature called Krylov method) to find the solution of a linear equation that defined as $$Mx=b$$ Instead of resolving entire elements of x (size $K \times 1$), we can ...
2
votes
0answers
46 views

Can the Lanczos algorithm converge very fast by taking a good initial guess?

Suppose I have the two lowest eigenvectors $v_1$, $v_2$ of a matrix $M$. If slightly change $M$ to $M'$. Can I use $v_1$ or $v_2$ as an initial guess for $M'$? If so, which one should be used, $v_1$ ...
2
votes
0answers
121 views

Saddle point problem (KKT) with block-diagonal matrix

Consider the following saddle point problem originating from an interior-point method algorithm: $$ \begin{bmatrix}\mathbf{H} & \mathbf{A}^{T}\\ \mathbf{A} & \mathbf{0} ...
2
votes
0answers
71 views

Numerically approximate the maximum of an element of a vector after a series of matrix multiplications.

Where S is a sigmoidal function, A_i is a matrix, and x is an input vector, and ...
2
votes
0answers
89 views

Effective computation of matrix commutator

Is there a faster way to compute the commutator of large (at least one of them sparse) matrices $[A,B]$ then to compute $AB$ ,$BA$ and subtract them?
2
votes
0answers
96 views

Kalman Filter Predict Update of LDL Decomposition of a Covariance Matrix

From the state predict equation: http://en.wikipedia.org/wiki/Kalman_filter# $$P_{n+1}=AP_nA^T + Q$$ Suppose the $LDL^{T}$ ( http://en.wikipedia.org/wiki/Cholesky_decomposition#LDL_decomposition_2 ) ...