Questions tagged [linear-algebra]

Linear algebra is concerned with vector spaces of all dimensions and linear transformations between them, including systems of linear equations, bases, dimensions, subspaces, matrices, determinants, traces, eigenvalues and eigenvectors, diagonalization, Jordan forms, etc. For questions specifically concerning matrices, use the (matrices) tag. For questions specifically concerning matrix equations, use the (matrix-equations) tag.

Filter by
Sorted by
Tagged with
609
votes
13answers
115k views

What's an intuitive way to think about the determinant?

In my linear algebra class, we just talked about determinants. So far I’ve been understanding the material okay, but now I’m very confused. I get that when the determinant is zero, the matrix doesn’t ...
337
votes
4answers
168k views

What is the intuitive relationship between SVD and PCA?

Singular value decomposition (SVD) and principal component analysis (PCA) are two eigenvalue methods used to reduce a high-dimensional dataset into fewer dimensions while retaining important ...
266
votes
26answers
60k views

If $AB = I$ then $BA = I$

If $A$ and $B$ are square matrices such that $AB = I$, where $I$ is the identity matrix, show that $BA = I$. I do not understand anything more than the following. Elementary row operations. Linear ...
246
votes
2answers
13k views

How does one prove the determinant inequality $\det\left(6(A^3+B^3+C^3)+I_{n}\right)\ge 5^n\det(A^2+B^2+C^2)$?

Let $\,A,B,C\in M_{n}(\mathbb C)\,$ be Hermitian and positive-definite matrices such that $A+B+C=I_{n},$ where $I_{n}$ is the identity matrix. Show that $$\det\left(6(A^3+B^3+C^3)+I_{n}\right)\...
238
votes
10answers
153k views

What is the importance of eigenvalues/eigenvectors?

What is the importance of eigenvalues/eigenvectors?
201
votes
0answers
11k views

Limit of sequence of growing matrices

Let $$ H=\left(\begin{array}{cccc} 0 & 1/2 & 0 & 1/2 \\ 1/2 & 0 & 1/2 & 0 \\ 1/2 & 0 & 0 & 1/2\\ 0 & 1/2 & 1/2 & 0 \end{array}\right), $$ $K_1=\left(\...
191
votes
6answers
13k views

Why does this matrix give the derivative of a function?

I happened to stumble upon the following matrix: $$ A = \begin{bmatrix} a & 1 \\ 0 & a \end{bmatrix} $$ And after trying a bunch of different examples, I noticed the ...
183
votes
4answers
46k views

Norms Induced by Inner Products and the Parallelogram Law

Let $ V $ be a normed vector space (over $\mathbb{R}$, say, for simplicity) with norm $ \lVert\cdot\rVert$. It's not hard to show that if $\lVert \cdot \rVert = \sqrt{\langle \cdot, \cdot \rangle}$ ...
181
votes
7answers
31k views

Why do we care about dual spaces?

When I first took linear algebra, we never learned about dual spaces. Today in lecture we discussed them and I understand what they are, but I don't really understand why we want to study them within ...
175
votes
6answers
9k views

How could we define the factorial of a matrix?

Suppose I have a square matrix $\mathsf{A}$ with $\det \mathsf{A}\neq 0$. How could we define the following operation? $$\mathsf{A}!$$ Maybe we could make some simple example, admitted it makes any ...
153
votes
7answers
110k views

How to intuitively understand eigenvalue and eigenvector?

I'm learning multivariate analysis and I have learnt linear algebra for two semester when I was a freshman. Eigenvalue and eigenvector is easy to calculate and the concept is not difficult to ...
149
votes
6answers
188k views

Eigenvectors of real symmetric matrices are orthogonal

Can someone point me to a paper, or show here, why symmetric matrices have orthogonal eigenvectors? In particular, I'd like to see proof that for a symmetric matrix $A$ there exists decomposition $A = ...
137
votes
1answer
10k views

Is the following matrix invertible?

$$\begin{bmatrix} 1235 &2344 &1234 &1990\\ 2124 & 4123& 1990& 3026 \\ 1230 &1234 &9095 &1230\\ 1262 &2312& 2324 &3907 \end{bmatrix}$$ Clearly, its ...
132
votes
4answers
90k views

What is the difference between linear and affine function

I am a bit confused. What is the difference between a linear and affine function? Any suggestions will be appreciated
132
votes
2answers
30k views

What is the geometric interpretation of the transpose?

I can follow the definition of the transpose algebraically, i.e. as a reflection of a matrix across its diagonal, or in terms of dual spaces, but I lack any sort of geometric understanding of the ...
128
votes
6answers
29k views

Is there a quick proof as to why the vector space of $\mathbb{R}$ over $\mathbb{Q}$ is infinite-dimensional?

It would seem that one way of proving this would be to show the existence of non-algebraic numbers. Is there a simpler way to show this?
127
votes
1answer
116k views

Derivative of Softmax loss function

I am trying to wrap my head around back-propagation in a neural network with a Softmax classifier, which uses the Softmax function: \begin{equation} p_j = \frac{e^{o_j}}{\sum_k e^{o_k}} \end{equation}...
125
votes
12answers
42k views

Why do we use the word “scalar” and not “number” in Linear Algebra?

During a year and half of studying Linear Algebra in academy, I have never questioned why we use the word "scalar" and not "number". When I started the course our professor said we would use "scalar" ...
125
votes
16answers
158k views

Where to start learning Linear Algebra? [closed]

I'm starting a very long quest to learn about math, so that I can program games. I'm mostly a corporate developer, and it's somewhat boring and non exciting. When I began my career, I chose it because ...
110
votes
10answers
191k views

Inverse of the sum of matrices

I have two square matrices - $A$ and $B$. $A^{-1}$ is known and I want to calculate $(A+B)^{-1}$. Are there theorems that help with calculating the inverse of the sum of matrices? In general case $B^{-...
109
votes
12answers
11k views

Is there an “inverted” dot product?

The dot product of vectors $\mathbf{a}$ and $\mathbf{b}$ is defined as: $$\mathbf{a} \cdot \mathbf{b} =\sum_{i=1}^{n}a_{i}b_{i}=a_{1}b_{1}+a_{2}b_{2}+\cdots +a_{n}b_{n}$$ What about the quantity? $$\...
108
votes
7answers
107k views

Proof that the Trace of a Matrix is the sum of its Eigenvalues

I have looked extensively for a proof on the internet but all of them were too obscure. I would appreciate if someone could lay out a simple proof for this important result. Thank you.
107
votes
3answers
74k views

Why is the eigenvector of a covariance matrix equal to a principal component?

If I have a covariance matrix for a data set and I multiply it times one of it's eigenvectors. Let's say the eigenvector with the highest eigenvalue. The result is the eigenvector or a scaled ...
105
votes
13answers
108k views

Why study linear algebra?

Simply as the title says. I've done some research, but still haven't arrived at an answer I am satisfied with. I know the answer varies in different fields, but in general, why would someone study ...
104
votes
18answers
41k views

What is the difference between a point and a vector?

I understand that a vector has direction and magnitude whereas a point doesn't. However, in the course notes that I am using, it is stated that a point is the same as a vector. Also, can you do ...
104
votes
20answers
198k views

Calculate Rotation Matrix to align Vector A to Vector B in 3d?

I have one triangle in 3d space that I am tracking in a simulation. Between time steps I have the the previous normal of the triangle and the current normal of the triangle along with both the current ...
96
votes
14answers
7k views

Why, intuitively, is the order reversed when taking the transpose of the product?

It is well known that for invertible matrices $A,B$ of the same size we have $$(AB)^{-1}=B^{-1}A^{-1} $$ and a nice way for me to remember this is the following sentence: The opposite of putting on ...
96
votes
14answers
30k views

Looking for an intuitive explanation why the row rank is equal to the column rank for a matrix

I am looking for an intuitive explanation as to why/how row rank of a matrix = column rank. I've read the proof at http://en.wikipedia.org/wiki/Rank_of_a_linear_transformation and I understand the ...
95
votes
7answers
42k views

Intuitively, what is the difference between Eigendecomposition and Singular Value Decomposition?

I'm trying to intuitively understand the difference between SVD and eigendecomposition. From my understanding, eigendecomposition seeks to describe a linear transformation as a sequence of three ...
93
votes
7answers
5k views

Is the “determinant” that shows up accidental?

Consider the class of rational functions that are the result of dividing one linear function by another: $$\frac{a + bx}{c + dx}$$ One can easily compute that, for $\displaystyle x \neq \frac cd$ $$\...
90
votes
6answers
8k views

Mathematicians' Tensors vs. Physicists' Tensors

It seems, at times, that physicists and mathematicians mean different things when they say the word "tensor." From my perspective, when I say tensor, I mean "an element of a tensor product of vector ...
90
votes
7answers
108k views

Show that the determinant of $A$ is equal to the product of its eigenvalues

Show that the determinant of a matrix $A$ is equal to the product of its eigenvalues $\lambda_i$. So I'm having a tough time figuring this one out. I know that I have to work with the characteristic ...
89
votes
7answers
111k views

Physical meaning of the null space of a matrix

What is an intuitive meaning of the null space of a matrix? Why is it useful? I'm not looking for textbook definitions. My textbook gives me the definition, but I just don't "get" it. E.g.: I think ...
89
votes
3answers
9k views

Why are vector spaces not isomorphic to their duals?

Assuming the axiom of choice, set $\mathbb F$ to be some field (we can assume it has characteristics $0$). I was told, by more than one person, that if $\kappa$ is an infinite cardinal then the ...
87
votes
18answers
24k views

Is arrow notation for vectors “not mathematically mature”?

Assuming that we can't bold our variables (say, we're writing math as opposed to typing it), is it "not mathematically mature" to put an arrow over a vector? I ask this because in my linear algebra ...
85
votes
2answers
34k views

GRE Subject Test - Past Papers, Books, Advice [closed]

This is not for the Maths part of the General GRE. This is for the GRE Subject Test in Maths. Feel free to add or comment. How do I know the definition of rings or of anything on the GRE given that ...
84
votes
3answers
4k views

Cute Determinant Question

I stumbled across the following problem and found it cute. Problem: We are given that $19$ divides $23028$, $31882$, $86469$, $6327$, and $61902$. Show that $19$ divides the following determinant: ...
84
votes
3answers
18k views

Why, historically, do we multiply matrices as we do?

Multiplication of matrices — taking the dot product of the $i$th row of the first matrix and the $j$th column of the second to yield the $ij$th entry of the product — is not a very ...
82
votes
9answers
297k views

What does it mean to have a determinant equal to zero?

After looking in my book for a couple of hours, I'm still confused about what it means for a $(n\times n)$-matrix $A$ to have a determinant equal to zero, $\det(A)=0$. I hope someone can explain this ...
79
votes
1answer
50k views

Simultaneously Diagonalizable Proof

Two $n\times n$ matrices $A, B$ are said to be simultaneously diagonalizable if there is a nonsingular matrix $S$ such that both $S^{-1}AS$ and $S^{-1}BS$ are diagonal matrices. a.) Show that ...
78
votes
3answers
17k views

Why did no student correctly find a pair of $2\times 2$ matrices with the same determinant and trace that are not similar?

I gave the following problem to students: Two $n\times n$ matrices $A$ and $B$ are similar if there exists a nonsingular matrix $P$ such that $A=P^{-1}BP$. Prove that if $A$ and $B$ are two ...
78
votes
10answers
5k views

Why does Friedberg say that the role of the determinant is less central than in former times?

I am taking a proof-based introductory course to Linear Algebra as an undergrad student of Mathematics and Computer Science. The author of my textbook (Friedberg's Linear Algebra, 4th Edition) says in ...
78
votes
9answers
121k views

When is matrix multiplication commutative?

I know that matrix multiplication in general is not commutative. So, in general: $A, B \in \mathbb{R}^{n \times n}: A \cdot B \neq B \cdot A$ But for some matrices, this equations holds, e.g. A = ...
77
votes
7answers
9k views

What is “Bra” and “Ket” notation and how does it relate to Hilbert spaces?

This is my first semester of quantum mechanics and higher mathematics and I am completely lost. I have tried to find help at my university, browsed similar questions on this site, looked at my ...
76
votes
9answers
112k views

Is the inverse of a symmetric matrix also symmetric?

Let $A$ be a symmetric invertible matrix, $A^T=A$, $A^{-1}A = A A^{-1} = I$ Can it be shown that $A^{-1}$ is also symmetric? I seem to remember a proof similar to this from my linear algebra class, ...
76
votes
4answers
48k views

Importance of matrix rank

What is the importance of the rank of a matrix? I know that the rank of a matrix is the number of linearly independent rows or columns (whichever is smaller). Why is it a problem if a matrix is rank ...
73
votes
6answers
84k views

Union of two vector subspaces not a subspace?

I'm having a difficult time understanding this statement. Can someone please explain with a concrete example?
72
votes
9answers
13k views

In Linear Algebra, what is a vector?

I understand that a vector space is a collection of vectors that can be added and scalar multiplied and satisfies the 8 axioms, however, I do not know what a vector is. I know in physics a vector is ...
72
votes
7answers
6k views

What exactly are eigen-things?

Wikipedia defines an eigenvector like this: An eigenvector of a square matrix is a non-zero vector that, when multiplied by the matrix, yields a vector that differs from the original vector at most ...
72
votes
5answers
7k views

linear algebra over a division ring vs. over a field

When I was studying linear algebra in the first year, from what I remember, vector spaces were always defined over a field, which was in every single concrete example equal to either $\mathbb{R}$ or $\...