Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [linear-algebra]

Linear algebra is concerned with vector spaces of all dimensions and linear transformations between them, including systems of linear equations, bases, dimensions, subspaces, matrices, determinants, traces, eigenvalues and eigenvectors, diagonalization, Jordan forms, etc. For questions specifically concerning matrices, use the (matrices) tag. For questions specifically concerning matrix equations, use the (matrix-equations) tag.

585
votes
13answers
110k views

What's an intuitive way to think about the determinant?

In my linear algebra class, we just talked about determinants. So far I’ve been understanding the material okay, but now I’m very confused. I get that when the determinant is zero, the matrix doesn’t ...
325
votes
4answers
159k views

What is the intuitive relationship between SVD and PCA?

Singular value decomposition (SVD) and principal component analysis (PCA) are two eigenvalue methods used to reduce a high-dimensional dataset into fewer dimensions while retaining important ...
254
votes
26answers
56k views

If $AB = I$ then $BA = I$

If $A$ and $B$ are square matrices such that $AB = I$, where $I$ is the identity matrix, show that $BA = I$. I do not understand anything more than the following. Elementary row operations. Linear ...
246
votes
2answers
12k views

How does one prove the determinant inequality $\det\left(6(A^3+B^3+C^3)+I_{n}\right)\ge 5^n\det(A^2+B^2+C^2)$?

Let $\,A,B,C\in M_{n}(\mathbb C)\,$ be Hermitian and positive-definite matrices such that $A+B+C=I_{n},$ where $I_{n}$ is the identity matrix. Show that $$\det\left(6(A^3+B^3+C^3)+I_{n}\right)\...
223
votes
10answers
146k views

What is the importance of eigenvalues/eigenvectors?

What is the importance of eigenvalues/eigenvectors?
189
votes
6answers
12k views

Why does this matrix give the derivative of a function?

I happened to stumble upon the following matrix: $$ A = \begin{bmatrix} a & 1 \\ 0 & a \end{bmatrix} $$ And after trying a bunch of different examples, I noticed the ...
182
votes
0answers
10k views

Limit of sequence of growing matrices

Let $$ H=\left(\begin{array}{cccc} 0 & 1/2 & 0 & 1/2 \\ 1/2 & 0 & 1/2 & 0 \\ 1/2 & 0 & 0 & 1/2\\ 0 & 1/2 & 1/2 & 0 \end{array}\right), $$ $K_1=\left(\...
174
votes
4answers
43k views

Norms Induced by Inner Products and the Parallelogram Law

Let $ V $ be a normed vector space (over $\mathbb{R}$, say, for simplicity) with norm $ \lVert\cdot\rVert$. It's not hard to show that if $\lVert \cdot \rVert = \sqrt{\langle \cdot, \cdot \rangle}$ ...
173
votes
7answers
28k views

Why do we care about dual spaces?

When I first took linear algebra, we never learned about dual spaces. Today in lecture we discussed them and I understand what they are, but I don't really understand why we want to study them within ...
172
votes
6answers
8k views

How could we define the factorial of a matrix?

Suppose I have a square matrix $\mathsf{A}$ with $\det \mathsf{A}\neq 0$. How could we define the following operation? $$\mathsf{A}!$$ Maybe we could make some simple example, admitted it makes any ...
143
votes
7answers
105k views

How to intuitively understand eigenvalue and eigenvector?

I'm learning multivariate analysis and I have learnt linear algebra for two semester when I was a freshman. Eigenvalue and eigenvector is easy to calculate and the concept is not difficult to ...
137
votes
18answers
29k views

Is there another simpler method to solve this elementary school math problem?

I am teaching an elementary student. He has a homework as follows. There are $16$ students who use either bicycles or tricycles. The total number of wheels is $38$. Find the number of students ...
135
votes
1answer
9k views

Is the following matrix invertible?

$$\begin{bmatrix} 1235 &2344 &1234 &1990\\ 2124 & 4123& 1990& 3026 \\ 1230 &1234 &9095 &1230\\ 1262 &2312& 2324 &3907 \end{bmatrix}$$ Clearly, its ...
135
votes
5answers
176k views

Eigenvectors of real symmetric matrices are orthogonal

Can someone point me to a paper, or show here, why symmetric matrices have orthogonal eigenvectors? In particular, I'd like to see proof that for a symmetric matrix $A$ there exists decomposition $A = ...
125
votes
2answers
28k views

What is the geometric interpretation of the transpose?

I can follow the definition of the transpose algebraically, i.e. as a reflection of a matrix across its diagonal, or in terms of dual spaces, but I lack any sort of geometric understanding of the ...
123
votes
12answers
40k views

Why do we use the word “scalar” and not “number” in Linear Algebra?

During a year and half of studying Linear Algebra in academy, I have never questioned why we use the word "scalar" and not "number". When I started the course our professor said we would use "scalar" ...
123
votes
16answers
150k views

Where to start learning Linear Algebra? [closed]

I'm starting a very long quest to learn about math, so that I can program games. I'm mostly a corporate developer, and it's somewhat boring and non exciting. When I began my career, I chose it because ...
123
votes
1answer
108k views

Derivative of Softmax loss function

I am trying to wrap my head around back-propagation in a neural network with a Softmax classifier, which uses the Softmax function: \begin{equation} p_j = \frac{e^{o_j}}{\sum_k e^{o_k}} \end{equation}...
117
votes
6answers
27k views

Is there a quick proof as to why the vector space of $\mathbb{R}$ over $\mathbb{Q}$ is infinite-dimensional?

It would seem that one way of proving this would be to show the existence of non-algebraic numbers. Is there a simpler way to show this?
110
votes
4answers
83k views

What is the difference between linear and affine function

I am a bit confused. What is the difference between a linear and affine function? Any suggestions will be appreciated
108
votes
12answers
11k views

Is there an “inverted” dot product?

The dot product of vectors $\mathbf{a}$ and $\mathbf{b}$ is defined as: $$\mathbf{a} \cdot \mathbf{b} =\sum_{i=1}^{n}a_{i}b_{i}=a_{1}b_{1}+a_{2}b_{2}+\cdots +a_{n}b_{n}$$ What about the quantity? $$\...
103
votes
9answers
179k views

Inverse of the sum of matrices

I have two square matrices - $A$ and $B$. $A^{-1}$ is known and I want to calculate $(A+B)^{-1}$. Are there theorems that help with calculating the inverse of the sum of matrices? In general case $B^{-...
103
votes
12answers
104k views

Why study linear algebra?

Simply as the title says. I've done some research, but still haven't arrived at an answer I am satisfied with. I know the answer varies in different fields, but in general, why would someone study ...
103
votes
3answers
70k views

Why is the eigenvector of a covariance matrix equal to a principal component?

If I have a covariance matrix for a data set and I multiply it times one of it's eigenvectors. Let's say the eigenvector with the highest eigenvalue. The result is the eigenvector or a scaled ...
95
votes
18answers
39k views

What is the difference between a point and a vector?

I understand that a vector has direction and magnitude whereas a point doesn't. However, in the course notes that I am using, it is stated that a point is the same as a vector. Also, can you do ...
93
votes
6answers
98k views

Proof that the Trace of a Matrix is the sum of its Eigenvalues

I have looked extensively for a proof on the internet but all of them were too obscure. I would appreciate if someone could lay out a simple proof for this important result. Thank you.
92
votes
7answers
4k views

Is the “determinant” that shows up accidental?

Consider the class of rational functions that are the result of dividing one linear function by another: $$\frac{a + bx}{c + dx}$$ One can easily compute that, for $\displaystyle x \neq \frac cd$ $$\...
92
votes
19answers
181k views

Calculate Rotation Matrix to align Vector A to Vector B in 3d?

I have one triangle in 3d space that I am tracking in a simulation. Between time steps I have the the previous normal of the triangle and the current normal of the triangle along with both the current ...
89
votes
10answers
7k views

Why, intuitively, is the order reversed when taking the transpose of the product?

It is well known that for invertible matrices $A,B$ of the same size we have $$(AB)^{-1}=B^{-1}A^{-1} $$ and a nice way for me to remember this is the following sentence: The opposite of putting on ...
89
votes
14answers
28k views

Looking for an intuitive explanation why the row rank is equal to the column rank for a matrix

I am looking for an intuitive explanation as to why/how row rank of a matrix = column rank. I've read the proof at http://en.wikipedia.org/wiki/Rank_of_a_linear_transformation and I understand the ...
88
votes
6answers
7k views

Mathematicians' Tensors vs. Physicists' Tensors

It seems, at times, that physicists and mathematicians mean different things when they say the word "tensor." From my perspective, when I say tensor, I mean "an element of a tensor product of vector ...
86
votes
18answers
23k views

Is arrow notation for vectors “not mathematically mature”?

Assuming that we can't bold our variables (say, we're writing math as opposed to typing it), is it "not mathematically mature" to put an arrow over a vector? I ask this because in my linear algebra ...
86
votes
7answers
101k views

Show that the determinant of $A$ is equal to the product of its eigenvalues

Show that the determinant of a matrix $A$ is equal to the product of its eigenvalues $\lambda_i$. So I'm having a tough time figuring this one out. I know that I have to work with the characteristic ...
86
votes
7answers
37k views

Intuitively, what is the difference between Eigendecomposition and Singular Value Decomposition?

I'm trying to intuitively understand the difference between SVD and eigendecomposition. From my understanding, eigendecomposition seeks to describe a linear transformation as a sequence of three ...
86
votes
3answers
9k views

Why are vector spaces not isomorphic to their duals?

Assuming the axiom of choice, set $\mathbb F$ to be some field (we can assume it has characteristics $0$). I was told, by more than one person, that if $\kappa$ is an infinite cardinal then the ...
83
votes
3answers
3k views

Cute Determinant Question

I stumbled across the following problem and found it cute. Problem: We are given that $19$ divides $23028$, $31882$, $86469$, $6327$, and $61902$. Show that $19$ divides the following determinant: ...
82
votes
7answers
98k views

Physical meaning of the null space of a matrix

What is an intuitive meaning of the null space of a matrix? Why is it useful? I'm not looking for textbook definitions. My textbook gives me the definition, but I just don't "get" it. E.g.: I think ...
82
votes
2answers
29k views

GRE Subject Test - Past Papers, Books, Advice [closed]

This is not for the Maths part of the General GRE. This is for the GRE Subject Test in Maths. Feel free to add or comment. How do I know the definition of rings or of anything on the GRE given that ...
81
votes
3answers
16k views

Why, historically, do we multiply matrices as we do?

Multiplication of matrices — taking the dot product of the $i$th row of the first matrix and the $j$th column of the second to yield the $ij$th entry of the product — is not a very ...
78
votes
10answers
5k views

Why does Friedberg say that the role of the determinant is less central than in former times?

I am taking a proof-based introductory course to Linear Algebra as an undergrad student of Mathematics and Computer Science. The author of my textbook (Friedberg's Linear Algebra, 4th Edition) says in ...
78
votes
8answers
277k views

What does it mean to have a determinant equal to zero?

After looking in my book for a couple of hours, I'm still confused about what it means for a $(n\times n)$-matrix $A$ to have a determinant equal to zero, $\det(A)=0$. I hope someone can explain this ...
77
votes
7answers
9k views

What is “Bra” and “Ket” notation and how does it relate to Hilbert spaces?

This is my first semester of quantum mechanics and higher mathematics and I am completely lost. I have tried to find help at my university, browsed similar questions on this site, looked at my ...
75
votes
3answers
16k views

Why did no student correctly find a pair of $2\times 2$ matrices with the same determinant and trace that are not similar?

I gave the following problem to students: Two $n\times n$ matrices $A$ and $B$ are similar if there exists a nonsingular matrix $P$ such that $A=P^{-1}BP$. Prove that if $A$ and $B$ are two ...
75
votes
9answers
116k views

When is matrix multiplication commutative?

I know that matrix multiplication in general is not commutative. So, in general: $A, B \in \mathbb{R}^{n \times n}: A \cdot B \neq B \cdot A$ But for some matrices, this equations holds, e.g. A = ...
73
votes
7answers
105k views

Is the inverse of a symmetric matrix also symmetric?

Let $A$ be a symmetric invertible matrix, $A^T=A$, $A^{-1}A = A A^{-1} = I$ Can it be shown that $A^{-1}$ is also symmetric? I seem to remember a proof similar to this from my linear algebra class, ...
72
votes
1answer
46k views

Simultaneously Diagonalizable Proof

Two $n\times n$ matrices $A, B$ are said to be simultaneously diagonalizable if there is a nonsingular matrix $S$ such that both $S^{-1}AS$ and $S^{-1}BS$ are diagonal matrices. a.) Show that ...
72
votes
7answers
6k views

What exactly are eigen-things?

Wikipedia defines an eigenvector like this: An eigenvector of a square matrix is a non-zero vector that, when multiplied by the matrix, yields a vector that differs from the original vector at most ...
72
votes
5answers
6k views

linear algebra over a division ring vs. over a field

When I was studying linear algebra in the first year, from what I remember, vector spaces were always defined over a field, which was in every single concrete example equal to either $\mathbb{R}$ or $\...
71
votes
6answers
79k views

Union of two vector subspaces not a subspace?

I'm having a difficult time understanding this statement. Can someone please explain with a concrete example?
71
votes
7answers
7k views

Is linear algebra more “fully understood” than other maths disciplines?

In a recent question, it was discussed how LA is a foundation to other branches of mathematics, be they pure or applied. One answer argued that linear problems are fully understood, and hence a ...