Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [linear-algebra]

Linear algebra is concerned with vector spaces of all dimensions and linear transformations between them, including systems of linear equations, bases, dimensions, subspaces, matrices, determinants, traces, eigenvalues and eigenvectors, diagonalization, Jordan forms, etc. For questions specifically concerning matrices, use the (matrices) tag. For questions specifically concerning matrix equations, use the (matrix-equations) tag.

573
votes
12answers
107k views

What's an intuitive way to think about the determinant?

In my linear algebra class, we just talked about determinants. So far I’ve been understanding the material okay, but now I’m very confused. I get that when the determinant is zero, the matrix doesn’t ...
318
votes
4answers
154k views

What is the intuitive relationship between SVD and PCA?

Singular value decomposition (SVD) and principal component analysis (PCA) are two eigenvalue methods used to reduce a high-dimensional dataset into fewer dimensions while retaining important ...
251
votes
25answers
53k views

If $AB = I$ then $BA = I$

If $A$ and $B$ are square matrices such that $AB = I$, where $I$ is the identity matrix, show that $BA = I$. I do not understand anything more than the following. Elementary row operations. Linear ...
246
votes
2answers
12k views

How does one prove the determinant inequality $\det\left(6(A^3+B^3+C^3)+I_{n}\right)\ge 5^n\det(A^2+B^2+C^2)$?

Let $\,A,B,C\in M_{n}(\mathbb C)\,$ be Hermitian and positive-definite matrices such that $A+B+C=I_{n},$ where $I_{n}$ is the identity matrix. Show that $$\det\left(6(A^3+B^3+C^3)+I_{n}\right)\...
218
votes
10answers
142k views

What is the importance of eigenvalues/eigenvectors?

What is the importance of eigenvalues/eigenvectors?
186
votes
6answers
12k views

Why does this matrix give the derivative of a function?

I happened to stumble upon the following matrix: $$ A = \begin{bmatrix} a & 1 \\ 0 & a \end{bmatrix} $$ And after trying a bunch of different examples, I noticed the ...
180
votes
0answers
10k views

Limit of sequence of growing matrices

Let $$ H=\left(\begin{array}{cccc} 0 & 1/2 & 0 & 1/2 \\ 1/2 & 0 & 1/2 & 0 \\ 1/2 & 0 & 0 & 1/2\\ 0 & 1/2 & 1/2 & 0 \end{array}\right), $$ $K_1=\left(\...
171
votes
4answers
42k views

Norms Induced by Inner Products and the Parallelogram Law

Let $ V $ be a normed vector space (over $\mathbb{R}$, say, for simplicity) with norm $ \lVert\cdot\rVert$. It's not hard to show that if $\lVert \cdot \rVert = \sqrt{\langle \cdot, \cdot \rangle}$ ...
170
votes
6answers
8k views

How could we define the factorial of a matrix?

Suppose I have a square matrix $\mathsf{A}$ with $\det \mathsf{A}\neq 0$. How could we define the following operation? $$\mathsf{A}!$$ Maybe we could make some simple example, admitted it makes any ...
169
votes
7answers
27k views

Why do we care about dual spaces?

When I first took linear algebra, we never learned about dual spaces. Today in lecture we discussed them and I understand what they are, but I don't really understand why we want to study them within ...
139
votes
6answers
102k views

How to intuitively understand eigenvalue and eigenvector?

I'm learning multivariate analysis and I have learnt linear algebra for two semester when I was a freshman. Eigenvalue and eigenvector is easy to calculate and the concept is not difficult to ...
135
votes
18answers
28k views

Is there another simpler method to solve this elementary school math problem?

I am teaching an elementary student. He has a homework as follows. There are $16$ students who use either bicycles or tricycles. The total number of wheels is $38$. Find the number of students ...
134
votes
1answer
9k views

Is the following matrix invertible?

$$\begin{bmatrix} 1235 &2344 &1234 &1990\\ 2124 & 4123& 1990& 3026 \\ 1230 &1234 &9095 &1230\\ 1262 &2312& 2324 &3907 \end{bmatrix}$$ Clearly, its ...
130
votes
4answers
170k views

Eigenvectors of real symmetric matrices are orthogonal

Can someone point me to a paper, or show here, why symmetric matrices have orthogonal eigenvectors? In particular, I'd like to see proof that for a symmetric matrix $A$ there exists decomposition $A = ...
123
votes
12answers
38k views

Why do we use the word “scalar” and not “number” in Linear Algebra?

During a year and half of studying Linear Algebra in academy, I have never questioned why we use the word "scalar" and not "number". When I started the course our professor said we would use "scalar" ...
122
votes
16answers
146k views

Where to start learning Linear Algebra? [closed]

I'm starting a very long quest to learn about math, so that I can program games. I'm mostly a corporate developer, and it's somewhat boring and non exciting. When I began my career, I chose it because ...
120
votes
2answers
27k views

What is the geometric interpretation of the transpose?

I can follow the definition of the transpose algebraically, i.e. as a reflection of a matrix across its diagonal, or in terms of dual spaces, but I lack any sort of geometric understanding of the ...
118
votes
1answer
102k views

Derivative of Softmax loss function

I am trying to wrap my head around back-propagation in a neural network with a Softmax classifier, which uses the Softmax function: \begin{equation} p_j = \frac{e^{o_j}}{\sum_k e^{o_k}} \end{equation}...
113
votes
6answers
26k views

Is there a quick proof as to why the vector space of $\mathbb{R}$ over $\mathbb{Q}$ is infinite-dimensional?

It would seem that one way of proving this would be to show the existence of non-algebraic numbers. Is there a simpler way to show this?
108
votes
12answers
10k views

Is there an “inverted” dot product?

The dot product of vectors $\mathbf{a}$ and $\mathbf{b}$ is defined as: $$\mathbf{a} \cdot \mathbf{b} =\sum_{i=1}^{n}a_{i}b_{i}=a_{1}b_{1}+a_{2}b_{2}+\cdots +a_{n}b_{n}$$ What about the quantity? $$\...
104
votes
4answers
79k views

What is the difference between linear and affine function

I am a bit confused. What is the difference between a linear and affine function? Any suggestions will be appreciated
102
votes
12answers
103k views

Why study linear algebra?

Simply as the title says. I've done some research, but still haven't arrived at an answer I am satisfied with. I know the answer varies in different fields, but in general, why would someone study ...
100
votes
9answers
171k views

Inverse of the sum of matrices

I have two square matrices - $A$ and $B$. $A^{-1}$ is known and I want to calculate $(A+B)^{-1}$. Are there theorems that help with calculating the inverse of the sum of matrices? In general case $B^{-...
99
votes
3answers
68k views

Why is the eigenvector of a covariance matrix equal to a principal component?

If I have a covariance matrix for a data set and I multiply it times one of it's eigenvectors. Let's say the eigenvector with the highest eigenvalue. The result is the eigenvector or a scaled ...
94
votes
18answers
37k views

What is the difference between a point and a vector?

I understand that a vector has direction and magnitude whereas a point doesn't. However, in the course notes that I am using, it is stated that a point is the same as a vector. Also, can you do ...
91
votes
7answers
4k views

Is the “determinant” that shows up accidental?

Consider the class of rational functions that are the result of dividing one linear function by another: $$\frac{a + bx}{c + dx}$$ One can easily compute that, for $\displaystyle x \neq \frac cd$ $$\...
91
votes
19answers
172k views

Calculate Rotation Matrix to align Vector A to Vector B in 3d?

I have one triangle in 3d space that I am tracking in a simulation. Between time steps I have the the previous normal of the triangle and the current normal of the triangle along with both the current ...
89
votes
10answers
6k views

Why, intuitively, is the order reversed when taking the transpose of the product?

It is well known that for invertible matrices $A,B$ of the same size we have $$(AB)^{-1}=B^{-1}A^{-1} $$ and a nice way for me to remember this is the following sentence: The opposite of putting on ...
88
votes
6answers
7k views

Mathematicians' Tensors vs. Physicists' Tensors

It seems, at times, that physicists and mathematicians mean different things when they say the word "tensor." From my perspective, when I say tensor, I mean "an element of a tensor product of vector ...
87
votes
6answers
94k views

Proof that the Trace of a Matrix is the sum of its Eigenvalues

I have looked extensively for a proof on the internet but all of them were too obscure. I would appreciate if someone could lay out a simple proof for this important result. Thank you.
85
votes
18answers
22k views

Is arrow notation for vectors “not mathematically mature”?

Assuming that we can't bold our variables (say, we're writing math as opposed to typing it), is it "not mathematically mature" to put an arrow over a vector? I ask this because in my linear algebra ...
84
votes
14answers
26k views

Looking for an intuitive explanation why the row rank is equal to the column rank for a matrix

I am looking for an intuitive explanation as to why/how row rank of a matrix = column rank. I've read the proof at http://en.wikipedia.org/wiki/Rank_of_a_linear_transformation and I understand the ...
84
votes
3answers
9k views

Why are vector spaces not isomorphic to their duals?

Assuming the axiom of choice, set $\mathbb F$ to be some field (we can assume it has characteristics $0$). I was told, by more than one person, that if $\kappa$ is an infinite cardinal then the ...
83
votes
3answers
3k views

Cute Determinant Question

I stumbled across the following problem and found it cute. Problem: We are given that $19$ divides $23028$, $31882$, $86469$, $6327$, and $61902$. Show that $19$ divides the following determinant: ...
82
votes
7answers
34k views

Intuitively, what is the difference between Eigendecomposition and Singular Value Decomposition?

I'm trying to intuitively understand the difference between SVD and eigendecomposition. From my understanding, eigendecomposition seeks to describe a linear transformation as a sequence of three ...
81
votes
2answers
27k views

GRE Subject Test - Past Papers, Books, Advice [closed]

This is not for the Maths part of the General GRE. This is for the GRE Subject Test in Maths. Feel free to add or comment. How do I know the definition of rings or of anything on the GRE given that ...
78
votes
7answers
97k views

Show that the determinant of $A$ is equal to the product of its eigenvalues

Show that the determinant of a matrix $A$ is equal to the product of its eigenvalues $\lambda_i$. So I'm having a tough time figuring this one out. I know that I have to work with the characteristic ...
78
votes
3answers
16k views

Why, historically, do we multiply matrices as we do?

Multiplication of matrices — taking the dot product of the $i$th row of the first matrix and the $j$th column of the second to yield the $ij$th entry of the product — is not a very ...
77
votes
10answers
5k views

Why does Friedberg say that the role of the determinant is less central than in former times?

I am taking a proof-based introductory course to Linear Algebra as an undergrad student of Mathematics and Computer Science. The author of my textbook (Friedberg's Linear Algebra, 4th Edition) says in ...
77
votes
7answers
8k views

What is “Bra” and “Ket” notation and how does it relate to Hilbert spaces?

This is my first semester of quantum mechanics and higher mathematics and I am completely lost. I have tried to find help at my university, browsed similar questions on this site, looked at my ...
77
votes
7answers
90k views

Physical meaning of the null space of a matrix

What is an intuitive meaning of the null space of a matrix? Why is it useful? I'm not looking for textbook definitions. My textbook gives me the definition, but I just don't "get" it. E.g.: I think ...
75
votes
7answers
266k views

What does it mean to have a determinant equal to zero?

After looking in my book for a couple of hours, I'm still confused about what it means for a $(n\times n)$-matrix $A$ to have a determinant equal to zero, $\det(A)=0$. I hope someone can explain this ...
74
votes
9answers
113k views

When is matrix multiplication commutative?

I know that matrix multiplication in general is not commutative. So, in general: $A, B \in \mathbb{R}^{n \times n}: A \cdot B \neq B \cdot A$ But for some matrices, this equations holds, e.g. A = ...
73
votes
5answers
6k views

linear algebra over a division ring vs. over a field

When I was studying linear algebra in the first year, from what I remember, vector spaces were always defined over a field, which was in every single concrete example equal to either $\mathbb{R}$ or $\...
72
votes
3answers
16k views

Why did no student correctly find a pair of $2\times 2$ matrices with the same determinant and trace that are not similar?

I gave the following problem to students: Two $n\times n$ matrices $A$ and $B$ are similar if there exists a nonsingular matrix $P$ such that $A=P^{-1}BP$. Prove that if $A$ and $B$ are two ...
72
votes
7answers
6k views

What exactly are eigen-things?

Wikipedia defines an eigenvector like this: An eigenvector of a square matrix is a non-zero vector that, when multiplied by the matrix, yields a vector that differs from the original vector at most ...
70
votes
9answers
11k views

In Linear Algebra, what is a vector?

I understand that a vector space is a collection of vectors that can be added and scalar multiplied and satisfies the 8 axioms, however, I do not know what a vector is. I know in physics a vector is ...
70
votes
7answers
101k views

Is the inverse of a symmetric matrix also symmetric?

Let $A$ be a symmetric invertible matrix, $A^T=A$, $A^{-1}A = A A^{-1} = I$ Can it be shown that $A^{-1}$ is also symmetric? I seem to remember a proof similar to this from my linear algebra class, ...
70
votes
7answers
6k views

Is linear algebra more “fully understood” than other maths disciplines?

In a recent question, it was discussed how LA is a foundation to other branches of mathematics, be they pure or applied. One answer argued that linear problems are fully understood, and hence a ...
70
votes
1answer
44k views

Simultaneously Diagonalizable Proof

Two $n\times n$ matrices $A, B$ are said to be simultaneously diagonalizable if there is a nonsingular matrix $S$ such that both $S^{-1}AS$ and $S^{-1}BS$ are diagonal matrices. a.) Show that ...