$\langle u,v \rangle = u^Tv$

I am wondering if this only true for column vector, or can this be extended to the matrix case?

  • 1
    $\begingroup$ There is an inner product on matrices of the same shape given by $\text{Tr}(A^\top B) = \sum_i \sum_j a_{ij} b_{ij}$. It is essentially the same as the inner product you have given, if you viewed the matrix as a big column vector. There are many other inner products. $\endgroup$ – angryavian Oct 2 '19 at 2:11

It is important to be aware that there are many kinds of inner products. All you need for an inner product (loosely speaking) is the following:

  1. Symmetry: $\langle x,y \rangle = \langle y,x \rangle$

  2. Positivity: $\langle x,x\rangle > 0$ if $x \neq 0$

  3. Linearity: $\langle x+y ,z \rangle = \langle x,z \rangle + \langle y,z \rangle$, and $\langle ax,y \rangle = a\langle x,y \rangle$

That is all!

And each of $x$ and $y$ are called vectors.

Now, in our standard geometry (Euclidean geometry), the inner product of vectors (where we define a vector to be a set of elements that look like: $x = [x_1,x_2,...,x_n]^T$, then yes, the most natural inner product for us to use is the dot product, which is as your write:

$\langle u,v \rangle = u^Tv$

Clearly this satisfies the above rules. Now why this particular form? (1) There is a nice geometrical intuition (cosine product rule) with this sort of expansion and (2) Simple to calculate.

Now if your default vectors were transposed (i.e. row -> column) then naturally it would need to be exapnded as $uv^T$.

Also I mentioned before you need to define how your vectors look. Because technically speaking a matrix can be considered to be a vector object, and so can a function etc... (these are all additive linear objects defined wrt a field etc...).

Now since a matrix can be considered to be a vector-object, it is possible to define an inner product over this also. But it won't look like $u^Tv$ since it is a different object. For real square matrices we usually define the following:

$\langle A,B \rangle = \text{tr}(AB^T)$

And I'm sure there are other definitions to the matrix inner product (because as you can see what constitutes an inner product is very flexible)

| cite | improve this answer | |
  • $\begingroup$ I feel I should point out that one property you left out about inner products is that they are scalar-valued. This seems important, because this is the major property that you lose when defining $\langle u, v \rangle = u^\top v$ for matrices $u, v$. Plenty of things still hold (sort of true)! Linearity holds. Symmetry is replaced by transpose symmetry. Even positivity sort of holds in the sense of $\langle u, u \rangle$ is positive semi-definite. $\endgroup$ – Theo Bendit Oct 2 '19 at 2:46
  • $\begingroup$ Yes! I forgot it needs to indeed evaluate to be from a member of the field (aka scalar). I won't edit the above post since I think you have explained it quite well in your comment already. $\endgroup$ – tisPrimeTime Oct 2 '19 at 3:14
  • $\begingroup$ Your definition of positivity is missing some important elements. $\endgroup$ – amd Oct 2 '19 at 4:37
  • $\begingroup$ I think you are referring to the condition of $x \neq 0$ since i have placed a strict greater than?. I have added that in now. :) $\endgroup$ – tisPrimeTime Oct 2 '19 at 4:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.