# inner product of a and b

$$\langle u,v \rangle = u^Tv$$

I am wondering if this only true for column vector, or can this be extended to the matrix case?

• There is an inner product on matrices of the same shape given by $\text{Tr}(A^\top B) = \sum_i \sum_j a_{ij} b_{ij}$. It is essentially the same as the inner product you have given, if you viewed the matrix as a big column vector. There are many other inner products. – angryavian Oct 2 '19 at 2:11

It is important to be aware that there are many kinds of inner products. All you need for an inner product (loosely speaking) is the following:

1. Symmetry: $$\langle x,y \rangle = \langle y,x \rangle$$

2. Positivity: $$\langle x,x\rangle > 0$$ if $$x \neq 0$$

3. Linearity: $$\langle x+y ,z \rangle = \langle x,z \rangle + \langle y,z \rangle$$, and $$\langle ax,y \rangle = a\langle x,y \rangle$$

That is all!

And each of $$x$$ and $$y$$ are called vectors.

Now, in our standard geometry (Euclidean geometry), the inner product of vectors (where we define a vector to be a set of elements that look like: $$x = [x_1,x_2,...,x_n]^T$$, then yes, the most natural inner product for us to use is the dot product, which is as your write:

$$\langle u,v \rangle = u^Tv$$

Clearly this satisfies the above rules. Now why this particular form? (1) There is a nice geometrical intuition (cosine product rule) with this sort of expansion and (2) Simple to calculate.

Now if your default vectors were transposed (i.e. row -> column) then naturally it would need to be exapnded as $$uv^T$$.

Also I mentioned before you need to define how your vectors look. Because technically speaking a matrix can be considered to be a vector object, and so can a function etc... (these are all additive linear objects defined wrt a field etc...).

Now since a matrix can be considered to be a vector-object, it is possible to define an inner product over this also. But it won't look like $$u^Tv$$ since it is a different object. For real square matrices we usually define the following:

$$\langle A,B \rangle = \text{tr}(AB^T)$$

And I'm sure there are other definitions to the matrix inner product (because as you can see what constitutes an inner product is very flexible)

• I feel I should point out that one property you left out about inner products is that they are scalar-valued. This seems important, because this is the major property that you lose when defining $\langle u, v \rangle = u^\top v$ for matrices $u, v$. Plenty of things still hold (sort of true)! Linearity holds. Symmetry is replaced by transpose symmetry. Even positivity sort of holds in the sense of $\langle u, u \rangle$ is positive semi-definite. – Theo Bendit Oct 2 '19 at 2:46
• Yes! I forgot it needs to indeed evaluate to be from a member of the field (aka scalar). I won't edit the above post since I think you have explained it quite well in your comment already. – tisPrimeTime Oct 2 '19 at 3:14
• Your definition of positivity is missing some important elements. – amd Oct 2 '19 at 4:37
• I think you are referring to the condition of $x \neq 0$ since i have placed a strict greater than?. I have added that in now. :) – tisPrimeTime Oct 2 '19 at 4:46