# Intuitive way to understand covariance and contravariance in Tensor Algebra

I'm trying to understand basic tensor analysis. I understand the basic concept that the valency of the tensor determines how it is transformed, but I am having trouble visualizing the difference between different valencies when it comes to higher order tensors.

I have this picture in my mind for the lower order tensors

$X^i = \left(\begin{array}{x} x^1 \\ x^2 \\ x^3\end{array}\right)$

$X_i = \left(\begin{array}{ccc} x_1 & x_2 & x_3\end{array}\right)$

$X^i_j = \left(\begin{array}{ccc} x^1_1 & x^1_2 & x^1_3 \\ x^1_2 & x^2_2 & x^3_2 \\ x^1_3 & x^2_3 & x^3_3\end{array} \right)$

for $X^{ij}$ and $X_{ij}$ they are represented in the same 2d array, but the action on a vector isn't defined in the same way as with matrices.

What I am having trouble with is intuitively understanding the difference between $X^{ijk}$, $X_{k}^{ij}$, $X_{jk}^{i}$, and $X_{ijk}$ (other permutations of the valence $(2,1)$ and $(1,2)$ omitted for brevity).

Since I am somewhat comfortable with tensor products in quantum mechanics, I can draw a parallel with the specific tensor space I'm used to.

If we consider a rank-5 tensor with a valence of (2,3) then we can consider it in the braket notation as

$\langle \psi_i \mid \otimes \ \langle \psi_j \mid \otimes \ \langle \psi_k \mid \otimes \mid \psi_l \rangle \ \otimes \mid \psi_m \rangle = X_{ijk}^{lm}$

Now if we operate with this tensor on rank-3 contravariant tensor, we are-left with a constant (from the inner product) and a rank-2 contravariant tensor, unmixed tensor product $\begin{eqnarray}(\langle \psi_i \mid \otimes \ \langle \psi_j \mid \otimes \ \langle \psi_k \mid \otimes \mid \psi_l \rangle \ \otimes \mid \psi_m \rangle)(\mid \Psi_i \rangle \ \otimes \mid \Psi_j \rangle \ \otimes \mid \Psi_k \rangle) &=& c \mid \psi_l \rangle \ \otimes \mid \psi_m \rangle \\ &=& X_{ijk}^{lm}\Psi^{ijk} = cX'^{lm}\end{eqnarray}$

If we were to further operate with a rank-2 covariant tensor (from the right, per convention that a covector and vector facing each other is an implied direct product) we would simply get a number out.

One thing I am confused about though, is that in one of the answer to this question there was a point made that we are taking tensor products of a Vector space with itself (and possibly it's dual), however in the quantum mechanics picture (although I didn't rely on it in this example) we often take tensor products between different, often disjoint, subspaces of the enormous Hilbert space that describes the quantum mechanical universe. Does the tensor picture change in this case?

Any comments on my example would be appreciated.

-
If $V$ and $W$ are different vector spaces, an element in the tensor product space $V^* \otimes W^*$ (for example) is a bilinear map which takes one vector from $V$ and one from $W$ and returns a scalar. You can still write mixed tensors like this in terms of components, if you introduce a basis $(e_1,\ldots,e_n)$ in $V$ and a basis $(f_1,\ldots,f_n)$ in $W$ and corresponding dual bases in $V^*$ and $W^*$. For example, $V \otimes W$ will have basis elements $e_i \otimes f_j$. Does that make things clearer? I don't know exactly what you mean by "tensor picture". –  Hans Lundmark Oct 28 '10 at 21:59

Since you asked for an intuitive way to understand covariance and contravariance, I think this will do.

First of all, remember that the reason of having covariant or contravariant tensors is because you want to represent the same thing in a different coordinate system. Such a new representation is achieved by a transformation using a set of partial derivatives. In tensor analysis, a good transformation is one that leaves invariant the quantity you are interested in.

For example, we consider the transformation from one coordinate system $x^1,...,x^{n}$ to another $x^{'1},...,x^{'n}$:

$x^{i}=f^{i}(x^{'1},x^{'2},...,x^{'n})$ where $f^{i}$ are certain functions.

Take a look at a couple of specific quantities. How do we transform coordinates? The answer is:

$dx^{i}=\displaystyle \frac{\partial x^{i}}{\partial x^{'k}}dx^{k}$

Every quantity which under a transformation of coordinates, transforms like the coordinate differentials is called a contravariant tensor.

How do we transform some scalar $\Phi$?

$\displaystyle \frac{\partial \Phi}{\partial x^{i}}=\frac{\partial \Phi}{\partial x^{'k}}\frac{\partial x^{'k}}{\partial x^{i}}$

Every quantity which under a coordinate transformation, transforms like the derivatives of a scalar is called a covariant tensor.

Accordingly, a reasonable generalization is having a quantity which transforms like the product of the components of two contravariant tensors, that is

$A^{ik}=\displaystyle \frac{\partial x^{i}}{\partial x^{'l}}\frac{\partial x^{k}}{\partial x^{'m}}A^{'lm}$

which is called a contravariant tensor of rank two. The same applies to covariant tensors of rank n or mixed tensor of rank n.

Having in mind the analogy to coordinate differentials and derivative of a scalar, take a look at this picture, which I think will help to make it clearer:

From Wikipedia:

The contravariant components of a vector are obtained by projecting onto the coordinate axes. The covariant components are obtained by projecting onto the normal lines to the coordinate hyperplanes.

Finally, you may want to read: Basis vectors

By the way, I don't recommend to rely blindly in the picture given by matrices, specially when you are doing calculations.

-
Thanks for the response. I wasn't implying I use the matrix picture for calculations, it was just my intuitive way to understand what a mixed rank-2 tensor represents. –  crasic Oct 28 '10 at 19:24
Yes, I understood what you mean. It was just a side note. –  Robert Smith Oct 28 '10 at 20:13

I prefer to think of them as maps instead of matrices. When you move to tensor bundles over manifolds, you won't have global coordinates, so it might be preferable to think this way.

So $x_i$ is a map which sends vectors to reals. Since it's a tensor, you're only concerned with how it acts on basis elements. It's nice to think of them in terms of dual bases: then $x_i(x^j)=\delta_{ij}$, which is defined as $1$ when $i=j$ and $0$ otherwise.

Similarly, $x^i$ is a map which sends covectors to reals, and is defined by $x^i(x_j)=\delta_{ij}$.

If you have more indices, then you're dealing with a tensor product $V^*\otimes\dotsb\otimes V^*\otimes V\otimes\dotsb\otimes V$, say with $n$ copies of the vector space and $m$ copies of the dual. An element of this vector space takes in $m$ vectors and gives you back $n$, again in a tensorial way. So, for example, $X_{ijk}$ is a trilinear map; $X^{ijk}$ is a trivector (an ordered triple of vectors up to linearity); $X_{ij}^k$ is a bilinear map taking two vectors to one vector; and so on.

It's worth thinking about these in terms of the tensors you've seen already. The dot product, for example, is your basic (0,2)-tensor. The cross product is a (1,2)-tensor. If you study Riemannian manifolds, it turns out you can use the metric to "raise and lower indices"; so the Riemannian curvature tensor, for example, is alternately defined as a (1,3)-tensor and a (0,4)-tensor, depending on the author's needs.

-
I agree, except that you seem to have swapped upper and lower indices: $X^i$ usually denotes (the components of) a (1,0)-tensor, that is a vector, that is a map which sends covectors to numbers. And so on. (But in the last paragraph you are back to standard usage again.) –  Hans Lundmark Oct 28 '10 at 10:14
Would it be fair to say that a (m,n) tensor $X_{*m}^{*n}$ takes n vectors to and m-tuple of vectors? What does this map do with covectors? –  crasic Oct 28 '10 at 10:38
Bleh, you're right. Indices bug me. I'll fix it. –  Paul VanKoughnett Oct 28 '10 at 10:38
An $(n,m)$ tensor takes $m$ vectors to an $n$-tuple of vectors. It would do the opposite on covectors, namely, take $n$ covectors to an $m$-tuple of covectors. –  Paul VanKoughnett Oct 28 '10 at 10:41
@crasic and @Paul: It's not correct to say that an $(n,m)$ tensor takes $m$ vectors to an $n$-tuple of vectors -- if you think of it this way, the output is actually an $n$-vector (an element of the $n$-fold tensor product of $V$ with itself), which can't be identified with an $n$-tuple. When $n$ is bigger than 1, in many ways it's easier to think of an $(m,n)$ tensor as taking $m$ vectors and $n$ covectors and yielding a number that depends linearly on each input separately. –  Jack Lee Oct 28 '10 at 16:58

(cross-posting a variation of my answer from this)

Both co-variant and contra-variant vectors are just vectors (or more generaly 1-order tensors).

Furthermore they are vectors which relate to the same underlying space (for example euclidean space or generally, a manifold)

Furthermore they relate to the same space but in different but dual ways (as such they have different, but dual, transformation laws)

Co-variant vectors are part of what is called the tangent space, which for an euclidean space coincides or is isomorphic to the space itself. And contra-variant vectors are part of the dual tangent space, called co-tangent space and which for an euclidean space also coincides or is also isomorphic to the space itself.

These spaces (and their vectors) are dual, in the algebraic sense, related through the norm (inner product) of the space. Plus are isomorphic to each other regardless if they are isomorphic to the underlying manifold itself (what is called raising and lowering indices)

A question is what these (associated) spaces represent, how are they related and what is the intuition behind their use?

Historicaly tensors and tensor analysis was initiated as a by-product of the theory of invariants. A way was needed to express invariant quantities under a change of representation (or change of underlying basis). Thus tensors were used. tensors represent quantities which transform under a change of representation in such ways as to make various quantities expressed in terms of them invariant.

Note, the terminology association with co-variant/contra-variant indices is largely an convention, any consistent convention will do.

This also gives the (intuitive) relation between co-variant tensors (vectors) and contra-variant tensors (vectors). When a co-variant vector (components) transform in one way, for example by a scaling factor $s$. The (associated) contra-variant vector (components) will have to transform by the inverse scaling factor $1/s$ in order for invariant quantities (for example an inner product $a^ib_i$) to remain invariant.

If one goes to higher-rank tensors (e.g $g_{ij}$, $R^{i}_{jkl}$) one can see an additional dual association as such:

Lets say we have a covariant tensor (i.e metric) $g_{ij}$ and want the contra-variant part ($g^{ij}$). if one part of the metric (e.g for $g_{12}$) represents an area (say $A_{12}$), the dual tensor at that same indices ($g^{12}$) represents the dual area (say $A^{12}$) which is the area remaing after the are $A_{12}$ is subtracted form the whole area (say $A$).

-