Strang, Introd. Linear Algebra, Section 2.7: Understanding example that uses $(Ax)^Ty=x^T(A^Ty)$ with calculus to obtain integration by parts.

Question

The following example appears in Strang's book Introduction to Linear Algebra in section 2.7.

Example 3 Will you allow me a little calculus? It is extremely important or I wouldn't leave linear algebra. (This is really linear algebra for functions $x(t)$). The difference matrix changes to a derivative $A=d/dt$. Its transpose will now come from $(dx/dt,y)=(x,-dy/dt)$.

The inner product changes from a finite sum of $x_ky_k$ to an intergral of $x(t)y(t)$.

Inner product of functions $$x^Ty=(x,y)=\int_{-\infty}^\infty x(t)y(t)dt \text{ by definition }\tag{1}$$

Transpose rule $(Ax)^Ty=x^T(A^Ty)$ $$\int_{-\infty}^\infty \frac{dx}{dt}y(t)dt=\int_{-\infty}^\infty x(t)\left (-\frac{dy}{dt}\right ) dt \text{ shows } A^T\tag{2}$$

I hope you recognize "integration by parts". The derivative moves from the first function $x(t)$ to the second function $y(t)$. During that move, a minus sign appears. This tells us the "transpose" of the derivative is minus the derivative.

Usually I preface my questions with a demonstration of some level of understanding of such a quoted snippet. In this case, I am quite at a loss. I know the concepts (I know calculus), but the writing is just atrocious.

Let me try to go through the text cited above.

The difference matrix changes to a derivative $A=d/dt$.

I assume he is referring to the fact that at some previous point we were analyzing a difference matrix that acted on a vector. Now we are analyzing what seems to be a derivative "operator" $A=d/dt$.

Its transpose will now come from $(dx/dt,y)=(x,-dy/dt)$.

The transpose of this derivative "operator" comes from this expression $(dx/dt,y)=(x,-dy/dt)$. How so?

Before we had an inner product that had a sum of terms of form $x_ky_k$, but in the new scenario we have an integral of $x(t)y(t)$.

(1) seems to show this, though I am confused by the equals signs and also the $(x,y)$.

Now, $Ax$ seems to be the derivative of $x$, $dx/dt$, and $(Ax)^Ty$ seems to be the integral of $dx/dt$ times $y(t)$, which, in analogy with the rule for matrices ($(Ax)^Ty=x^T(A^Ty)$) is equal to the integral of $x(t)$ times what $(A^Ty)$ is in the new scenario, which apparently is $-\frac{dy}{dt}$.

I am looking for an answer that makes sense of all of this.

Answer 1

§1. Rigorously, what we are (implicitly) considering is the set of differentiable and square-integrable* real functions on $\mathbb{R}$. Let us call it $V$. Given $x,y\in V$ and $\lambda\in\mathbb{R}$, we define operations pointwise, i.e.: $$(x+y)(t):=x(t)+y(t)\qquad (\lambda x)(t):=\lambda \cdot x(t) $$ These operations make $V$ into a (infinite-dimensional) vector space over $\mathbb{R}$. Now we define an inner product given by

$$\langle x,y\rangle:=\int_{-\infty}^{+\infty}x(t)y(t)\mathrm{d}t$$ You could check that this definition satisfies the usual axioms of inner product, i.e. (for all $x,y\in V$ and $\lambda \in \mathbb{R}$)

• $\langle x,y\rangle=\langle y,x\rangle$
• $\langle \lambda x + z, y\rangle=\lambda\langle x,y\rangle + \langle z,y\rangle$
• $\langle x,x\rangle>0$ if $x\neq 0_V$

However, the inner product is well defined only if we assume that the improper integral always converges. This leads to the technical request (omitted by Strang) that all the elements of $V$ be square-integrable.

§2. We define an operator** $A:V\to V$ given by $Ax=dx/dt$. It is easily seen that $A$ is linear (by elementary properties of derivatives): $$\frac{d}{dt}(x+y)=\frac{dx}{dt}+\frac{dy}{dt} ,\qquad \frac{d}{dt}(\lambda x)=\lambda\frac{dx}{dt}$$

We want to study the interaction of this operator with the inner product previously defined. Notice that there are many other possible definitions of an inner product on $V$, and even though $A$ is linear whatever the inner product on $V$ (since linearity does not involve inner products), what we are going to say about $A^{*}$ depends on our specific choice of inner product.

§3. In general, given any vector space $W$ with an inner product $\langle\cdot,\cdot\rangle$ and a linear operator $T$ on $W$, we denote with $T^{*}$ the operator such that $$\langle Tx,y\rangle=\langle x,T^{*}y\rangle$$ for all $x,y\in W$.

If $W$ is finite dimensional (e.g. $W=\mathbb{R}^n$), then we can choose a basis such that $T$ is represented by multiplication by a matrix $M$ and $\langle x,y\rangle$ is the usual dot product of components. In this case, $T^*$ is represented by the matrix $M^T$ (the transpose), and this explains the notation of Strang (who writes $A^T$ even though $A$ is not a matrix).

§4. Concering our case, we want to find the operator $A^*$ such that $$\langle Ax,y\rangle=\langle x,A^{*}y\rangle$$ for all $x,y\in V$.

We recall then integration by parts, i.e. (with our notation):

$$\langle Ax,y\rangle=x(t)y(t)\Big|_{-\infty}^{+\infty} -\langle x,Ay\rangle$$

Since $x$ and $y$ are square-integrable, we have $$\lim_{t \to \pm\infty}x(t)=0$$ and the same for $y$. Therefore $x(t)y(t)\Big|_{-\infty}^{+\infty}=0$ and we deduce that

$$\langle Ax,y\rangle=-\langle x,Ay\rangle$$

that is $\langle Ax,y\rangle=\langle x,-Ay\rangle$.

Hence we have found that $A^*=-A$.

---

(*) We say that $f:\mathbb{R}\to\mathbb{R}$ is square-integrable on $(-\infty,+\infty)$ if $$\int_{-\infty}^{+\infty}f(t)^2\mathrm{d}t<\infty$$

(**) Operator here means nothing else than function or mapping $V\to V$. In this context it is usual to use the term functions for the elements of $V$ (functions $\mathbb{R}\to\mathbb{R}$) and operator for mappings $V\to V$.