Strang, Introd. Linear Algebra, Section 2.7: Understanding example that uses $(Ax)^Ty=x^T(A^Ty)$ with calculus to obtain integration by parts.

The following example appears in Strang's book Introduction to Linear Algebra in section 2.7.

Example 3 Will you allow me a little calculus? It is extremely important or I wouldn't leave linear algebra. (This is really linear algebra for functions $$x(t)$$). The difference matrix changes to a derivative $$A=d/dt$$. Its transpose will now come from $$(dx/dt,y)=(x,-dy/dt)$$.

The inner product changes from a finite sum of $$x_ky_k$$ to an intergral of $$x(t)y(t)$$.

Inner product of functions $$x^Ty=(x,y)=\int_{-\infty}^\infty x(t)y(t)dt \text{ by definition }\tag{1}$$

Transpose rule $$(Ax)^Ty=x^T(A^Ty)$$ $$\int_{-\infty}^\infty \frac{dx}{dt}y(t)dt=\int_{-\infty}^\infty x(t)\left (-\frac{dy}{dt}\right ) dt \text{ shows } A^T\tag{2}$$

I hope you recognize "integration by parts". The derivative moves from the first function $$x(t)$$ to the second function $$y(t)$$. During that move, a minus sign appears. This tells us the "transpose" of the derivative is minus the derivative.

Usually I preface my questions with a demonstration of some level of understanding of such a quoted snippet. In this case, I am quite at a loss. I know the concepts (I know calculus), but the writing is just atrocious.

Let me try to go through the text cited above.

The difference matrix changes to a derivative $$A=d/dt$$.

I assume he is referring to the fact that at some previous point we were analyzing a difference matrix that acted on a vector. Now we are analyzing what seems to be a derivative "operator" $$A=d/dt$$.

Its transpose will now come from $$(dx/dt,y)=(x,-dy/dt)$$.

The transpose of this derivative "operator" comes from this expression $$(dx/dt,y)=(x,-dy/dt)$$. How so?

Before we had an inner product that had a sum of terms of form $$x_ky_k$$, but in the new scenario we have an integral of $$x(t)y(t)$$.

(1) seems to show this, though I am confused by the equals signs and also the $$(x,y)$$.

Now, $$Ax$$ seems to be the derivative of $$x$$, $$dx/dt$$, and $$(Ax)^Ty$$ seems to be the integral of $$dx/dt$$ times $$y(t)$$, which, in analogy with the rule for matrices ($$(Ax)^Ty=x^T(A^Ty)$$) is equal to the integral of $$x(t)$$ times what $$(A^Ty)$$ is in the new scenario, which apparently is $$-\frac{dy}{dt}$$.

I am looking for an answer that makes sense of all of this.

• @ancientmathematician Yes, you are right. I've corrected the typo in my post.
– xoux
Commented Dec 14, 2022 at 19:42
• The adjoint (transpose in your setting) of a linear operator $A$ defined in an inner product space $H$ is defined as the linear operator $A^*$ such that $(Ax,y)=(x,A^*y)$ for all $x,y\in H$, where $(x,y)$ is the inner product of vectors $x$ and $y$. What Gil says is that for the derivative operator acting on some space of $C^1(\mathbb{R})$ functions that are squared integrable (he is committing all this technical mumbo-jumbo) equipped with the inner product $(x,y)=\int_{\mathbb{R}} x(t)y(t)\,dt$ one has from integration by parts ($(x',y)=-(x,y')$), that is, if $A=\frac{d}{dt}$, then $A^*=-A$ Commented Dec 14, 2022 at 20:18

§1. Rigorously, what we are (implicitly) considering is the set of differentiable and square-integrable* real functions on $$\mathbb{R}$$. Let us call it $$V$$. Given $$x,y\in V$$ and $$\lambda\in\mathbb{R}$$, we define operations pointwise, i.e.: $$(x+y)(t):=x(t)+y(t)\qquad (\lambda x)(t):=\lambda \cdot x(t)$$ These operations make $$V$$ into a (infinite-dimensional) vector space over $$\mathbb{R}$$. Now we define an inner product given by

$$\langle x,y\rangle:=\int_{-\infty}^{+\infty}x(t)y(t)\mathrm{d}t$$ You could check that this definition satisfies the usual axioms of inner product, i.e. (for all $$x,y\in V$$ and $$\lambda \in \mathbb{R}$$)

• $$\langle x,y\rangle=\langle y,x\rangle$$
• $$\langle \lambda x + z, y\rangle=\lambda\langle x,y\rangle + \langle z,y\rangle$$
• $$\langle x,x\rangle>0$$ if $$x\neq 0_V$$

However, the inner product is well defined only if we assume that the improper integral always converges. This leads to the technical request (omitted by Strang) that all the elements of $$V$$ be square-integrable.

§2. We define an operator** $$A:V\to V$$ given by $$Ax=dx/dt$$. It is easily seen that $$A$$ is linear (by elementary properties of derivatives): $$\frac{d}{dt}(x+y)=\frac{dx}{dt}+\frac{dy}{dt} ,\qquad \frac{d}{dt}(\lambda x)=\lambda\frac{dx}{dt}$$

We want to study the interaction of this operator with the inner product previously defined. Notice that there are many other possible definitions of an inner product on $$V$$, and even though $$A$$ is linear whatever the inner product on $$V$$ (since linearity does not involve inner products), what we are going to say about $$A^{*}$$ depends on our specific choice of inner product.

§3. In general, given any vector space $$W$$ with an inner product $$\langle\cdot,\cdot\rangle$$ and a linear operator $$T$$ on $$W$$, we denote with $$T^{*}$$ the operator such that $$\langle Tx,y\rangle=\langle x,T^{*}y\rangle$$ for all $$x,y\in W$$.

If $$W$$ is finite dimensional (e.g. $$W=\mathbb{R}^n$$), then we can choose a basis such that $$T$$ is represented by multiplication by a matrix $$M$$ and $$\langle x,y\rangle$$ is the usual dot product of components. In this case, $$T^*$$ is represented by the matrix $$M^T$$ (the transpose), and this explains the notation of Strang (who writes $$A^T$$ even though $$A$$ is not a matrix).

§4. Concering our case, we want to find the operator $$A^*$$ such that $$\langle Ax,y\rangle=\langle x,A^{*}y\rangle$$ for all $$x,y\in V$$.

We recall then integration by parts, i.e. (with our notation):

$$\langle Ax,y\rangle=x(t)y(t)\Big|_{-\infty}^{+\infty} -\langle x,Ay\rangle$$

Since $$x$$ and $$y$$ are square-integrable, we have $$\lim_{t \to \pm\infty}x(t)=0$$ and the same for $$y$$. Therefore $$x(t)y(t)\Big|_{-\infty}^{+\infty}=0$$ and we deduce that

$$\langle Ax,y\rangle=-\langle x,Ay\rangle$$

that is $$\langle Ax,y\rangle=\langle x,-Ay\rangle$$.

Hence we have found that $$A^*=-A$$.

(*) We say that $$f:\mathbb{R}\to\mathbb{R}$$ is square-integrable on $$(-\infty,+\infty)$$ if $$\int_{-\infty}^{+\infty}f(t)^2\mathrm{d}t<\infty$$

(**) Operator here means nothing else than function or mapping $$V\to V$$. In this context it is usual to use the term functions for the elements of $$V$$ (functions $$\mathbb{R}\to\mathbb{R}$$) and operator for mappings $$V\to V$$.