What does a positive definite matrix have to do with Cauchy-Schwarz Inequality?

In my text book, Cauchy-Schwarz Inequality is extended to a positive definite matrix.

But I neither understand what the relationship between Cauchy-Schwarz Inequality and a positive definite matrix nor the sentence underlined in red, I am not a strong linear algebra person, I have just started studying it.

I'd like post the page here so that I might get a better explanation. Thank you.

As a complement to the other answers, I just wanted to note that reasoning a bit more abstractly you can get this result from the ordinary Cauchy-Schwarz inequality for free, without any computation.

The Cauchy-Schwarz inequality is valid in any inner product space (Euclidean vector space if you are working over the real numbers). In a given an inner product space (such as $\Bbb R^n$ with the standard inner product), what a positive definite symmetric linear operator $B$ (given by a positive definite symmetric matrix in the case of $\Bbb R^n$) gives you is precisely another inner product on the same space, defined by $(x\mid y)_B\overset{\rm def}=(x\mid By)=( Bx\mid y)$. Now the Cauchy-Schwarz inequality holds for this inner product as well: $$(x\mid y)_B^2 \leq (x\mid x)_B ~~(y\mid y)_B \qquad\text{with equality only if x \parallel y}$$ (the notation $x \parallel y$ here means the vectors $x,y$ are linearly dependent, one being a scalar multiple of the other). Now taking $\def\b{\mathbf b}x=\b$ and $\def\d{\mathbf d}y=B^{-1}\d$, this becomes $$(\b\mid B^{-1}\d)_B^2 \leq (\b\mid \b)_B ~~(B^{-1}\d\mid B^{-1}\d)_B \qquad\text{with equality only if \b \parallel B^{-1}\d.}$$ which expanding the definition of $(\mid )_B$ simplifies to $$(\b\mid\d)^2 \leq (\b\mid B\b) ~~(\d\mid B^{-1}\d) \qquad\text{with equality only if \b \parallel B^{-1}\d,}$$ which is your "extended" Cauchy-Schwarz inequality.

The sentence you underlined in red abbreviates these steps:

$$(\mathbf{B}^{1/2} \mathbf{b} \cdot \mathbf{B}^{-1/2} \mathbf{d})^2 \leq (\mathbf{B}^{1/2} \mathbf{b} \cdot \mathbf{B}^{1/2} \mathbf{b})(\mathbf{B}^{-1/2} \mathbf{d} \cdot \mathbf{B}^{-1/2} \mathbf{d})$$

This is the Cauchy-Schwarz inequality applied as requested. Now we use the transpose-based definition of the dot product $(\mathbf{a} \cdot \mathbf{b}) = \mathbf{a}^T \mathbf{b}$:

$$(\mathbf{b}^T (\mathbf{B}^{1/2})^T \mathbf{B}^{-1/2} \mathbf{d})^2 \leq (\mathbf{b}^T (\mathbf{B}^{1/2})^T \mathbf{B}^{1/2} \mathbf{b}) (\mathbf{d}^T (\mathbf{B}^{-1/2})^T \mathbf{B}^{-1/2} \mathbf{d})$$

Finally we take advantage of the the following observations:

• $\mathbf{B}$ is positive definite and therefore symmetric
• $\mathbf{B}^{1/2}$ is also positive definite and symmetric
• The inverse of a positive definite matrix is positive definite

As a result:

$$(\mathbf{B}^{1/2})^T \mathbf{B}^{-1/2} = \mathbf{B}^{1/2} \mathbf{B}^{-1/2} = \mathbf{I}$$

$$(\mathbf{B}^{1/2})^T \mathbf{B}^{1/2} = \mathbf{B}^{1/2} \mathbf{B}^{1/2} = \mathbf{B}$$

$$(\mathbf{B}^{-1/2})^T \mathbf{B}^{-1/2} = \mathbf{B}^{-1/2} \mathbf{B}^{-1/2} = \mathbf{B}^{-1}$$

Completing the proof.

You can see how this relates to the regular Cauchy-Schwarz inequality (for the dot product) by considering what it states when $\mathbf{B} = \mathbf{I}$ (Since $\mathbf{I}$ is a positive definite matrix). In this case we have

$$(\mathbf{b} \cdot \mathbf{d})^2 \leq (\mathbf{b} \cdot \mathbf{b}) (\mathbf{d} \cdot \mathbf{d})$$

Which is precisely the usual Cauchy-Schwarz inequality for the dot product.

Isn't CSI $x'y\leq \sqrt{x'\cdot x}\sqrt{y'\cdot y}$? Square it to get $$(x'\cdot y)^2\leq (x'\cdot x)(y'\cdot y)$$ Now apply this inequality by letting the vectors $x=B^{1/2} b$ and $y=B^{-1/2}d.$ You get $$b'\cdot d \leq (b'Bb)(d'B^{-1}d)$$ as required.

If you put $B = I$ then you recover the usual Cauchy–Schwarz inequality, so this is a generalization. You can write the Cauchy–Schwarz inequality in the form $$(x'y)^2 \leq (x'x)(y'y).$$ Taking $x = B^{1/2} b$ and $y = B^{-1/2} d$, we get $$(b'd)^2 = ((B^{1/2} b)'(B^{-1/2} d))^2 \leq ((B^{1/2}b)'B^{1/2}b)((B^{-1/2}d)'D^{-1/2}d) = \\ (b' B^{1/2} B^{1/2} b) (d'B^{-1/2} B^{-1/2} d) = (b' B b) (d' B^{-1} d).$$