Understanding positive definite kernel

From Mercer's Theorem:

A kernel is a symmetric continuous function $K: [a,b] \times [a,b] \rightarrow \mathbb{R}$, so that $K(x, s) = K(s, x)$ ($\forall s,x \in [a,b]$).

$K$ is said to be non-negative definite (or positive semi-definite) if and only if $$\sum_{i=1}^n\sum_{j=1}^n K(x_i, x_j) c_i c_j \geq 0$$ for all finite sequences of points $x_1, ..., x_n$ of $[a, b]$ and all choices of real numbers $c_1, ..., c_n$.

Let $\{ H_n \}_{n \in {\mathbb Z}}$ be a sequence of (complex) Hilbert spaces and $\mathcal{L}(H_i, H_j)$ be the bounded operators from $H_i$ to $H_j$.

A map $A$ on ${\mathbb Z} \times {\mathbb Z}$ where $A(i,j)\in\mathcal{L}(H_i, H_j)$ is called a positive definite kernel if for all $m > 0$ and $h_i \in H_i$, the following positivity condition holds: $$\sum_{-m \leq i\quad\, \atop j \leq m} \langle A(i,j) h_i, h_j \rangle \geq 0.$$

1. I wonder if the two definitions for positive-definite kernel agree with each other and how?
2. Is a positive-definite kernel related to a positive-definite bilinear form on a vector space?
3. What is the definition of kernel in its most general case, i.e. for the general Hilbert space case?

References are also appreciated. Thanks and regards!

Those definitions are very closely related, in that they can be put into the same general framework.

Let $X$ be a set, let $(H_x)_{x\in X}$ be a family of Hilbert spaces indexed by $X$, and for each $(x,y)\in X\times X$, let $K(x,y)$ be an element of $\mathcal{L}(H_x,H_y)$. Then $K$ is called a positive (semidefinite) kernel if for all finite sequences $x_1,\ldots x_n$ in $X$ and $h_1,\ldots,h_n$ with $h_i\in H_{x_i}$,

$$\sum_{i,j=1}^n\langle K(x_i,x_j)h_i,h_j\rangle\geq 0.$$ This is equivalent to requiring that the matrix $(K(x_j,x_i))_{ij}$ represents a positive operator on the Hilbert space $H_{x_1}\oplus\cdots\oplus H_{x_n}$. (There's a distracting transpose there, but I'm trying to make this consistent with the definition you gave.)

You get your second example by taking $X=\mathbb Z$. You get your first example by taking $X=[a,b]$ and $H_x=\mathbb R$ (as a real Hilbert space) for all $x$.

It is common (for example in the context mentioned below) that there is a single Hilbert space $H$ such that $H_x=H$ for all $x\in X$. In that case, $K:X\times X\to \mathcal{L}(H)$ is a positive kernel if for each finite sequence $x_1,\ldots,x_n$ in $X$, the matrix $(K(x_j,x_i))_{ij}$ represents a positive operator on the Hilbert space $H^{(n)}$.

So far I hope I have somewhat answered questions 1 and 3. Now I will discuss one context where such functions arise, leading to a partial answer to question 2.

One place where positive kernel functions arise is in the study of reproducing kernel Hilbert spaces. If $X$ is a set, $H$ is a Hilbert space, and $E$ is a Hilbert space whose elements are $H$-valued functions on $X$, then $E$ is called a (vector-valued) reproducing kernel Hilbert space if for each $x\in X$, evaluation at $x$ is a bounded linear operator from $E$ to $H$.

Suppose that $E$ satisfies this definition, and for each $x\in X$, let $\mathrm{ev}_x\in\mathcal{L}(E,H)$ be evaluation at $x$, $\mathrm{ev}_x(f)=f(x)$. Let $K:X\times X\to \mathcal{L}(H)$ be defined by $K(x,y)=\mathrm{ev}_x\mathrm{ev}_y^*$. Then $K$ is a positive kernel on $X$, called the reproducing kernel for $E$. The reason for the name "reproducing kernel" is that the evaluations of elements of $E$ can be "reproduced" from $K$ and the inner product on $E$. For each $x\in X$ and $h\in H$, the function $k_{x,h}:X\to H$ defined by $k_{x,h}(y)=K(y,x)h$ is in $E$. If $f$ is in $E$, then $\langle f(x),h\rangle=\langle f,k_{x,h}\rangle$. (In fact, note that $k_{x,h}=\mathrm{ev}_x^*h$.) This property uniquely characterizes $K$.

Conversely, if $X$ is a set, $H$ is a Hilbert space, and $K:X\times X\to \mathcal{L}(H)$ is a positive kernel function, then there is a unique reproducing kernel Hilbert space $E_K$ of $H$-valued functions on $X$ such that for all $f\in E$, $x\in X$, and $h\in H$, $k_{x,h}$ as defined above is in $E_K$, and $\langle f(x),h\rangle=\langle f,k_{x,h}\rangle$. In other words, $K$ is the (unique) reproducing kernel for a (unique) reproducing kernel Hilbert space. For the construction of $E_K$, a positive bilinear (or sesquilinear) form, i.e. an inner product, is defined on a free vector space whose formal generators "wind up" being the functions $k_{x,h}$ after completion. The positivity of $K$ is precisely what is needed to make the inner product work, so this might be a partial answer to your question 2. (I am leaving this part vague for now, but if you're interested I can elaborate or provide a reference.)

There are further generalizations that have appeared in operator theory and operator algebras literature, and there are certainly more types and applications of positive kernel functions than I am even aware of, let alone mentioned here. I may add more references at some point (I certainly will if you ask), but for now I will point out a couple that I have found particularly useful:

You might also be interested in a couple of past questions, here and here, which were about scalar-valued reproducing kernel Hilbert spaces (and hence, at least implicitly, involved positive scalar-valued kernels).

• I'm not sure if this is already covered in the sources you give: The book on Kazhdan's property (T) by Bekka-de la Harpe-Valette contains a short introduction to kernel constructions in appendix C with applications to unitary group representations in mind. I found the abstract GNS construction quite appealing, especially its use in Chapters C.4 and C.5. – t.b. Aug 20 '11 at 7:20
• @Theo: Thanks, I hadn't given any references that relate to unitary group representations. I am confused at the naming "GNS" there. I don't quite see the analogy to what is usually called GNS, namely representations constructed from states on C*-algebras. If I'm understanding correctly, what they call GNS is what I have seen called "Kolmogorov decomposition," and it is proved by essentially constructing the RKHS corresponding to the kernel. Regardless of that nitpick, it is a great reference! – Jonas Meyer Aug 20 '11 at 7:43
• I see what you mean. I didn't look that closely at the references they give, but this may simply be a matter of background/interests. There was an earlier version of that book by de la Harpe and Valette (Asterisque, mid-80s) that essentially contained theorem C.4.10 under the name GNS and I think this is justified because it is the "group version" of GNS (for the group algebra $L^1(G)$, expressed in non-$\ast$-algebraic terms). Then I'd say they view Theorem C.1.4 as an abstraction of that. The entire section is very close to parts of Dixmier's book on $C^{\ast}$-algebras, e.g. Ch 13.4 – t.b. Aug 20 '11 at 7:59
• Note that they don't speak of RKHS'es (is that the proper plural?) and the reference to the earlier book is: La propriété $(T)$ pour les groupes localement compacts, Astérisque 175, Soc. Math. France, 1989. ([HarVa-89] in the book I linked to) – t.b. Aug 20 '11 at 8:03
• I see, thanks. I hadn't looked in C.4 yet, and with that in mind GNS makes sense. They don't speak of RKHS (I guess this could be the plural if you just make that last initial stand for "spaces"), but the $\mathcal{H}$ of Theorem C.1.4 essentially is the RKHS constructed from $\Phi$. It makes sense not to emphasize (or even name) this perspective when they don't need it, but, e.g., they do say that $\mathcal{H}$ "can be realized as a space of functions," and this realization would be precisely the RKHS of $\Phi$. (I'm probably not saying anything you hadn't noticed.) – Jonas Meyer Aug 20 '11 at 19:31