2
$\begingroup$

Q1: Let $C(\mathcal{X})$ be the space of continuous functions $f:[a,b]\to\mathbb{R}$. Let $F(\mathcal{X})$ be a subset of $C(\mathcal{X})$ that is dense in $C(\mathcal{X})$ w.r.t. metric $d_{\infty}(\cdot,\cdot)$. Is $B=\{f\in F(\mathcal{X}): \lVert{f}\rVert^2\leq 1\}$ dense in $A=\{f\in C(\mathcal{X}): \lVert{f}\rVert^2\leq 1\}$? ($\lVert{\cdot}\rVert^2$ is L2-norm.)

Q2: Assume the answer for Q1 is yes. Let $B,A$ be defined as Q1. For two points $x_1,x_2\in [a,b]$, we have \begin{align*} \sup_{f\in A} \lvert f(x_1)-f(x_2)\rvert =\sup_{f\in B} \lvert f(x_1)-f(x_2) \rvert.\tag{*} \end{align*}

Let $F(\mathcal{X})=\overline{\mathrm{span}}\{k_x: x\in[a,b]\}$, where $k_x$ is a feature map induced by a Gaussian kernel $\langle k_x,k_y\rangle=k(x,y)=\mathrm{e}^{{-\alpha(x-y)}^2}$. Then $F(\mathcal{X})$ is a RKHS and it has been proven that it is dense in $C(\mathcal{X}$) for any positive constant $\alpha$. Then I get: \begin{align*} \sup_{f\in B} \lvert f(x_1)-f(x_2) \rvert &=\sup_{f\in B} \langle k_{x_1}-k_{x_2}, f\rangle\\ &=\sqrt{\langle k_{x_1}-k_{x_2},\ k_{x_1}-k_{x_2}\rangle}\quad\text{ (**)}\\ &=\sqrt{2-2\mathrm{e}^{{-\alpha(x_1-x_2)}^2}}. \end{align*} Now the result depends on $\alpha$. It is expected to get a constant value for different $\alpha$. What's wrong with the process?

$\endgroup$
2
  • $\begingroup$ What norm do you use in the definition of $A,B$? If you use the sup-norm, then the answer is yes to Q1. $\endgroup$
    – daw
    May 3, 2021 at 9:58
  • $\begingroup$ It's actually L2 norm. Is the claim still true? $\endgroup$ May 3, 2021 at 10:43

1 Answer 1

1
$\begingroup$

I think you mix up the two different(!) norms $\lVert f\rVert_{L^2}$ and the norm from the Gaussian Kernel $\lVert f\rVert_{\exp}$. Your equation (*) assumes you work with the unit balls of the $L^2$-norm while (**) is only true for the unit ball in the RKHS norm.

With respect to (**): Let $\langle\cdot,\cdot\rangle_{\exp}$ be the scalar product of the Gaussian RKHS. Then the equation should read/is only true for: $$ \sup_{\lVert f\rVert_{\exp}\leq 1} \langle k_{x_1}-k_{x_2}, f\rangle_{\exp} =\sqrt{\langle k_{x_1}-k_{x_2},\ k_{x_1}-k_{x_2}\rangle_{\exp}}.$$

And since $\lVert f\rVert_{L^2} \neq \lVert f\rVert_{\exp}$ the unit balls $B= \{f\mid \lVert f\rVert_{L^2} \leq 1\}$ and $\{f\mid \lVert f\rVert_{\exp} \leq 1\}$ are not equal. That is why $$ \sup_{f\in B} \langle k_{x_1}-k_{x_2}, f\rangle_{\exp} \neq \sqrt{\langle k_{x_1}-k_{x_2},\ k_{x_1}-k_{x_2}\rangle_{\exp}}.$$

(And $\alpha$ has to be negative)

$\endgroup$
2
  • $\begingroup$ Thank you for the answer. I'm still a bit confused. $F(\mathcal{X})$ induced by Gaussian kernel is dense in $C(\mathcal{X})$ in terms of $d_\infty(\cdot,\cdot)$. Both the elements in $A$ and $B$ are restricted to be from unit ball (L2-norm). The functions in $A$ uniformly converge to a function in $B$. That's why the equation $(∗)$ holds. $(∗∗)$ is just a solution for the RHS of $(∗)$. The problem is that the solution changes with $\alpha$, which is supposed to be a constant according to equation $(∗)$. $\endgroup$ May 3, 2021 at 11:08
  • $\begingroup$ see my edits I hope it is clearer now. $\endgroup$
    – g g
    May 3, 2021 at 12:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .