The following text discusses that the ARD kernel is a regular gaussian kernel but one where $\Sigma$ is diagnonal and one where the $\sigma$'s go to infinity. It seems that the $\kappa$(x,x') would simply be 1 in that case? why would this be a useful kernel? 
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
An ARD Kernel simply expresses each dimension as being independent from the others. As such $\Sigma$ is diagonal. They are not saying that $\sigma_j$ is $\infty$ they are simply saying that is that was the case then the $j^{th}$ dimension would be ignored. If every dimension was ignored then the kernel would be a constant as you suggest but this is a trivial case and you are right it would not be useful. If, however, there is a dimension that has useful information then you would optimise for $\sigma_j$ and find that it simply scales that dimension appropriately.
