You should not forget the meaning of $\delta$. It is a linear functional on (for instance) the compactly supported smooth functions on $\mathbf R^d$. That is, $LE = \delta$ means for all such $\psi$ that
$$\langle LE, \psi \rangle = \langle \delta, \psi \rangle = \psi(0).$$
Also, the support of $\delta$ is $\{0\}$ as can be seen above. Hence, if we test against $\psi$ which vanishes in $0$ we get
$$\langle LE, \psi \rangle = 0.$$
Obtaining that $LE = 0$ outside the support of $\delta$, that is, outside $\{0\}$.
Concerning your second question, the question reduces to "which class of functions do you test again", as recall that, distributions are functionals that have a certain domain. If $k \neq 0$, this only works on classes of test functions vanishing in $0$.