The problem is to determine whether the given differential operator $L[y]$, whose domain consists of all functions that have continuous second derivatives on the interval $[0,\pi]$ and satisfy the given boundary conditions, is selfadjoint.
$$L[y]=y''+\lambda y;\;\;\;\;y(0)+y'(\pi)=0,\;\;\;\;y'(0)+y(\pi)=0$$
For an operator to be selfadjoint over an interval, it must satisfy the equation $(u,L[v])=(L[u],v)$ for all $u,v$ in the domain, where the inner product over the interval $[a,b]$ is defined as:
$$(f,g)=\int_a^b f(x)g(x)dx$$
Applying integration-by-parts twice to the integral for $(u,L[v])$ goes as follows:
$$ (u,L[v])= \int_0^\pi u(v''+\lambda v)dx = \int_0^\pi uv''\;dx + \int_0^\pi \lambda v\;dx = uv'|_0^\pi - \int_0^\pi u'v'\;dx + \int_0^\pi \lambda v\;dx $$
$$ = uv'|_0^\pi - u'v|_0^\pi + \int_0^\pi u''v\;dx + \int_0^\pi \lambda v\;dx = (uv'-u'v)|_0^\pi + (L[u],v) $$
Thus, the operator is selfadjoint iff $(uv'-u'v)|_0^\pi=0$. The boundary conditions can be used to show that $uv'|_0^\pi=-u'v|_0^\pi$, leading to the following condition for selfadjointness:
$$uv'|_0^\pi=0$$
This seems promising, but I'm not sure where to go from here. I can't even think of functions that satisfy the boundary conditions, which makes finding a counter-example difficult.
Edit:
It looks like the following is a counter-example:
$$u(x)=2\cos{2x}-\sin{2x},\;\;\;\;v(x)=\cos{x}+\sin{x}$$ $$ uv'|_0^\pi = (2\cos{2x}-\sin{2x})(\cos{x}-\sin{x})|_0^\pi = -4$$
Unfortunately, I don't find this particularly enlightening. Is there any way to show that the operator is not selfadjoint without a counter-example?