- The function \begin{array}{ccccc}
K & : & (0,\infty)\times \mathbb R^+ & \to & \mathbb R \\
& & (x,t) & \mapsto & \arctan(t/x)e^{-t} \\
\end{array}
is clearly continuous.
Furthermore, $\forall (x,t)\in (0,\infty)\times \mathbb R^+,|K(x,t)|\leq \frac{\pi}{2}e^{-t}$
Since the function $t\to \frac{\pi}{2}e^{-t}$ is integrable over $\mathbb R^+$, the previous inequality grants domination.
You can then deduce that $f$ is continuous.
- Let $x,y\in (0,\infty)$ such that $x<y$ and $t\in \mathbb R^+$
Then, $1/y <1/x \Rightarrow \arctan(t/y)e^{-t}< \arctan(t/x)e^{-t}$
Integrating the inequality with respect to $t$ yields $f(y)< f(x)$
$f$ is consequently decreasing.
- Fix some arbitrary non-negative $t$.
We need to prove the function $g_t:\displaystyle x\to\arctan(\frac{t}{x})e^{-t}$ is convex
The derivative of $g_t$ is $\displaystyle g_t'(x)=\frac{-te^{-t}}{t^2+x^2}$ which an increasing function. Hence the convexity of $g$.
The following inequality holds $\forall \lambda \in [0,1], \forall (x,y)\in (0,\infty), g_t((1-\lambda)x+\lambda y)\leq (1-\lambda)g_t(x)+\lambda g_t(y) \; \; \; \; (1)$
Now, $\displaystyle \forall \lambda \in [0,1], \forall (x,y)\in (0,\infty), f((1-\lambda)x+\lambda y) = \int_0^{\infty} g_t((1-\lambda)x+\lambda y) dt \; \; \; \; (2)$
$(1)$ and $(2)$ imply $\displaystyle \forall \lambda \in [0,1], \forall (x,y)\in (0,\infty), f((1-\lambda)x+\lambda y) \leq (1-\lambda)f(x)+\lambda f(y) $
Hence the convexity of $f$.
- I'm going to do it the hard way (by means of sequences)
Let $(x_n)$ be an arbitrary sequence of positive numbers that goes to $0$
Let us prove that $f(x_n)$ goes to $\displaystyle \frac{\pi}{2}$
Consider \begin{array}{ccccc}
K_n & : & \mathbb R^+ & \to & R\\
& & t & \mapsto & \arctan(t/x_n)e^{-t} \\
\end{array}
$K_n$ is a continous function for each $n$, and the sequence $(K_n)$ converges pointwise to $\displaystyle t\to \frac{\pi}{2}e^{-t} $, which is continuous as well.
Moreover, $|K_n(x)|\leq \frac{\pi}{2}e^{-t} $ and $\displaystyle t\to \frac{\pi}{2}e^{-t} $ is integrable over $\mathbb R^+$ which yields domination.
Therefore, $\displaystyle \lim_{n\to \infty} \int_{0}^{\infty} K_n(t) dt = \int_{0}^{\infty} \lim_{n\to \infty} K_n(t) dt$
Which can be rewritten as $\displaystyle \lim_{n\to \infty} \int_{0}^{\infty} \arctan(t/x_n)e^{-t} dt = \int_{0}^{\infty} \frac{\pi}{2}e^{-t}$
This, in turn, is equivalent to $\displaystyle \lim_{n\to \infty} f(x_n)= \frac{\pi}{2} \int_{0}^{\infty} e^{-t} = \frac{\pi}{2}$
I've proven that for any sequence of positive numbers $(x_n)$ that goes to $0$, $f(x_n)$ goes to $\frac{\pi}{2}$. This implies that $\displaystyle \lim_{x \rightarrow 0}f(x)=\frac{\pi}{2}$
It is easy to prove that $\forall y \geq 0, \arctan(y) \leq y$
Hence $\displaystyle \int_{0}^{\infty} \arctan(t/x)e^{-t} dt \leq \int_{0}^{\infty} \frac{t}{x}e^{-t} dt $
So $\displaystyle f(x) \leq \frac{1}{x} \int_{0}^{\infty}te^{-t} dt$
And $\displaystyle 0\leq f(x) \leq \frac{1}{x}$
Taking limits as $x\to \infty$, $\displaystyle \lim_{x \rightarrow \infty}f(x)=0$
- $f$ is a decreasing continuous function over $(0,\infty)$.
Hence $\displaystyle \qquad \sup_{x>0} f(x) = \lim_{x \rightarrow 0}f(x)=\frac{\pi}{2}$
And $\displaystyle \qquad \inf_{x>0} f(x) = \lim_{x \rightarrow \infty}f(x)=0$
I leave the last part to you. You just need to justify differentiation under the integral sign.