Here's a proof of a more general result. Here I let $\alpha \in \Omega^1(M)$, I define
$$s_\alpha: M \longrightarrow T^\ast M,$$
$$p \mapsto (p, \alpha_p),$$
and I write $M_\alpha$ for the image of $s_\alpha$. Recall that the canonical symplectic form $\omega_{\text{can}}$ on $T^\ast M$ is exact,
$$ \omega_{\text{can}} = - d\lambda,$$
where $\lambda$ is the Liouville form given locally by
$$\lambda = \sum_{i = 1}^n \xi_i ~dx_i$$
and globally by
$$\lambda_{(x,\xi)} = (d\pi_{(x,\xi)})^\ast \xi,$$
where $\pi: T^\ast M \longrightarrow M$ is the bundle projection.
With the above setup, I claim that
$$s_\alpha^\ast \lambda = \alpha.$$
To see this, recall that by definition,
$$\lambda_p = (d\pi_p)^\ast \xi$$
where $p = (x, \xi) \in T^\ast X$ and $\pi: T^\ast X \longrightarrow X$ is the bundle projection. Then we have
$$\lambda_{s_\alpha(x)} = (d\pi_{s_\alpha(x)})^\ast \alpha_x,$$
and hence
\begin{align*}
(s^\ast_\alpha \lambda)_x & = (ds_\alpha)^\ast_x \lambda_p \\
& = (ds_\alpha)^\ast_x (d\pi_{s_\alpha(x)})^\ast \alpha_x \\
& = (d\pi_{s_\alpha(x)} \circ (ds_\alpha)_x)^\ast \alpha_x \\
& = d(\pi \circ s_\alpha)_x^\ast \alpha_x \\
& = \alpha_x,
\end{align*}
since $s_\alpha$ is a section and thus $\pi \circ s_\alpha = \mathrm{Id}_X$. Therefore
$$s_\alpha^\ast \lambda = \alpha.$$
Theorem. Let $\alpha \in \Omega^1(M)$. Then $M_\alpha$ is a Lagrangian submanifold of $(T^\ast M, \omega_\mathrm{can})$ if and only if $\alpha$ is closed.
Proof. Since $M_\alpha$ is the image of the closed embedding $s_\alpha$, we have that
\begin{align*}
M_\alpha \text{ is Lagrangian} & \iff s_\alpha^\ast \omega_\text{can} = 0 & & \\
& \iff s_\alpha^\ast d\lambda = 0 & & (\omega_\text{can} = -d\lambda) \\
& \iff ds_\alpha^\ast \lambda = 0 & & (d \text{ commutes with pullbacks}) \\
& \iff d\alpha = 0, & & (s_\alpha^\ast \lambda = \alpha)
\end{align*}
from which the result follows.
