Why is every $f_*\mathcal{O}_X$-module also an $\mathcal{O}_X$-module?

Let $f:X \rightarrow Y$ be a finite morphism of locally Noetherian schemes. If $\mathcal{F}$ is an $f_*\mathcal{O}_X$-module, why is $\mathcal{F}$ endowed with the structure of an $\mathcal{O}_X$-module?

This should be really basic, but I struggle to see this. For each open subset $U \subset Y$ we can multiply $\mathcal{F}(U)$ by scalars in $\mathcal{O}_X(f^{-1}(U))$, thus $\mathcal{F}$ gives us for each open subset in $Y$ an $\mathcal{O}_X$-module, but not for each open subset of $X$...

Why is $\mathcal{F}$ endowed with the structure of an $\mathcal{O}_X$-module?

When people say this, this is a slight abuse of language. Strictly speaking, a sheaf on $Y$ cannot be an $\mathcal{O}_X$-module unless $X = Y$. However, for finite (more generally, affine) morphisms $f\colon X\to Y$, the direct image functor induces an equivalence between quasi-coherent $\mathcal{O}_X$-modules and quasi-coherent $f_*\mathcal O_X$-modules. That is, for every $f_*\mathcal O_X$-module $\mathcal F$ there is a unique (up to isomorphism) $\mathcal O_X$-module $\mathcal G$ such that $f_*\mathcal G\cong \mathcal F$ We say that $f_* \mathcal G$ has an $\mathcal{O}_X$-module structure since this is what happens in the affine world: if $X = \mathrm{Spec}(A)$ and $Y = \mathrm{Spec}(B)$, then $f$ corresponds to a ring homomorphism $B\to A$ and if $\mathcal G = M^\sim$, then $f_*\mathcal G = M_B^\sim$ where $M_B$ is just $M$, but considered as a $B$-module.