This is a bit technical. I will try to outline a proof. Let $\mathbb{R}^{n}$ and $\mathbb{R}^{n+1}$ be both endowed with the maximum norm. First consider an open ball $B((x_0,y_0),r)$ in $\mathbb{R}^{n+1}$ such that $C:=\overline{B((x_0,y_0),r)}\subseteq O$. Now $g$ is bounded on the compact set $C$ by $b> 0$, say:
$$
\|g(x,y)\|_\infty \le b \quad ((x,y) \in C).
$$
By the multivariate Weierstraß Approximation Theorem there is a sequence of polynomials $(p_m)_{m=1}^\infty$ such that
$$
\|p_m(x,y)-g(x,y)\|_\infty \le \frac{1}{m} \quad ((x,y) \in C,~ m \in \mathbb{N}).
$$
In particular
$$
\|p_m(x,y)\|_\infty \le 1+b \quad ((x,y) \in C, ~ m \in \mathbb{N}).
$$
By the Picard-Lindelöf Theorem each IVP
$$
u'(x)=p_m(x,u(x)), \quad u(x_0)=y_0
$$
has a solution $u_m:I:=[x_0-\delta,x_0+\delta] \to \mathbb{R}^{n}$ with $\delta:=r/(1+b)$. Note that $(x,u_m(x)) \in C$ $(x \in I)$. Each $u_m$ is Lipschitz continuous on $I$ with constant $1+b$ and $u_m(x_0)=y_0$. Hence $\{u_m:m \in \mathbb{N}\}$ is a bounded and equicontinuous subset of $C(I, \mathbb{R}^{n})$, hence relatively compact by Arzela Ascoli. Let $(u_{m_k})$ be a uniformly convergent subsequence of $(u_m)$ with limit $v\in C(I, \mathbb{R}^{n})$, say. Note that then also $g(x,u_{m_k}(x)) \to g(x,v(x))$ uniformly on $I$ as $k \to \infty$.
Now for $x \in I$:
$$
\|v(x)-y_0-\int_{x_0}^x g(t,v(t))dt\|_\infty
$$
$$
\le \|v(x)-u_{m_k}(x)\|_\infty + \|\int_{x_0}^x p_{m_k}(t,u_{m_k}(t))-g(t,u_{m_k}(t))dt\|_\infty
+ \|\int_{x_0}^x g(t,u_{m_k}(t))-g(t,v(t))dt\|_\infty
$$
$$
\le \|v(x)-u_{m_k}(x)\|_\infty + \frac{1}{m_k}|x-x_0|
+ \|\int_{x_0}^x g(t,u_{m_k}(t))-g(t,v(t))dt\|_\infty \to 0 \quad (k \to \infty).
$$
Thus
$$
v(x)=y_0 + \int_{x_0}^x g(t,v(t))dt \quad (x \in I).
$$
Edit: I meanwhile looked in the book of Royden and Fitzpatrick, and in fact the exercise is a bit missleading. The function $g$ there is assumed to be continuous (nothing else). In this exercise the authors talk about "a form of Picard's existence theorem" which is incorrect then; it is Peano's existence theorem. If in addition $g(x,y)$ is assumed to be Lipschitz in $y$ an approximation by Lipschitz continuous functions makes no sense to me. Moreover the proofs of the Picard Lindelöf theorem for $n=1$ and $n>1$ are almost the same, just by replacing the absolut value by a norm. So my answer shows a way to prove Peano's theorem via the Picard Lindelöf theorem, but after all I am not sure what this exercise is asking for. I think that this exercise is just poorly designed.