# Setting

One version of the Kolmogorov-Arnold representation theorem (cf. [1, Thm. 1]) states

Theorem: Fix $$d \geq 2$$. There are real numbers $$a, b_p, c_q$$ and a continuous and monotone function $$\psi: \mathbb{R} \rightarrow \mathbb{R}$$ such that for any continuous function $$f: [0, 1]^d \rightarrow \mathbb{R}$$, there exists a continuous function $$g: \mathbb{R} \rightarrow \mathbb{R}$$ with $$f(x_1, \ldots, x_d) = \sum_{q = 0}^{2d} g\bigg( \sum_{p = 1}^{d} b_p \psi(x_p + qa) + c_q \bigg). \tag{1}$$

Definition: Two-hidden-layer feedforward neural networks with activation functions $$\sigma_1$$ and $$\sigma_2$$ (in the second resp. the first layer) are defined as

$$f(\mathbf{x}) = \sum_{q = 1}^{m_1} d_q\, \sigma_1\bigg( \sum_{p = 1}^{m_2} b_{pq}\, \sigma_2(\mathbf{w}_p^T\mathbf{x} + a_p) + c_q \bigg) \tag{2}$$ where $$\mathbf{w}_p \in \mathbb{R}^d$$, $$a_p, b_{pq}, c_q, d_q \in \mathbb{R}$$ are parameters.

# Question

According to [1], the RHS of (1) can be represented via a two-hidden-layer feedforward neural network with activation functions $$g$$ and $$\psi$$ (in the second resp. the first layer). I agree that it looks similar to (2) but I could not find a sensible choice of the parameters.

Choosing $$m_1 = 2d + 1, m_2 = d, d_q = 1, \sigma_1 = g, \sigma_2 = \psi$$, $$b_{pq} = b_p$$ where $$b_p$$ are independent of $$q$$ and $$\mathbf{w}_p$$ being the $$p$$-th unit vector, then I get $$f(\mathbf{x}) = \sum_{q = 1}^{2d + 1} g\bigg( \sum_{p = 1}^{d} b_{p} \psi(x_p + a_p) + c_q \bigg). \tag{3}$$ However, I am stuck now because I don't understand why $$a_p$$ can be chosen dependent on $$q$$ (which choosing $$a_p = qa$$ would do). Is that allowed in a neural network setting?

# References

[1] https://arxiv.org/abs/2007.15884 (published in Neural Networks 137 (2021) 119–126)