# Solving an inverse problem with machine learning

I am running up against a very tough inverse problem that I suspect might be solvable using machine learning. Here is the problem.

Overview

I am studying an object $X$ which, internally, is composed of two functions $f_1(x)$ and $f_2(x)$, $x \in [0,1]$. I want to know what $f_1$ and $f_2$ are, but I can't measure them directly.

What I can measure is a list of numbers $\omega_1, \omega_2, \ldots, \omega_N$ which are determined by $f_1$ and $f_2$. Using these numbers $\vec \omega$, I want to infer $f_1$ and $f_2$ the best I can.

The current approach

When $N$ is very large and the measurement errors on each of the $\omega$'s is small, the following method works well.

I have a model that I consider to be a good but not exact description of $X$. It differs in its internal $f_1$ and $f_2$'s, which causes differences in $\omega$. Additionally, there is an arbitrary polynomial difference in $\omega$ whose form is known but whose coefficients are unknown.

Using the relative differences between the $\vec \omega$'s that come from the model and the $\vec \omega$'s that I observe in $X$, and also fitting for the polynomial difference in $\omega$, I can calculate the relative difference in $f_1$ and $f_2$ between the model and $X$ using the following system of equations:

$$\delta \omega_i = c_1(\delta\omega)^{-1} + c_2(\delta\omega)^3 + \int (K^{f_1}_i \cdot \delta f_1 + K^{f_2}_i \cdot \delta f_2) \; \text{d}x, \qquad i=1,\ldots,N$$ where $c_1$ and $c_2$ are constants that have to be determined (in the least squares sense) along with $\delta f_1$ and $\delta f_2$. The functions $K^{f_1}_i$ and $K^{f_2}_i$ are known functions called kernel functions that do not need to be estimated - they come for free from the model.

Clearly this is an inverse problem because $\delta f_1$ and $\delta f_2$ are trapped in that integral. However, by representing $\delta f_1$ and $\delta f_2$ by basis functions, we can estimate the coefficients of the basis functions using normal techniques (e.g. regularized least squares) and then determine $f_1$ and $f_2$.

A new approach?

I am studying $X$'s for which $N$ is small and the measurement error on the $\omega$'s is large. The previous method fails to give reliable estimates, which I can test by using two models and pretending one of them is $X$.

I am thinking that a machine learning approach might work here. In particular, I can calculate a large number of models and ask how their values of $\vec \omega$ correspond to $f_1$ and $f_2$'s. Then, I can put in the $\vec \omega$ that I observe and get out the real $f_1$ and $f_2$. That is, I will train a machine learning algorithm to predict

$$f(\vec \omega) = [f_1(x_i), f_2(x_i)], \; x_i \in [0.01, 0.02, \ldots, 1.00]$$

based on a large number of simulations.

The problem that I am coming up against is the arbitrary function of $\omega$ that I mentioned in the previous section. There is always a difference in practice between the real $\vec \omega$ and the model $\vec \omega$. We know the form of the difference is a polynomial with a cubic term and an inverse term, but we don't know the coefficients - they are determined simultaneously when determining the differences due to $f_1$ and $f_2$.

Any advice?

• Many kinds of statistical questions get great answers on this site, but for machine learning maybe you should also post on Cross Validated. – BruceET Apr 9 '17 at 6:57