# Estimate Gaussian Mixture Model (GMM) Parameters Embedded in Linear System

For the linear equation $$Ax=b,$$ where $$A$$ is a 100 x 64 matrix, $$b$$ is a 100 x 1 vector, $$x$$ is a 64 x 1 vector, unknown. In my situation, x consists of no more than 5 Gaussian basis. $$x=\Sigma_{i=1}^5 F_i \exp\left(-\frac{(T-T \mathrm{mean}_i)^2}{W_i}\right)$$ Here T is a known 64 x 1 vector, $$W_i$$ controls the width of the distribution and $$F_i$$ is the amplitude of the Gaussian basis. $$F_i$$, $$W_i$$ and $$T\mathrm{mean}_i$$ are undetermined parameters.

$$A$$,$$b$$,$$T$$ are attached here.

What I am interested in is the $$T\mathrm{mean}_i$$, i.e. the number of Gaussian basis and the center of the existing Gaussian ($$A_i>0$$).

• This is a non linear optimization problem.
– Royi
Commented Oct 25, 2022 at 6:09
• Cannot be solved by gradient-based optimization algorithms? @Royi Commented Nov 7, 2022 at 0:48
• The function is smooth, so you can do gradient descent. But the convergence is to a local minima, not the global one.
– Royi
Commented Nov 7, 2022 at 11:21
• I think your y.csv file has some issues. You should remove all ,. Also the first 7 values are integers numbers (The format is float, yet the values are integers). Does it make sense?
– Royi
Commented Jun 30 at 10:22

I can see 2 approaches to tackle this:

1. Solve in 2 Steps
Basically $$\boldsymbol{x}$$ is a 1D Gaussian Mixture Model (GMM) data.
So you can find $$\hat{\boldsymbol{x}}$$ using Linear System Solver and in the next step apply GMM to find the amplitude, mean and variance of the data.
2. Solve Using Non Linear Least Squares
The model has $$5 \times 3$$ parameters.
One could use Non Linear Least Squares method to estimate the parameters.
One could use (1) as a starting point.

The y.csv (Which is $$\boldsymbol{b}$$ in the question) required removing the , from the file before reading.

I used Julia to solve the problem.
At first I tried Non Linear Least Squares solvers: LsqFit.jl and LeastSquaresOptim.jl.
The problem was easily solved by them so no need to do option (1):

MSE of LeastSquaresOptim with LM: 0.003
MSE of LeastSquaresOptim with Dogleg: 0.005
MSE of LsqFit with LM: 0.006
The LeastSquaresOptim with LM solution run time: 0.14207 [Sec]
The LeastSquaresOptim with Dogleg solution run time: 0.06752 [Sec]
The LsqFit with LM solution run time: 0.14235 [Sec]

As one can see, the fit is almost perfect (The dashed lines are on the purple line of the measurements).

The code is available on my StackExchange Mathematics GitHub Repository (Look at the Mathematics\Q4555441 folder).