I am trying to solve a high dimensional (up to ~50) class of data fitting & modelling problems. The user specifies the problem, so I would like to make the configuration as easy as possible, and the method as robust as possible. Some of the potential inputs are distributions which makes MCMC an attractive algorithm for fitting without specifying a potentially innaccurate model. I have corruption in the input data of up to an order of magnitude so theoretically well founded detection of outliers is also a very attractive aspect of MCMC. The data spans ~5 orders of magnitude within a variable. Some of the variables are bounded on the [0,1] internal, some bounded positive and some unbounded.
Transform unbounded candidates from MCMC to bounded variables as required with tan and log transforms. The user is required to specify the type of variable.
Use Parallel Tempering Ensamble MCMC (pt variant of the MCMC Hammer) on a gaussian mixture model per http://dan.iel.fm/posts/mixture-models/
The log-likelihood is specified in terms of the probability of non-zero difference between x/y and y/x to minimise scaling problems and avoid specifying a relative standard deviation, which feels arbitrary
Determine a Principle Component mapping in either the bounded or unbounded space (not sure yet) to reduce the dimensionality of the problem
Re-run the ptMCMC Hammer in the reduced dimensional space - I can now do a longer run given RAM constraints because each draw from the posterior is smaller
Map back to high dimensional real space to report distributions and best estimates for the fit
As required, predict based on interpolation in the reduced dimensional space
My questions are:
Does this sound like a good or even plausible algorithm to an expert? There is non-trivial work involved in fully testing this approach, so I'd appreciate an opinion.
Do the transforms from bounded to unbounded space (or any non-linear transform in general) invalidate the results of MCMC in the sense of getting draws from the true stable posterior? I know that the uninformative distribution is no longer flat (ie. no longer maxent), which is concerning. Can I fix this with a prior?
Per (2) for PCA?
Is any other step of the algorithm fatally flawed?
I appreciate your time and apologise for my poor use of Bayesian language. My first language is Chem Eng. I wouldn't be trying to do this if the problem didn't justify it. Please tell me if the questions are ill-posed or there is insufficient information.