Right now I am coding up optimization for a 70 dimension nonlinear optimization, where the analytical gradient is unavailable.
I have some non-linear constraints that maps the structural parameters into reduced-form matrices in a non-trivial way. Sometimes one of the reduced-form matrices is close to singular, and I need to take the inverse of the matrix.
To avoid the issue, I currently check the conditioning number of the matrix in question, and if it is sufficiently large I create a discontinuity in the objective function.
I've learned that gradient-descent algorithms don't play well with discontinuities, but I am not sure what the best way to code this constraint with my solver.
BTW I am using MATLAB's FMINCON's Interior-Point solver.