2
$\begingroup$

It is well known that the maximum likelihood estimator of logistic regression does not admit a closed form solution, at least in the general case where the predictors are not binary or categorical. Whereas, ordinary least squares regression does have a closed form solution in terms of matrix inverses.

Is there a proof that logistic regression can't have a closed form solution or it is simply that none have been found after an extensive search? Proofs of the non-existence of things are presumably difficult in general. Is there at least an agreed upon definition of closed form in this context?

$\endgroup$
2
  • $\begingroup$ B/c convergence of iterative methods is not guaranteed (at least according to the wikipedia article), then it seems like the corrsponding optimization is non-convex. This is by no means a proof, but generally speaking non-convex optimization problems usually don't have "easy exact solutions" which usually rules out the existence of a "closed form solution", if for no other reason than the existence of a "closed form solution" would imply an "easy" algorithm for the solution. math.stackexchange.com/questions/1985008/… $\endgroup$ Feb 15 at 1:49
  • $\begingroup$ Cf. also arxiv.org/abs/1712.07897 Although anyway apparently I am wrong and the log-likelihood for logistic regression is convex stats.stackexchange.com/questions/295920/… -- admittedly it is still fairly common that convex optimization problems lack closed form solutions, but that being said because most are not NP-hard it follows that NP-hardness can not be used as a "proof" of "non-existence of a closed form". Anyway I agree with you that this is a good question and evidently I don't know the answer. Sorry $\endgroup$ Feb 15 at 1:53

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.