Given a non-convex differentiable function $f$ and a set $S$ of possible input vectors for function $f$. I am trying to find the (global) optimum input vector out of the possibilities in set $S$. Linear search can't be applied because of the size of set $S$. So probably the most common way to do it is using e.g. heuristics, but then IMHO there are two possible approaches, and I am not sure which is better (or if maybe there are other ways):
- Only trying the possible vectors (from set $S$) in my heuristic and iteratively getting to some approximate
- By not considering it as a combinatorial problem, and just finding the approximate optimum of $f$ and afterwards searching for the nearest neighbor in set $S$ (e.g. maybe using R* trees or similar data structures)
Which solution is better? I feel like solution 1 doesn't use the differentiable property of function $f$ in any way. Are there better solutions, even exact ones?