What is the difference between strong and weak local optima? I have seen these term being used, what are their differences?
What does it mean to find a strong local optima?
 A: If you are talking about an optimization problem, you should know that a local optimum is a solution which is optimal with respect to a neighboring set (https://en.wikipedia.org/wiki/Local_optimum)
The distinction between weak and strong local optima is simply the fact whether they are the unique such points in the neighborhood.
The local optima in the following image are strong 

whereas, if this function had an interval with constant value (and minimum value in that interval) then those points would be weak local optima
A: In the calculus of variation, the definition of strong or weak local optima refers to the norm chosen on the space of differentiable functions.
Let $E=C^1([a,b])$ for some $b>a$. On the space $E$, we can define the norm $\|f\|_{C^0}=\max_{x \in [a,b]}|f(x)|$, and the norm $\|f\|_{C^1}=\max_{x \in [a,b]}|f(x)|+\max_{x \in [a,b]}|f'(x)|$ (in the latter case, $E$ is complete).
Now, let $A$ be a subset of $E=C^1([a,b])$, and $I$ a functional on $A$.
One says that $f_0$ is a weak miminum of the functional $I$ if there exists $R>0$ such that
$I[f]\geq I[f_0]$  for all $f\in A$ such that $\|f-f_0\|_{C^1}<R$.
One says that $f_0 \in A$ is a strong miminum of the functional $I$ if there exists $R>0$ such that
$I[f]\geq I[f_0]$ for all $f\in A$ such that $\|f-f_0\|_{C^0}<R$.
Since the $C^0$ norm is weaker than the $C^1$ norm, then a strong minimum is a fortiori a weak minimum.
