Does the equation $\displaystyle min_{x,y} f(x,y)=min_{y}min_{x}f(x,y)=min_{x}min_{y}f(x,y)$ always hold for a minimization problem? For a minimization problem of function $f(x,y)$, does the equation $min_{x,y} f(x,y)=min_{y}min_{x}f(x,y)=min_{x}min_{y}f(x,y)$ always hold?
I saw this equation somewhere. It does the minimization in a sort of iterated way, treating either $x$ or $y$ as a constant first. But what about a saddle point? If I use Hessian matrix to consider the convexity of $f(x,y)$, it will let me know it doesn't work if there's a saddle point, while this iterated method may still give me a value.
I think this kind of problems falls into optimization, and I also wonder if  $min_{y}max_{x}f(x,y)=max_{x}min_{y}f(x,y)$ holds.
 A: Let $X$ and $Y$ be arbitrary non-empty sets and let $f:X\times Y\to\mathbb R$ be a function.

Claim: $\inf_{(x,y)\in X\times Y}f(x,y)=\inf_{x\in X}\inf_{y\in Y}f(x,y)$.


This includes the possibility that any of the infima is $-\infty$. If all of the infima involved are actually attained, then you can replace “$\inf$ ” with “$\min$.”
Therefore, the answer to your question is affirmative, with the caveat that $\min_{y\in Y}f(x,y)$ may not exist (because the infimum value is not attained by any $y\in Y$) for some of the $x\in X$, even if the joint minimum $\min_{(x,y)\in X\times Y}f(x,y)$ does exist.

Proof: For any $x_0\in X$ and and $y_0\in Y$, it is clear that $$f(x_0,y_0)\geq\inf_{(x,y)\in X\times Y}f(x,y).$$ Leaving $x_0$ fixed and allowing $y_0$ to vary, we then have that $$\inf_{y\in Y}f(x_0,y)\geq\inf_{(x,y)\in X\times Y}f(x,y).$$ This is true for any fixed $x_0\in X$, so we can now take infimum over $X$ to obtain $$\inf_{x\in X}\inf_{y\in Y}f(x,y)\geq\inf_{(x,y)\in X\times Y}f(x,y).$$
For the reverse inequality, fix again $x_0\in X$ and $y_0\in Y$. Clearly, we have that $$\inf_{x\in X}\inf_{y\in Y}f(x,y)\leq\inf_{y\in Y}f(x_0,y)\leq f(x_0,y_0).$$ Now we can take infimum on the right-hand side over all of $X\times Y$ to obtain that $$\inf_{x\in X}\inf_{y\in Y}f(x,y)\leq\inf_{(x,y)\in X\times Y}f(x,y),$$ which completes the proof. $\quad\blacksquare$

For the interchangeability of the maximum and minimum operators, please consult this thread.
