Intuition behind a nonlinear trajectory of a minimum point of a linear combination of two quadratic equations Say there are two quadratic equations
$$f_1(x)=x^T A_1 x + b_1^T x + c_1 $$
$$f_2(x)=x^T A_2 x + b_2^T x + c_2 $$
where $A_1$ and $A_2$ are symmetric positive semi-definite matrices.
I am inspecting a trajectory of
$$\arg\min_x f_1(x)+\lambda f_2(x)$$
as I vary $\lambda$.
A visual example is shown below, where there are contours of two 2D quadratic formulas, and the trajectory of the critical point is shown in red (sweeping from $\lambda=0$ to $\lambda \rightarrow \infty$).

I am trying to have an intuitive understanding of this trajectory. This is by no means an unexpected/surprising result, but why is the trajectory nonlinear? Why does it seem to align itself to the larger principal axes of the contours? I think it should be explainable with eigenvalues and eigenvectors of $A_1+\lambda A_2$, but I need some help proceeding from here. Perhaps the intuition can be provided through answers to the following questions that I think are interesting:

*

*For a 2D $x=[x_1,x_2]$, it is possible for the trajectory to be nonmonotonic in both coordinates ($x_1$ and $x_2$)?


*For $n$D (n>2), what is the maximum number of coordinates where the trajectory is moving non-monotonically?
 A: Some notations first. Let
$$f(x):=f_1(x)+\lambda f_2(x)$$
Let $M_1=\arg\min_x f_1(x)$, $M_2=\arg\min_x f_2(x)$ and $M_{\lambda}=\arg\min_x (f_1(x)+\lambda f_2(x))$.
A) Explicit computations:
$$f(x)=x^T\underbrace{(A_1+\lambda A_2)}_{A_{\lambda}}x+\underbrace{{(b_1+\lambda b_2)}^T}_{b_{\lambda}^T}x+(c_1+\lambda c_2)$$
has its minimum where the gradient is $0$, i.e., for an $x$ such that $$2A_{\lambda}x+b_{\lambda}=0 \tag{1}$$
(1) gives the solution:
$$x=M_{\lambda}=-\frac12 A_{\lambda}^{-1}b_{\lambda}$$
Taking generic matrices, one obtains a parameterization of the (red) curve of $M_{\lambda}$:
$$M_{\lambda}=\begin{pmatrix}x(\lambda)\\ y(\lambda)\end{pmatrix}=-\tfrac12 {\underbrace{\begin{pmatrix}r+\lambda r'&b+\lambda s'\\s+\lambda s'&t+\lambda t'\end{pmatrix}}_{A_{\lambda}}}^{-1}\underbrace{\begin{pmatrix}u+\lambda u'\\v+\lambda v'\end{pmatrix}}_{b_{\lambda}}\tag{2}$$
which, being of the form
$$\begin{cases}x(\lambda)=p(\lambda)/d(\lambda)\\ y(\lambda)=q(\lambda)/d(\lambda)\end{cases} \ \text{where} \ deg(p), deg(q), deg(d) \le 2,\tag{2'}$$
is a conic curve (in fact an ellipse because $d(\lambda)$ being proportional to the determinant of a symmetric positive-definite matrix is never $O$).
Now, let us come back to (2). Its derivative with respect to $\lambda$ is the derivative of a product, giving:
$$S(\lambda)=-\frac12 \left(-(A^{-1}(\lambda)A'(\lambda)A^{-1}(\lambda))b(\lambda)+A(\lambda)b'(\lambda)\right)\tag{3}$$
Taking $\lambda=0$ in (3), we obtain the initial speed vector of parametric curve (2) as being:
$$S(0)=\frac12 \left(A_1^{-1}A_2A_1^{-1}b_1-A_1^{-1}b_2\right)\tag{4}$$
which shows that the initial direction is not in general an eigenvector of quadratics $f_1$ nor of $f_2$. The same kind of answer could be given "mutatis mutandis" for the final direction.
Remark: In (1) we have an absolute minimum because we work with positive definite matrices.
B) On the side of intuition:
Think to a representation in terms of surfaces
$$z=f(x)$$
Let us take $\lambda \to -\infty$ which does not change the issue and will be easier to explain.
This surface is a "blend" of 2 paraboloids. More precisely, according to the value of $\lambda$:

*

*$\lambda$ small: The surface has a minimum close to the minima of $f_1$, i.e., $M_{\lambda}\approx M_1$, with a little bump in the direction of $M_2$.


*$\lambda$ around $1$: The surface looks like a big trouser pocket shaped as a "thalweg with a slope" at the bottom; $M_{\lambda}$ is approximately in between $M_1$ and $M_2$.


*$\lambda >> 1$: in this case, we have the inverse of the first case: the first little pocket has been almost resorbed :$M_{\lambda}\approx M_2.$
Edit: Motivation for (2'): Let us take $t:=\lambda$. One can start from the Weierstrass substitution formulas $$\begin{cases}x=\cos \theta = \dfrac{1-t^2}{1+t^2}\\y=\sin \theta=\dfrac{2t}{1+t^2}\end{cases}$$ giving the unit circle. Then apply to vector $\binom{x}{y}$ a linear or affine transformation transforming into any other ellipse with still the form $p(t)/r(t),q(t)/r(t).$
