Optimization approaches to solving PDEs

Question

In modern numerical methods, a PDE is often recast into the form of a variational problem, which is sometimes equivalent to a minimization problem. However in my courses on numerical analysis (say, finite element methods) the focus is not (apparently) on developing optimization techniques to minimize the arosen energy functional, but rather on approximating the variational problem on a smaller subspace.

Are there interesting approaches that focus on the minimization of the energy directly? Is research being done in this field, and could you maybe provide some reference?

Answer 1

I'll try to answer to your questions following the order of their appearance.

Are there interesting approaches that focus on the minimization of the energy directly?

Yes, there are several approaches to the solution of this problem: they are all known under the collective name of "Variational Method". This "method" has its root in the direct method in the calculus of variation which was pioneered by Leonida Tonelli, and aims to find the minimum of a functional by directly evaluating its value on a properly defined (sub)sequence of functions for which it is defined, without the need to calculate its functional derivative and solve the associated Euler-Lagrange DEs. Contrary to some widespread belief, the variational method is widely applicable to general (system of) PDEs, as we'll see below.

Let's start our description by saying that, when a functional $F$ which is equivalent to the given (system of) PDEs has been constructed, the general variational method can be described by mimicking the standard approach to direct method in the calculus of variation:$\DeclareMathOperator{\DM}{\mathrm{Dom}}$

1. show that the functional $F$ is (sequentially) lower semicontinuous (or upper semicontinuous, if we are searching for a maximum instead of a minimum) respect to a topology chosen on its domain of definition $\DM F$.

2. chose a minimizing sequence i.e. a sequence of functions $\{u_n\}$ contained in the domain $\DM F$ such that $$ \lim_{n\to +\infty} F[u_n]=\inf_{u\in\DM F}F(u) $$

3. Show that the chosen sequence $\{u_n\}$ admits a subsequence $\{u_{n_k}\}$ covering to $u\in \DM F$ respect to the given topology. Then $u$ is the sought for solution.

Differently from what happens in the direct method, where the functional is customarily known from the start, step 1. for the variational method is customarily done during the construction of $F$. For example, the very general method for giving a variational formulation to any operator equation worked out by Enzo Tonti (see for example [3] and [1], chapter IV, §17, pp. 168-182) produces the a functional which has a minimum $u$ if and only if $u$ solves the initial operator equation, implicitly proving its semicontinuity.

Finally, what are the differences between the various procedures that go under this collective name? They use different methods to construct the minimizing sequence and its subsequence: for example the method of orthonormal series, the Ritz (or Rayleigh-Ritz) Method, the Galerkin Method or its variants as the Faedo- Bunbnov- Galerking Methods, the Least Squares Method, The Courant Method and The Method of Steepest Descent are all variational methods.

Is research being done in this field, and could you maybe provide some reference?

Yes, there is currently a large deal of investigation on the variational method, due to its wide applicability to standard problems in engineering and physics: try googling one of the names given above, just to see the number of the results that you'll get. As a basic reference for the standard techniques, I'd suggest [2], part III, chapters 9-17, pp. 113-178, while the problem of constructing a functional "equivalent" to a given (system of) PDEs, technically called "the inverse problem of the calculus of variation", is comprehensively addressed in references [1] and [3]: the first one gives also a very general procedure for solving very general nonlinear equations (chapter IV, §18, pp. 182-198) due to A. D. Lyashko.

Final notes

• The approach used in the solution of the inverse problem of the calculus of variation is briefly addressed also in this and this Q&A.
• As I briefly alluded to in the answer to the first part of the question, it is a widespread belief that only some PDEs (or more generally some operator equations) admit a variational formulation, even if the works of Tonti, Filippov and others have shown that the inverse problem of the calculus of variation has a generalized solution for large classes of operator equations. However, the belief is not entirely unjustified since it is true that there are large classes of functional formulations of PDE problems for which the functional derivative does not lead to the Euler-Lagrange equations: for example Copson [A1] proves that the ordinary heat equation (and more generally equations of parabolic type) is not the Euler-Lagrange equation of any functional. But it also clear from the works [1] and [2] that such kind of equations do admit a variational formulation.

References

[1] Filippov, Vladimir Mikhailovich, Variational principles for nonpotential operators. With an appendix by the author and V. M. Savchin, Transl. from the Russian by J. R. Schulenberger. Transl. ed. by Ben Silver, Translations of Mathematical Monographs, 77. Providence, RI: American Mathematical Society (AMS). pp. xiii+239 (1989), ISBN: 0-8218-4529-2. MR1013998, Zbl 0682.35006.

[2] Karel Rektorys, Variational methods in mathematics, science and engineering, Translated from the Czech by Michael Basch. 2nd ed. (English), Dordrecht - Boston - London: D. Reidel Publishing Company, pp. 571 (1980), ISBN: 90-277-1060-0, MR0596582, Zbl 0481.49002.

[3] Tonti, Enzo, "Extended variational formulation", Vestnik Rossiĭskogo Universiteta Druzhby Narodov, Seriya Matematika 2, No. 2, 148-162 (1995). Zbl 0965.35036.

Addendum reference

[A1] Edward Thomas Copson, "Partial differential equations and the calculus of variations" Proceedings of the Royal Society of Edinburgh 46, 126-135 (1926), JFM 52.0509.01.

Answer 2

Many physical phenomena are a result of dynamic/static equilibrium energy/work conservation in physics/mathematical physics. A pde represents a phenomenon based on the laws of physics that obey the phenomenon.

This answer would do little justice to so many phenomena in Science.

Kontorovich's text book in Variational methods is informative in geometry. A particular case from geometry in linear plate theory by Sophie Germaine:

$$\left(\dfrac {\partial^2 }{\partial x^2}+\dfrac {\partial^2 }{\partial y^2} \right)\left(\dfrac {\partial^2 w}{\partial x^2}+\dfrac {\partial^2 w}{\partial y^2} \right)= \dfrac{p}{D}$$

where $ \text{ bending deformation w, pressure p, D plate bending rigidity} $ are obtained by neglecting second order terms from

$$ A \Delta(e_i e_j) = B \Delta(e_i \kappa_j) = D \Delta(\kappa_i \kappa_j) $$

where $(A,B,D) $ are in-plane , cross coupling and stiffness coefficients relating double strain, cross strain-curvature and Gauss curvature that are involved on von Kármán's equations of pure geometric non-linearity...

Navier-Stokes equation has not been generally solved using known mathematical functions.

Answer 3

A good start to gather some insight into PDE is partial_differential_equation. This shows up that still the knowledge based start is most important. Identify what the PDE under consideration is. Retrieve the historical solution and improvements and then pose your own specifics in details.

There are existence and uniqueness consideration and a wellposed examination needed.

Following the successful and skillfull classification there is for a mathematician in need of reputation the calculation of analytically closed solution. Based on this for inhomogenities a variation problem is to be solved.

If this is not possible some numerical experiments can be conducted. Most of these method solve the pde on finite points and lead to an interpolation function as a result.

The energy method can be used to verify well-posedness of initial-boundary-value-problems. That is a subset of pde and one that does fit into the finite kompartment methodologies too. Energy principles in structural mechanics lists some of the successful applications. This again follows the knowlegde approach. The examples are of eternal importance but the references in the article are a little outdated.

Whether pde or energy principle is easier or less complex in the solution depends on the CAS or FEM system in use. Many of these have the knwolegde incorporated into their codes and the learning curves are steep and their are tutorials available that recompile the knowledge to the extent to provides solutions in these systems.

If there is a trend than in modularity. Most designers of course know that their may be two or more to be successfully sold to the interested ambitioned scientist. A comparison of both article shows that the pde methodologies are described to some depth while the energy principle is not. Why is this the case? The pde methodologies are used more often and considered more comfortable. They are indeed posed more to the community. The energy principle on the other is an overview page and have below each specification another more detailed page for the problem categories covered.

It is necessary to use such great tools a mind maps on the wikipedia graph of linkes articles to gain more overview. The energy principles does no exist independent of the pde page but the path is longer than reproduced in the given question. This path has to be studied and that is the designers of modern pde courses much to much content. They are in need to over the steep learning of the FEM systems or programmin languages too.

The interested and ambitioned student can use the literature to fill in the gaps. Put aside the question of moderness of the literature. The wikipedia links offer a sorrow and convinient introduction and give opportunity to speed plenty of time to start with correspondences up to cohesion.