Finite Element Method Versus Finite Difference Method It has been a while that I have gained some elementary knowledge of finite element methods (FEMs) and finite difference methods in solving boundary value problems (BVPs), eigen-value problems (EVPs) and initial boundary value problems (IBVPs). Although FEM seems to be much more complicated and involved, it has got much more popularity than FDM in these days.
Here is a question which I couldn't get a nice answer for myself.

What is the advantage of FEM over FDM in solving such problems or vice versa?

 A: This question has been answered many times throughout the years on this forum and others. The most nuanced answers separate BVPs, EVPs and IBVPs, as the required properties of a successful method varies a little depending on the type of problem. I will focus on IBVPs here.
Most arguments for FEM follow a line of reasoning involving "provable stability" and "applicable in complex geometries", which (it is argued) would be difficult to achieve using FDM. Such an argument would have been sufficiently informative up to about 25 years ago. Today, it is outdated.
The simple truth is that anything that can be done with FEM can also be done with FDM, and vice versa, with roughly the same difficulty. This is due to the fact that the properties of a given method that allows us to attack a given IBVP is not implicit to either FDM or FEM (apart, possibly, from consistency of approximation, which often is straightforward to ensure for both methods) but to certain symmetries of the resulting derivative approximations. These symmetries allow the method to mimic the properties of the IBVP that are necessary for e.g. stability and convergence, which is necessary for a successful approximation of the solution.
What must be realised here is that 'the finite difference method' and 'the finite element method' are not singular methods but families of methods, some of which are useful and some of which are not. The distinguishing factors between a useful method and a (for the lack of a more diplomatic term) useless one is whether or not it satisfies the Summation-by-Parts (SBP) property (a set of symmetry requirements), and the way in which boundary/interface conditions are implemented (strongly vs. weakly, the latter of which is the proper way). This is what is needed to prove stability for essentially any well-posed linear IBVP, and non-linear ones as far as that is possible. It is also what is needed to couple elements together in a stable and accurate way, irrespective of the method used (it is perfectly fine to construct finite difference methods on elements as well). Every method that has enjoyed some success in solving IBVPs since the mid 90's turns out to follow the SBP formalism. This includes certain finite difference methods, certain spectral methods, certain finite element methods, certain finite volume methods, certain discontinuous Galerkin methods, certain flux reconstruction methods...
Now to answer your actual question, I will assume that we are dealing with a method that has the SBP property and where boundary conditions are implemented in a proper way, else a fair comparison simply cannot be made. Let's start with advantages of FEM:
1) There is some evidence suggesting FEM outperforms FDM for parabolic problems.
2) FEM can be applied to complex geometries with relative ease. Now, the 'easy' part is due to the fact that the FEM operators are straight forward to construct from interpolation theory and the Galerkin procedure. Complex geometries can be handled by FDM in more or less exactly the same way as FEM, however the theoretical background leading up to the definition of the finite difference operators on, say, simplex elements is more involved and the theory has not yet properly matured.
3) It is possible to implement FEM in ways that allows us to exploit GPU acceleration to a very large extent.
Now, to the advantages of FDM:
1) There is some evidence suggesting that FDM outperforms FEM for hyperbolic problems.
2) The implementation of FDM is usually simpler and less time consuming than FEM.
3) On simple geometries, FDM is typically a bit more efficient than FEM, both in terms of computational speed and memory handling.
Other than that, it is hard to come up with any 'fundamental' advantages of the two methods. There are probably exceptions to each one I've listed available in the literature as well. The point is; it makes little sense to ask for advantages of FEM or FDM in relation to each other since the properties that they enjoy and that make them useful in the first place are the same for the two methods. If I'm allowed to speculate a little; give it five or ten years and we will probably see that the two methods have fused into one, the difference being the approximation spaces they are based upon, or the type of interpolation used to define them.
