Why define norm in $L_p$ in that way? Who first defined the norm in $L_p$ space as $$\left(\int{\lvert f(x) \rvert^p}\right)^{1/p}$$
Is there any reference for this? Is it just an simple extension from $L_2$? 
$L_p$ space has some really nice properties. I think it relies on the definition of this norm, but this definition does not seem to come naturally or maybe I just can't see how it comes naturally. I understand this definition gives a norm. I just want to know the underlying thought of it. 
Are those nice properties just by chance or does it actually have a relationship with this definition?
 A: The cases $p=1,2$ are quite natural: the former is a very reasonable definition of "how large a function is" and the latter arises from Hilbert space theory: a well developed and useful theory. If you want to generalize these for arbitrary $p$, you are forced to consider $\int |f(x)|^p$. This is not a norm for arbitrary $p$ because $\int |a f(x)|^p = |a|^p \int |f(x)|^p$, and there is exactly one way to remedy this: by introducing the power $\frac{1}{p}.$ Miraculously (or maybe not?) the $L^p$ norm we constructed is actually a norm. 
The most curious case is $p=\infty$. At a glance, there is no reason to call the essential supremum norm $L^\infty$ as it has nothing to do with integrals. The following result shows that under certain conditions the infinity norm is the limit case of the $p$ norm:
If $f\in L^p \cap L^\infty$, $p<q<\infty$ then $f \in L^q$ and $\lim_{q \to \infty} \|f\|_q = \|f\|_\infty$
So, naming $L^{\infty}$ like that is justified. 
That being said, $L^{\infty}$ is often the worst space to work in: see @Ian's answer for details. 
A: If p is close to 1, the p-norm gives more importance to the regions where the function is small and less importance to regions where it is large. If P is large, then the norm gives less importance to the regions where the function is small and more to those where it is large.
Thus varying to allows us to decide what we want to focus on.
A: Lp spaces were first introduced by Frigyes Riesz (1910). (F&M Riesz both produced many important contributions to mathematics, and it is difficult to remember, which was which.)
The question in this link says that Riesz had worked on the moment problem in 1907. Later he generalized those results, and that required Minkowski and Hölder inequalities, so he developed the Lp spaces (1910).
Where does the $L^p$ norm come from?
Lp spaces are named for Henri Lebesgue, although Riesz was first. Lebesgue introduced the Lebesgue integral in 1904.
Should Rogers and Hölder share the main credit for the "p" in Lp spaces and Lp norms with Riesz?
"Hölder's inequality was first found by Leonard James Rogers (1888). Inspired by Rogers' work, Hölder (1889) gave another proof as part of a work developing the concept of convex and concave functions and introducing Jensen's inequality,[1] which was in turn named for work of Johan Jensen building on Hölder's work.[2]" https://en.wikipedia.org/wiki/H%C3%B6lder%27s_inequality
Hermann Minkowski showed the Minkowski inequality for infinite sums in 1896. https://de.wikipedia.org/wiki/Minkowski-Ungleichung
It all stemmed from L2 results?
L2 Hölder inequality (i.e., Cauchy-Schwarz inequality) "for sums was published by Augustin-Louis Cauchy (1821). The corresponding inequality for integrals was published by Viktor Bunyakovsky (1859) and Hermann Schwarz (1888). Schwarz gave the modern proof of the integral version."
Hölder was German, Minkowski was German (& Polish & Jewish), Rogers was British, Riesz was Hungarian-Jewish, Lebesgue was French.
A: In a certain sense, there are basically $4$ $L^p$ spaces with $p \geq 1$: $L^1,L^2,L^\infty$, and everything else. Specifically:


*

*$L^2$ is the only "Hilbertable" $L^p$ space, which gives it a wealth of additional properties.

*For "typical" measure spaces, all $L^p$ spaces except $L^\infty$ are separable. 

*For any measure space, $L^p$ is reflexive for all $1<p<\infty$. For "typical" measure spaces, $L^1$ and $L^\infty$ are both not reflexive.

*$L^1$ loses some theorems held by $L^p$ spaces with $1<p<\infty$ because for "typical" measure spaces it is not isomorphic to the dual space of any Banach space. (It "would" be the dual of $L^\infty$ but it is usually too small.) This means that the theorem "the unit ball in a space which is the dual of something is weak-* compact" does not apply to $L^1$, and thus neither do many of its consequences.

*$L^\infty$ also loses a lot of other theorems held by other $L^p$ spaces...and frankly I have some trouble giving a simple explanation for why. In PDE theory many of the relevant theorems fail because approximation by compactly supported smooth functions fails in $L^\infty$, but there is something more general that makes theorems in $L^\infty$ more elusive.


Here any time I say "typical" my statements include the Lebesgue measure, all measures which are mutually absolutely continuous with respect to it, and restrictions of such measures to positive measure subsets.
I know that one place where "unusual" values of $p$ (i.e. not $1,2$ or $\infty$) pop up is in Sobolev space theory, specifically with Sobolev embedding. Here a Sobolev space in sufficiently high dimension will embed into a Sobolev space with less regularity and more integrability. For example, in dimension $6$, $W^{1,2}$ embeds into $L^3$. In dimension $2$, $W^{1,2}$ embeds into $L^p$ for all $1 \leq p < \infty$.
A: In a very rough and informal way, this is how I understand this definition (and I find it very natural)
Starting from finite dimension space, suppose a function $f$ has finite components, say $f_1, ...f_n$. You want to understand its certain behavior thru its components. One way of doing this is by averaging all its components in some way (here we choose $|f_i|^p$). You may choose "arithmetic average" $\displaystyle \frac {1}{p} \sum\limits_{i=0}^n |f_i|^p $, or "geometric average" $\displaystyle \big(\sum\limits_{i=0}^n |f_i|^p\big)^{1/p} $. The former is not so suitable because the usual average is not efficient to control the growth of exponential function, whereas the latter controls that much better, thus be a better candidate for a norm.
Now move to infinite dimension space. You would have "infinite" components in the norm, so you would end up with something like $||f||_p = \big(\sum\limits_{i=0}^\infty |f_i|^p\big)^{1/p}$. This makes you think of integral notation, because integral is essentially an infinite sum (think of how Riemann integral is constructed). Also you will have no more "components" but $f$ itself. Finally, you end up with something like the definition.
