I have solved many problems regarding uniform continuity, but still I can't understand the following:
Is there any practical application of this concept, or it is just a theoretical concept? Is there any wide application of this concept in any theorem or problem? In short, how did the concept of uniform continuity arise?
A Parable (that makes no claims of historical accuracy): Suppose your mathematical universe contains only the rational numbers. You study rational-valued functions of rational numbers, and discover the $\varepsilon$-$\delta$ definition of continuity.
Then you discover the real numbers, and the fact that the rationals are dense in the reals. It's natural to ask: If $f$ is a continuous function (on the rationals), does $f$ extend to a continuous function on the reals? Perhaps surprisingly, "no". For example, $f(x) = 1/(x^{2} - 2)$ is a continuous (!), rational-valued function on the set of rational numbers that is easily seen not to extend to a continuous function on the reals.
There are many ways to interpret "what went wrong" in this example, but the explanation serving the preceding parable comes down to the following:
Question 1: Suppose $f$ is a continuous (rational- or real-valued) function on some domain $D$, and suppose $(x_{k})$ is a Cauchy sequence in $D$. Is the image sequence $\bigl(f(x_{k})\bigr)$ Cauchy?
If you don't know the answer, it's worth making a serious effort at a proof before reading further. There's an obvious strategy: Fix $\varepsilon > 0$ arbitrarily. Since $f$ is continuous, we can make $|f(y) - f(x)| < \varepsilon$ by taking $y$ sufficiently close to $x$. And a sequence is Cauchy if "its terms can be made as close as we like". It all seems to work at first glance....
The problem is, the answer is "no". An easy example is to take $f(x) = 1/x$ on the set of positive numbers, and to take $x_{k} = 1/k$. Or, if $f(x) = 1/(x^{2} - 2)$, use the recursive rational sequence $x_{1} = 2$, $x_{k+1} = \frac{1}{2}(x_{k} + \frac{2}{x_{k}})$.
The attempted proof above is a cautionary lesson about why mathematicians become so pedantic when teaching analysis: It's necessary to formulate and use precise definitions to avoid going logically awry. In the attempt to answer Question 1, the proof hits a snag because a Cauchy sequence in $D$ need not converge to a point $\ell$ of $D$, so we can't pick a $\delta > 0$ such that if $|x - \ell| < \delta$, then $|f(x) - f(\ell)| < \varepsilon$. And then, after careful examination, we discover a second, fatal snag: If $\delta_{k} > 0$ satisfies the definition of continuity at $x_{k}$ (i.e., if $|y - x_{k}| < \delta_{k}$ implies $|f(y) - f(x_{k})| < \varepsilon$), nothing ensures the sequence $(\delta_{k})$ has a positive lower bound.
Well, what if we require more of $f$, not merely continuity, but a condition that for every $\varepsilon > 0$, there exists a single $\delta > 0$ that "works throughout $D$" in the sense that if $x$ and $y$ are points of $D$ such that $|x - y| < \delta$, then $|f(x) - f(y)| < \varepsilon$.
Ah: We've just discovered uniform continuity on the set $D$.
Theorem: If $f$ is uniformly continuous on $D$, and if $(x_{k})$ is a Cauchy sequence in $D$, then $\bigl(f(x_{k})\bigr)$ is a Cauchy sequence.
Proof: See above, except now there are no snags.
All right, a (locally) uniformly continuous function maps Cauchy sequence to Cauchy sequences; so what?
Well, now we can have some mathematical fun. First, the definitions above extend with no effort to an arbitrary metric space $(X, d)$, and the proofs go through with obvious modifications (i.e., writing "$d(x, y)$" instead of "$|x - y|$").
Every metric space $(X, d)$ embeds isometrically into a complete space $(\overline{X}, d)$. One construction of $\overline{X}$ is to use equivalence classes of Cauchy sequences in $X$. A bit of analytic yoga shows that if $f:X \to Y$ is a uniformly continuous mapping into a complete metric space $(Y, d')$, there exists a unique (uniformly) continuous mapping $\overline{f}:\overline{X} \to Y$ that extends $f$.
Here are two "interesting" applications:
If $D$ is a bounded set of real numbers and $f$ is a real-valued function on $D$, then $f$ is uniformly continuous on $D$ (if and) only if $f$ extends continuously to the closure of $D$. This gives an essentially visual criterion for detecting uniform continuity (in the same sense that continuity is detectable by "the graph having no breaks", or differentiability is detectable by "the graph having a tangent line at each point"). For example, if $D$ denotes the set of non-zero real numbers, the functions $$ f(x) = \sin\frac{1}{x},\qquad g(x) = \frac{x}{|x|} $$ are continuous in $D$, but (by a glance at their graphs) not uniformly continuous.
The space $X$ of continuous, real-valued functions on $[0, 1]$ is a vector space with distance function $$ d(f, g) = \int_{0}^{1} |f(t) - g(t)|\, dt. $$ The integration operator $I(f) = \int_{0}^{1} f(t)\, dt$ is (more-or-less obviously) uniformly continuous on $(X, d)$. Now, $(X, d)$ is not complete, but if $(f_{k})$ is a Cauchy sequence in $(X, d)$, then $\bigl(I(f_{k})\bigr)$ is a Cauchy sequence of real numbers, and therefore converges. This allows us to extend the Riemann integral (uniquely by continuity) to the completion of $(X, d)$.
The completion turns out to be $L^{1}[0, 1]$, the space of Lebesgue-integrable functions on $[0, 1]$, and the Lebesgue integral is the unique continuous extension of the Riemann integral.
Uniform continuity is a much stronger condition than continuity and it is used in lots of places. One very fundamental usage of uniform continuity is in the proof that every continuous function of a closed interval is Riemann integrable.
It is a theorem in analysis that all functions that are continuous on a closed interval (or more generally, a compact set) are also uniformly continuous on that closed interval (resp. compact set).
As mentioned in the other answers, uniform continuity is much stronger than other notions of continuity. One of the reasons it is useful is that if we are working with a continuous function on a closed interval, we get uniform continuity for free.
