If we the function $f(x)$ in the interval $x \in [\frac{\pi}{4}, \frac{7\pi}{4}]$, will $\frac{\pi}{4}$ be one of the local minima and $\frac{7\pi}{4}$ one of the local maxima?
Also if the interval was $x \in (\frac{\pi}{4}, \frac{7\pi}{4})$, will the number just little more than $\frac{\pi}{4}$ be one of the local minima and number just little less than $\frac{7\pi}{4}$ one of the local maxima? If yes, is there anyway to denote that number using limits or something?
No, for instance, $f(x) = \sin x$ contradicts both of these. Also you say "the number just a little more than $\frac{\pi}{4}$" as if there is a unique such number; but the interval $(\frac{\pi}{4}, \frac{7\pi}{4})$ has no least element. Indeed, if $\xi \in (a,b)$ then $\xi'=\frac{a + \xi}{2} \in (a,b)$ and $\xi'<\xi$.
Now, if it is continuous $f(x)$ must have a maximum and minimum value in $[a,b]$ and it must attain these values (but this need not happen at the end points). This is an elementary property of continuous functions whose proof can be found in all introductory analysis texts. However it need not have a local maximum or minimum in $(a,b)$; for instance, $f(x) = \frac{1}{x}$ has no maximum value in $(0,1)$ and it is strictly decreasing on this interval so it certainly has no local maxima.
No, in general. Those are simply the endpoints of the interval. Whether or not any maxima and minima coincide with them or not depends on the function.
And what do you mean by "number just a little more"? There is no such unique number. It sounds like you're thinking of something infinitesimal-related here, but infinitesimals don't exist in the reals nor do numbers "just next to" reals if by that you mean there's nothing between that number and the number to which it is adjacent.
I think your doubt may be a follows: when we have end point(s) in the function's domain, the end points always, by definition, are critical points = points that are candidate to be local extrema. You will have to evaluate the function at these points, then *if*i the function is derivable in its domain without the ends you can use the first derivative test, second one and etc. to find other critic points more, and at the end you have to choose the smaller value(s), that'll be minima, and the bigger ones that'll be maxima.
If the function's domain is an open interval then you directly can use the 1st derivative test as the end points aren't contained in the domain. There's not such thing as a real number "a littler more or a little less" than any other, in general.
