Esoteric knowledge regarding statistic tests like $F$-test, $t$-test and $X^2$ (Chi-Square) etc. For a year or two I've been doing/learning statistics using books written both for engineers and semi-professionals. I know how to apply most of the theory that statisticians use on job, but I'm still searching for answers why the theory behind statistics is the way it is. 
In books it is written when to use an $F$-test, $t$-test or $X^2$ (Chi-Square) etc. to solve problems, but the reason why these tests are used are never stated and the theory behind them is completely left out.
Why are the theory that goes behind statistics so esoteric (hidden) ? Is it because it is very hard to understand ? Why are the theory / reasons for using these tests not shown in books ? Could someone tell me where to look up the theory, since doing something where you don't have the "background" knowledge seems unappealing to me.
 A: One standard text is "Introduction to Mathematical Statistics" by Hogg and Craig.  I see it is now in its seventh edition.  Unlike some other texts, it does not use measure theory, so I suppose it is not totally rigorous, but I think you will find the theory behind the standard statistical tests clearly explained.
A: Very valid concern (and not only for statistics).
In facts, I used these tests for a long time, before I had the idea to read the original works. Very instructive. I realized I missed the whole concept, the all reason-why, of statistics since a very long time.
Student's job was to compare Guiness beer quality. Suppose he had a stout of average $\bar x = 85$ to compare to a stout of average $\bar y = 95$. His question was whether $\bar x - \bar y =10$ were a big difference or not? His answer was to compare the difference $\bar x - \bar y$ to the standard deviation $\sigma$ by computing the ratio $t=\frac{\bar x - \bar y}\sigma$.
A big $t$ means that either the numerator $\bar x - \bar y$ is big (meaning the difference is real), either the denominator $\sigma$ is small, (meaning that, although the difference may not be very high, it is very accurate). In both cases, he could conclude that the stouts were different.
On the opposite, a small $t$ means that either $\bar x - \bar y$ is small (because the stouts are similar for real), either that $\sigma$ is big, (meaning that, although the difference may seems important, it stays below the level of statistical fluctuation). In both cases, he could not distinguish between the stouts. Note that he still did not know if the stouts were really different, or if he could not distinguish between them because of the statistical uncertainty, somehow for the same reason that the two lights of car in the night appear like one large spot when the car is very far.
Student then faced the question "for which values of $t$ should I consider it is big?". As a rule of thumb, used in the Guiness bear factory, when the number of observations is large, so that $\bar x - \bar y$ is almost gaussian, the threshold is 2 : when $t>2$ the stouts are different, when $t<2$ they could not be distinguished, requesting more sample. The exact value (look in a table) is not 2, but 1.96 for a level of confidence of 95%. And the exact sentence is : if $t>1.96$ you have a 95% probability to say something wrong when saying "the stouts are different".
When the number of observations is small, however, the gaussian approximation is not valid, and Student devised a full theory and computed tables giving the threshold for $t$ as function of the number of observations and the requested level of confidence.
When reading Student paper, Fisher had the idea to apply the same concept to compare variances. In this case, the indicator is $F = \frac {\sigma ^2 (Y)} {\sigma ^2 (X)}$ which is small is X vary less than Y, and large if X vary more than Y. It is used, for example, to measure if the knowledge of the factor $X$ reduces the variance of the variable $Y$, which is the least to be expected when $X$ is expected to have an explanatory power on $Y$. Again Fisher devised a full theory and computed tables giving the threshold for $F$ as function of the number of observations (for $X$ and $Y$) and the requested level of confidence.
I do agree with you that, very unfortunately, an important part of the interpretation is lost when learning from the books only. Without this interpretation, the more theory you receive, the more confuse you get.
A: The primary reason for what you have observed in your studies is that deriving the appropriate sampling distribution for a particular test statistic is in general not an easy thing to do, but once known, it need not be derived again in order for the result to be applied in practice.
By analogy, this is a bit like calculus versus real analysis:  in calculus, we are primarily concerned with how to compute integrals, derivatives, limits, and series expansions, but in real analysis, we learn about the mathematical principles that make such operations rigorous (e.g., properties of the Riemann and Lebesgue integrals, uniform continuity, etc).
So, if you consult any reasonably comprehensive text in mathematical statistics (as opposed to applied statistics), then you should find the relevant proofs for the F-test, t-test, chi-squared test, and so forth.  Some of the simpler tests such as the Z-test are easy enough to illustrate in the applied texts.  You will encounter more advanced concepts such as the delta method, asymptotic convergence, and Satterwaithe's approximation, because some of these tests apply only to large sample sizes because the exact sampling distribution is not "simple."
