I've written down some questions that I had earlier when I was learning about probability density functions and approximating discrete distributions to continuous distributions.

Q1. Probability Percentiles:

In my earliest studies of statistics, we defined the $xth$ percentile as the value which $x\% $ of the data lies under. So, $P_{40} = 0.4n$ where $n$ is the sum of the of the number of data values in our data set. We have now transferred the concept of Percentiles to probability density functions, which I don't quite know how to interpret. Should we interpret $xth$ percentile as the value which $x \% $ of the probability lies under?

Q2. Probability Density Functions:

I have also been having trouble interpreting 'Probability density functions' $f(x)$. I comprehend the fact that $f(x)$ is not equal to a probability, and is in fact equal to $$f(x) = \lim_{c.w.->0} \frac{\frac{f}{\sum{f}}}{c.w.} $$ where $c.w.$ stands for the class width and $\frac{f}{\sum{f}}$ is relative frequency or probability. However, I have been having trouble accepting that the mode, is simply the highest point on $f(x)$. My basic definition for the mode back when I was working with data sets, was it's the value (in our data set) which occurs the most frequently. So, my intuition as to what the mode is/means when dealing in probability, is the value in our data set that has the highest chance of happening or the value with the highest probability. Since $f(x) \not= Probability $ I struggle to understand how the highest point on a probability density function is equivalent to the mode.

Q3. Continuous Uniform Distributions:

Let's suppose we have continuous uniform distribution, we know that it takes the form $$ f(x) = \frac{1}{b-a} \thinspace a\leq x \leq b$$ Obviously, the mean $E(X)$ can simply be found by deriving $$\int_a^b xf(x) \thinspace dx$$ However, it has been said numerous amounts of times in my lectures that it can easily be deduced that the mean is simply $\frac{a+b}{2} $ by looking at the curve. I have looked at the curve for the continuous uniform distribution and don't see how this is so.

Q4. Continuity Corrections:

So, we apparently can approximate binomial with a normal if certain conditions are met regarding the value of $n$ and $p$ where $$X \sim B[n,p]$$. Now, I would like to know what the purpose of continuity corrections are? Suppose we were asked to find $P(X<32)$ and we wanted to approximate this to a normal, why do we have to round up or down to $P(X<31.5)$ or $P(X<32.5)$???

  • $\begingroup$ Usually it is best to ask one question at a time and to give more information about what your difficulty is. I will give brief answers. $\endgroup$
    – BruceET
    May 22 '18 at 3:31

About density functions. A density function $f_X(x)$ of a continuous random variable $X$ has three defining properties:

(1) $f_X(x) \ge 0,$ (2) $\int_{-\infty}^{\infty} f_X(x)\,dx = 1,$ and (3) $P(a < X \le b) = \int_a^b f_X(x)\,dx.$

Accordingly, probability of intervals are defined, but individual points have $0$ probability. If you say that reaction times in a particular situation for individuals in a population are exponentially distributed with mean $\mu = 0.1$s (or rate $\lambda =10$ per second), then the probability that a particular individual shows reaction time $X=.12$ is $P(X = 0.12) = 0.$ You might say it is impossible to measure that reaction time as exactly 0.12000000s. If you can measure within $\pm.005$s, you could make numerical sense of this by saying her reaction time has $P(.115 < X \le .125) = \int_.115^.125 10e^{-10x}\,dx = 0.0301.$

Percentiles. Percentiles for small discrete samples are defined in various ways, roughly as you you mentioned. (Exact methods vary slightly among textbooks and statistical software packages.) However, the definition for continuous random variables is precise. We say that the 90th percentile of the distribution of a random variable $X$ is $q,$ if $P(X \le q) = .90.$ For the distribution $\mathsf{Exp}(\lambda = 10),$ the 90th percentile is $q = 0.23026.$ You can get this by integration or using software. (The computation in R statistical software is shown below.)

qexp(.9, 10)
## 0.2302585
pexp(.23026, 10)
## 0.9000015

enter image description here

As another example, heights of men in a population might be approximately distributed as $Y \sim \mathsf{Norm}(\mu = 59, \sigma=3.5)$ in inches. Then the probability a randomly chosen man is within half an inch of 58" tall is $P(57.5 < Y \le 58.5) = 0.1090$ and that the 80th percentile of the population is about 62 inches.

diff(pnorm(c(57.5, 58.5), 59, 3.5))
## 0.1090839
qnorm(.8, 59, 3.5)
## 61.94567

enter image description here

Software or tables are required for computations involving a normal distribution. For technical reasons, calculus cannot be used.

Modes. For a discrete sample such as 1, 2, 2, 3, 3, 3, 3, 4, 7, the mode is the most frequently occurring value (if there is one); here the mode is 3. It is customary to define the mode of a continuous distribution (if it exists) as the location where the density function reaches a maximum. The mode of $\mathsf{Norm}(59, 3.5)$ is at $\mu = 59.$

Some texts say that $\mathsf{EXP}(\lambda=10)$ takes values in $(0, \infty)$ and others this distribution takes values in $[0, \infty),$ so that the value $0$ is theoretically possible. In the latter case one would say that the mode is at $0.$

Continuity correction. I will illustrate the idea of 'continuity correction' for the approximation of a binomial distribution by a normal distribution. Suppose you toss a fair coin $n = 10$ times. Then the number $H$ of heads you see is distributed $H \sim \mathsf{Binom}(n=10, p = 1/2).$ If you want to find $P(X \le 4),$ you can sum five terms of the the binomial PDF (or PMF) to get $$P(X \le 4) = P(X = 0) + P(X=1) + \cdots + P(X=4) = 0.3770,$$ which would require a moderate amount of computation with an ordinary calculator.

pbinom(4, 10, .5)
## 0.3769531

Because $E(H) = np= 10(.5) = 5,\,$ $Var(H) = np(1-p) =2.5,$ and $SD(H) = \sqrt{2.5} = 1.5811,$ and because $n = 10$ is barely large enough to use a normal approximation, we can say that $H$ is approximately distributed as $\mathsf{Norm}(\mu=5,\,\sigma=1.5811).$ Then we can express our problem in terms of $H$ as $P(H \le 4) = P(H \le 4.5) = P(H < 5).$ Because we are using the continuous normal distribution to approximate a probability for a discrete binomial distribution, it is best to use the second form of the probability.

$$P(H \le 4.5) = P\left(\frac{N-np}{\sqrt{np(1-p)}} \le {4.5 - 5}{1.5811 = -0.316}\right) \approx P(Z \le -0.316) = 0.3795,$$ where $Z$ has a standard normal distribution, with values available in printed tables. Often normal approximations using the 'continuity correction' give two places of accuracy for binomial probabilities. Here the exact and approximate values are both about $0.38.$

enter image description here

The actual probability is the area of the histogram bars at $H = 4$ and below; the approximate probability is the area under the normal density curve to the left of the vertical dotted red line. If we had used the first or last form of the binomial statement, we would have wrongly excluded the area between 4 and 4.5, or wrongly included the area between 4.5 and 5. The continuity correction is simply a matter of coordinating binomial and normal probabilities for the most accurate result.

  • $\begingroup$ Excellent, thank you so much. $\endgroup$
    – Uni
    May 22 '18 at 4:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.