3
$\begingroup$

I wish to derive a finite (that is, which has a finite support) discrete approximation to a normal distribution, with the following considerations:

  1. It should have exactly the same mean and variance
  2. It must be symmetric
  3. It should resemble the normal distribution in some sense (unimodal pmf, etc.)
  4. It should be discrete and finite (having finite support), with a pre-determined set which contains the support (for example, the integers).

Naive attempt

Here's a naive attempt. Suppose we wish to give an approximation to $\mathcal{N}(\mu,\sigma^2)$. Let the support be $S=\left[\lfloor \mu-d\sigma \rfloor,\lceil \mu+d\sigma\rceil\right]$ (for some natural $d$, perhaps $3$), and define the following pmf: \begin{equation*} f(k) = \begin{cases} \Phi_{\mu,\sigma^2}\left(k+\frac{1}{2}\right) & k = \min{S}\\ \Phi_{\mu,\sigma^2}\left(k+\frac{1}{2}\right) - \Phi_{\mu,\sigma^2}\left(k-\frac{1}{2}\right) & \min{S} < k < \max{S}\\ 1 - \Phi_{\mu,\sigma^2}\left(k-\frac{1}{2}\right) & k = \max{S} \end{cases} \end{equation*} where $\Phi_{\mu,\sigma^2}$ is the cdf of $\mathcal{N}(\mu,\sigma^2)$. This is a legitimate pmf (sums to $1$), it is symmetric, unimodal, discrete and finite, and has mean $\mu$ -- but it does not have variance $\sigma^2$ (I think it always has a larger variance).

Can you fix this naive solution somehow?

$\endgroup$
  • $\begingroup$ Do you mean it has finite support? (i.e.$f(k)=0$ for $|k|\geq K$ for some $K>0$) $\endgroup$ – Dan Sep 5 '14 at 8:59
  • $\begingroup$ Indeed, I'll clarify this. $\endgroup$ – Bach Sep 5 '14 at 9:00
  • $\begingroup$ If you wish to change the variance for a specified support you will have to change the shape of your distribution away from a Gaussian over that interval. An easy way (but almost certainly not what you're after) is to add two delta functions with suitable coefficients to each end of the support. $\endgroup$ – Dan Sep 5 '14 at 9:05
  • 1
    $\begingroup$ Your requirements cannot be met for arbitrary mean $\mu$. This is because a symmetric approximating distribution with support $\{ a_1, ... a_n\} $ will have a mean that's necessarily either $a_{(n+1)/2}$ (odd $n$) or $\frac{1}{2}(a_{n/2} + a_{n/2 + 1})$ (even $n$); e.g., if the support is a set of consecutive integers, then the mean must be a multiple of $\frac{1}{2}$. $\endgroup$ – r.e.s. Sep 7 '14 at 16:59
  • $\begingroup$ (cont'd) In fact, your "naive solution" is not symmetric, and yet does not exactly match an arbitrary mean. For example, with $\mu = 4.9, \sigma = 1$, and support $\{\lfloor \mu- 3 \sigma \rfloor, ..., \lceil \mu + 3 \sigma\rceil\}= \{1,...,8\}$, it gives $f(1)=0.0003369... \ne f(8)=0.004661...$ (not symmetric) and mean $=4.8998... \ne 4.9$ (not an exact match). $\endgroup$ – r.e.s. Sep 7 '14 at 16:59
0
$\begingroup$

Intuitively, two distributions which closely approximate one another would have similar PDF (or PMF).

$$P(X=k)$$

A literal one to one numerical comparison cannot strictly be made between a PMF and a PDF, yet a simple comparison can be made for statistical measures: mean, variance, skewness, and kurtosis.

$$\mu, \sigma^2, \frac{\mu_3}{\sigma^3}, \frac{\mu_4}{\sigma^4}$$

A given Normal distribution $X \sim \mathcal{N}(\mu, {\sigma}^2)$ corresponds to the following measures, respectively.

$$\mu, {\sigma}^2, 0, 3$$

General form

In exploring possible solutions, I arrived at this general template.

Given $m_0, m_n \in \mathbb{Z}$, $\mu_0 \in \mathbb{R}$ and $\sigma_0 \in \mathbb{R}^{+}$ as inputs, where $m_0 \lt \mu \lt m_n$.

Construct a function $f(k) \ge 0$ defined over a finite support $S$ of size $n+1$.

$$k \in S = \{m_0, m_1, \ldots, m_n\}$$

$$ m_i = m_0 + i $$

And then define a discrete distribution with a PMF proportional to $f(k)$ as follows.

$$P(X=k) = \frac{f(k)}{C_S}$$

$$C_S = \sum_{k \in S}f(k)$$

As desired, the inputs may be restricted further as follows. Given $d \in \mathbb{R}^{+}$ choose $m_0, m_n$ such that

$$m_0 = \lfloor \mu_0 - \sigma_0 d \rfloor$$

$$m_n = \lceil \mu_0 + \sigma_0 d \rceil$$

Acceptance testing of $f(k)$

This general form clearly meets question requirement #4, so for each $f(k)$ I want to evaluate how well the associated distribution meets the question requirements #1, #2 and #3. I am interpreting requirement #2 more specifically as follows:

  1. It must be symmetric where possible ($2\mu \in \mathbb{Z}$) or at least have skewness $\frac{\mu_3}{\sigma^3} \approx 0$

Fixing the naive attempt

With this general form, some improvements to the naive attempt become apparent.

  • There is a free parameter $d$. So instead of a conservative $d=3$, choose to broaden the support to at least $d=6$
  • The general form preserves the unit sum of a PMF. So having the tails of the normal distribution capped into the endpoints of the PMF by a piecewise function is unnecessary.
  • The inputs $\mu_0, \sigma_0$ may not exactly equal $\mu, \sigma$, but an appropriate choice of inputs may target the desired $\mu, \sigma$ correctly.

Discrete Normal PMF ($f_1(k)$)

Summary: (Fix the naive attempt:) Create a PMF over a finite support that is exactly proportional to the continuity correction values obtained from a Normal CDF of the same mean. The variance increases by this approximation, so find a $\sigma_0$ that gives the desired $\sigma$.

Given the general form, let $f_1(k)$ be defined as follows.

$$f_1(k) = \Phi_{\mu_0,\sigma_0}(k+\frac{1}{2})-\Phi_{\mu_0,\sigma_0}(k-\frac{1}{2})$$

In testing a range of inputs, I have found the observed variance is too big. Specifically, I find

$$\{\mu, \sigma^2, \frac{\mu_3}{\sigma^3}, \frac{\mu_4}{\sigma^4}\} \approx \{\mu_0, \sigma_0^2+\frac{1}{12}, 0, 3\}$$

So I require inputs such that $\mu_0=\mu$ and $\sigma_0=\sqrt{\sigma^2 - \frac{1}{12}}$.

The approximation appears to improve as $\sigma d \to \infty$.

Now, does $f_1(k)$ meet the requirements?

  1. Yes, except variance is not exact (but could be improved by refinements to $\sigma_0$)
  2. Yes (as reposed in this answer)
  3. Yes (but one Normal is employed to approximate the statistics of another, begging the question, "Which Normal is being approximated?")

And does $f_1(k)$ have any limitations?

  • $\sigma_0 \ge 1$ and $d \ge 6$ (so that the variance is approximately correct)

Improving on $f_1(k)$

I mentioned that a continuity correction of the Normal distribution gives us $f_1(k)$, but I might have arrived at the same function a different way. It turns out if I integrate the PDF of $\mathcal{N}$, and then approximate the derivative using $h=1$ and the following formula, that I will get the same function.

$$g'(a)\approx\frac{g(a+\frac{h}{2})-g(a-\frac{h}{2})}{h}$$

This reposing begs a question of, "Why not create the PMF directly from the PDF?"

Discrete Normal PMF ($f_2(k)$)

Summary: Create a PMF over a finite support that is exactly in proportion to a Normal PDF and which (in consequence) has statistical measures that match (allowing some margin of error).

Given the general form, let $f_2(k)$ be defined as follows.

$$f_2(k) = e^{\frac{-(x-\mu_0)^2}{2{\sigma_0}^2}}$$

In testing a range of inputs, I have found decent agreement. Specifically, the statistical measures of the PMF are approximate to that of the Normal PDF from which they are derived.

$\{\mu, \sigma^2, \frac{\mu_3}{\sigma^3}, \frac{\mu_4}{\sigma^4}\} \approx \{\mu_0, \sigma_0, 0, 3\}$

The approximation appears to improve as $d\sigma \to \infty$.

Now, does $f_2(k)$ meet the requirements?

  1. Approximately, but not exact (but could be improved by numerical methods)
  2. Yes (as reposed in this answer)
  3. Yes (the PMF is in exact proportion to the PDF of the Normal being approximated)

And does $f_2(k)$ have any limitations?

  • $\sigma_0 \ge 1$ and $d \ge 6$ (so that the variance is approximately correct)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.