2
$\begingroup$

I have this histogram of data...what is the most proper way to prepare it for consumption in a neural network? I know how to normalize/standardize other types of data, but I'm wondering what to do with this kind of distribution.

$\endgroup$
5
  • $\begingroup$ So you want to transform exponential data into normally distributed data? Why? $\endgroup$
    – knrumsey
    Feb 13, 2017 at 21:13
  • $\begingroup$ So that I can feed it into a neural network which needs data to be standardized. $\endgroup$ Feb 13, 2017 at 21:18
  • $\begingroup$ It seems there are various ways people normalize data for their neural networks. Perhaps you should describe how you normalize a few other types of data. Otherwise you might get an answer that normalizes the data to the wrong range for you. $\endgroup$
    – David K
    Feb 13, 2017 at 21:35
  • $\begingroup$ So far most of the data has been either normal looking (which I set to mean of 0 and sd of 1) or uniform looking (which I scale to be between 0 and 1). Other data what is categorical I one-hot-encode. $\endgroup$ Feb 13, 2017 at 21:37
  • $\begingroup$ So you don't want the data to be Normally distributed... Just standardized someway? The so-called "Standard Exponential" is just exponential with mean 1. So you could divide by the mean. $\endgroup$
    – knrumsey
    Feb 13, 2017 at 22:11

2 Answers 2

2
$\begingroup$

When normalising inputs to a Neural net, you want the numbers to be in a similar range across different inputs, so inputs which tend to have much larger absolute values don't dwarf the contributions from smaller ones. You need to preserve the Y values (frequencies) but can change the X scale with various transforms (potentially non-linear ones).

Here you want to "squash" the distribution along the x axis, so the larger X values are "squashed" more, and so the range of X is of the same order of magnitude as the other inputs, without destroying the frequency information.

Taking a log (base 2 or 10) of X is the obvious way to do that. If you use log10 your data will range from 0->2.4 or so.You can get away with this in your distribution because the lowest value is 1. It's trickier if your min value is zero or close to zero.

You may then want to do a further normalisation of subtracting the mean, and dividing by the standard deviation, so the variance is 1 - the most common "standard" nn normalisation technique.

$\endgroup$
2
$\begingroup$

This might be a somewhat naive method, but it will certainly work. We will use the Probability Integral Transformation.

If $X$ is a continuous random variable with cdf $F_X(x)$, then the random variable $U = F_X(X) \sim \mathrm{Uniform} (0,1)$. Similarly, $X = F^{-1}(U) \sim F_X$.

Step 1, convert your data to Uniform.

Use maximum likelihood estimator $\lambda = \bar{X}$

\begin{equation*} u_i = 1 - e^{x_i/\lambda} \end{equation*}

Convert the Uniform data to Standard Normal data. Let $\Phi(z)$ be the standard normal CDF.

\begin{equation*} z_i = \Phi^{-1}(u_i) \end{equation*}

It works like so...

Transformation process

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .