Can a probability density function diverge at an endpoint? The essence of my question is in the title.
Suppose we want to draw numbers from the unit interval $[0,1]$; can $p(x)$ be, say, of the form
$$
p_1(x; \varepsilon) = \frac{\varepsilon}{x^{1-\varepsilon}},
$$
despite the fact that $p_1(x)$ is unbounded in the $x \to 0$ limit?
Is there any 'legal' way to do away with the regulator $\varepsilon$ and take $\varepsilon \to 0$? (maybe this limit is just the Dirac $\delta(x)$, but it's certainly not a standard delta-sequence example)
Clearly if we put a 'cut' on the possible values of $x$ and restricted ourselves to $[\varepsilon, 1]$ we could also do something like
$$
p_2 (x; \varepsilon) = \frac{1}{-\log \varepsilon} \frac{1}{x};
$$
I would expect this to have the same limit, though as we've regulated it in a different way it has different properties.
Any insight or references on this would be appreciated.
 A: There's no problem saying (modifying your notation) $$p_\epsilon(x)=\frac{\epsilon}{x^{1-\epsilon}};$$no rule says a density has to be bounded. And yes this does tend weakly to a "delta function". Whether this is "standard" is not so clear to me, but it's a special case of something that seems pretty standard:


True Fact. Say $(\mu_n)$ is a sequence of (Borel, etc.) probability measures on $[0,1]$. Suppose that for every $\lambda\in(0,1]$ we have $\lim_{n}\mu_n([\lambda,1])=0$. Then $\mu_n\to\delta_0$ weakly.


A: For any $\varepsilon>0$, define $\mu_\varepsilon(dx)=\varepsilon x^{\varepsilon-1}\mathbb{1}_{[0,1]}(x)dx$. A simple calculations shows that each $\mu_\varepsilon$ is a probability measure supported in $[0,1]$ and its density is unbounded in a neighborhood of $0$.
We show that $\mu_\varepsilon\Longrightarrow\delta_0$ as $\varepsilon\rightarrow0$, that is $\mu_\varepsilon$ converges weakly (the team convergence in distribution is also used) to the Dirac measure $\delta_0$ defined as
$$\delta_0(A)\left\{\begin{matrix} 1 & \text{if}& x\in A\\
0 & \text{if} & x\notin A
\end{matrix}
\right.
$$
for any set $A$.
To do so, we appeal to the following result in the theory of weak convergence in probability:

Theorem: Let $(S,d)$ be a separable metric space and $\{\mu_n,\mu:n\in\mathbb{N}\}$  Borel probability measure on $S$. Let $\mathscr{U}$ be a class of sets such that

*

*$\mathscr{U}$ is closed under finite intersection,

*For any $x\in S$ and any $\varepsilon>0$ there is $A\in\mathscr{U}$  such that
$$x\in\operatorname{Int}(A)\subset A\subset B(x;\delta)$$
If $\lim_{n\rightarrow\infty}\mu_n(A)=\mu(A)$ for all $A\in\mathscr{U}$, then $\mu_n\Longrightarrow\mu$.


This result can be found in the classical book Billingsley, P., Convergence of Probability Measurers, John Wiley and Sons, New York, 1968 pp. 14-15.
We apply the result above to $([0,1],\mathscr{B}([0,1])$ with probability measures $\{\mu_{\varepsilon}, \delta_0:\varepsilon>0\}$, and collection   $\mathscr{U}=\{(a,b]\cap[0,1]: a,b\in\mathbb{R},\,a<b\}$. That $\mathscr{U}$ satisfies conditions (1) and (2) of the theorem is easy to check.
Let $(\varepsilon_n:n\in\mathbb{N})$ be any sequence in $(0,1)$ converging to $0$.
If $I\in\mathscr{U}$ and $0\in I$, then $I=[0,b]$ for some $0<b<1$ and
$$\lim_{n\rightarrow\infty}\mu_{\varepsilon_n}([0,b])=\lim_{n\rightarrow \infty}b^{\varepsilon_n}=1=\delta_0([0,b])$$
On the other hand, if $0\notin I$, then $I=(a,b]$ for some $0<a<b\leq 1$ and
$$\lim_{n\rightarrow\infty}\mu_{\varepsilon_n}((a,b])=\lim_{n\rightarrow\infty}b^{\varepsilon_n}-a^{\varepsilon_n}=1-1=0=\delta_0((a,b])$$
Hence $\mu_{\varepsilon_n}\Longrightarrow\delta_0$.
