Questions tagged [statistical-inference]
The area of statistics that focuses on taking information from samples of a population, in order to derive information on the entire population.
1,415
questions with no upvoted or accepted answers
7
votes
0
answers
2k
views
How are category theory and probability theory related?
How are category theory and probability theory related ? Category theory seems very useful for understanding objects with definite relationships, whereas probability theory (particular Bayesian ...
- 1,659
7
votes
1
answer
99
views
Going Through Yellows
I have observed that I am almost never the last car through a traffic light. Sometimes I stop (because it is yellow or red), in which case, of course, the car behind me also stops and the car in front ...
- 609
7
votes
1
answer
146
views
Find a function such that follows to normal in distribution
Suppose that $X_{n}\sim \text{Binomial}(n,\theta)$, where $n=1,2,\ldots$ and $0<\theta<1$. Find a function $g$ such that $\sqrt{n}(g(\frac{1}{n}X_n)-g(\theta))\xrightarrow{D} N(0,1)$ for each ...
- 215
7
votes
0
answers
1k
views
The distribution of the ith order statistic for discrete random variables
Assume $(X_i)_{i=1,...,n}$ are a sequence of real iid random variables with continuous density $p_x$.
We know that $$Y:=\sum_{i=1}^n 1\{X_i\leq u\}\sim Bin(n,F_x(u)),$$ since $1\{X_i\leq u\}\sim Ber(...
- 767
6
votes
0
answers
57
views
What mathematics will help me predict sales curves?
I'm a programmer and have a client who annually releases new products which have "long tail" type of sales curves, very heavy when initially released, tapering out until discontinued years later. He ...
- 263
6
votes
0
answers
76
views
single variable is significant but overall test is not
I do a multiple regression with 3 independent variables $X_1$, $X_2$ and $X_3$. The correlation between $Y$ and $X_1$, $Y$ and $X_2$, and $Y$ and $X_3$, are each large and statistically significant. ...
- 494
6
votes
1
answer
242
views
Divergence based robust inference
The term 'divergence' means a function $D$ which takes two probability distributions $g,f$ as input and puts out a non-negative real number $D(g,f)$. I have learnt that the inference based on ...
- 1,931
5
votes
0
answers
105
views
Farmer wants to know how wet their field is
Problem
A farmer wants a better understanding of rainfall on their field. Assuming rain falls randomly and with equal likelihood over the entire field, the farmer thinks they can model the volume of ...
- 201
5
votes
0
answers
299
views
Product of correlated random variables and its transformation
There is an interesting result, saying that if $Z_1, Z_2$ are standard normal random variables with a correlation $\rho\in (-1,1)$, then the product $Z=Z_1Z_2$ has a density function explicitly given ...
- 771
5
votes
0
answers
100
views
Minimax Estimator for Normal Random Vector
Question. Suppose $Y_i \sim N(\mu_1, 1)$. Let $Y := (Y_1, Y_2)$, and $T_y = (Y_1, 0)$. Denote $\Theta$ as the space of all estimators $\mu := (\mu_1, \mu_2)$. Is it necessarily true that $\hat{\mu}$ ...
- 311
5
votes
0
answers
105
views
Bayes Estimator under $L_{\eta}$
I am wondering if the following loss function is well known and if it is, does it have a standard name:
$$
L_{\eta} (\theta, a) = (\theta-a) (\eta - \mathbb{I}_{(-\infty, a)} (\theta) ), \quad \eta \...
- 6,136
5
votes
0
answers
152
views
Likelihood function & MLE without known values of observed data
Question: Let $X_1,\dots,X_n$ be iid exponential rate $\lambda$. Suppose we don't know the observed values of our experiments, but we know that $k$ values were $\le M$ and the remaining $n-k$ were $&...
- 2,006
5
votes
0
answers
223
views
Intuitive explanation of requirement for achieving the Cramer Rao Lower Bound
this question relates to the requirement for achieving CRLB.
I know that for a random sample $Y_1, \ldots, Y_n$, an estimator $U$ of $g(\theta)$ is MVUE (i.e. it is unbiased and also $\operatorname{...
- 329
5
votes
1
answer
3k
views
Kruskal Wallis - Effect size
I analyse 4 algorithms and 3 sets of metrics for each algorithm in which I apply the non-parametric Kruskal-Wallis test for each metric to detect any differences in performance between these ...
- 153
4
votes
0
answers
78
views
MVUE problem related to splitting joint variables into independent ones.
Hi please help me with this problem. With the random samples $X_1,\dots,X_n$ from $\operatorname{Exp}(\mu, \sigma)$, I need to attain the MVUE of $\eta = \mathbb{P}(X_1>a)$. I used the Lehmann-...
- 68
4
votes
0
answers
144
views
Generalizations of Stein's identity to product of functions of gaussian vector
Given a $d$-dimensional Gaussian $X \sim N(\mu, \Sigma)$ and two real-valued differentiable functions $f,g$ with bounded first derivatives, I am wondering if there is a simple expression for the ...
- 6,136
4
votes
0
answers
72
views
Given n iid Pareto distributed random variables, find the UMP one sided test of the first moment
Given $X_1,...,X_n$ ($n\geq 2$) are iid and each have density:
$f_X(x) = \frac{c^\theta \theta}{x^{1+\theta}}\mathbb{1}(x> c)$ for known $c$ and $\theta > 1$
then we can easily find the first ...
- 71
4
votes
0
answers
112
views
Measurability of maximum likelihood estimator. Is there a mistake in Lehmann's "Theory of point estimation"?
I'm trying to prove that MLE from the proof of one theorem in Lehmann's "Theory of point estimation" (the theorem is below) is a measurable function. I know that under some regularity ...
- 5,690
4
votes
0
answers
156
views
Generalization of Cramer Rao Lower Bound.
Let $B(p)$ be a Bernoulli R.V. with mean $p$. Using the Cramer-Rao lower bound we have that for every unbiased estimator $\hat{\theta}$ of the parameter $p$ it holds
$$
E[(\hat{\theta}_n - p)^2] = ...
- 161
4
votes
0
answers
80
views
(Conditional) uniform asymptotic inference
Let $(\Omega,\mathcal{F})$ be measurable space and $\mathcal{P}$ be a family of probability measures on $(\Omega,\mathcal{F})$. A "typical" statistical problem is to show that
$$\tag{1}\label{1}
\...
- 756
4
votes
1
answer
2k
views
ML estimator of an double exponential distribution
Im trying to figure out the ML estimator of $$f_X(x)=\frac{1}{2\beta}\exp\left(-\frac{|x|}{\beta}\right)$$ as well as the variance of this estimator.
So far I have
$$L(\beta;x)=\prod_{i=1}^n\frac{1}{...
- 79
4
votes
0
answers
107
views
Random Variables and Statistic
I'm studying Statistical Inference by Casella and I'm confused with the definitions of random variable & statistic.
So let we have the probability space $(\Omega, F, P)$ where $\Omega$ is the ...
- 2,817
4
votes
0
answers
467
views
Nikolski class of probability measures - Metric and Topological Properties
I am reading a book about non-parametric statistics (Tsybakov's Introduction to Non-Parametic Estimation), and in order to prove some important inequalities on mean-squared error, different classes of ...
- 376
4
votes
0
answers
107
views
Teaching Student's distribution
While it is fairly straightforward to show the basics of the normal distribution in a first year undergraduate course, how does a teacher provide good intuition when the Student distribution comes in? ...
- 167
3
votes
0
answers
38
views
The sufficient statistic and unbiased estimator of normal variance
Suppose we have a normal distribution with mean $\theta_1$ and variance $\theta_2$.
I know that $\frac{1}{n-1}\sum_{i=1}^n (X_i-\bar{X})^2$ is an unbaised estimator of $\theta_2$ and has a variance $2\...
- 2,023
3
votes
0
answers
40
views
Best estimator of a matrix signal with binary entries
Setup: Given that we have a noisy matrix signal $\breve{B}\in\mathbb{R}^{p\times L}$ of the true signal $B\in\mathbb{R}^{p\times L}$, where the empirical distribution of the rows of $B$ converge to $\...
- 816
3
votes
0
answers
71
views
Show that $\bar{Y} - \min(Y_{1}, \dots, Y_{n})$ is independent of $\min(Y_{1}, \dots, Y_{n})$
Suppose that $Y_1, \dots, Y_n$ are i.i.d observations from the density $f(y, \theta, \beta) = \beta e^{-\beta(y - \theta)}I_{[y \geq\theta]}$
where $\beta \gt 0$, $\theta \in \mathbb{R}$ are unknown ...
- 41
3
votes
0
answers
100
views
Bound of Renyi divergence under addition of random variables?
Consider two random variables, $X$ with $p_X(x)$ and $Y$ with $p_Y(y)$. These random variables have Renyi divergence at level $\alpha$ of $R_1 = D_\alpha(p_X || p_Y)$
Now noise is added:
$X’ = X+A$
$Y’...
- 161
3
votes
0
answers
57
views
Bias be larger than variance in ERM
Given a convex set $S\subset \mathbb{R}^n$ and some $\theta\in S$, consider the observation $y=\theta+\epsilon$ where $\epsilon\sim \mathcal{N}(0,I)$, the ERM estimator is
$$\hat{\theta}=\arg \min_{x\...
- 117
3
votes
0
answers
234
views
Finding densities to estimate parameters using the Maximum likelihood technique
Consider the following autoregressive process with normal errors:
\begin{equation}\label{7YlUV4i8nuO}\tag{I}
y_t = \phi y_{t-1}+ u_t, \quad u_t \overset{iid}{\sim} N(0,\sigma^2)
\end{equation}
We ...
- 544
3
votes
0
answers
60
views
Relationship between the Fundamental Lemma of Calculus of Variations and Completeness in Statistical Inference
I've been studying the Fundamental Lemma of Calculus of Variations and the concept of completeness in statistical inference, and I've noticed that both concepts involve the idea that an integral (or ...
- 59
3
votes
0
answers
101
views
HMM, reverse engineering the transition matrix
I fitted a 2-states-HMM model last week, and generate a bunch of 1s and 0s, but I forgot to store its parameters (transition matrix). Now, I only got these 1s and 0s, how do I backward/reverse-...
- 159
3
votes
0
answers
180
views
Find a two-dimensional minimal sufficient statistic for $(\theta,j)$.
Let $(X_1,X_2,...,X_n)$ be a random sample from a distribution with discrete probability $f_{\theta,j}$, where $\theta \in (0,1)$, $j=1,2$, $f_{\theta,1}$ is the Poisson distribution with mean $\theta$...
- 721
3
votes
0
answers
130
views
Inference and Simple Gaussian Mixture
I am stuck with the following question. I would be grateful for any help. Assume, we have a simple Gaussian mixture:
\begin{align}
\text{with prob. } \pi: y &=\mu+\epsilon\\
\text{with prob. } 1-\...
- 319
3
votes
0
answers
441
views
Inverse of the transform in the Box-Muller transform
I am following this writeup of the Box-Muller method but I am confused how they derived the inverse. In this method, they write
Box-Muller Algorithm is a classic method to generate identical and ...
- 3,071
3
votes
0
answers
27
views
Correlation between correlations
Suppose
\begin{align*}
\begin{pmatrix}
X_i \\ Y_i \\ Z_i
\end{pmatrix} \sim N\left(\begin{pmatrix}
\mu_X \\ \mu_Y \\ \mu_Z
\end{pmatrix}, \sigma^2\begin{pmatrix}
1 & \rho_{XY} & \rho_{XZ} \\ \...
- 4,740
3
votes
0
answers
112
views
Fisher Information and Cramér-Rao lower bound problem
Suppose $X_1,...,X_n$ are random samples from $N(\mu, \sigma^2)$, where both $\mu$ and $\sigma \gt 0$ are unknown, and let $\theta = \sigma^p$ for some $p \gt 0$. I want to find the Fisher Information ...
- 183
3
votes
0
answers
176
views
What are books/notes on coordinate-free statistics?
There are some books here, but all focus on linear models.
Do we have books/notes with broader coverages?
In case of linear models, “coordinate-free” basically means “matrix-free” and uses the theory ...
- 59
3
votes
0
answers
142
views
How to logically justify the rejection of null hypothesis?
The rejection of null hypothesis will be valid only if this is true:
If the probability of the obtained result given that an assumption is true is very low, then the probability that the assumption is ...
- 151
3
votes
1
answer
451
views
Calculating the inverse of planar flows
I am trying to find a way to calculate the inverse of a planar flow.
In general, I understand that with normalizing flows, one can simply go from one distribution to the other with the change of ...
- 171
3
votes
0
answers
135
views
Uncertainty quantification Frequentist vs Bayesian
Is it actually possible to quantify the uncertainty in a frequentist setting? (e.g. using Maximum Likelihood Estimator).
Say that we have a dataset $\mathcal{D} = \{(x_i,y_i)\}_{i=1}^n$ and assume ...
- 1,963
3
votes
0
answers
90
views
Help with this simple theorem related to the Wald test
I'm studying the book all statistics by Wasserman and I'm trying to prove the theorem following the definition of the Wald Test on page 153.
10.3 Definition. The Wald Test
Consider testing
$H_0:\...
- 23.8k
3
votes
0
answers
200
views
Gaussian Mixture Division
In Bayesian inference, it is sometimes necessary to divide a Gaussian mixture (GM) posterior distribution by a GM prior. If the posterior GM is given by $$p_{1}(x) = \sum_{i=1}^{n} \alpha_{i} \mathcal{...
- 168
3
votes
0
answers
159
views
Proof of Theorem 4.16 from Mathematical Statistics by Jun Shao (Second Edition, Section 4.5.1, p.287)
First I would like to state the Theorem - it reads as follows: Let $X_{1}, \dotsc, X_{n}$ be i.i.d. from a p.d.f. $f_{\theta}$ w.r.t. a $\sigma$-finite measure $\nu$ on $(\mathcal{R},\mathcal{B})$ ...
3
votes
0
answers
141
views
Looking for the proof of theorem 5.2.11 of Casella, Berger, Statistical Inference
Theorem 5.2.11 Suppose $X_1,\dots, X_n$ is a random sample from a pdf or pmf $f(x\mid \theta)=h(x)c(\theta)\exp(\sum_{i=1}^kw_i(\theta)T_i(x))$ is in exponential family. Define statistics $T_i=\...
- 8,580
3
votes
0
answers
209
views
Confidence interval for generalized variance (determinant of covariance matrix)
Let $X\sim N(\mu,\Sigma)$ be a random variable in $\mathbb{R}^d$ with multivariate normal distribution. Let $\hat\Sigma$ be the maximum likelihood estimator for $\Sigma$,
\begin{align}
\hat\Sigma=\...
- 1,019
3
votes
0
answers
278
views
Why can we treat Cox's partial likelihood as a full likelihood?
I am doing some self study on Cox regression, and am trying to figure out how we can derive the partial likelihood for the Cox model from the full likelihood. Generally, I know that to get a partial ...
- 149
3
votes
1
answer
283
views
Generalized Least Squares results
So, I've got the next problem:
Let $Y\sim N_n(X\beta, \sigma^2 V)$. Prove that, if $\hat{\beta} = (X^{\prime}V^{-1}X)^{-1}X^{\prime}V^{-1}Y$ then:
$SSR = (Y-X\hat{\beta})^{\prime}V^{-1}(Y-X\hat{\...
- 610
3
votes
1
answer
96
views
Bayesian hypothesis testing and posterior distribution
Let $X$ be a random variable with a probability density $f(\cdot;\theta)$ where $\theta \in \Theta \subset \mathbb R$ is an unknown parameter. Suppose that we have a prior density $\pi(\theta)$, with ...
- 416
3
votes
0
answers
455
views
Relationship between gamma and poisson identity
Let $X_1, \dots, X_n$ be a random sample from a Poisson population with parameter $\lambda$ and define $Y = \sum_i X_i$. Y is sufficient for $\lambda$ and $Y \text{~} \text{Poisson}(n \lambda)$.
...
- 4,962