Questions tagged [maximum-likelihood]
For questions that use the method of maximum likelihood for estimating the parameters of a statistical model with given data.
1,459
questions
-3
votes
1
answer
177
views
Restricted covariance matrix [closed]
I want to define linear restrictions on a symmetric $3\times3$ covariance matrix such that matrix $A$ is equal to matrix $B$.
\begin{align}
A &= \begin{bmatrix} \sigma^2_{\nu_0} & \cdot & \...
- 1
2
votes
0
answers
38
views
Predicting if an Optimization Algorithm is "Doomed to Fail?
Suppose we have a linear combination weighted (with weights $\pi_i$ ) sum of Normal Distributions (Mixture Distributions https://en.wikipedia.org/wiki/Mixture_distribution):
\begin{align*}
p(x|\theta) ...
- 2,358
0
votes
0
answers
9
views
How to understand asymptotic normality under constraints?
Consider the following constrained maximum likelihood problem:
\begin{align*}
\min\limits_{\theta \in \mathbb R^d}~ & -\log p(x_{1:n};\theta) \\
{\rm s.t.} ~~& f(\theta)=0.
\end{align*}
Let $F(...
0
votes
0
answers
7
views
Does the Tyler's M-estimator lose the estimator of scale?
I was learning some robust estimation methods dealing with outliers and heavy-tail. I noticed that Tyler's M-estimator, whose key idea is to standardize the sample data by the distance to the mean, ...
0
votes
1
answer
83
views
How do we know the maximum likelihood estimator of a multivariate gaussian distribution is an argmax?
I was thinking about an optimization question that arises in parametric statistics.
Suppose that you have $(x_i)_{1\leq i \leq n}$, $n$ i.i.d observations in $\mathbb{R}^d$ that follow a multivariate ...
- 3
0
votes
0
answers
18
views
intuitive explanation/derivation of likelihood function for logistic regression
I'm struggling to wrap my head around the intuitiveness of the likelihood function for logistic regression shown below. If you could please explain why: A) you want to have a joint probability ...
0
votes
0
answers
16
views
Pivotal Quantity for Normal Distribution
Suposse a random sample of size $n$ from a Nomal distribution $X_{i}\sim N(\mu,\sigma^{2})$, for the following random variables:
(1) $\frac{\overline{X}-\mu}{S/\sqrt{n}}\sim t(n-1)$ and (2) $\frac{(n-...
- 21
1
vote
0
answers
28
views
Deriving Maximum Likelihood Estimators for a Linear Model with Exponential Error Term
I am currently working on a problem where I need to derive the maximum likelihood estimators for a linear model with an exponential error term. Here's the problem:
A machine sequentially performs two ...
- 11
0
votes
0
answers
13
views
Likelihood ratio as minimal sufficient statistics in infinite parameter space
Consider a family of density functions $f(x|\theta)$ where the parameter space for $\theta$ is finite, that is, $\theta \in \{\theta_0, \cdots, \theta_p\}$. Assume that $\theta_0$ is such that $f(x|\...
- 35
1
vote
2
answers
132
views
Given $x>0,y>0$, AND $\frac{3}{2 x^2+3 xy}+\frac{5}{3 xy+4 y^2} =2$, So max{xy}?
Given $x>0;y>0$; if $\frac{3}{2 x^2+3 xy}+\frac{5}{3 xy+4 y^2} =2$, Find max{xy}?
Here is my try:
Solution 1:
$xy=t$,
$\frac{3}{2 x^2+3t}+\frac{5}{3t+4y^2} =2$
$y = \frac{t}{x}$
$\frac{3}{2 x^...
- 211
-2
votes
1
answer
40
views
How to find the likelihood of a uniform random variable with support of length 1, but where we don't know where it starts.
It is said for the above question the answer is Option A) A horizontal line between $x_1$ and $x_1-1$.
I am unable to understand how the likelihood is being expressed in terms of $x_1$ rather than it ...
2
votes
0
answers
57
views
MLE of $ \theta$ in $ N(\theta, \theta^2) $ and Asymptotic Distribution of $\hat{{\theta}}_{\text{MLE}}$
Question
Let $( X_1, X_2, \ldots, X_n )$ be an independent random sample from $N(\theta, \theta^2)$ where $ \theta \neq 0 $.
Find the MLE for $\theta$ and find the asymptotic distribution of the MLE
...
0
votes
1
answer
29
views
Peak location of the maximum likelihood estimator's sampling distribution
Let's say we obtained a point maximum likelihood estimation $\hat{\theta}_\mathrm{MLE}\left(\mathbf{x}\right)$ from a set of measurements $\mathbf{x} = \left[x_1, x_2, \cdots, x_n \right]$ that ...
- 13
0
votes
0
answers
50
views
Integration issue with the Gamma statistical model
I need to verify if an MLE is biased for this Gamma statistical model.
\begin{align*}
\mathbb{E}\left[\frac{1}{\bar{X}}\right]&=\int^\infty_0\frac{1}{s}\frac{(n\beta)^{2n}}{\Gamma(2n)}s^{(2n-1)}e^{...
- 1,431
1
vote
1
answer
43
views
Asymptotic confidence interval using MLE and Fisher Information
We have observed x1, x2, ..., xn, independent samples from a Poisson distribution with an unknown mean λ > 0. Let $z_{1-α/2}$ denote the $1-\frac{α}{2}$ quantile of the standard normal distribution....
- 51
0
votes
0
answers
27
views
Finding the Maximum Likelihood Estimator (MLE) When the Likelihood is the Same
The Setup:
Let $X_1, \dots, X_n$ be IID with pdf $f_{\theta}$, where $\theta \in {0, 1}$ is unknown.
Given $f_{0}(x) \equiv 1_{0 < x < 1}$ and $f_{1}(x) \equiv \frac{1}{\sqrt{2x} }1_{0 < x &...
- 37
0
votes
0
answers
78
views
M.L.E of Pareto distribution
I must find the M.L.E for the Pareto distribution
$$f(x|x_0)=\begin{cases}
\frac{\alpha x_0^{\alpha}}{x^{\alpha+1}} & \text{ if } x\geq x_0, \\
0 & \text{ if } x<x_0.
\end{cases}$$
I end ...
- 981
1
vote
1
answer
129
views
Regarding the proof of James-Stein estimator
I'm currently struggling to understand the james stein estimator.
For $N \ge 3$ and the James-Stein estimator $\hat \mu^{JS} = (1-\frac{N-2}{\sum z_i^2})z$, where $z \sim N_N (\mu , I)$,
$$E[\Vert \...
- 605
2
votes
0
answers
35
views
Uncertainty analysis in maximum likelihood estimation under constraint
I'm not from a statistical background so you might have to excuse me for my somewhat inaccurate (or even erroneous) phrasing, I'll try the phrase my problem as I understand it.
The maximum likelihood ...
- 31
1
vote
1
answer
63
views
Find the maximum likelihood estimate of...
Suppose you have $n$ i.i.d random variables $X_1,\dots,X_n$ that are normally distributed with mean $\mu$ and variance $\sigma^2$. Thus,
$$ f_{X_i} (\mu , \sigma^2) = \left( \frac{1}{\sigma \sqrt{2\pi}...
- 11
0
votes
2
answers
67
views
Can a normal density be proportional to another normal density?
Suppose I have $X \sim \mathcal{N}(\mu, \sigma^2) = f_1$. Basically, in plain English, I have a density call it $f_1$ which is normal with mean $\mu$ and variance $\sigma^2$.
$$\Pr(\mu-\sigma < X &...
- 127
0
votes
0
answers
21
views
Estimating a Constant Duration (T) from Sequential Sampling
I have a statistical problem I've been grappling with, and I'd appreciate some guidance on how to approach it. Here's the scenario:
I have a binary variable, let's call it $x$, which can be in one of ...
- 43
0
votes
0
answers
17
views
GLRT for a one-sided composite hypothesis for $N(\mu, \sigma^2)$, $H_0 : \sigma \leq \sigma_0$
This is sort of related to (different hypothesis though): GLRT statistic for composite normal hypothesis, two unknowns
GLRT statistic for composite normal hypothesis, two unknowns
Problem
I am ...
- 55
1
vote
0
answers
15
views
The difference between the Bayesian estimator and MLE multiplied by $\sqrt{n}$ converges to zero.
Let $\theta$ be a random parameter with support $[0,1]$ and positive density, and let $X_1,X_2,\ldots \sim \rm N(\theta,1)$ be its i.i.d. observations. Define
$$ \delta_n := \sqrt{n} \Big(\hat \...
- 1,545
0
votes
0
answers
21
views
Expected squared difference between between the ML estimator and the posterior expectation.
Let $\theta$ be a random parameter with support $[0,1]$ and positive density, and let $X_1,X_2,\ldots ~ \rm N(\theta,1)$ be its i.i.d. observations. Does
$$ \rm E\Big[n\cdot \Big(\hat \theta_n(X_1,\...
- 1,545
0
votes
1
answer
37
views
Finding the GLRT for a one-sided hypothesis using uniform distribution
The Problem
Let $X_1, \dots, X_n \sim U(\theta, 5)$ where $0 < \theta < 5$ with pdf
$$
f(x;\theta) = \dfrac{1}{5-\theta} \quad \theta < x < 5
$$
Find the generalized likelihood ratio test ...
- 55
0
votes
0
answers
58
views
Maximizing the likelihood over the truncated support always leads to strictly greater probability on the truncated region than original pdf?
Suppose $\mathbf{X}$ is a random variable with a finite support $\Omega$ and with some pdf $f(\cdot; \mathbf{v}_0)$ where $\mathbf{v}_0$ is the parameter. Define, $\mathcal{A}:= \{\mathbf{x}:S(\mathbf{...
- 127
2
votes
1
answer
263
views
Is the maximum likelihood estimator for the mean equal to the sample mean under all densities?
Suppose, I have a sample from an unknown distribution. I want to prove/disprove (mathematically!) the following statement:
The maximum likelihood (ML) estimator for the (unknown) population mean will ...
- 127
0
votes
0
answers
28
views
Performing Maximum A Posteriori estimation on a set of dice results.
I have a set of data obtained from rolling a 20 sided dice 1000 times. I understand that ideally a dice would have a uniform distribution, and that forms my prior belief. But how exactly does one go ...
0
votes
0
answers
45
views
Definition of likelihood function on Wikipedia
In Likelihood function page of Wikipedia, the definition of likelihood is written as below:
Given a probability density or mass function
$x \mapsto f(x\mid\theta),$ (1)
where x is a realization of ...
2
votes
0
answers
31
views
Distribution of concominant order statistics
Motivating problem: We have $n$ students writing a mock test, and a day after, they write a final test. Let $X_i$ represent the grade (continuous from $0$ to $\infty$) of $i$-th student from the first ...
- 722
2
votes
1
answer
62
views
Estimation of exponential distribution parameter from smallest $n$ out of N observations
I am interested in estimating the parameter $\lambda$ of an exponential distribution based on the smallest $n$ out of a total of $N$ observations.
In mathematical terms: let $X$ be distributed ...
- 3,591
0
votes
0
answers
60
views
True value of a parameter in statistics
Let us consider a statistics model
$X_1,X_2,\cdots,X_n\sim^{i.i.d} f(x,\theta)$,
where $f(x,\theta)$ is a probability density function with a parameter $\theta$.
Let $\theta_0$ be a true value of $\...
1
vote
0
answers
50
views
Neyman-Pearson Lemma clarification
A randomized Neyman-Pearson test is of the form
$$ \phi(X_1,\ldots,X_n):= 1 \text{ if } \frac{p_1(X)}{p_0(X)} > c_0, \ q \text{ if } \frac{p_1(X)}{p_0(X)} = c_0, \ 0 \text{ if } \frac{p_1(X)}{p_0(X)...
- 317
0
votes
0
answers
71
views
Testing if the mean of a Laplace distributed random variable is zero
Suppose I have a sample $x=(x_1,x_2,...,x_n)^T$ of size $n$. Suppose I perdormed a statistical test and successfully showed that the sampling comes from a Laplace distribution with parameters that are ...
- 157
1
vote
1
answer
57
views
Maximum likelihood of normal distribution with mean $0$
If
$$f(x;\theta) = \frac{1}{\sqrt{2\pi\theta}}e^{-x^2/(2\theta)},$$
what is the maximum likelihood estimator of $\theta$?
(I answered my own question, since writing all this code made me aware the ...
user952685
0
votes
1
answer
200
views
Is Maximum likelihood estimation "better" than the method of moments?
We are given a random sample $\mathbf x=(x_1,\dots,x_N)$ from a parameterized distribution $x_n\sim F_\theta$ and asked to estimate the parameters $\theta\in\Bbb R^m$.
Method of moments (MoM) ...
- 302
0
votes
1
answer
58
views
Probability density functions for the maximum likelihood density estimation
I have the following constrained optimization problem corresponding to the maximum likelihood density estimation:
\begin{equation}
\begin{aligned}
&\text{maximize} && L(f) \\
&\...
- 171
1
vote
0
answers
34
views
Likelihood Ratio Test for Comparing Two Poisson Distributions
I am currently studying statistical hypothesis testing, specifically the Likelihood Ratio Test (LRT), and I've come across a problem that I'm struggling to solve. I would greatly appreciate any ...
0
votes
1
answer
82
views
How to calculate an integral with an unknown number of integration variables?
How to calculate the following integral, which has an unknown number of integration variables?
$$
\int\limits_{-\infty}^{+\infty}\cdots\int\limits_{-\infty}^{+\infty}\exp\left[-\dfrac{1}{2\theta}\sum_{...
- 87
1
vote
1
answer
114
views
How to calculate Fisher Information of exponential family w.r.t. mean parameterization in maximum likelihood estimation?
We have the exponential family:
$$
f_\mathbf{X}(x;\theta) = h(x)\exp\{\langle\theta, T(x)\rangle-A(\theta)\}
$$
where the parameter vector $\theta$ is often referred to as canonical parameter or ...
- 11
0
votes
0
answers
48
views
Can the parameter in maximum likelihood function be infinty if that's when the maximum occurs?
I was working on this problem:
Find the maximum likelihood estimator for the parameter $a$, in the distribution
$$f(x) =\begin{cases}
3a^3\cdot x^{−4} & \text{if } x ≥ a \\
0 & \text{...
- 1
2
votes
2
answers
89
views
Estimate unknown parameter by maximum likelihood method and moment's method
There is a random sample $X_1, X_2, ..., X_n$ distributed with:
$ x $
-3
0
3
$ P(X = x) $
$ \frac{1}{3} - \theta $
$ \frac{1}{3} + 2 \theta $
$ \frac{1}{3} - \theta $
Estimate unknown parameter by ...
- 175
1
vote
1
answer
96
views
Maximum likelihood estimator with indicator
Let $X_1,...,X_n$ be the sample from distribution with density
$$
p_{\alpha,\beta}(x) = \frac{1}{\alpha}e^{(\beta−x)/\alpha}I_{[\beta,+\infty)}(x).
$$
where $θ = (\alpha,\beta)$ is a two-dimensional ...
- 471
0
votes
0
answers
194
views
Can You Multiply Different Probability Distributions Together?
Suppose there is set of $k$ different countries. Let's say that the average income of a specific country can be given by:
$$X_i \sim N(\mu, \sigma_e^2 + \sigma_i^2)$$
Where:
$X_i$ is the average ...
- 2,358
0
votes
0
answers
27
views
How do you infer the model of a car based on prior information?
Sorry if this does not quite make sense as I am still wrapping my head around it as well. Suppose I have j car models (i.e. different brands, builds etc.) such that $\textbf{m} = {m_1, m_2, . . . , ...
- 21
0
votes
0
answers
29
views
log-likelihood: changing relative contribution of two summation terms
Suppose I have a log-likelihood of the form
$$\mathcal{L} = \sum_{i = 1}^{n} a_i + \sum_{j = 1}^{m} b_j,$$
where $a_i$ and $b_j$ are some independent log-probabilities. The problem is, the second sum $...
1
vote
1
answer
50
views
Find the maximum likelihood estimator for θ
We have a simple random sample of size n from a distribution with pmf 𝑝(𝑥) = $\theta{(1-\theta)}^{x-1}$ for 𝑥 = 1,2, …. Find the MLE[𝜃]
My try:
$
L\ =\ \theta{(1-\theta)}^{1-1}\times\theta{(1-\...
- 23
0
votes
0
answers
47
views
Fisher Information Matrix singular but unbiased estimator exists? Please help me figure out where I've gone wrong
I'm doing some research into the Cramer-Rao bound for time of arrival localization and have come across a rather strange result: the FIM is singular, but there exists an unbiased estimator. My ...
- 11
0
votes
1
answer
92
views
Maximum likehood estimate
I have a question regarding the correction of my exercise:
Exercise 6. Let $Y_1,\dots,Y_n$ be i.i.d. such that $Y_i$ equals $1$ with probability $p$ and $-1$ with probability $1-p$, for all $i\in[n]$....
