Questions tagged [maximum-likelihood]
For questions that use the method of maximum likelihood for estimating the parameters of a statistical model with given data.
1,482
questions
0
votes
1
answer
23
views
Argmax of sum of log functions
I am trying to find the below with the context that each $\pi_s$ is a probability. (Trying to assign probabilities so I get maximum likelihood)
$$\underset{\bf{\pi}}{\operatorname{argmax}} \ \sum_{s} ...
0
votes
1
answer
12
views
Prove that MLEs of two independent samples are independent
I am trying to prove that if I have two samples of iid random variables, then MLE's based on these two samples will be also independent.
More formally, let
$$\mathbf{x} = (x_i)^T_{i = 1} \stackrel{iid}...
0
votes
0
answers
14
views
Pareto MLE with restricted range
The $\text{Pareto}(1, \alpha)$ MLE is $\frac{n}{\sum_{i = 1}^{n} \log x_{i}}$, where $X_{1}, X_{2}, ..., X_{n}$ are i.i.d. with $f(x | \alpha) = \alpha x^{-\alpha - 1}$. Further, add the constraint ...
0
votes
0
answers
21
views
Maximum likelihood estimator in both sides of the equation
I am following Mixed Models theory and applications with r, and there is an MLE that I am struggling to get, and it seems that the parameter appears in both sides of the equation.
We have the model $...
0
votes
0
answers
56
views
Independence of MLEs for subsamples
Suppose I have a sample $\textbf{x} = (x_1, \dots, x_n)$ from stationary and $\beta$-mixing time series. I want to estimate the scalar parameter $\theta$ of the distribution using MLE, i.e.,
$$ \hat{\...
0
votes
1
answer
31
views
Maximum Likelihood - Information Matrix Identity Derivation
I try to derive the information matrix equality for the Poisson distribution with the log-Likelihood:
$$\mathcal{L}(\lambda; x_1, x_2, \ldots, x_n) = \sum_{i=1}^{n} \left[-\lambda + x_i \log(\lambda) -...
0
votes
0
answers
17
views
Matlab fminunc stops at first iteration [closed]
The assignment is:
Suppose yit = 1 (β0 + βitxit + εit > 0), where i indexes an individual and t indexes time.We observe each individual twice (t = 1, 2). We continue to assume that β0 = 1, βit ∼ N (...
0
votes
0
answers
8
views
Logit panel with Simulated Maximum Likelihood
I'm testing a very basic logit panel model in matlab. The setup is as follows: We observe a binary variable $y_{it} = 1(\beta_0 + \beta_{it}x_{it} + \varepsilon_{it} > 0)$ where i is individual and ...
1
vote
1
answer
55
views
Binary choice with mixture of normal errors estimation
My problem is simplified as follows: I'm estimating a binary choice probit model where the error term follows a mixture of two normal distributions. This is:
$y_i = \mathbf{1} \left( \beta_0 + \beta_1 ...
-2
votes
0
answers
29
views
Find Logarithmic Maximum Likelihood Estimation (Log MLE) for a piecewise Probability Density Function (PDF) with zero in one of it's rules.
Consider random variable Y having PDF
$$
f_{Y}(y|\theta) = \begin{cases}
\\
\frac{1}{\theta}ry^{r-1}e^{ \frac{-y^r}{\theta}}, & y , \theta> 0 \\\\
0, & \text{otherwise}
\end{cases}
$$
...
0
votes
0
answers
34
views
Maximizing the expectation in CE Importance sampling.
Suppose the following maximization:
$$v_t = \arg \max_{v} E_{v_{t-1}} 1\{S(x) \geq \gamma\} \frac{f(x;u)}{f(x;v_{t-1})}\ln f(x;v) = max_{v} E_{v_{t-1}} 1\{S(x) \geq \gamma\} W(u;v_{t-1}) \ln f(x;v),$$
...
0
votes
1
answer
31
views
Deriving the log-likelihood of a multivariate normal distribution by \mu
I have a sample from multiple groups, $k=\{1,2,..,K\}$. I need to find the maximum likelihood estimates for $\mu^{(k)}$ and for $V$. Looking at here I can see how the entire right part becomes $Nd; N=...
1
vote
1
answer
71
views
Calculating MSE of MOM and MLE of a Uniform Distribution
Let $X_1, X_2, X_3$ be a random sample of size three from a $uniform(θ, 2θ)$ distribution, where $θ > 0$
I solved to get $\tilde{\theta}_{MoM}$ to be $\frac{2\bar{x}}{3}$.
Also, I got $\hat{\theta}...
1
vote
1
answer
32
views
0
votes
1
answer
122
views
Statistical framework for using MSE for linear classification
We learned in class to use a linear model to to predict a real target value y. We made the assumption that $$ y = w^Tx + w_0 + \epsilon $$ where $x$ is the input vector,$w$ is the vector of the ...
0
votes
0
answers
29
views
How to find the maximum likelihood estimate with sample minimum??
Consider the independent non-identical random variables $X_i \sim Exp(i/\mu)$
, enumerated by $i \in \mathbb{N}$. Here, $\mu > 0$ is an unknown
parameter. Instead of the entire sample $x_1, . . . , ...
4
votes
1
answer
36
views
Maximizing log-likelihood: Determinining whether critical points are maximums
Consider the following proof of the fact that $\bar{X}$, the sample mean, is the MLE of parameter $\lambda$ in a Poisson distribution.
Let $x_1, \ldots, x_n$ be the observations of $X_1, \ldots, X_n$ ...
0
votes
0
answers
32
views
What does singular hessian in optimization tell me
I am doing optimization using maximum likelihood estimation, and when I am trying to get the standard errors of estimates using hessian matrix, I get non-invertible/singular hessian warning.
After I ...
0
votes
0
answers
16
views
Deriving MLE & asymptotic variance manually - for which distributions/cases is it possible?
I am interested whether you know any distributions or special cases where the maximum likelihood estimator, the theoretical and estimated asymptotic variance can be fully derived manually, i.e. which ...
0
votes
0
answers
26
views
Estimation of the variance of ML estimator (linear regression)
Given the following likelihood:
$$\prod_{i=1}^{N} \frac{1}{\sqrt{2 \pi \sigma^2}} \exp\left\{-\frac{(y_i - x_i' \beta)^2}{2\sigma^2}\right\}$$
Thanks to the information matrix equality we have two ...
0
votes
1
answer
30
views
Finding a function whose product with itself has a higher value at an optimal point for a different parameter value
Suppose we have a function $f(x;\theta)$ that depends on a parameter $\theta$. We also have another function $P(x, y;\theta)=f(x;\theta)\cdot f(y;\theta)$. For a fixed value of $\theta$, say $\theta_1$...
0
votes
0
answers
11
views
Derive the asymptotic variance for ML estimator with BHHH (Poisson)
I tried to apply the BHHH method (outer products of gradients) to derive the asymptotic variance of the Poisson distribution:
$\hat{\text{avar}}(\hat{\lambda}) = \left( \sum_i s_{i}(\hat{\lambda}_{i}) ...
2
votes
0
answers
63
views
Method of Moment and MLE of Bernoulli
Given n observations $X_1, X_2, \dots, X_n$ from a random sample with Bernoulli probability function
$$
\operatorname{Pr}(X=k) = p^k(1-p)^{1-k}, \text{ for }k=0,1.
$$
Denote the method of moment ...
0
votes
0
answers
11
views
Difference between Likelihood Estimation and CRLB Estimation for Cooperative Radar
I do not know if this question fits this stack but I do not know if there's other place where I can ask.
The question is about the difference between the cooperative/collaborative radar system when ...
0
votes
0
answers
35
views
Application of the Generalized Method of Moments
The professor gave us an exercise but for me it's not totally clear. I have a distribution function which depend on a parameter $x$ that I need to estimate. The function describes the distribution of ...
1
vote
1
answer
77
views
Show Hodges estimator is not a regular estimator
The question is related to Semiparametric Theory Ch3 by Tsiatis
First consider a Hodge estimator:
Let $Z_1, \ldots, Z_n$ be iid $N(\mu, 1), \mu \in \mathbb{R}$. For this simple model, we know that ...
0
votes
2
answers
70
views
Show MLE $\bar Z_n$ of normal $N(0,1)$ has $P(|\bar{Z_n}| < n^{-1/4}) \to 1$.
Recall Let $Z_1, \ldots, Z_n$ be iid $N(\mu, 1), \mu \in \mathbb{R}$. For this simple model, we know that the maximum likelihood estimator (MLE) of $\mu$ is given by the sample mean $\bar{Z}_n=n^{-1} \...
3
votes
1
answer
53
views
Find conditional MLE of AR time series
I was given a model $r_{t} = ϕ_{0} + ϕ_{2}r_{t-2} + ϵ_{t}$ with $\epsilon_t \sim N(0,\sigma^2)$ and have to derive the likelihood of $(r_{3}, r_{4}, . . . , r_{T})$ conditional on $(r_{1}, r_{2})$ and ...
0
votes
0
answers
11
views
Find maximum likelihood estimator of joint probability distribution of Bernoulli variables (for use in mutual information/feature selection)
When performing feature selection by finding mutual information estimates for class C and feature U (both binary), we need to estimate joint probabilities like P(C=1, U=1). This site claims that the ...
0
votes
1
answer
108
views
likelihood of data given parameter or likelihood of parameter given data?
What is more correct? Likelihood of data given parameter or likelihood of parameter given data? In https://en.wikipedia.org/wiki/Likelihood_function for instance, we see both
likelihood of $\hat{\...
0
votes
0
answers
78
views
Finding Asymptotic Confidence Intervals for Parameter θ in Uniform and Exponential Distributions
Let $(X_1, \ldots, X_n)$ be an i.i.d. random sample. Determine asymptotic confidence Let $(X_1, \ldots, X_n)$ be an i.i.d. random sample. Determine asymptotic confidence intervals at level $\gamma \in ...
2
votes
0
answers
39
views
Predicting if an Optimization Algorithm is "Doomed to Fail?
Suppose we have a linear combination weighted (with weights $\pi_i$ ) sum of Normal Distributions (Mixture Distributions https://en.wikipedia.org/wiki/Mixture_distribution):
\begin{align*}
p(x|\theta) ...
0
votes
0
answers
18
views
How to understand asymptotic normality under constraints?
Consider the following constrained maximum likelihood problem:
\begin{align*}
\min\limits_{\theta \in \mathbb R^d}~ & -\log p(x_{1:n};\theta) \\
{\rm s.t.} ~~& f(\theta)=0.
\end{align*}
Let $F(...
0
votes
0
answers
11
views
Does the Tyler's M-estimator lose the estimator of scale?
I was learning some robust estimation methods dealing with outliers and heavy-tail. I noticed that Tyler's M-estimator, whose key idea is to standardize the sample data by the distance to the mean, ...
0
votes
1
answer
102
views
How do we know the maximum likelihood estimator of a multivariate gaussian distribution is an argmax?
I was thinking about an optimization question that arises in parametric statistics.
Suppose that you have $(x_i)_{1\leq i \leq n}$, $n$ i.i.d observations in $\mathbb{R}^d$ that follow a multivariate ...
0
votes
0
answers
23
views
intuitive explanation/derivation of likelihood function for logistic regression
I'm struggling to wrap my head around the intuitiveness of the likelihood function for logistic regression shown below. If you could please explain why: A) you want to have a joint probability ...
0
votes
0
answers
39
views
Pivotal Quantity for Normal Distribution
Suposse a random sample of size $n$ from a Nomal distribution $X_{i}\sim N(\mu,\sigma^{2})$, for the following random variables:
(1) $\frac{\overline{X}-\mu}{S/\sqrt{n}}\sim t(n-1)$ and (2) $\frac{(n-...
1
vote
0
answers
44
views
Deriving Maximum Likelihood Estimators for a Linear Model with Exponential Error Term
I am currently working on a problem where I need to derive the maximum likelihood estimators for a linear model with an exponential error term. Here's the problem:
A machine sequentially performs two ...
0
votes
0
answers
25
views
Likelihood ratio as minimal sufficient statistics in infinite parameter space
Consider a family of density functions $f(x|\theta)$ where the parameter space for $\theta$ is finite, that is, $\theta \in \{\theta_0, \cdots, \theta_p\}$. Assume that $\theta_0$ is such that $f(x|\...
1
vote
2
answers
135
views
Given $x>0,y>0$, AND $\frac{3}{2 x^2+3 xy}+\frac{5}{3 xy+4 y^2} =2$, So max{xy}?
Given $x>0;y>0$; if $\frac{3}{2 x^2+3 xy}+\frac{5}{3 xy+4 y^2} =2$, Find max{xy}?
Here is my try:
Solution 1:
$xy=t$,
$\frac{3}{2 x^2+3t}+\frac{5}{3t+4y^2} =2$
$y = \frac{t}{x}$
$\frac{3}{2 x^...
-2
votes
1
answer
44
views
How to find the likelihood of a uniform random variable with support of length 1, but where we don't know where it starts.
It is said for the above question the answer is Option A) A horizontal line between $x_1$ and $x_1-1$.
I am unable to understand how the likelihood is being expressed in terms of $x_1$ rather than it ...
2
votes
0
answers
110
views
MLE of $ \theta$ in $ N(\theta, \theta^2) $ and Asymptotic Distribution of $\hat{{\theta}}_{\text{MLE}}$
Question
Let $( X_1, X_2, \ldots, X_n )$ be an independent random sample from $N(\theta, \theta^2)$ where $ \theta \neq 0 $.
Find the MLE for $\theta$ and find the asymptotic distribution of the MLE
...
0
votes
1
answer
32
views
Peak location of the maximum likelihood estimator's sampling distribution
Let's say we obtained a point maximum likelihood estimation $\hat{\theta}_\mathrm{MLE}\left(\mathbf{x}\right)$ from a set of measurements $\mathbf{x} = \left[x_1, x_2, \cdots, x_n \right]$ that ...
0
votes
0
answers
54
views
Integration issue with the Gamma statistical model
I need to verify if an MLE is biased for this Gamma statistical model.
\begin{align*}
\mathbb{E}\left[\frac{1}{\bar{X}}\right]&=\int^\infty_0\frac{1}{s}\frac{(n\beta)^{2n}}{\Gamma(2n)}s^{(2n-1)}e^{...
1
vote
1
answer
80
views
Asymptotic confidence interval using MLE and Fisher Information
We have observed x1, x2, ..., xn, independent samples from a Poisson distribution with an unknown mean λ > 0. Let $z_{1-α/2}$ denote the $1-\frac{α}{2}$ quantile of the standard normal distribution....
0
votes
0
answers
30
views
Finding the Maximum Likelihood Estimator (MLE) When the Likelihood is the Same
The Setup:
Let $X_1, \dots, X_n$ be IID with pdf $f_{\theta}$, where $\theta \in {0, 1}$ is unknown.
Given $f_{0}(x) \equiv 1_{0 < x < 1}$ and $f_{1}(x) \equiv \frac{1}{\sqrt{2x} }1_{0 < x &...
0
votes
0
answers
173
views
M.L.E of Pareto distribution
I must find the M.L.E for the Pareto distribution
$$f(x|x_0)=\begin{cases}
\frac{\alpha x_0^{\alpha}}{x^{\alpha+1}} & \text{ if } x\geq x_0, \\
0 & \text{ if } x<x_0.
\end{cases}$$
I end ...
1
vote
1
answer
285
views
Regarding the proof of James-Stein estimator
I'm currently struggling to understand the james stein estimator.
For $N \ge 3$ and the James-Stein estimator $\hat \mu^{JS} = (1-\frac{N-2}{\sum z_i^2})z$, where $z \sim N_N (\mu , I)$,
$$E[\Vert \...
2
votes
0
answers
45
views
Uncertainty analysis in maximum likelihood estimation under constraint
I'm not from a statistical background so you might have to excuse me for my somewhat inaccurate (or even erroneous) phrasing, I'll try the phrase my problem as I understand it.
The maximum likelihood ...
1
vote
1
answer
64
views
Find the maximum likelihood estimate of...
Suppose you have $n$ i.i.d random variables $X_1,\dots,X_n$ that are normally distributed with mean $\mu$ and variance $\sigma^2$. Thus,
$$ f_{X_i} (\mu , \sigma^2) = \left( \frac{1}{\sigma \sqrt{2\pi}...