Skip to main content

Questions tagged [maximum-likelihood]

For questions that use the method of maximum likelihood for estimating the parameters of a statistical model with given data.

Filter by
Sorted by
Tagged with
-1 votes
1 answer
35 views

Likelihood of Bayes' theorem [closed]

When estimating the parameter (hypothesis), I thought it was correct to compare the values of "P(hypothesis_i | observed data)" by changing i for each hypothesis However, when applying Bayes'...
hi-jin's user avatar
  • 1
0 votes
1 answer
23 views

Question about likelhood function of discriminative models

Im a little confused with the likelihood function. For discriminative models, we have a hypothesis function $h_{\theta}(x) = p(y \mid x ; \theta)$. Using the principles of maximim likelihood we want ...
Joe Jameson's user avatar
0 votes
0 answers
34 views

Trigamma-free Negative Binomial regression: doubts on Hessian and Fisher Information Matrix in the dispersion parameter

I have been looking at alternative versions of the Hessian and Fisher (expected) Information Matrix for the Negative Binomial regression specification, which are given by widely-cited academic sources ...
DrEti's user avatar
  • 63
4 votes
3 answers
96 views

Maximum Likelihood Estimation for Poisson Mean with Given Observations

You have a sample of $n$ i.i.d. realizations of the random variable $X$ distributed as a Poisson with parameter $\lambda$. It is known that: $n_1$ values are greater than or equal to $2$; $n_2$ ...
Emalas's user avatar
  • 43
1 vote
1 answer
25 views

Rejection region in hypothesis test using LRT [closed]

I have $X_i \sim Bi(1, \theta)$ and want to test $H_0: \theta = \theta_0$ vs $H_1: \theta \neq \theta_0$ using the LRT test. I've found that $\lambda = \frac{sup_{\theta \in \Theta_0}L(\theta)}{sup_{\...
Peter Sampodiras's user avatar
0 votes
1 answer
43 views

Prove that the MLE for variance is consistent

Suppose $X_1, ..., X_n$ are independent random variables with expected value $\mu$ and variance $\sigma^2$: I would like to show the $$ \hat\sigma^2 = \frac{1}{n} \sum_{i=1}^{n}\left(X_i - \bar X_n\...
user007's user avatar
  • 615
0 votes
0 answers
50 views

Finding the Cramer Rao bound

Let $x=(x_1,\dots,x_n)$ be a sample of i.i.d random variables with pdf $$f(x;\theta)=(1-\theta)\chi_{[-1/2,0]}+(1+\theta)\chi_{[0,1/2]}$$ where $\theta\in(-1,1)$. Find the Cramer Rao bound. So to do ...
Tutusaus's user avatar
  • 657
2 votes
2 answers
128 views

Computing the likelihood function

Let $x=(x_1,\dots,x_n)$ be a sample from $X_1,\dots,X_n$ independent identically distributed random variables with pdf $$f(x)=(1-\theta)1_{[-\frac{1}{2},0]}(x)+(1+\theta)1_{(0,\frac{1}{2}]}(x)$$ where ...
Tutusaus's user avatar
  • 657
5 votes
2 answers
191 views

Finding MLE given dependent observations from uniform distribution $U(0,\theta)$ [closed]

Suppose we are given random variables $X_1,...,X_n$ that are uniformly distributed on the interval $[0,\theta]$, with $\theta >0$ unknown. I know that if the $X_1,...,X_n$ are furthermore ...
user007's user avatar
  • 615
0 votes
0 answers
15 views

Deriving posterior distribution with explicit constants in closed form using Jacobian method compared to Bayes' rule

Question on Bayesian Inference Deriving posterior distribution with explicit constants in closed form using Jacobian method compared to Bayes' rule I am working on a Bayesian ...
Alireza's user avatar
  • 311
0 votes
0 answers
13 views

Show how to approximate the likelihood of a Stochastic Volatility model with Gaussian AR(1) log-volatility and Gaussian errors

hi i am very new to SVM and are asked the following question: $\textbf{(i) Show how to approximate the likelihood of a Stochastic Volatility model with Gaussian AR(1) log-volatility and Gaussian ...
Robbert's user avatar
  • 25
1 vote
1 answer
29 views

Check MLE estimator is asymptotically normal

Let $X_i$ be i.i.d. random variables, with parameter $\theta > 0$, such that: $$X_i \sim \mathrm B\left(\frac{1}{\theta}, 1\right)$$ with PDF: $$f(x,\theta) = \frac{x^\frac{1}{\theta}}{x\theta}\Bbb ...
meerkat's user avatar
  • 358
1 vote
1 answer
28 views

Min-max optimization and prediction of a parameter in a mathematical model

Context Hello, everyone; let me preface this by saying that my background is in CS and not mathematics, but I do have a background in calculus, statistics, and discrete mathematics. The issue at hand ...
A.Sal's user avatar
  • 13
2 votes
0 answers
35 views

Difference between compensator of point process under real parameter an its MLE estimator

Suppose we have some point process $N=N_{\theta_0}$ on the real line, driven by a conditional intensity $\lambda_{\theta_0}$ dependent on some finite-dimensional parameter $\theta_0\in\Theta\subset\...
Václav Mordvinov's user avatar
2 votes
1 answer
29 views

Efficient and unbiased estimation of the location ($\mu$) of truncated normal distribution with known scale ($\sigma^2$) and truncation points

I have one observation $x$ which I know comes from the following truncated normal distribution: $$x \sim TN(\mu, \sigma^2, -\delta, \delta) \;\textrm{ where }\; \delta > 0$$ In my problem, the ...
Tele's user avatar
  • 21
4 votes
2 answers
257 views

Given the maximum likelihood function- estimate the value of the parameter

Lets say I have the pdf and maximum likelihood function: $ f_X(x) = \begin{cases} \frac{\alpha \beta^\alpha}{x^{\alpha+1}}, & x > \beta, \\ 0, & x \leq \beta. \end{cases} $ $ \begin{...
Need_MathHelp's user avatar
0 votes
0 answers
12 views

Nonparametric likelihood function $\mathcal{L}_n(f) = \prod_{i=1}^nf(X_i)$ doesnt attain maximum in set of all densities

Let $X_1, \dots, X_n$ be i.i.d random variables with distribution function $F$, and $\mathcal{L}_n(f) = \prod_{i=1}^nf(x_i)$ it's likelihood function. Let $\mathcal{F}$ be the family of all possible ...
nicoyanovsky's user avatar
2 votes
1 answer
36 views

How to derive likelihood function

I have been struggling a lot with the concept of likelihood and I'd really appreciate it if someone could verify if my understanding is correct and give input. If I understand this correcly, we pick ...
Need_MathHelp's user avatar
1 vote
1 answer
46 views

Maximum likelihood on Ticket Collector

There are $N$ types of tickets. A ticket is drawn, noted, and kept back. Any ticket may be observed equally likely on a given draw, regardless of what happened in the previous draws. $n=519$ tickets ...
yeetcode's user avatar
  • 143
1 vote
1 answer
32 views

Asymptotic distribution of MLE of $\theta$ for $f(x) = (1-\theta)1_{(-1/2,0)}(x) + (1+\theta)1_{(0,1/2)}(x)$

I am trying to find the asymptotic distribution for $\theta$ given $f(x) = (1-\theta)1_{(-1/2,0)}(x) + (1+\theta)1_{(0,1/2)}(x)$. I've shown that $\hat{\theta} = \frac{T_2 - T_1}{n}$, where $T_1 = \...
Peter Sampodiras's user avatar
2 votes
1 answer
45 views

Mean squared error of MLE of $\theta$ where $f(x) = 3x^2 \theta e^{-\theta x^3} 1_{(0,\infty)}(x)$

Given $f(x) = 3x^2 \theta e^{-\theta x^3} 1_{(0,\infty)}(x)$ I want to find the MSE of the MLE estimator for $\theta$. I've found that $\hat{\theta} = \frac{n}{\sum_{i=1}^n X_i^3} = \frac{1}{\bar{X^3}}...
Saim Faigol's user avatar
1 vote
0 answers
15 views

MLE and limit distribution of ratio of parameters

I am solving an estimation problem and I can't make any progress. I have $ (X_{i1},X_{i2})^T $ iid from $\mathcal{N}_2 ( \mu , \Sigma)$. Define $$ \lambda_j = \frac{\mu_j}{\sqrt{\sigma_{jj}^2}} , \...
daniel's user avatar
  • 753
0 votes
1 answer
53 views

Maximum Likelihood Estimation of median for an exponential distribution

Given data x1, ... xn i.i.d. with exponential distribution and unknown parameter λ, determine maximum likelihood estimation of θ given the observed data where theta is the median of the distribution. ...
Michael Williams's user avatar
0 votes
1 answer
28 views

Maximum Likelihood Estimation based on the sample minimum

I have $x_1, \dots, x_n$ being observations independently drawn from $X \sim Exp(\lambda)$ and I am trying to calculate MLE for $\lambda$, only knowing $s = \min\{x_1, \dots, x_n\}$. I suppose the ...
timten's user avatar
  • 3
0 votes
0 answers
16 views

Estimating the Parameter $\eta$ in a Mixture Model Involving Gaussian Noise

Problem Description: Consider the mixture model defined by the equation: $$ z = x + \eta \cdot (y - x) + k \cdot \sqrt{\eta} \cdot n $$ where: $ x, y, z $ are known D-dimensional vectors. $ n $ is a ...
BinChen's user avatar
  • 628
1 vote
0 answers
29 views

MLE of the covariance of the probabilistic PCA (multivariate normal distribution)

To get to the MLE of probalistic PCE, we start with the marginal distribution, which is defined with $x \sim \mathcal N(b, C)$ with $C=WW^\top + \sigma^2 I_d \text{ and } b \in \mathbb{R}^d, W \in \...
Lopsio's user avatar
  • 85
0 votes
0 answers
7 views

Standard practice to show Biased CRBs

I have a problem with four-parameter estimation. I have derived the variances for the estimated parameters using Monte Carlo simulations (numerical ones) and theoretical ones using the inverse of the ...
CfourPiO's user avatar
  • 109
0 votes
0 answers
20 views

Why is maximizing the Cox partial likelihood meaningful

So the likelihood of seeing our data in survival analysis is (assuming simple case of no censorship and no ties): ...
user23773995's user avatar
0 votes
1 answer
132 views

Likelihood ratio test of Poisson distribution

I'm given this problem: Let $X_1,...X_{100}$ be a random sample from a Poisson distribution with mean $\lambda$. Consider testing the hypothesis $H_0$: $\lambda=1$ vs $H_1$: $\lambda<1$. Consider ...
lcthaha's user avatar
0 votes
0 answers
57 views

Conditional likelihood, conditional independence and joint independence

Consider a sequence of data samples generated from $n$ independent random vectors $(X_1, Y_1), (X_2,Y_2), (X_3,Y_3) ...$ $$D = (x_1,y_1), (x_2,y_2), (x_3,y_3) ...$$ Where $(X_i, Y_i)$ - is a random ...
spie227's user avatar
  • 21
1 vote
1 answer
54 views

How to visualize conditional maximum likelihood estimation?

In Probabilistic Machine Learning (Murphy, 2022, p. 8) I'm stuck in this part: 1.2.1.6 Maximum likelihood estimation When fitting probabilistic models, it is common to use the negative log ...
filip augusto's user avatar
0 votes
0 answers
46 views

Maximum Likelihood Estimation and Unbiased Estimator for drawing balls without replacement

For $N\geq 1$ there is an urn with $N$ balls labeled with the numbers $1,...,N$ and we want to estimate $N$ by randomly choosing $n\leq N$ balls without replacement. Determine the corresponding ...
Lu1998's user avatar
  • 27
2 votes
1 answer
79 views

MLE of $\theta$ from $N(\theta+2, \theta^2)$

Let $X_1, X_2, ..., X_n$ be a random sample from $N(\theta+2, \theta^2)$. Find the MLE of $\theta$. I went through some work and I could not solve for $\theta$. Am I doing anything wrong? $$L( \theta )...
Brian Lam's user avatar
0 votes
1 answer
30 views

Argmax of sum of log functions

I am trying to find the below with the context that each $\pi_s$ is a probability. (Trying to assign probabilities so I get maximum likelihood) $$\underset{\bf{\pi}}{\operatorname{argmax}} \ \sum_{s} ...
haklek1's user avatar
0 votes
1 answer
18 views

Prove that MLEs of two independent samples are independent

I am trying to prove that if I have two samples of iid random variables, then MLE's based on these two samples will be also independent. More formally, let $$\mathbf{x} = (x_i)^T_{i = 1} \stackrel{iid}...
Grigori's user avatar
  • 159
0 votes
0 answers
19 views

Pareto MLE with restricted range

The $\text{Pareto}(1, \alpha)$ MLE is $\frac{n}{\sum_{i = 1}^{n} \log x_{i}}$, where $X_{1}, X_{2}, ..., X_{n}$ are i.i.d. with $f(x | \alpha) = \alpha x^{-\alpha - 1}$. Further, add the constraint ...
narwahl's user avatar
  • 13
0 votes
0 answers
23 views

Maximum likelihood estimator in both sides of the equation

I am following Mixed Models theory and applications with r, and there is an MLE that I am struggling to get, and it seems that the parameter appears in both sides of the equation. We have the model $...
David Harar's user avatar
0 votes
0 answers
59 views

Independence of MLEs for subsamples

Suppose I have a sample $\textbf{x} = (x_1, \dots, x_n)$ from stationary and $\beta$-mixing time series. I want to estimate the scalar parameter $\theta$ of the distribution using MLE, i.e., $$ \hat{\...
Grigori's user avatar
  • 159
0 votes
1 answer
33 views

Maximum Likelihood - Information Matrix Identity Derivation

I try to derive the information matrix equality for the Poisson distribution with the log-Likelihood: $$\mathcal{L}(\lambda; x_1, x_2, \ldots, x_n) = \sum_{i=1}^{n} \left[-\lambda + x_i \log(\lambda) -...
Marlon Brando's user avatar
0 votes
0 answers
11 views

Logit panel with Simulated Maximum Likelihood

I'm testing a very basic logit panel model in matlab. The setup is as follows: We observe a binary variable $y_{it} = 1(\beta_0 + \beta_{it}x_{it} + \varepsilon_{it} > 0)$ where i is individual and ...
Mark F's user avatar
  • 33
1 vote
1 answer
59 views

Binary choice with mixture of normal errors estimation

My problem is simplified as follows: I'm estimating a binary choice probit model where the error term follows a mixture of two normal distributions. This is: $y_i = \mathbf{1} \left( \beta_0 + \beta_1 ...
Mark F's user avatar
  • 33
0 votes
0 answers
35 views

Maximizing the expectation in CE Importance sampling.

Suppose the following maximization: $$v_t = \arg \max_{v} E_{v_{t-1}} 1\{S(x) \geq \gamma\} \frac{f(x;u)}{f(x;v_{t-1})}\ln f(x;v) = max_{v} E_{v_{t-1}} 1\{S(x) \geq \gamma\} W(u;v_{t-1}) \ln f(x;v),$$ ...
entropy's user avatar
  • 147
0 votes
1 answer
73 views

Deriving the log-likelihood of a multivariate normal distribution by \mu

I have a sample from multiple groups, $k=\{1,2,..,K\}$. I need to find the maximum likelihood estimates for $\mu^{(k)}$ and for $V$. Looking at here I can see how the entire right part becomes $Nd; N=...
David Harar's user avatar
1 vote
1 answer
109 views

Calculating MSE of MOM and MLE of a Uniform Distribution

Let $X_1, X_2, X_3$ be a random sample of size three from a $uniform(θ, 2θ)$ distribution, where $θ > 0$ I solved to get $\tilde{\theta}_{MoM}$ to be $\frac{2\bar{x}}{3}$. Also, I got $\hat{\theta}...
Maale Faustus's user avatar
1 vote
1 answer
46 views

"How to solve a system of equations of estimators (alpha and beta) using the method of maximum likelihood (beta distribution) in R?"

...
becky rose's user avatar
0 votes
1 answer
123 views

Statistical framework for using MSE for linear classification

We learned in class to use a linear model to to predict a real target value y. We made the assumption that $$ y = w^Tx + w_0 + \epsilon $$ where $x$ is the input vector,$w$ is the vector of the ...
Tomer's user avatar
  • 436
0 votes
0 answers
58 views

How to find the maximum likelihood estimate with sample minimum??

Consider the independent non-identical random variables $X_i \sim Exp(i/\mu)$ , enumerated by $i \in \mathbb{N}$. Here, $\mu > 0$ is an unknown parameter. Instead of the entire sample $x_1, . . . , ...
Sanad Twijiri's user avatar
4 votes
1 answer
37 views

Maximizing log-likelihood: Determinining whether critical points are maximums

Consider the following proof of the fact that $\bar{X}$, the sample mean, is the MLE of parameter $\lambda$ in a Poisson distribution. Let $x_1, \ldots, x_n$ be the observations of $X_1, \ldots, X_n$ ...
lafinur's user avatar
  • 3,468
0 votes
0 answers
44 views

What does singular hessian in optimization tell me

I am doing optimization using maximum likelihood estimation, and when I am trying to get the standard errors of estimates using hessian matrix, I get non-invertible/singular hessian warning. After I ...
jasmine's user avatar
  • 123
0 votes
0 answers
21 views

Deriving MLE & asymptotic variance manually - for which distributions/cases is it possible?

I am interested whether you know any distributions or special cases where the maximum likelihood estimator, the theoretical and estimated asymptotic variance can be fully derived manually, i.e. which ...
Marlon Brando's user avatar

1
2 3 4 5
30