Questions tagged [maximum-likelihood]

For questions that use the method of maximum likelihood for estimating the parameters of a statistical model with given data.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
20 views

EM Algorithm for Missing Data in Multivariate Data [closed]

I am currently trying to understand the EM algorithm and solve the following problem: I have a dataset with 10 different variables in which there can be missing values. The problem is that I think the ...
user avatar
  • 1
3 votes
1 answer
74 views

Show that $\log |\Sigma|+\text{Tr}(\Sigma^{-1}V)$ is uniquely minimized

Let $V$ be an $n\times n$ fixed positive definite matrix and let $f$ be the function defined by $$f(\Sigma)=\log |\Sigma|+\text{Tr}(\Sigma^{-1}V)$$ over the set $P^+$ of $n\times n$ positive definite ...
user avatar
  • 3,654
1 vote
0 answers
17 views

MLE of Geometric variable derived from exponential variable

A call center waiting time is independently distributed $\ \sim \exp( \theta ) $, and after some $\ a $ minutes of waiting, the call gets disconnected. After the call gets disconnected, the client ...
user avatar
  • 1,417
1 vote
0 answers
28 views

Existence of maximum likelihood estimator in factor analysis

Let $Y_1,\dots,Y_n$ be i.i.d. $N(0,\Sigma$) random variables, where $\Sigma=FF'+ D$ with $F$ $m\times k$ and $D$ is diagonal positive definite. Let $V=\frac{1}{n}\sum_{i=1}^n Y_iY_i^\top $ and define ...
user avatar
  • 3,654
0 votes
0 answers
15 views

How do you show the "true parameter" is a critical value of the log likelihood function as n tends to infinity?

These are the first steps in a simple sketch of the proof that, under reasonable conditions, the max likelihood estimate is consistent. I follow the steps all the way down to entering the true ...
user avatar
1 vote
0 answers
53 views

Rigorous definition of joint PDF and likelihood function?

Let $x=(x_1,x_2,...x_n)$ be a vector of arbitrary random variables. The parameter $\theta$ in the likelihood function induces a measure $\mu_{\theta}$ on the underlying measure space. Let $\mu_{\theta}...
user avatar
0 votes
2 answers
38 views

Differential of a matrix term wrt a matrix

Say we have a term as $P = XCX^t-XB-B^tX^t+C$ where all three matrices $X, C, B$ are square $n\times n$ matrices. Matrix $C$ is symmetric and $X,B$ are asymmetric with $X$ having zero as diagonal ...
user avatar
  • 251
2 votes
1 answer
65 views

MLE of cdf, consistency and asymptotic confidence interval

Let $\{X_i\}_{i=1}^{n}$ be i.i.d. random variables with distribution $N(\mu, 1)$. Let $p = \mathbb{P}[X_i > 0]$. Find the MLE for $p$ and compute the 95% asymptotic confidence interval. My attempt: ...
user avatar
  • 590
0 votes
0 answers
52 views

Mixture of Markov Models question

This is a follow up from this question. Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in ...
user avatar
  • 29
1 vote
0 answers
59 views

How to get Fisher information matrix from Likelihood?

Since det $R(k) = (1 + \sum_i S/N_i) det N(k)$, only the ex- ponential part of the density function will depend on the delays. Let the signal delay vector D be defined as The likelihood function for $...
user avatar
  • 401
0 votes
1 answer
21 views

Sufficient statistic and the maximum likelihood estimator of the probability of having an infectious disease when people are grouped and tested

Suppose N students arriving at a college are all equally likely to have a particular disease with an unknown probability p. The disease status (affected / not affected) of all students are independent....
user avatar
  • 1,569
0 votes
0 answers
19 views

Constrained MLE in multinormal gaussian [closed]

Suppose I have a two dimensional Gaussian and I want to solve for the MLE of covariance matrix under the constrain that the covariance matrix is diagonal. Is there a way to formulate this constrain in ...
user avatar
0 votes
0 answers
16 views

Integrating multivatiate gaussian for likelihood

I want to calulate the likelihood by evaluating the second integral , I am supposed to get the third step but I don't seem to get it , how do I evaluate the integra l
user avatar
  • 11
0 votes
0 answers
16 views

Likelihood in MAP and MLE for linear regression

In MAP estimation for linear regression task, the posterior of the weight given the data is written as $p(w|X,Y)=\frac{p(Y|X,w)p(w)}{p(Y|X)}$, why the likelihood is not $p(X,Y|w)$? From my ...
user avatar
0 votes
1 answer
38 views

Most likely estimate of maximum of discrete uniform distribution

For my homework I have to answer the following question: A company has manufactured certain objects and has printed a serial number on each manufactured object. The serial numbers start at 1 and end ...
user avatar
2 votes
0 answers
33 views

Solving for Fokker-Planck coeficients given two observations.

Imagine that I have a stochastic process that I hypothesize evolves according to the 1-Dimensional Fokker-Planck equation: Let $x$ denote the physical property. Let $t$ denote time, $t_0$ denote a ...
user avatar
  • 129
0 votes
0 answers
20 views

MLE is undefined for "densityless" distribution like Cantor distribution

I thought of a situation where we are given a random variable $X$ that has a Cantor "rescaled" distribution. That means that for a parameter $p>0$, $X$ has CDF $F_X(x)=C(\frac xp)$. This ...
user avatar
  • 1,072
0 votes
0 answers
36 views

What is the prior of the Lp Space?

Pretty much the title. Not much to add here I am aware that L1-norm is equivalent to Laplacian prior and L2-norm is equivalent to Gaussian prior. The latter was in my stats textbook and it even had a ...
user avatar
1 vote
1 answer
33 views

Maximum Likelihood Estimator for a Random Sample from Bernoulli distribution

Given a random sample $X_1, X_2,..., X_n$ from Bernoulli distribution. The log-likelihood function is: $\mathcal{L}(\theta) = \sum_1^n x_i^*\log{\theta} + (n - \sum_1^n x_i^*)\log{(1-\theta)}$ Score ...
user avatar
0 votes
0 answers
27 views

Condition for existence of mle with a cauchy distribution

I'm solving some problem from statistical theory about maximum likelihood estimater. Suppose $X_1,...,X_n$ are independent samples from $f(x;\alpha,\beta)$. ($n$ is an odd number.) $$ f(x;\alpha,\beta)...
user avatar
  • 437
0 votes
0 answers
15 views

Conceptual problem in consecutive poisson processes

Imagine we observe a duck for a few hours in the evening. This special duck quacks at random with a certain rate $r_1$, but at some point it changes its mind and starts quacking at a lower rate $r_2$, ...
user avatar
0 votes
1 answer
27 views

Find mle of theta from some mixed density

I'm trying to find a mle from: $$ P_\theta(x) = (1 + \theta) I(0 \leq x \leq 1/2) + (1 - \theta) I(1/2 < x \leq 1)$$ Then, \begin{align*} L(\theta) &= \Pi_{i=1}^n P_\theta(x_i) = (1+\theta)^k (...
user avatar
  • 437
0 votes
1 answer
48 views

Can I derive the first moment when I am know the second and forth?

So I know what $E[X^2]$ and $E[X^4]$ are and I need the first moment. Would it be a right approach for me to use the equality below $$E[X^4] - E[X^2] = (E[X^2] - E[X^1])\times(E[X^2] + E[X^1])$$ If ...
user avatar
  • 45
1 vote
0 answers
34 views

Gradient vector for maximum likelihood estimation of a dynamic probit model

I am using the following specification to estimate a binary choice Probit model: $$ P(y_t=1|x_t) = \Phi(\pi_t), $$ where $\Phi$ is the cumulative distribution function of the normal distribution. My ...
user avatar
1 vote
0 answers
43 views

Neyman-Pearson lemma for the following most powerful test

Let $X_1 ... X_n$ denote independent observations and suppose that $X_i$ has a $N(\mu_i, 1)$ distribution, $i = 1, ..., n$. Show that, according to the Neyman-Perason Lemma, the most powerful test (...
user avatar
4 votes
1 answer
93 views

Estimate Poisson parameter

1. Background: Given a parameter $k\in\mathbb{R}^+$. I have a bunch of positive data pairs $\{(a_i,b_i)|a_i\in\mathbb{R}^+, b_i\in\mathbb{R}^+\}_{i=1}^N$. For each pair $(a_i,b_i)$, an observation ...
user avatar
  • 447
1 vote
0 answers
65 views

Mathematical Statistics - Jun Shao, Theorem 5.4

I am studying Jun Shao's Mathematical Statistics and got a bit stuck in the proof of Theorem 5.4, which states: Let $u$ be a Borel function on $\mathbb{R}^d$ satisfying $\int u(x)dF = 0$ and $\hat{F}$...
user avatar
  • 547
1 vote
1 answer
21 views

Is it a problem that we have to *choose* reference measure in Likelihood function?

Suppose we have some parametric model $\{P_\theta \ \colon \theta \in \Theta\}$ and a sample $X$. If we suppose as usual in classical statistics that $P_\theta << \lambda$ for all $\theta \in \...
user avatar
  • 823
0 votes
0 answers
55 views

Difference in Joint Probability vs. Likelihood?

Conceptually, I am trying to understand the difference between Probabilities and Likelihood. For instance, suppose I am flipping a 2 sided coin with equal probabilities - this corresponds to a ...
user avatar
0 votes
0 answers
27 views

Deriving estimators for functions of an unknown parameter $\theta$

I am wondering if there is an easier way to find an estimator of a function of an unknown parameter $\theta$ for a particular statistical model. For example, if I derive the Maximum Likelihood ...
user avatar
  • 1,175
2 votes
0 answers
33 views

Why does the unbiased statistic in this example be MVUE immediately?

I am reading "Introduction to Mathematical Statistics" to familiarize myself with sufficient statistics. I got stuck by an example in Sec. 7.3 of the book. Below are page 427 and 428 that ...
user avatar
  • 21
0 votes
1 answer
23 views

Maximum likelihood estimator - Partial derivative

I've trying to get the maximum likelihood estimator of $\theta_{MLE}$ but after doing the final derivate step. I've got -1. Am I doing the partial derivative wrongly? What is the MLE of $\theta$? $$ \...
user avatar
  • 105
0 votes
0 answers
21 views

Find $\mathbb{E}[\hat{\alpha}]$. Suggest a new estimate $\check{\alpha}$ of $\alpha$ which is unbiased, such that $\mathbb{E}[\check{\alpha}]=a$

I'm currently working on the following question: $$Define \: m=\sum_{i=1}^n log(\frac{\theta+X_i}{\theta}) \sim Pareto(n,\theta)\: and \: i.i.d$$ $$Let \: \hat{\alpha}=\frac{n}{\sum_{i=1}^n log(\frac{\...
user avatar
  • 3
3 votes
1 answer
66 views

Convergence of sum of MLE's over all possible $0/1$ sequences

Fix $N\in\mathbb N$ and take $\Theta=\{\frac1{N+1},\ldots,\frac N{N+1}\}$. For $y^T\in\{0,1\}^T$, let $\hat\theta(y^T)$ be the number in $\Theta$ that is closest to $\frac1T\sum_{i=1}^Ty^T_i$. Let $\...
user avatar
0 votes
0 answers
15 views

Derivation of the maximum likelihood function: SEM

I was wondering if anyone knows the derivation of the maximum likelihood estimator of SEM: $$ F_{\mathrm{ML}}=\log |\hat{\boldsymbol{\Sigma}}|+\operatorname{tr}\left[\mathbf{S} \hat{\boldsymbol{\Sigma}...
user avatar
  • 23
2 votes
0 answers
51 views

Find asymptotic variance of moment estimator

I have that $$f(x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}x^2}$$ I have the conditional distribution: $$f_{\beta}(y|x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}(y-\beta_0-\beta_1x-\beta_2x^2)^2}$$ and we ...
user avatar
  • 655
1 vote
1 answer
38 views

Likelihood of censored exponential random variables

Consider $X_1, \dots, X_n \stackrel{\text{iid}}{\sim} \text{Exp}(\lambda)$ and define \begin{equation*} Y_i = \begin{cases} X_i & X_i \leq c \\ c & X_i > c \end{cases} \end{equation*} for ...
user avatar
0 votes
0 answers
26 views

Find fisher information matrix for minimum estimator

I have that $$f(x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}x^2}$$ I have the conditional distribution: $f_{\beta}(y|x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}(y-\beta_0-\beta_1x-\beta_2x^2)^2}$ and we have ...
user avatar
  • 655
0 votes
0 answers
43 views

Sampling distribution of GBM Maximum-Likelihood estimator

Given the geometric Brownian diffusion $$ X_t = \mu X_t dt + \sigma X_t d W_t$$ I learnt that its maximum likelihood estimators are the following as this web article suggests $$\hat \mu = \frac{\delta ...
user avatar
  • 296
1 vote
1 answer
53 views

Maximum likelihood for two dependent variables

Suppose you have a box containing 10 balls, of which $\theta$ are white and the rest are green. Suppose we take two balls for without replacement and let $X_i = 1$ if the i-th drawn ball is white and $...
user avatar
0 votes
0 answers
19 views

Why do GLS and ML estimators coincide for the estimation of a VAR(p) model?

When estimating the coefficients in a VAR(p) model (assuming normality), the coefficient estimators using GLS and MLE coincide. Could anyone explain why this is the case?
user avatar
0 votes
0 answers
11 views

Give an estimator of the average of seismic activity for hour

I have this problem and an attempt, but I'm not sure that is correct, and I don't know how to answer the last question. I hope you can help me, please. The Seismology institute has randomly observed ...
user avatar
0 votes
2 answers
41 views

MLE for uniform distribution

Suppose we have a sample of $X_i = x_i$ for $i \in [1,n]$ where $\forall X_i$ they are iid with uniform distribution on the interval $[a,b]$ Now we want to find the MLE for the uniform distrubtion. We ...
user avatar
  • 1,462
4 votes
2 answers
137 views

EM Algorithm vs Gradient Descent

I was reading about the EM algorithm (https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm) - this algorithm is used for optimizing functions (e.g. the Likelihood Functions ...
user avatar
  • 1,904
0 votes
0 answers
11 views

MLE & MLP Comparison

I am studying the allele frequency in population genetics and I have some questions: When using a conjugate prior and comparing the MLE and MLP results for allele frequency, how does the mean squared ...
user avatar
1 vote
0 answers
26 views

Differentiation of logLikelihood MLE

Given a log likelihood function, $$l=\sum_{i=1}^m\log[\frac{n!}{X_{i,1}!X_{i,2}!(n-X_{i,1}-X_{i,2})!}p_1^{X_{i,1}}p_2^{X_{i,2}}(1-p_1-p_2)^{n-X_{i,1}-X_{i,2}}]$$ To derive: with respect to $p_1$, $\...
user avatar
  • 21
1 vote
0 answers
34 views

(Log) Likelihood as a Loss Function?

I'm trying to understand the relation between the theory of statistical decision problems and the theory of regression of distributions. Recall that a statistical decision problem consists of a ...
user avatar
1 vote
1 answer
48 views

Derivation of the Conditional Maximum Entropy distribution

I am trying to derive the conditional maximum entropy distribution in the discrete case, subject to marginal and conditional empirical moments. We assume that we have access to the empirical moments, $...
user avatar
  • 611
1 vote
0 answers
44 views

Renewal process with inter-arrival time distributed as gamma: Model estimation

Let's start with the Poisson process: If $N_t$ is a Poisson process with parameter $\lambda$, then we know that the inter-arrival time distribution is an exponential distribution with parameter $\...
user avatar
0 votes
0 answers
42 views

Finding the Mean Squared Error of Estimator

The positive random variables $X_{1}, X_{2},...X_{n}$ are independent and identically distributed as $Ge(\theta)$. The maximum likelihood estimator of $\psi = \frac{(1 - \theta)}{\theta}$ is the ...
user avatar

1
2 3 4 5
27