Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [maximum-likelihood]

For questions that use the method of maximum likelihood for estimating the parameters of a statistical model with given data.

0
votes
0answers
8 views

Inference Statistic - Likelihood Function

Can anyone help me understand this? Consider the four observations from de Normal Distribution with variance equal to one $y_1 < 10$$, y_2 > 10 $, $5 < y_3 < 10 $ and $ y_4 = 10$. ...
0
votes
0answers
21 views

Deriving the Maximum Likelihood Estimation (MLE) of a parameter for an Inverse Gaussian Distribution

Given the following likelihood function $$f(y|x,\tau) = \prod_{i=0}^Nf_T(u_i-x_i-\tau) \tag{1}$$ where, $f_T(t)$ is the probability density function of an Inverse Gaussian distribution given by ...
1
vote
1answer
23 views

Dealing with MLE with multiple distribution

I am looking for advice to make sure I am understanding an example. If I am not , I am looking for advice on where I can fix things or how to think about it in a different way. Suppose we have a ...
1
vote
0answers
28 views

MLE of a certain probablity

Let $X_1,X_2, \dots ,X_n$ be a random sample from the pdf $$f(x,\gamma)=\frac{2\gamma+1}{2\gamma}x^{\frac{1}{2\gamma}},\quad 0\leq x \leq 1,\gamma>0,$$ what are the steps to find the Maximum ...
1
vote
1answer
45 views

MLE and method of moments estimator (example)

Let $X_1,X_2,\dots ,X_n$ be a random sample from the Gamma distribution $$ f(x,\theta)=\theta^2 x e^{-\theta x},\quad x>0$$ To find the Maximum Likelihood Estimator, we define the likelihood ...
0
votes
2answers
19 views

Maximum Likelihood Estimation of P(x|theta) = 1/(1-theta)

I want to calculate the maximum likelihood estimation of $P(x|\theta) = 1/(1-\theta)$ for $\theta<= x <=1$. I end up with $log1 - nlog(1-\theta)$ and when I want to take the derivitie I end up ...
3
votes
1answer
67 views

MLE of $\mu$ given $X_1\sim \mathsf N(\mu, 4)$ and $X_2\sim \mathsf{N}(\mu, 16)$

Let $X_1\sim \mathsf N(\mu, 4)$ and $X_2\sim \mathsf{N}(\mu, 16)$ where $X_1$ and $X_2$ are independent. Find the maximum likelihood estimator $\hat{\mu}$ of $\mu$ if it exists. We have $$\...
1
vote
1answer
20 views

MAP Solution for Linear Regression - What is a Gaussian prior?

I am looking at some slides that compute the MLE and MAP solution for a Linear Regression problem. It states that the problem can be defined as such: We can compute the MLE of w as such: Now they ...
-1
votes
0answers
29 views

Properties of MLE on a restricted parameter space

So, I bumped into this question of my Statistics Course: Consider a random sample $X_1, X_2, ..., X_n$ from a one dimensional density $f_\theta$ with $\theta > 0$, or $\Theta = (0, \infty)$. The ...
1
vote
0answers
27 views

Likelihood function with inequality

Suppose $Y_1, \dots, Y_n$ are i.i.d. bernoulli random variables. Also, $Y=\sum Y_i \sim binom(n, \theta)$ and we have a prior beta distribution $\theta\sim beta(a,b)$. I want to compute $P(\theta>0....
0
votes
0answers
10 views

Can someone recommend a book that covers (and proofs) asymptotic properties of Maximum Likelihood Estimation?

I am taking an econometrics course and we are currently discussing Non-linear least squares (NLS). I have trouble understanding the asymptotic properties of Maximum likelihood estimation (MLE). I ...
0
votes
0answers
58 views

Finding MLE when there are different distributions for different values of parameter

Suppose a random sample of size $n$ is drawn from a population having different distributions, one for different values of the parameter $\theta$. Let's say I consider the case when there are two ...
0
votes
0answers
7 views

Constraint on bayesian likelihood expression

I'm doing MCMC simulation but I'm confused in some part of my model. I dont know which of my likelihood expression is right. My model is as following. $\gamma$ = $(\gamma_{1},\cdots,\gamma_{K})$ and ...
2
votes
0answers
29 views

As n approaches infinity, the likelihood function at true parameter values is maximum among any other feasible choice of the parameters

Assume that a density $f(x;\theta)$ is distinct for different values of parameter $\theta$ and has common support for all $\theta$. To show that for $\theta \neq \theta_0$ (=true value that generates ...
0
votes
0answers
24 views

Detection of Gaussian Signal in Gaussian Noise

I am constructing a hypothesis test for detection of zero-mean Gaussian signals in zero-mean Gaussian noise under $L$ number of observations, where the noise variance is known but the signal variance ...
1
vote
1answer
42 views

Simplify exponential expression

Consider an exponential model with density $\theta e^{-\theta x}$ with $x> 0$ and $\theta >0 $. Derive LR test of approximate level $\alpha$ (For large sample size) for the hypothesis problem $...
0
votes
0answers
27 views

Likelihood ratio test - exercise for two independent means

Let $X_1$,...,$X_n$, respectively $Y_1$,...,$Y_n$, be two independent data groups of iid data following a normal distribution with paramater $\mu_1,\sigma^2$, respectively $\mu_2,\sigma^2$. Derive the ...
4
votes
0answers
35 views

How can I write my optimization problem as a SDP problem?

I have the following maximum likelihood optimization problem: $$ \hat{X}=\underset{X}{\text{argmax}} \left ( T_{1}X^{T}GX+T_{2}X^{T}Y \right ) \\ \text{s.t.} \left\{\begin{matrix} X\in \{\pm 1\}^{n}...
0
votes
0answers
6 views

Asymptotically biased maximum likelihood estimator

I have a model such that a single experiment $\boldsymbol{y}$ driven by an unknown parameter vector $\boldsymbol{\theta} \in \mathbb{R}^4$ consists of $N$ binary observations $\boldsymbol{y} = (y_1, ...
0
votes
0answers
75 views

Solving simultaneous quadratic equations (“almost” linear…), arising from maximum likelihood estimation problem

I have been staring at this for a while and getting nowhere, and Mathematica is also struggling to produce something useful, so I am beginning to suspect that I am asking the impossible, but just in ...
0
votes
0answers
61 views

Minimax bounds on estimation of Gaussian mean

Suppose $X_1, \dots, X_n \sim^{\text{iid}} \mathcal{N}(\theta, \sigma^2 I_p)$ and $\sigma^2$ is known. Define the risk as follows $$ \mathcal{R}(\theta_1, \theta_2) = \mathbb{E} \|\theta_1 - \theta_2\|...
0
votes
0answers
70 views

How to find the MLE of the parameters of an inverse Gaussian distribution?

The pdf is $(\frac{\lambda}{2\pi x^3})^\frac{1}{2}$exp$(\frac{-\lambda (x-\mu)^2}{2 \mu^2 x})$ I believe $\hat\mu$=$\bar X$ but I can't seem to find $\hat \lambda$
2
votes
1answer
42 views

Maximum Likelihood Estimate for 2 Coins Combination (Bernoulli Trials)

Given: 2 coins: $C_1$ and $C_2$ $p$: probability of choosing $C_1$ to flip. $p_1$: probability of heads landing on $C_1$. $p_2$: probability of heads landing on $C_2$. All trials are Bernoulli. Which ...
1
vote
1answer
33 views

Maximum Likelihood Estimation Question 3

I have given the following density function: $$ f_c(x) = \begin{cases} \frac{2c^2}{x^3} & \;\mbox{for}\; x\geq {c}\\ 0 &\;\mbox{else}\end{cases} $$ with $c > 0$ I need now to find a ...
-2
votes
1answer
55 views

How to calculate the generator matrix,parity check matrix and the maximum likelihood decoding

An unknown encoding device for a binary linear block code has 4 bits of input-data pins and 8 bits of output data pins. If we send the messages $$u_1=(1 1 0 0),\, u_2=(1010),\, u_3=(1001),\, u_4=(...
0
votes
1answer
30 views

Estimate how many tracks the city has.(Almost done, but can not find whether estimator is biased or not)

I try to estimate the number of tracks in the city by observing their serial numbers. Assume that the serial numbers are drawn from a uniform probability density ranging from 0 to an unknown parameter ...
1
vote
2answers
25 views

ML estimation with given samples

Let $X_i,...,X_n$ be a random independent sample from a distribution with pdf $$ f(x;\theta)= (\theta + 1)x^{-(\theta+2)},$$ where $x>0$, and $\theta > 0$. What is the ML estimate for the ...
0
votes
1answer
25 views

derive asymptotic distribution of the ML estimator

Let $x$ be a random variable with probability density (pdf) $$f(x)= (\theta +1)x^\theta $$ where $\theta >-1$. The expressions for its mean and variance are $$E(X)= \frac{\theta + 1}{\theta +...
1
vote
2answers
41 views

MLE of continuous uniform distribution

A series of $n$ geomagnetic readings are taken from a meter, but the readings are judged to be approximate and unreliable. The chief scientist does know however that the true values are all positive ...
0
votes
1answer
17 views

asymptotic distribution for MLE - Borel distribution

Suppose we have sample $y_1,y_2,...y_n$ from a borel distribution $$P(Y=y;\alpha)= \dfrac{1}{y!}(\alpha y)^{y-1}e^{-\alpha y} , y=1,2..$$ The MLE of $\alpha$ is $\hat{\alpha} = 1-\dfrac{1}{\bar{y}} ...
0
votes
1answer
44 views

Efficient and stable estimation of restricted weighted multivariate regression model

First, I will show what I currently do in a (non-redistricted) weighted multivariate regression model. Then I will pose my question as I need the notation I introduce. Suppose we have weighted ...
0
votes
0answers
16 views

Understanding the likelihood of an inhomogenous poisson spatial point process

This was recently posted on cross validated... I've since deleted that question and reposted here as there seems to be more of a focus on proofs on this site, as opposed to crossvalidated which ...
1
vote
0answers
44 views

Matrix differentiation in mle

I want to find the MLE for $\beta,\rho,\sigma^2$ and $v^2$, where $\beta$ is a vector. The log-likelihood is given by $$ \ell \propto -\frac{1}{2}\ln\left(\left|\sigma^2exp\left(\frac{-\textbf{D}...
0
votes
1answer
48 views

Getting different parameters of distribution when using different methods

I have $n$ observations of variable $X$ and now I want to estimate parameters $a$ and $\lambda$ of gamma distribution ($a$ is more known as $\alpha$ and $\lambda$ as $\beta$ but that is how I was ...
1
vote
1answer
100 views

Prove Neg. Log Likelihood for Gaussian distribution is convex in mean and variance.

I am looking to compute maximum likelihood estimators for $\mu$ and $\sigma^2$, given n i.i.d random variables drawn from a Gaussian distribution. I believe I know how to write the expressions for ...
0
votes
0answers
19 views

Calibrating the likelihood ratio test

Background Let $x_1,....x_n$ be real-valued random variables that are distributed according to either measure $P_0$ or $P_1$, and let these measures have densities $f_0$ and $f_1$ respectively. The ...
0
votes
1answer
26 views

Find the maximum likelihood estimator for $\theta \geq 0$ and unknown $\lambda$.

Given the random samples $\{X_1,...,X_n\}$ with p.d.f.: \begin{align} f(x_i) = \left\{ \begin{array}{cc} \lambda e^{-\lambda(x_i-\theta)}, & \hspace{5mm} x_i \geq\theta \\ ...
0
votes
0answers
17 views

A function that is a product of densities and probabilities, can it be considered as a likelihood function?

I have defined a function that is the product of densities of measured/tested values that follow a normal distribution BY a probability (CDF of a normal distribution). $$L(\theta |x_1,x_2,x_3,...,x_n)...
0
votes
0answers
35 views

Maximum likelihood estimate of standard deviation, given uncertain, correlated data

There's a stochastic variable $X$ which I assume is normally distributed, and I'd like to know the parameters of that normal distribution. (Mean $\mu_X$ and variance $\sigma_X^2$.) I can't actually ...
0
votes
0answers
27 views

Bayes vs. MLE (used in machine learning)

I am having a hard time understanding the connection/difference between MLE and the Bayes rule. MLE, as I understand it, is a way to find the most likely value of a parameter. Now, the Bayes rule is ...
4
votes
3answers
71 views

Integrate the likelihood function

We know that for a density function $f(x\mid\theta)$ we have $$\int^{\infty}_{-\infty}f(x\mid\theta)\, dx=1$$ Do we also have for the likelihood function $L(\theta\mid x)$ that $$\int^{\infty}_{-\...
1
vote
0answers
22 views

Is my understanding correct? (Goodness of Fit question)

The Question: Suppose $X_1,\dots,X_{n}$ are independent and identically distributed random variables that take values in $\{0,1,2,\dots \}$. We gather the following data: \begin{array} \\ \text{...
1
vote
0answers
27 views

Understanding terminology of Mixtures of Gaussians model

I'm following this lecture notes to understand mixture of Gaussian model and EM (Expectation Maximization) algorithm to fit it. I understand the complete intuition behind this algorithm, which is ...
1
vote
1answer
106 views

Maximum Likelihood Estimator (MLE) of $ \theta $ for the PDF $ f( x; \theta) = \frac{1}{2}(1+\theta x)$

I need to find de maximum likelihood estimator of $\theta$ for $f(x)=\frac{1}{2}(1+\theta x)$, $-1 \leq x \leq 1$ I start with: $L(\theta)=f(x_1,\theta)f(x_2,\theta)\cdots f(x_n,\theta)$ $$L(\theta)=...
0
votes
0answers
27 views

Likelihood function uder Brownian motion

Say we have a set of observations $\{X_i : i=0,...,N\}$ from a simple Brownian motion (scaled Winner process) $$dX_t = \sigma dW_t .$$ If we discretise this to $$X_i-X_{i-1} = \sigma\sqrt{\Delta}Z_i$...
0
votes
1answer
33 views

Confusing summation of $i$ not $j$, but $j$ is not defined

I am trying to write out the following log-likelihood: $$\mathcal L(\vec{x}, \vec{y}) = \sum_{i} \left[ k_{i}^{out} (\boldsymbol{A}^*) \ln x_i + k_i^{in} (\boldsymbol{A}^*) \ln y_i\right] - \sum_{i \...
0
votes
0answers
17 views

Notation for Maximum Likelihood

I am following Ian Goodfellow's lecture on GANs. In ~13:34 there exist a notation for maximum likelihood I do not understand. Ian states that they are trying to maximize the $\log$ of the ...
3
votes
3answers
228 views

Maximum Likelihood Estimator for Poisson Distribution

Studying for my upcoming exams I came upon this weird MLE exercise : Let the random variable $X$ follow the Poisson Distribution with unknown parameter $\theta >0$. In $50$ observations of $X$, ...
1
vote
0answers
62 views

Empirical distribution and Kullback–Leibler divergence

In the article Interpreting Kullback-Leibler divergence with the Neyman-Pearson lemma is written: Given a random sample $x_1, x_2, · · · , x_n$ from the underlying distribution $P$, let $P_n$ be ...
2
votes
0answers
24 views

Constructing a symmetrical $100(1-a)\%$ confidence interval for $\theta$.

Exercise : Let $X_1, \dots, X_n$ be a random sample from the distribution function $F(x) =1 - \frac{\theta^3}{x^3}, \; x \geq \theta$ where $\theta >0$ unknown parameter. (i) Find a maximum ...