Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

0
votes
0answers
7 views

Comparison of Cramer Rao bound - deduction and conceptual question

The CRB gives the variance of the estimation error of the estimates and a lower value is preferred. I have computed the cramer rao bound (CRB) of the estimates of the coefficients $\mathbf{h^T}$ for ...
1
vote
0answers
21 views

How to bound $E \left[\left(E[Z^2\mid Y] \right)^2\right]- 2E \left[ |E[Z\mid Y]| \sqrt{E[Z^2\mid Y]} \right]$

I am looking for an upper bound on the following quantity \begin{align*} A=E \left[\left(E[Z^2\mid Y] \right)^2\right]- 2E \left[ |E[Z\mid Y]| \sqrt{E[Z^2\mid Y]} \right] \end{align*} where $Z$ is ...
-1
votes
0answers
21 views

Confidence interval of median of log-normal distribution

Assume 101 families ($n$) were questioned about their spendings in the month of July, and the amount of money they earn. Say $X$ is the difference in monthly income and their expenses in July. This ...
0
votes
2answers
37 views

Estimation, bias, and mean square error

Let $X$ be a continuous random variable with pdf $f(x) =\frac{1}{2}(1+ \theta x)$, for $-1 < x < 1$, and $-1 < \theta < 1$ (a) Show that $E(X) = \frac{\theta}{3}$. (b) Given a random ...
1
vote
0answers
20 views

Proof of differentialbility in mean square calculus?

let $x_t$ be a mean squared Riemann integrable over $[a, t]$ for every $t\in[a,b]$. Then $y_t=\int\limits_a^t x_\tau d\tau\ $ is mean squared continuous on $[a, b]$. Furthermore, if $x_t $ is mean ...
-2
votes
1answer
49 views

How to determine the MLE of $E(x)$ [closed]

Let $x_1, x_2,...,x_n$ be a random sample from a log normal distribution $$F(x,\mu,\sigma)=\frac{1}{x\sqrt{2\pi\sigma^2}}e^{\left(-\frac{(\ln(x)-\mu)^2}{2\sigma^2}\right)}$$ Find the maximum ...
0
votes
1answer
23 views

Bayes estimator under squared error loss

Consider one random variable X from the Bernoulli distribution with parameter θ. Let p, the prior density, be equal to 6θ(1 − θ), for θ ∈ (0, 1). Under squared error loss, L(t, θ) = (t − θ)$^2$, the ...
2
votes
0answers
18 views

MLE of two-dimensional distribution

Let $X_1, ..., X_n$ be a random sample from a continuous distribution with pdf $$f_{\theta,\kappa}(x) = \frac{\kappa\theta^\kappa}{x^{\kappa+1}}, x\geq \theta, \theta > 0, \kappa > 0.$$ How to ...
1
vote
2answers
45 views

Maximum a Posteriori (MAP) Estimator of Exponential Random Variable with Uniform Prior

What would be the Maximum a Posteriori (MAP) estimator for $ \lambda $ for IID $ \left\{ {x}_{i} \right\}_{i = 1}^{N} $ where $ {x}_{i} \sim \exp \left( \lambda \right), \; \lambda \sim U \left[ ...
2
votes
0answers
27 views

Conditional distribution [closed]

I am trying to figure out the derivation of Kalman filter based on Bayesian estimator. As we know, the assumption of Gauss-Markov model is used, then, the conditional distribution p(x(t)|Y(t-1))can be ...
1
vote
0answers
34 views

How to calculate Fisher Information (FI) matrix for Multivariate Normal Distribution (MN)

Below is the gradient (score) of the MN log likelihood function L for n=1 observation. I originally attempted to calculate the Hessian matrix but ran into difficulty calculating 2nd order derivatives ...
4
votes
2answers
115 views

Estimating the “step size” of a grid

Suppose one is given a set of $M$ points distributed on a "grid", i.e: $$x_i = x_0 + \alpha n_i + \epsilon_i, \quad n_i\in\mathbb{Z}$$ This might like something like this: $\quad\ ...
3
votes
2answers
40 views

Difficult to understand difference between the estimates on E(X) and V(X) and the estimates on variance and std.dev. on lambda-hat

I'm having a very hard time to separate estimates on population values versus estimates on sample values. I'm struggling with this exercise (not homework, self-study for my exam in introductionary ...
1
vote
0answers
17 views

bias reduction when the bias depends on the true parameter

Let's say we estimate a parameter, $\theta$, by $\hat{\theta}$. For this estimator we have the following property that $$\hat{\theta}\to_{p}\theta+f(\theta)$$ where $\to_{p}$ denotes convergence in ...
2
votes
1answer
41 views

How to prove that the maximum likelihood estimator of $\theta$ is aysmptotically unbiased and cosistent

In a class we looked at this example: Let $X_1,...,X_n\sim U(0,\theta)$. Then the maximum likelihood function is $\mathcal{L}(\theta) = \begin{cases} \dfrac{1}{\theta^{n}} & \text{if } ...
0
votes
1answer
60 views

Maximum likelihood estimator and confidence interval

Let $\theta$ be an unknown constant. Let $W_1,…,W_n$ be independent exponential random variables each with parameter $1$. Let $X_i=θ+W_i$. First, I need to find $\hat\theta _{ML}(x_1,\ldots ,x_ n)$. ...
1
vote
0answers
26 views

MLE of variance for a spherical Gaussian

I am trying to implement the X-Means clustering algorithm. In it, the authors use the BIC to determine which model fits the data best. It is explained here: ...
1
vote
1answer
56 views

Sum of variances of multinomial distribution.

I've k fair coins, and I would like to know the number of heads obtained in $n$ trials. But that is simple binomial distribution. But if I want to find out how much it varies from binomial ...
1
vote
1answer
34 views

Dice roll, estimator, epsilon

We roll a non-symmetric die. Let $X_n$ be the reulst of $n$-th roll. $$P(X_n = 6)= \frac{1}{6} + \varepsilon, \ P(X_n = 1) = \frac{1}{6} - \varepsilon, \ P(X_n=2) = ... = P(X_n = 5) = \frac{1}{6} $$ ...
0
votes
0answers
40 views

Most efficient estimator

$X_1,...X_n$ is a random sample of size $n$ from a population with mean $\mu$ and variance $\sigma^2$.There are three estimators for $\mu$:  $\hat\mu _1=\frac{x_1+x_2}{2}$ $\hat\mu ...
0
votes
0answers
23 views

Variance estimation of a diffusion process

The framework of this question is a 1 dimensional diffusion process, defined ny the following equation: $dx_t=adt+bdw_t$ Where $w_t$ is a standard berownian motion and and $a$ is a constant drif ...
0
votes
0answers
4 views

A mix between the Horvitz-Thompson and ordinary estimator

I have asked this question on mathoverflow, but got no answer. Here I have corrected some mistakes and wish to hear any ideas that may bring at least numerical result: The data I have two samples: ...
2
votes
0answers
73 views

Near-Application of Cauchy-Schwarz Inequality

I have the following situation: I have two estimators of $\alpha$, both via maximum likelihood of the density: $$ f(x,y\mid \alpha,\beta) = f(y \mid x,\alpha,\beta)f(x \mid \alpha) $$ One uses only ...
0
votes
2answers
70 views

How to find the bias, variance and MSE of $\hat p$?

If $X_1,\dots,X_n$ are iid $\mathrm{Binomial}(3,p)$, then the maximum likelihood estimator of $p$ is $$\hat p = \frac{1}{n}\sum_i X_i$$ Find the bias, variance and MSE of $\hat p$? We are asked to ...
0
votes
1answer
35 views

linear least square estimation with random sum

Let $N$ be a geometric r.v. with mean $1/p$; let $A1,A2,… $be a sequence of i.i.d. random variables, all independent of $N$, with mean $1$ and variance $1$; let $B1,B2,… $be another sequence of i.i.d. ...
0
votes
0answers
19 views

Second partial derivative of a minimum function

I am reading a book on detection and estimation theory, and the author had this to say in the derivation of the white noise process from the Wiener process: We can formally obtain the covariance ...
0
votes
0answers
6 views

Function of efficient estimator

Say I have an efficient estimator $\theta$, call it $\hat{\theta}$. If I wanted to estimate another quantity, call it $\delta = g(\theta)$, is there any result which allows me to say that ...
2
votes
1answer
138 views

Hypothesis test between two normal distributions

Let $T_1,T_2,\ldots ,T_ n$ be i.i.d. observations, each drawn from a common normal distribution with mean zero. With probability $1/2$ this normal distribution has variance $1$, and with probability ...
0
votes
1answer
70 views

Trajectory estimation

The vertical coordinate (“height") of an object in free fall is described by an equation of the form $x(t) = \theta _0 + \theta _1t + \theta _2 t^2,$ We assume that $\theta_0$ is a known constant. We ...
1
vote
1answer
39 views

Unbiased estimator for the sum of numbers

Let $\alpha_1, \dots, \alpha_n \in \mathbb{R}$. We want to approximate the sum as follows $$ S = \sum_{i=1}^{n} \alpha_i \approx \dfrac{n}{c} \sum_{i=1}^{c} \alpha_i, $$ where $\alpha_i$ is picked ...
1
vote
0answers
33 views

Hammersley–Chapman–Robbins bound for Rice distribution

I am trying to evaluate the Hammersley–Chapman–Robbins bound for the variance of an unbiased estimate $\hat{\alpha}$ of $\alpha$ (for a given $\sigma$) for the Rice distribution: $$p(x|\alpha,\sigma) ...
0
votes
0answers
29 views

Expectation of inverse of a symmetric matrix with gaussian elements

Is there any way to calculate: \begin{equation} \mathbb{E} \; ( H^{T}H )^{-1} \end{equation} assuming that the entries of the matrix $H$ are gaussian random variables with unknown means but same ...
0
votes
0answers
18 views

ML estimator for correlated random vectors

Suppose, we have two random vectors in $\mathbb{R}^2 $ denoted by $(X_1,Y_1)$ and $(X_2,Y_2)$ . The individual random variables can only take value in $\lbrace 0, 1 \rbrace$. We get samples of the ...
2
votes
1answer
64 views

UMVUE for pdf $f_{\theta}(x) = \theta e^{-\theta x}, x>0$

Let $X_1,\ldots,X_n$ be a random sample from a pdf $f_{\theta}(x) = \begin{cases} \theta e^{-\theta x}, & x>0 \\ 0, & \text{otherwise} \end{cases}$, where $\theta>0$ is an unknown ...
1
vote
0answers
19 views

Fisher Expected Information for a Gaussian Process model

Suppose I have a two dimensional Gaussian process model (GP), defined by a squared exponential correlation function s.t: $$R(x_{i},x_{j}) = \exp\left(-\frac{|x_{i} - x_{j}|^2}{2}\right).$$ I am ...
0
votes
1answer
34 views

calculate The maximum likelihood estimator of parameter $\mu$ according to $T$

suppose $X_1,X_2,\ldots,X_n$ be a random sample of $N(\mu,1)$. if $T=\sum_{i=1}^n I_{(X_i<0)}$ how can I calculate The maximum likelihood estimator of parameter $\mu$ according to $T$. ($\Phi$ is ...
0
votes
0answers
12 views

Finding a bayes estimator

Let $X_1,...,X_n|\eta~\exp(1,\eta)$ and $\eta$~$N(\mu,1)$, where $\mu\epsilon\Re$. Find the Bayes estimator $\eta$ under the squared error loss. After finding the joint likelihood of $exp(1,\eta)$ ...
-2
votes
1answer
55 views

calculating $\mathbb E\left(\exp\left(\frac{1}{2}\sum_{i=1}^n X_i^2\right)\right)$ [closed]

suppose $X_1,X_2,\ldots,X_n \sim \mathcal N(0,\sigma^2)$. How can I calculate $$\mathbb E\left(\exp\left(\frac{1}{2}\sum_{i=1}^n X_i^2\right)\right)$$
0
votes
1answer
24 views

calculating UMVUE of parameter $(1-\sigma^2)^-\frac{n}{2}$.

suppose $X_1,X_2,\ldots,X_n$ be random sample of $N(0,\sigma^2)$. how can I calculate UMVUE of parameter $(1-\sigma^2)^-\frac{n}{2}$. I know $T=\sum_{i=1}^n X_i^2$ is Sufficient and complete ...
1
vote
1answer
28 views

How to evaluate the goodness of Fit of parameters obtained from EM algorithm

I have a set of observations $\mathcal{Y} = {Y_1, \ldots, Y_T}$. I am running EM algorithm to fit the observations to the following Hidden Markov Model $$A = [a_{ij}]_{N \times N}, a_{ij} = P(X_{k+1} ...
0
votes
1answer
24 views

Hypothesis Test on Gaussian Mixture

I have data blocks being received at a node which can be presumed to be Gaussian sequence, $X(m_1,sigma_1).$ In some of the blocks a separate Gaussian stream $Y(m_2,sigma_2)$ adds to the original ...
1
vote
1answer
34 views

Compound distribution with unknown distribution of its hyperparameter

Suppose $X\sim \mathcal{N}(0,\sigma)$, and $\sigma$ is another random variable in a sense that we only know that it is some constant random variable with finite support, i-e $\sigma \in [\sigma_\max, ...
0
votes
0answers
15 views

What is the test statistic mentioned in the paper by Dai and Singleton?

I have read the paper with title 'Specification Analysis of Affine Term Structure Models' (2000) by Qiang Dai and Kenneth J. Singleton. On page 22 there's a table which include test statistics, these ...
0
votes
0answers
37 views

probability distribution estimation from correlated samples

I am looking to solve the following estimation problem. Consider a blackbox where (given below) given an input X, its N observations are recorded as output. These observations are denoted by $Y_1, ...
1
vote
0answers
15 views

Rao-Blackwell improvement for a nonrandomized estimator

Context: please consider a parametric statistical model $(\mathcal{Y},\{P_\theta:\theta\in\Theta\})$ and suppose that we are estimating $g(\theta)$. Associated with this is the set of decisions ...
0
votes
1answer
95 views

Estimator for second moment for Poisson random variable

Let $X \sim Poiss(\lambda)$. As, $\displaystyle \sum_{i=1}^{N} X_i $ is sufficient statistic for both mean (and variance) of $Y$, so we can define the unbiased estimate for mean as , $ s=\frac{1}{N} ...
0
votes
0answers
65 views

Determining the actual number of observations in a dataset

I have two datasets one is a dataset with doctors in which I have the procedures they have performed at a given hospital where the actual number of procedures is not captured by this data since it is ...
0
votes
0answers
23 views

What is the nonlinear estimator for Gaussian Random variable?

I know that the best estimator is $g(x)=E\{Y|X=x\}$ and the conditional density for jointly Gaussian random variables is known to be Gaussian with mean and variance given by ...
2
votes
0answers
24 views

ML estimation for Weibull

What are the maximum likelihood estimators of $\eta$ and $\beta$ ($\eta>0$ and $\beta>0$) for an i.i.d. sample of size $n$ from the following density: $f(y_i)=\frac{\beta x_i^{\beta-1} }{\eta ^ ...
0
votes
1answer
21 views

Finding an maximum likelihood estimator (Bernouilli problem)

Could someone point me in the right direction? Suppose we compare 2 treatments. For each patient we observe $(Y_i,R_i)$ where $Y_i$ denotes if the treatment was succesfull ($Y_i=1$) or not ...