Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

0
votes
0answers
15 views

UMVUE of parameter $(1-\sigma^2)^{-\frac{n}{2}}$

suppose $X_1,X_2,\ldots,X_n$ be random sample of $N(0,\sigma^2)$. how can I calculate UMVUE of parameter $(1-\sigma^2)^{-\frac{n}{2}}$
0
votes
0answers
19 views

Decompose summation of signals

Imagine a summation of three distinct signals such as in the following graphic. Is it possible to estimate the original signals? Below is a matlab-code to generate the image: I have found similar ...
1
vote
1answer
33 views

Bias, SE and MSE of Uniform Distribution

Let $X_1,\ldots,X_n$ be an i.i.d. sequence of Uniform $(\mu,2\mu)$ and let an estimator be $\hat{\mu} = \frac{1}{2} \max\{X_1,\ldots,X_n\}$. Find the bias, SE, and MSE of this estimator. Hint: Let ...
0
votes
1answer
18 views

Estimating variance of estimator of bernoulli process

The maximum likelihood estimate of a Bernoulli process is simply given by $\hat{\theta}=\frac{\sum X_i}{N}$, where N is the total number of bernoulli trial and $X_i$ is the outcome of each trial. ...
1
vote
0answers
36 views

minimum kullback leibler estimator

Suppose that one has independent and identically distributed samples $x_i,i=1,...,n$ from some unknown density and one wants to fit a probability distribution $f_\theta(x)$, where $\theta$ is a ...
1
vote
0answers
43 views

Origins of Kalman filter Algorithms in his paper in 1960

Concerning Kalman's original paper published in 1960, "A New Approach to Linear Filtering and Prediction Problems", it seems the majority is to show the orthogonal projection is the optimal estimation ...
4
votes
0answers
45 views

Properties of an MLE based on likelihood constructed from both PDF and CDF

For continuous RV the likelihood function is (typically) given by a product of PDFs, i.e. $$L(\theta; x_1,x_2, ..., x_n) = \prod_{i=1}^n f(x_i\mid \theta) $$ However, in survival analysis with ...
1
vote
0answers
16 views

Estimation of unbiased estimator for population variance in case of poisson distribution

sample variance is normally biased estimator for population variance. but in case of poisson distribution sample variance is unbiased for population variance. how do you prove this?
0
votes
0answers
8 views

Comparison of Cramer Rao bound - deduction and conceptual question

The CRB gives the variance of the estimation error of the estimates and a lower value is preferred. I have computed the cramer rao bound (CRB) of the estimates of the coefficients $\mathbf{h^T}$ for ...
1
vote
0answers
22 views

How to bound $E \left[\left(E[Z^2\mid Y] \right)^2\right]- 2E \left[ |E[Z\mid Y]| \sqrt{E[Z^2\mid Y]} \right]$

I am looking for an upper bound on the following quantity \begin{align*} A=E \left[\left(E[Z^2\mid Y] \right)^2\right]- 2E \left[ |E[Z\mid Y]| \sqrt{E[Z^2\mid Y]} \right] \end{align*} where $Z$ is ...
0
votes
2answers
37 views

Estimation, bias, and mean square error

Let $X$ be a continuous random variable with pdf $f(x) =\frac{1}{2}(1+ \theta x)$, for $-1 < x < 1$, and $-1 < \theta < 1$ (a) Show that $E(X) = \frac{\theta}{3}$. (b) Given a random ...
1
vote
0answers
20 views

Proof of differentialbility in mean square calculus?

let $x_t$ be a mean squared Riemann integrable over $[a, t]$ for every $t\in[a,b]$. Then $y_t=\int\limits_a^t x_\tau d\tau\ $ is mean squared continuous on $[a, b]$. Furthermore, if $x_t $ is mean ...
-2
votes
1answer
50 views

How to determine the MLE of $E(x)$ [closed]

Let $x_1, x_2,...,x_n$ be a random sample from a log normal distribution $$F(x,\mu,\sigma)=\frac{1}{x\sqrt{2\pi\sigma^2}}e^{\left(-\frac{(\ln(x)-\mu)^2}{2\sigma^2}\right)}$$ Find the maximum ...
0
votes
1answer
27 views

Bayes estimator under squared error loss

Consider one random variable X from the Bernoulli distribution with parameter θ. Let p, the prior density, be equal to 6θ(1 − θ), for θ ∈ (0, 1). Under squared error loss, L(t, θ) = (t − θ)$^2$, the ...
2
votes
0answers
19 views

MLE of two-dimensional distribution

Let $X_1, ..., X_n$ be a random sample from a continuous distribution with pdf $$f_{\theta,\kappa}(x) = \frac{\kappa\theta^\kappa}{x^{\kappa+1}}, x\geq \theta, \theta > 0, \kappa > 0.$$ How to ...
1
vote
2answers
55 views

Maximum a Posteriori (MAP) Estimator of Exponential Random Variable with Uniform Prior

What would be the Maximum a Posteriori (MAP) estimator for $ \lambda $ for IID $ \left\{ {x}_{i} \right\}_{i = 1}^{N} $ where $ {x}_{i} \sim \exp \left( \lambda \right), \; \lambda \sim U \left[ ...
2
votes
0answers
27 views

Conditional distribution [closed]

I am trying to figure out the derivation of Kalman filter based on Bayesian estimator. As we know, the assumption of Gauss-Markov model is used, then, the conditional distribution p(x(t)|Y(t-1))can be ...
1
vote
0answers
56 views

How to calculate Fisher Information (FI) matrix for Multivariate Normal Distribution (MN)

Below is the gradient (score) of the MN log likelihood function L for n=1 observation. I originally attempted to calculate the Hessian matrix but ran into difficulty calculating 2nd order derivatives ...
4
votes
2answers
115 views

Estimating the “step size” of a grid

Suppose one is given a set of $M$ points distributed on a "grid", i.e: $$x_i = x_0 + \alpha n_i + \epsilon_i, \quad n_i\in\mathbb{Z}$$ This might like something like this: $\quad\ ...
3
votes
2answers
41 views

Difficult to understand difference between the estimates on E(X) and V(X) and the estimates on variance and std.dev. on lambda-hat

I'm having a very hard time to separate estimates on population values versus estimates on sample values. I'm struggling with this exercise (not homework, self-study for my exam in introductionary ...
1
vote
0answers
17 views

bias reduction when the bias depends on the true parameter

Let's say we estimate a parameter, $\theta$, by $\hat{\theta}$. For this estimator we have the following property that $$\hat{\theta}\to_{p}\theta+f(\theta)$$ where $\to_{p}$ denotes convergence in ...
2
votes
1answer
42 views

How to prove that the maximum likelihood estimator of $\theta$ is aysmptotically unbiased and cosistent

In a class we looked at this example: Let $X_1,...,X_n\sim U(0,\theta)$. Then the maximum likelihood function is $\mathcal{L}(\theta) = \begin{cases} \dfrac{1}{\theta^{n}} & \text{if } ...
0
votes
1answer
61 views

Maximum likelihood estimator and confidence interval

Let $\theta$ be an unknown constant. Let $W_1,…,W_n$ be independent exponential random variables each with parameter $1$. Let $X_i=θ+W_i$. First, I need to find $\hat\theta _{ML}(x_1,\ldots ,x_ n)$. ...
1
vote
0answers
26 views

MLE of variance for a spherical Gaussian

I am trying to implement the X-Means clustering algorithm. In it, the authors use the BIC to determine which model fits the data best. It is explained here: ...
1
vote
1answer
64 views

Sum of variances of multinomial distribution.

I've k fair coins, and I would like to know the number of heads obtained in $n$ trials. But that is simple binomial distribution. But if I want to find out how much it varies from binomial ...
1
vote
1answer
36 views

Dice roll, estimator, epsilon

We roll a non-symmetric die. Let $X_n$ be the reulst of $n$-th roll. $$P(X_n = 6)= \frac{1}{6} + \varepsilon, \ P(X_n = 1) = \frac{1}{6} - \varepsilon, \ P(X_n=2) = ... = P(X_n = 5) = \frac{1}{6} $$ ...
0
votes
0answers
49 views

Most efficient estimator

$X_1,...X_n$ is a random sample of size $n$ from a population with mean $\mu$ and variance $\sigma^2$.There are three estimators for $\mu$:  $\hat\mu _1=\frac{x_1+x_2}{2}$ $\hat\mu ...
0
votes
0answers
32 views

Variance estimation of a diffusion process

The framework of this question is a 1 dimensional diffusion process, defined ny the following equation: $dx_t=adt+bdw_t$ Where $w_t$ is a standard berownian motion and and $a$ is a constant drif ...
0
votes
0answers
5 views

A mix between the Horvitz-Thompson and ordinary estimator

I have asked this question on mathoverflow, but got no answer. Here I have corrected some mistakes and wish to hear any ideas that may bring at least numerical result: The data I have two samples: ...
2
votes
0answers
73 views

Near-Application of Cauchy-Schwarz Inequality

I have the following situation: I have two estimators of $\alpha$, both via maximum likelihood of the density: $$ f(x,y\mid \alpha,\beta) = f(y \mid x,\alpha,\beta)f(x \mid \alpha) $$ One uses only ...
0
votes
2answers
75 views

How to find the bias, variance and MSE of $\hat p$?

If $X_1,\dots,X_n$ are iid $\mathrm{Binomial}(3,p)$, then the maximum likelihood estimator of $p$ is $$\hat p = \frac{1}{n}\sum_i X_i$$ Find the bias, variance and MSE of $\hat p$? We are asked to ...
0
votes
1answer
36 views

linear least square estimation with random sum

Let $N$ be a geometric r.v. with mean $1/p$; let $A1,A2,… $be a sequence of i.i.d. random variables, all independent of $N$, with mean $1$ and variance $1$; let $B1,B2,… $be another sequence of i.i.d. ...
0
votes
0answers
19 views

Second partial derivative of a minimum function

I am reading a book on detection and estimation theory, and the author had this to say in the derivation of the white noise process from the Wiener process: We can formally obtain the covariance ...
0
votes
0answers
7 views

Function of efficient estimator

Say I have an efficient estimator $\theta$, call it $\hat{\theta}$. If I wanted to estimate another quantity, call it $\delta = g(\theta)$, is there any result which allows me to say that ...
2
votes
1answer
142 views

Hypothesis test between two normal distributions

Let $T_1,T_2,\ldots ,T_ n$ be i.i.d. observations, each drawn from a common normal distribution with mean zero. With probability $1/2$ this normal distribution has variance $1$, and with probability ...
0
votes
1answer
74 views

Trajectory estimation

The vertical coordinate (“height") of an object in free fall is described by an equation of the form $x(t) = \theta _0 + \theta _1t + \theta _2 t^2,$ We assume that $\theta_0$ is a known constant. We ...
1
vote
1answer
41 views

Unbiased estimator for the sum of numbers

Let $\alpha_1, \dots, \alpha_n \in \mathbb{R}$. We want to approximate the sum as follows $$ S = \sum_{i=1}^{n} \alpha_i \approx \dfrac{n}{c} \sum_{i=1}^{c} \alpha_i, $$ where $\alpha_i$ is picked ...
1
vote
0answers
36 views

Hammersley–Chapman–Robbins bound for Rice distribution

I am trying to evaluate the Hammersley–Chapman–Robbins bound for the variance of an unbiased estimate $\hat{\alpha}$ of $\alpha$ (for a given $\sigma$) for the Rice distribution: $$p(x|\alpha,\sigma) ...
0
votes
0answers
35 views

Expectation of inverse of a symmetric matrix with gaussian elements

Is there any way to calculate: \begin{equation} \mathbb{E} \; ( H^{T}H )^{-1} \end{equation} assuming that the entries of the matrix $H$ are gaussian random variables with unknown means but same ...
0
votes
0answers
18 views

ML estimator for correlated random vectors

Suppose, we have two random vectors in $\mathbb{R}^2 $ denoted by $(X_1,Y_1)$ and $(X_2,Y_2)$ . The individual random variables can only take value in $\lbrace 0, 1 \rbrace$. We get samples of the ...
2
votes
1answer
71 views

UMVUE for pdf $f_{\theta}(x) = \theta e^{-\theta x}, x>0$

Let $X_1,\ldots,X_n$ be a random sample from a pdf $f_{\theta}(x) = \begin{cases} \theta e^{-\theta x}, & x>0 \\ 0, & \text{otherwise} \end{cases}$, where $\theta>0$ is an unknown ...
1
vote
0answers
20 views

Fisher Expected Information for a Gaussian Process model

Suppose I have a two dimensional Gaussian process model (GP), defined by a squared exponential correlation function s.t: $$R(x_{i},x_{j}) = \exp\left(-\frac{|x_{i} - x_{j}|^2}{2}\right).$$ I am ...
0
votes
1answer
37 views

calculate The maximum likelihood estimator of parameter $\mu$ according to $T$

suppose $X_1,X_2,\ldots,X_n$ be a random sample of $N(\mu,1)$. if $T=\sum_{i=1}^n I_{(X_i<0)}$ how can I calculate The maximum likelihood estimator of parameter $\mu$ according to $T$. ($\Phi$ is ...
0
votes
0answers
13 views

Finding a bayes estimator

Let $X_1,...,X_n|\eta~\exp(1,\eta)$ and $\eta$~$N(\mu,1)$, where $\mu\epsilon\Re$. Find the Bayes estimator $\eta$ under the squared error loss. After finding the joint likelihood of $exp(1,\eta)$ ...
-2
votes
1answer
55 views

calculating $\mathbb E\left(\exp\left(\frac{1}{2}\sum_{i=1}^n X_i^2\right)\right)$ [closed]

suppose $X_1,X_2,\ldots,X_n \sim \mathcal N(0,\sigma^2)$. How can I calculate $$\mathbb E\left(\exp\left(\frac{1}{2}\sum_{i=1}^n X_i^2\right)\right)$$
0
votes
1answer
30 views

calculating UMVUE of parameter $(1-\sigma^2)^-\frac{n}{2}$.

suppose $X_1,X_2,\ldots,X_n$ be random sample of $N(0,\sigma^2)$. how can I calculate UMVUE of parameter $(1-\sigma^2)^-\frac{n}{2}$. I know $T=\sum_{i=1}^n X_i^2$ is Sufficient and complete ...
1
vote
1answer
33 views

How to evaluate the goodness of Fit of parameters obtained from EM algorithm

I have a set of observations $\mathcal{Y} = {Y_1, \ldots, Y_T}$. I am running EM algorithm to fit the observations to the following Hidden Markov Model $$A = [a_{ij}]_{N \times N}, a_{ij} = P(X_{k+1} ...
0
votes
1answer
25 views

Hypothesis Test on Gaussian Mixture

I have data blocks being received at a node which can be presumed to be Gaussian sequence, $X(m_1,sigma_1).$ In some of the blocks a separate Gaussian stream $Y(m_2,sigma_2)$ adds to the original ...
1
vote
1answer
36 views

Compound distribution with unknown distribution of its hyperparameter

Suppose $X\sim \mathcal{N}(0,\sigma)$, and $\sigma$ is another random variable in a sense that we only know that it is some constant random variable with finite support, i-e $\sigma \in [\sigma_\max, ...
0
votes
0answers
15 views

What is the test statistic mentioned in the paper by Dai and Singleton?

I have read the paper with title 'Specification Analysis of Affine Term Structure Models' (2000) by Qiang Dai and Kenneth J. Singleton. On page 22 there's a table which include test statistics, these ...