Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

1
vote
1answer
33 views

How do you show that the estimator for the covariance matrix is unbiased?

So according to Wikipedia (Here) the sample covariance matrix is an unbiased estimator of the covariance matrix, but how do I prove this mathematically?
0
votes
0answers
10 views

Estimate function given PDF and covariance

Let's say $h(x)$, random variable, represents the height of a surface, with x being the usual x-axis. The probability distribution function is: $P(h) = Ke^{-\frac{h^2}{2s^2}}$ is Gaussian, where $K$ ...
0
votes
0answers
44 views

Minimum-variance and minimum-divergence estimator

Given a parametric family of distributions $\{P_\theta \colon \theta \in \Theta\}$ and a sample $X \sim P_\theta$, an estimator $T^\star(X)$ of the parameter $\theta$ is said to be a uniformly ...
0
votes
0answers
36 views

Kalman Filter with State Constrained to a Surface

I have a state that represents a direction vector condtrained to the surface of a unit sphere . In the update step of a Kalman filter, the state estimate is the sum of two values, like this ...
-3
votes
1answer
51 views

Sufficient statistic for $N(\theta,\theta^2)$ [closed]

Let $X_1,\ldots,X_n$ be a randon sample of the normal distribution with parameters $(\theta,\theta^2)$ How can I find a sufficient statistic for $\theta$? Is there an easy way to do it?
2
votes
1answer
88 views

Expectation of largest and smallest order statistic from uniform distribution

Given is a random sample of size n from a uniform distribution with parameters $-\theta$ and $\theta$, $\theta>0$. I'm asked to find a constant $c$ such that $c(X_{n:n}-X_{1:n})$ is an unbiased ...
1
vote
0answers
23 views

Covariance estimation and Graphical Modelling

I've started reading on Convariance Matrix estimation through Graphical model in high-dimensional situation. But I have several questions. Suppose, $X_i \overset{iid}{\sim} N_p(\mu,\Sigma)$, ...
1
vote
0answers
60 views

Is $\theta_{mle}$ the UMVUE of $\theta$

I am working through some practice problems before an exam, and am getting stuck. The pdf is $f(x|\theta)=\theta x^{\theta-1}$ where $0 < x < 1$. I have already found my MLE as ...
1
vote
1answer
34 views

Improving estimatied parameters of a known distribution

Assume there is a Set of data which follows a known distribution (e.g. normal distribution). $$S = \left\{ a_0,a_1 ... a_n \right\}$$ When taking a subset from S $$S_k = \left\{ a_0,a_1 ... a_k ...
0
votes
0answers
5 views

How to interprete Non-uniform kernel in kernel density estimation?

I understand that uniform kernel is that we count the number of points in the neighbor of $x$ then divided by the total of points to estimate the probability density distribution of $x$. This is ...
4
votes
1answer
76 views

MLE for the PDF $f_\theta(x)=\theta x$ on $0\leq x\leq\sqrt{2/\theta}$: where is the mistake?

Consider $f_X(x;\theta)=\theta\cdot x$, $x\leq\sqrt{\frac{2}{\theta}}$. Find the maximum likelihood for the estimator $\hat{\theta}$ of $\theta$. By definition, the MLE of ...
0
votes
2answers
103 views

Variance of Variance Estimation Test

I am trying to verify, through numerical simulation, the expression for the variance of the variance estimation, namely: Var(s^2)=2/n sigma^4 where n is the number of samples, and sigma is the ...
0
votes
0answers
17 views

MMSE detector for elliptical distribution.

Suppose we have $Y=HX+W$ where dimension of $Y$ is $N$ and $W$ is elliptically distributed $H$ is also elliptically distributed $X$ is uniformly distributed. We want to estimate $\hat{X}$ using MMSE ...
0
votes
1answer
51 views

(UPDATED): Find the minimum-variance unbiased estimator of a given function

Let $X_1, X_2, ... , X_n$ be a random sample form an exponential distribution $E(\theta)$, $\theta>0$. Obtain minimum-variance unbiased estimator of a function $g(\theta)=\frac{1}{\theta^2}$ ($E(X) ...
2
votes
1answer
43 views

Nice applications of estimation theory and hypothesis testing

As a mathematics professor in an engineeer school, I want to write some lab work for students in Statistics. This work should last four hours and will be made in a language such as Matlab or Python. ...
1
vote
0answers
45 views

UMVUE of parameter $(1-\sigma^2)^{-\frac{n}{2}}$

suppose $X_1,X_2,\ldots,X_n$ be random sample of $N(0,\sigma^2)$. how can I calculate UMVUE of parameter $(1-\sigma^2)^{-\frac{n}{2}}$
0
votes
0answers
29 views

Decompose summation of signals

Imagine a summation of three distinct signals such as in the following graphic. Is it possible to estimate the original signals? Below is a matlab-code to generate the image: I have found similar ...
2
votes
1answer
290 views

Bias, SE and MSE of Uniform Distribution

Let $X_1,\ldots,X_n$ be an i.i.d. sequence of Uniform $(\mu,2\mu)$ and let an estimator be $\hat{\mu} = \frac{1}{2} \max\{X_1,\ldots,X_n\}$. Find the bias, SE, and MSE of this estimator. Hint: Let ...
0
votes
1answer
43 views

Estimating variance of estimator of bernoulli process

The maximum likelihood estimate of a Bernoulli process is simply given by $\hat{\theta}=\frac{\sum X_i}{N}$, where N is the total number of bernoulli trial and $X_i$ is the outcome of each trial. ...
2
votes
1answer
76 views

minimum kullback leibler estimator

Suppose that one has independent and identically distributed samples $x_i,i=1,...,n$ from some unknown density and one wants to fit a probability distribution $f_\theta(x)$, where $\theta$ is a ...
1
vote
0answers
88 views

Origins of Kalman filter Algorithms in his paper in 1960

Concerning Kalman's original paper published in 1960, "A New Approach to Linear Filtering and Prediction Problems", it seems the majority is to show the orthogonal projection is the optimal estimation ...
4
votes
0answers
59 views

Properties of an MLE based on likelihood constructed from both PDF and CDF

For continuous RV the likelihood function is (typically) given by a product of PDFs, i.e. $$L(\theta; x_1,x_2, ..., x_n) = \prod_{i=1}^n f(x_i\mid \theta) $$ However, in survival analysis with ...
1
vote
0answers
29 views

Estimation of unbiased estimator for population variance in case of poisson distribution

sample variance is normally biased estimator for population variance. but in case of poisson distribution sample variance is unbiased for population variance. how do you prove this?
1
vote
0answers
32 views

How to bound $E \left[\left(E[Z^2\mid Y] \right)^2\right]- 2E \left[ |E[Z\mid Y]| \sqrt{E[Z^2\mid Y]} \right]$

I am looking for an upper bound on the following quantity \begin{align*} A=E \left[\left(E[Z^2\mid Y] \right)^2\right]- 2E \left[ |E[Z\mid Y]| \sqrt{E[Z^2\mid Y]} \right] \end{align*} where $Z$ is ...
0
votes
2answers
51 views

Estimation, bias, and mean square error

Let $X$ be a continuous random variable with pdf $f(x) =\frac{1}{2}(1+ \theta x)$, for $-1 < x < 1$, and $-1 < \theta < 1$ (a) Show that $E(X) = \frac{\theta}{3}$. (b) Given a random ...
1
vote
0answers
23 views

Proof of differentialbility in mean square calculus?

let $x_t$ be a mean squared Riemann integrable over $[a, t]$ for every $t\in[a,b]$. Then $y_t=\int\limits_a^t x_\tau d\tau\ $ is mean squared continuous on $[a, b]$. Furthermore, if $x_t $ is mean ...
0
votes
1answer
54 views

Bayes estimator under squared error loss

Consider one random variable X from the Bernoulli distribution with parameter θ. Let p, the prior density, be equal to 6θ(1 − θ), for θ ∈ (0, 1). Under squared error loss, L(t, θ) = (t − θ)$^2$, the ...
2
votes
0answers
30 views

MLE of two-dimensional distribution

Let $X_1, ..., X_n$ be a random sample from a continuous distribution with pdf $$f_{\theta,\kappa}(x) = \frac{\kappa\theta^\kappa}{x^{\kappa+1}}, x\geq \theta, \theta > 0, \kappa > 0.$$ How to ...
1
vote
2answers
416 views

Maximum a Posteriori (MAP) Estimator of Exponential Random Variable with Uniform Prior

What would be the Maximum a Posteriori (MAP) estimator for $ \lambda $ for IID $ \left\{ {x}_{i} \right\}_{i = 1}^{N} $ where $ {x}_{i} \sim \exp \left( \lambda \right), \; \lambda \sim U \left[ ...
3
votes
0answers
38 views

Conditional distribution [closed]

I am trying to figure out the derivation of Kalman filter based on Bayesian estimator. As we know, the assumption of Gauss-Markov model is used, then, the conditional distribution p(x(t)|Y(t-1))can be ...
1
vote
0answers
283 views

How to calculate Fisher Information (FI) matrix for Multivariate Normal Distribution (MN)

Below is the gradient (score) of the MN log likelihood function L for n=1 observation. I originally attempted to calculate the Hessian matrix but ran into difficulty calculating 2nd order derivatives ...
4
votes
2answers
139 views

Estimating the “step size” of a grid

Suppose one is given a set of $M$ points distributed on a "grid", i.e: $$x_i = x_0 + \alpha n_i + \epsilon_i, \quad n_i\in\mathbb{Z}$$ This might like something like this: $\quad\ ...
3
votes
2answers
49 views

Difficult to understand difference between the estimates on E(X) and V(X) and the estimates on variance and std.dev. on lambda-hat

I'm having a very hard time to separate estimates on population values versus estimates on sample values. I'm struggling with this exercise (not homework, self-study for my exam in introductionary ...
1
vote
0answers
24 views

bias reduction when the bias depends on the true parameter

Let's say we estimate a parameter, $\theta$, by $\hat{\theta}$. For this estimator we have the following property that $$\hat{\theta}\to_{p}\theta+f(\theta)$$ where $\to_{p}$ denotes convergence in ...
2
votes
1answer
62 views

How to prove that the maximum likelihood estimator of $\theta$ is aysmptotically unbiased and cosistent

In a class we looked at this example: Let $X_1,...,X_n\sim U(0,\theta)$. Then the maximum likelihood function is $\mathcal{L}(\theta) = \begin{cases} \dfrac{1}{\theta^{n}} & \text{if } ...
0
votes
1answer
71 views

Maximum likelihood estimator and confidence interval

Let $\theta$ be an unknown constant. Let $W_1,…,W_n$ be independent exponential random variables each with parameter $1$. Let $X_i=θ+W_i$. First, I need to find $\hat\theta _{ML}(x_1,\ldots ,x_ n)$. ...
1
vote
0answers
62 views

MLE of variance for a spherical Gaussian

I am trying to implement the X-Means clustering algorithm. In it, the authors use the BIC to determine which model fits the data best. It is explained here: ...
1
vote
1answer
114 views

Sum of variances of multinomial distribution.

I've k fair coins, and I would like to know the number of heads obtained in $n$ trials. But that is simple binomial distribution. But if I want to find out how much it varies from binomial ...
1
vote
1answer
39 views

Dice roll, estimator, epsilon

We roll a non-symmetric die. Let $X_n$ be the reulst of $n$-th roll. $$P(X_n = 6)= \frac{1}{6} + \varepsilon, \ P(X_n = 1) = \frac{1}{6} - \varepsilon, \ P(X_n=2) = ... = P(X_n = 5) = \frac{1}{6} $$ ...
2
votes
0answers
77 views

Near-Application of Cauchy-Schwarz Inequality

I have the following situation: I have two estimators of $\alpha$, both via maximum likelihood of the density: $$ f(x,y\mid \alpha,\beta) = f(y \mid x,\alpha,\beta)f(x \mid \alpha) $$ One uses only ...
0
votes
2answers
148 views

How to find the bias, variance and MSE of $\hat p$?

If $X_1,\dots,X_n$ are iid $\mathrm{Binomial}(3,p)$, then the maximum likelihood estimator of $p$ is $$\hat p = \frac{1}{n}\sum_i X_i$$ Find the bias, variance and MSE of $\hat p$? We are asked to ...
0
votes
1answer
67 views

linear least square estimation with random sum

Let $N$ be a geometric r.v. with mean $1/p$; let $A1,A2,… $be a sequence of i.i.d. random variables, all independent of $N$, with mean $1$ and variance $1$; let $B1,B2,… $be another sequence of i.i.d. ...
2
votes
1answer
200 views

Hypothesis test between two normal distributions

Let $T_1,T_2,\ldots ,T_ n$ be i.i.d. observations, each drawn from a common normal distribution with mean zero. With probability $1/2$ this normal distribution has variance $1$, and with probability ...
0
votes
1answer
181 views

Trajectory estimation

The vertical coordinate (“height") of an object in free fall is described by an equation of the form $x(t) = \theta _0 + \theta _1t + \theta _2 t^2,$ We assume that $\theta_0$ is a known constant. We ...
1
vote
1answer
59 views

Unbiased estimator for the sum of numbers

Let $\alpha_1, \dots, \alpha_n \in \mathbb{R}$. We want to approximate the sum as follows $$ S = \sum_{i=1}^{n} \alpha_i \approx \dfrac{n}{c} \sum_{i=1}^{c} \alpha_i, $$ where $\alpha_i$ is picked ...
1
vote
0answers
72 views

Hammersley–Chapman–Robbins bound for Rice distribution

I am trying to evaluate the Hammersley–Chapman–Robbins bound for the variance of an unbiased estimate $\hat{\alpha}$ of $\alpha$ (for a given $\sigma$) for the Rice distribution: $$p(x|\alpha,\sigma) ...
1
vote
0answers
118 views

Expectation of inverse of a symmetric matrix with gaussian elements

Is there any way to calculate: \begin{equation} \mathbb{E} \; ( H^{T}H )^{-1} \end{equation} assuming that the entries of the matrix $H$ are gaussian random variables with unknown means but same ...
2
votes
1answer
226 views

UMVUE for pdf $f_{\theta}(x) = \theta e^{-\theta x}, x>0$

Let $X_1,\ldots,X_n$ be a random sample from a pdf $f_{\theta}(x) = \begin{cases} \theta e^{-\theta x}, & x>0 \\ 0, & \text{otherwise} \end{cases}$, where $\theta>0$ is an unknown ...
2
votes
0answers
40 views

Fisher Expected Information for a Gaussian Process model

Suppose I have a two dimensional Gaussian process model (GP), defined by a squared exponential correlation function s.t: $$R(x_{i},x_{j}) = \exp\left(-\frac{|x_{i} - x_{j}|^2}{2}\right).$$ I am ...
0
votes
1answer
49 views

calculate The maximum likelihood estimator of parameter $\mu$ according to $T$

suppose $X_1,X_2,\ldots,X_n$ be a random sample of $N(\mu,1)$. if $T=\sum_{i=1}^n I_{(X_i<0)}$ how can I calculate The maximum likelihood estimator of parameter $\mu$ according to $T$. ($\Phi$ is ...