Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

0
votes
0answers
18 views

Unbiased estimation of parameter with singular matrix.

Given sample $N_p(A\theta,Q)$, where $\theta, Q$ - unknown. A - known $q*p$ matrix, $rankA = q, q<p$. The question is: how can I find unbiased estimation $\hat\theta$ of $\theta$? It seems easy ...
0
votes
0answers
27 views

Cluster sampling: Compare efficiencies

A company operates from 12 branches, and the numbers of cars, $N_i$ and means $\bar{X}_i$ and variances $S_i^2$ of miles driven last year for each brand, are as follows Branch: $N_i$; $\bar{X}_i$ ; ...
0
votes
0answers
29 views

Simple Random Sampling: Find the variance

I have trouble answering this simple question. There is a total of 280 trees. The assessed total yield is at 432,6 tons. 25 trees are picked at random and their timber yields are accurately ...
1
vote
0answers
33 views

Higher Order Estimation Errors

I well know estimation measure is the so called minimum mean square error (MMSE) defined as: \begin{align} E[|W-\hat{W}(V)|^2] \end{align} where $W$ is a random variable (that we want to estimate) and ...
2
votes
1answer
55 views

What is the concentration result of the entropy?

Let $X_1, X_2, \ldots, X_n$ be i.i.d. binary variables with $Pr(X_i=1)=p$ and $Pr(X_i=0)=1-p$. The famous result about $p$ is $$Pr\left(\left|\frac{1}{n}\sum_{i=1}^n X_i-p\right|>\epsilon\right)\...
0
votes
1answer
36 views

Maximum Likelihood Estimator of $\theta$

I have the following question I tried to answer I got answer that same like this answer Is this true answer? (Note that: in the question $0<p<\frac{1}{2}$, but in this answer $...
1
vote
1answer
15 views

Does the correlation between the measurement noise influence the result of the LS?

Consider a linear measurement process with some noise: $$y=Hx+v$$ with $$v \sim \mathcal{N}(0,\Sigma)$$ the covariance matrix $\Sigma$ is not a diagonal matrix. As we know, using LS, the $\hat{x}$ is ...
1
vote
1answer
35 views

How do I compute a realization of h(x) given its PDF and covariance?

Desription of problem I've added a picture of what I want to compute. In the nomenclature of the picture, I want to compute a realization of y(x) given the known distributions and constants. Let's ...
1
vote
1answer
34 views

How do you show that the estimator for the covariance matrix is unbiased?

So according to Wikipedia (Here) the sample covariance matrix is an unbiased estimator of the covariance matrix, but how do I prove this mathematically?
0
votes
0answers
10 views

Estimate function given PDF and covariance

Let's say $h(x)$, random variable, represents the height of a surface, with x being the usual x-axis. The probability distribution function is: $P(h) = Ke^{-\frac{h^2}{2s^2}}$ is Gaussian, where $K$ ...
0
votes
0answers
47 views

Minimum-variance and minimum-divergence estimator

Given a parametric family of distributions $\{P_\theta \colon \theta \in \Theta\}$ and a sample $X \sim P_\theta$, an estimator $T^\star(X)$ of the parameter $\theta$ is said to be a uniformly minimum-...
0
votes
0answers
38 views

Kalman Filter with State Constrained to a Surface

I have a state that represents a direction vector condtrained to the surface of a unit sphere . In the update step of a Kalman filter, the state estimate is the sum of two values, like this $$\hat{x_{...
-3
votes
1answer
54 views

Sufficient statistic for $N(\theta,\theta^2)$ [closed]

Let $X_1,\ldots,X_n$ be a randon sample of the normal distribution with parameters $(\theta,\theta^2)$ How can I find a sufficient statistic for $\theta$? Is there an easy way to do it?
2
votes
1answer
88 views

Expectation of largest and smallest order statistic from uniform distribution

Given is a random sample of size n from a uniform distribution with parameters $-\theta$ and $\theta$, $\theta>0$. I'm asked to find a constant $c$ such that $c(X_{n:n}-X_{1:n})$ is an unbiased ...
1
vote
0answers
23 views

Covariance estimation and Graphical Modelling

I've started reading on Convariance Matrix estimation through Graphical model in high-dimensional situation. But I have several questions. Suppose, $X_i \overset{iid}{\sim} N_p(\mu,\Sigma)$, $i=1(1)n$...
1
vote
0answers
60 views

Is $\theta_{mle}$ the UMVUE of $\theta$

I am working through some practice problems before an exam, and am getting stuck. The pdf is $f(x|\theta)=\theta x^{\theta-1}$ where $0 < x < 1$. I have already found my MLE as $\hat{\theta}_{...
1
vote
1answer
35 views

Improving estimatied parameters of a known distribution

Assume there is a Set of data which follows a known distribution (e.g. normal distribution). $$S = \left\{ a_0,a_1 ... a_n \right\}$$ When taking a subset from S $$S_k = \left\{ a_0,a_1 ... a_k \...
0
votes
0answers
5 views

How to interprete Non-uniform kernel in kernel density estimation?

I understand that uniform kernel is that we count the number of points in the neighbor of $x$ then divided by the total of points to estimate the probability density distribution of $x$. This is ...
4
votes
1answer
76 views

MLE for the PDF $f_\theta(x)=\theta x$ on $0\leq x\leq\sqrt{2/\theta}$: where is the mistake?

Consider $f_X(x;\theta)=\theta\cdot x$, $x\leq\sqrt{\frac{2}{\theta}}$. Find the maximum likelihood for the estimator $\hat{\theta}$ of $\theta$. By definition, the MLE of $f(x_1\ldots,x_n;\hat{\...
0
votes
2answers
110 views

Variance of Variance Estimation Test

I am trying to verify, through numerical simulation, the expression for the variance of the variance estimation, namely: Var(s^2)=2/n sigma^4 where n is the number of samples, and sigma is the ...
0
votes
0answers
17 views

MMSE detector for elliptical distribution.

Suppose we have $Y=HX+W$ where dimension of $Y$ is $N$ and $W$ is elliptically distributed $H$ is also elliptically distributed $X$ is uniformly distributed. We want to estimate $\hat{X}$ using MMSE ...
0
votes
1answer
51 views

(UPDATED): Find the minimum-variance unbiased estimator of a given function

Let $X_1, X_2, ... , X_n$ be a random sample form an exponential distribution $E(\theta)$, $\theta>0$. Obtain minimum-variance unbiased estimator of a function $g(\theta)=\frac{1}{\theta^2}$ ($E(X) ...
2
votes
1answer
43 views

Nice applications of estimation theory and hypothesis testing

As a mathematics professor in an engineeer school, I want to write some lab work for students in Statistics. This work should last four hours and will be made in a language such as Matlab or Python. ...
1
vote
0answers
45 views

UMVUE of parameter $(1-\sigma^2)^{-\frac{n}{2}}$

suppose $X_1,X_2,\ldots,X_n$ be random sample of $N(0,\sigma^2)$. how can I calculate UMVUE of parameter $(1-\sigma^2)^{-\frac{n}{2}}$
0
votes
0answers
29 views

Decompose summation of signals

Imagine a summation of three distinct signals such as in the following graphic. Is it possible to estimate the original signals? Below is a matlab-code to generate the image: I have found similar ...
2
votes
1answer
319 views

Bias, SE and MSE of Uniform Distribution

Let $X_1,\ldots,X_n$ be an i.i.d. sequence of Uniform $(\mu,2\mu)$ and let an estimator be $\hat{\mu} = \frac{1}{2} \max\{X_1,\ldots,X_n\}$. Find the bias, SE, and MSE of this estimator. Hint: Let $...
0
votes
1answer
45 views

Estimating variance of estimator of bernoulli process

The maximum likelihood estimate of a Bernoulli process is simply given by $\hat{\theta}=\frac{\sum X_i}{N}$, where N is the total number of bernoulli trial and $X_i$ is the outcome of each trial. ...
2
votes
1answer
79 views

minimum kullback leibler estimator

Suppose that one has independent and identically distributed samples $x_i,i=1,...,n$ from some unknown density and one wants to fit a probability distribution $f_\theta(x)$, where $\theta$ is a (...
1
vote
0answers
93 views

Origins of Kalman filter Algorithms in his paper in 1960

Concerning Kalman's original paper published in 1960, "A New Approach to Linear Filtering and Prediction Problems", it seems the majority is to show the orthogonal projection is the optimal estimation ...
4
votes
0answers
59 views

Properties of an MLE based on likelihood constructed from both PDF and CDF

For continuous RV the likelihood function is (typically) given by a product of PDFs, i.e. $$L(\theta; x_1,x_2, ..., x_n) = \prod_{i=1}^n f(x_i\mid \theta) $$ However, in survival analysis with ...
1
vote
0answers
29 views

Estimation of unbiased estimator for population variance in case of poisson distribution

sample variance is normally biased estimator for population variance. but in case of poisson distribution sample variance is unbiased for population variance. how do you prove this?
1
vote
0answers
39 views

How to bound $E \left[\left(E[Z^2\mid Y] \right)^2\right]- 2E \left[ |E[Z\mid Y]| \sqrt{E[Z^2\mid Y]} \right]$

I am looking for an upper bound on the following quantity \begin{align*} A=E \left[\left(E[Z^2\mid Y] \right)^2\right]- 2E \left[ |E[Z\mid Y]| \sqrt{E[Z^2\mid Y]} \right] \end{align*} where $Z$ is ...
0
votes
2answers
52 views

Estimation, bias, and mean square error

Let $X$ be a continuous random variable with pdf $f(x) =\frac{1}{2}(1+ \theta x)$, for $-1 < x < 1$, and $-1 < \theta < 1$ (a) Show that $E(X) = \frac{\theta}{3}$. (b) Given a random ...
1
vote
0answers
26 views

Proof of differentialbility in mean square calculus?

let $x_t$ be a mean squared Riemann integrable over $[a, t]$ for every $t\in[a,b]$. Then $y_t=\int\limits_a^t x_\tau d\tau\ $ is mean squared continuous on $[a, b]$. Furthermore, if $x_t $ is mean ...
0
votes
1answer
55 views

Bayes estimator under squared error loss

Consider one random variable X from the Bernoulli distribution with parameter θ. Let p, the prior density, be equal to 6θ(1 − θ), for θ ∈ (0, 1). Under squared error loss, L(t, θ) = (t − θ)$^2$, the ...
2
votes
0answers
31 views

MLE of two-dimensional distribution

Let $X_1, ..., X_n$ be a random sample from a continuous distribution with pdf $$f_{\theta,\kappa}(x) = \frac{\kappa\theta^\kappa}{x^{\kappa+1}}, x\geq \theta, \theta > 0, \kappa > 0.$$ How to ...
1
vote
2answers
433 views

Maximum a Posteriori (MAP) Estimator of Exponential Random Variable with Uniform Prior

What would be the Maximum a Posteriori (MAP) estimator for $ \lambda $ for IID $ \left\{ {x}_{i} \right\}_{i = 1}^{N} $ where $ {x}_{i} \sim \exp \left( \lambda \right), \; \lambda \sim U \left[ {u}_{...
3
votes
0answers
38 views

Conditional distribution [closed]

I am trying to figure out the derivation of Kalman filter based on Bayesian estimator. As we know, the assumption of Gauss-Markov model is used, then, the conditional distribution p(x(t)|Y(t-1))can be ...
1
vote
0answers
305 views

How to calculate Fisher Information (FI) matrix for Multivariate Normal Distribution (MN)

Below is the gradient (score) of the MN log likelihood function L for n=1 observation. I originally attempted to calculate the Hessian matrix but ran into difficulty calculating 2nd order derivatives ...
4
votes
2answers
141 views

Estimating the “step size” of a grid

Suppose one is given a set of $M$ points distributed on a "grid", i.e: $$x_i = x_0 + \alpha n_i + \epsilon_i, \quad n_i\in\mathbb{Z}$$ This might like something like this: $\quad\ \quad\quad\quad\...
3
votes
2answers
49 views

Difficult to understand difference between the estimates on E(X) and V(X) and the estimates on variance and std.dev. on lambda-hat

I'm having a very hard time to separate estimates on population values versus estimates on sample values. I'm struggling with this exercise (not homework, self-study for my exam in introductionary ...
1
vote
0answers
24 views

bias reduction when the bias depends on the true parameter

Let's say we estimate a parameter, $\theta$, by $\hat{\theta}$. For this estimator we have the following property that $$\hat{\theta}\to_{p}\theta+f(\theta)$$ where $\to_{p}$ denotes convergence in ...
2
votes
1answer
62 views

How to prove that the maximum likelihood estimator of $\theta$ is aysmptotically unbiased and cosistent

In a class we looked at this example: Let $X_1,...,X_n\sim U(0,\theta)$. Then the maximum likelihood function is $\mathcal{L}(\theta) = \begin{cases} \dfrac{1}{\theta^{n}} & \text{if } \text{...
0
votes
1answer
71 views

Maximum likelihood estimator and confidence interval

Let $\theta$ be an unknown constant. Let $W_1,…,W_n$ be independent exponential random variables each with parameter $1$. Let $X_i=θ+W_i$. First, I need to find $\hat\theta _{ML}(x_1,\ldots ,x_ n)$. ...
1
vote
0answers
63 views

MLE of variance for a spherical Gaussian

I am trying to implement the X-Means clustering algorithm. In it, the authors use the BIC to determine which model fits the data best. It is explained here: http://www.cs.cmu.edu/~dpelleg/download/...
1
vote
1answer
115 views

Sum of variances of multinomial distribution.

I've k fair coins, and I would like to know the number of heads obtained in $n$ trials. But that is simple binomial distribution. But if I want to find out how much it varies from binomial expectation,...
1
vote
1answer
39 views

Dice roll, estimator, epsilon

We roll a non-symmetric die. Let $X_n$ be the reulst of $n$-th roll. $$P(X_n = 6)= \frac{1}{6} + \varepsilon, \ P(X_n = 1) = \frac{1}{6} - \varepsilon, \ P(X_n=2) = ... = P(X_n = 5) = \frac{1}{6} $$ ...
2
votes
0answers
77 views

Near-Application of Cauchy-Schwarz Inequality

I have the following situation: I have two estimators of $\alpha$, both via maximum likelihood of the density: $$ f(x,y\mid \alpha,\beta) = f(y \mid x,\alpha,\beta)f(x \mid \alpha) $$ One uses only ...
0
votes
2answers
150 views

How to find the bias, variance and MSE of $\hat p$?

If $X_1,\dots,X_n$ are iid $\mathrm{Binomial}(3,p)$, then the maximum likelihood estimator of $p$ is $$\hat p = \frac{1}{n}\sum_i X_i$$ Find the bias, variance and MSE of $\hat p$? We are asked to ...
0
votes
1answer
67 views

linear least square estimation with random sum

Let $N$ be a geometric r.v. with mean $1/p$; let $A1,A2,… $be a sequence of i.i.d. random variables, all independent of $N$, with mean $1$ and variance $1$; let $B1,B2,… $be another sequence of i.i.d. ...