Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

2
votes
2answers
23 views

Is “non-random parameter estimation” the same thing as maximum likelihood estimation?

In one book and a few papers, mostly on navigational tracking, I have found reference to the method of "non-random parameter estimation" but this term is not on the Wikipedia and not in a lot of ...
0
votes
0answers
9 views

How to define a likelihood function for an EM algorithm

Assuming $A$ a set of vectors from a normal distribution, and $X$ a projection matrix and $B$ a set of projected vectors of $A$ using $X$: $B=A*X$ Using an EM approach and by initializing X from ...
0
votes
0answers
13 views

Checking if estimators are sufficient

For an i.i.d. sample of random variables Xi distributed according to a normal distribution, known variance. I found a sufficient statistic—the sample mean. How do I check if other statistic like ...
0
votes
2answers
47 views

check if estimation is unbiased?

Assume we that we calculate the expected value of some measurements $x=\dfrac {x_1 + x_2 + x_3 + x_4} 4$. what if we dont include $x_3$ and $x_4$, but instead we use $x_2$ as $x_3$ and $x_4$. Then We ...
0
votes
0answers
20 views

Error Covariance of Minimum-Variance Estimate

I'm working my way through Luenberger's "Optimization by Vector Space Methods". On chapter 4, "Least-Squares Estimation", Section 4.5., Theorem 1, Luenberger shows that given a measurement setup of ...
0
votes
0answers
17 views

Unbiased estimation of parameter with singular matrix.

Given sample $N_p(A\theta,Q)$, where $\theta, Q$ - unknown. A - known $q*p$ matrix, $rankA = q, q<p$. The question is: how can I find unbiased estimation $\hat\theta$ of $\theta$? It seems easy ...
0
votes
0answers
24 views

Cluster sampling: Compare efficiencies

A company operates from 12 branches, and the numbers of cars, $N_i$ and means $\bar{X}_i$ and variances $S_i^2$ of miles driven last year for each brand, are as follows Branch: $N_i$; $\bar{X}_i$ ; ...
0
votes
0answers
25 views

Simple Random Sampling: Find the variance

I have trouble answering this simple question. There is a total of 280 trees. The assessed total yield is at 432,6 tons. 25 trees are picked at random and their timber yields are accurately ...
1
vote
0answers
31 views

Higher Order Estimation Errors

I well know estimation measure is the so called minimum mean square error (MMSE) defined as: \begin{align} E[|W-\hat{W}(V)|^2] \end{align} where $W$ is a random variable (that we want to estimate) and ...
2
votes
1answer
51 views

What is the concentration result of the entropy?

Let $X_1, X_2, \ldots, X_n$ be i.i.d. binary variables with $Pr(X_i=1)=p$ and $Pr(X_i=0)=1-p$. The famous result about $p$ is $$Pr\left(\left|\frac{1}{n}\sum_{i=1}^n ...
0
votes
1answer
28 views

Maximum Likelihood Estimator of $\theta$

I have the following question I tried to answer I got answer that same like this answer Is this true answer? (Note that: in the question $0<p<\frac{1}{2}$, but in this answer ...
1
vote
1answer
15 views

Does the correlation between the measurement noise influence the result of the LS?

Consider a linear measurement process with some noise: $$y=Hx+v$$ with $$v \sim \mathcal{N}(0,\Sigma)$$ the covariance matrix $\Sigma$ is not a diagonal matrix. As we know, using LS, the $\hat{x}$ is ...
1
vote
1answer
30 views

How do I compute a realization of h(x) given its PDF and covariance?

Desription of problem I've added a picture of what I want to compute. In the nomenclature of the picture, I want to compute a realization of y(x) given the known distributions and constants. Let's ...
1
vote
1answer
21 views

How do you show that the estimator for the covariance matrix is unbiased?

So according to Wikipedia (Here) the sample covariance matrix is an unbiased estimator of the covariance matrix, but how do I prove this mathematically?
0
votes
0answers
9 views

Estimate function given PDF and covariance

Let's say $h(x)$, random variable, represents the height of a surface, with x being the usual x-axis. The probability distribution function is: $P(h) = Ke^{-\frac{h^2}{2s^2}}$ is Gaussian, where $K$ ...
0
votes
0answers
37 views

Minimum-variance and minimum-divergence estimator

Given a parametric family of distributions $\{P_\theta \colon \theta \in \Theta\}$ and a sample $X \sim P_\theta$, an estimator $T^\star(X)$ of the parameter $\theta$ is said to be a uniformly ...
0
votes
0answers
26 views

Kalman Filter with State Constrained to a Surface

I have a state that represents a direction vector condtrained to the surface of a unit sphere . In the update step of a Kalman filter, the state estimate is the sum of two values, like this ...
-3
votes
1answer
38 views

Sufficient statistic for $N(\theta,\theta^2)$ [closed]

Let $X_1,\ldots,X_n$ be a randon sample of the normal distribution with parameters $(\theta,\theta^2)$ How can I find a sufficient statistic for $\theta$? Is there an easy way to do it?
2
votes
1answer
65 views

Expectation of largest and smallest order statistic from uniform distribution

Given is a random sample of size n from a uniform distribution with parameters $-\theta$ and $\theta$, $\theta>0$. I'm asked to find a constant $c$ such that $c(X_{n:n}-X_{1:n})$ is an unbiased ...
1
vote
0answers
22 views

Covariance estimation and Graphical Modelling

I've started reading on Convariance Matrix estimation through Graphical model in high-dimensional situation. But I have several questions. Suppose, $X_i \overset{iid}{\sim} N_p(\mu,\Sigma)$, ...
1
vote
0answers
45 views

Is $\theta_{mle}$ the UMVUE of $\theta$

I am working through some practice problems before an exam, and am getting stuck. The pdf is $f(x|\theta)=\theta x^{\theta-1}$ where $0 < x < 1$. I have already found my MLE as ...
1
vote
1answer
31 views

Improving estimatied parameters of a known distribution

Assume there is a Set of data which follows a known distribution (e.g. normal distribution). $$S = \left\{ a_0,a_1 ... a_n \right\}$$ When taking a subset from S $$S_k = \left\{ a_0,a_1 ... a_k ...
0
votes
0answers
4 views

How to interprete Non-uniform kernel in kernel density estimation?

I understand that uniform kernel is that we count the number of points in the neighbor of $x$ then divided by the total of points to estimate the probability density distribution of $x$. This is ...
4
votes
1answer
69 views

MLE for the PDF $f_\theta(x)=\theta x$ on $0\leq x\leq\sqrt{2/\theta}$: where is the mistake?

Consider $f_X(x;\theta)=\theta\cdot x$, $x\leq\sqrt{\frac{2}{\theta}}$. Find the maximum likelihood for the estimator $\hat{\theta}$ of $\theta$. By definition, the MLE of ...
0
votes
2answers
61 views

Variance of Variance Estimation Test

I am trying to verify, through numerical simulation, the expression for the variance of the variance estimation, namely: Var(s^2)=2/n sigma^4 where n is the number of samples, and sigma is the ...
0
votes
0answers
15 views

MMSE detector for elliptical distribution.

Suppose we have $Y=HX+W$ where dimension of $Y$ is $N$ and $W$ is elliptically distributed $H$ is also elliptically distributed $X$ is uniformly distributed. We want to estimate $\hat{X}$ using MMSE ...
0
votes
1answer
42 views

(UPDATED): Find the minimum-variance unbiased estimator of a given function

Let $X_1, X_2, ... , X_n$ be a random sample form an exponential distribution $E(\theta)$, $\theta>0$. Obtain minimum-variance unbiased estimator of a function $g(\theta)=\frac{1}{\theta^2}$ ($E(X) ...
2
votes
1answer
41 views

Nice applications of estimation theory and hypothesis testing

As a mathematics professor in an engineeer school, I want to write some lab work for students in Statistics. This work should last four hours and will be made in a language such as Matlab or Python. ...
1
vote
0answers
42 views

UMVUE of parameter $(1-\sigma^2)^{-\frac{n}{2}}$

suppose $X_1,X_2,\ldots,X_n$ be random sample of $N(0,\sigma^2)$. how can I calculate UMVUE of parameter $(1-\sigma^2)^{-\frac{n}{2}}$
0
votes
0answers
28 views

Decompose summation of signals

Imagine a summation of three distinct signals such as in the following graphic. Is it possible to estimate the original signals? Below is a matlab-code to generate the image: I have found similar ...
1
vote
1answer
158 views

Bias, SE and MSE of Uniform Distribution

Let $X_1,\ldots,X_n$ be an i.i.d. sequence of Uniform $(\mu,2\mu)$ and let an estimator be $\hat{\mu} = \frac{1}{2} \max\{X_1,\ldots,X_n\}$. Find the bias, SE, and MSE of this estimator. Hint: Let ...
0
votes
1answer
34 views

Estimating variance of estimator of bernoulli process

The maximum likelihood estimate of a Bernoulli process is simply given by $\hat{\theta}=\frac{\sum X_i}{N}$, where N is the total number of bernoulli trial and $X_i$ is the outcome of each trial. ...
2
votes
1answer
64 views

minimum kullback leibler estimator

Suppose that one has independent and identically distributed samples $x_i,i=1,...,n$ from some unknown density and one wants to fit a probability distribution $f_\theta(x)$, where $\theta$ is a ...
1
vote
0answers
67 views

Origins of Kalman filter Algorithms in his paper in 1960

Concerning Kalman's original paper published in 1960, "A New Approach to Linear Filtering and Prediction Problems", it seems the majority is to show the orthogonal projection is the optimal estimation ...
4
votes
0answers
52 views

Properties of an MLE based on likelihood constructed from both PDF and CDF

For continuous RV the likelihood function is (typically) given by a product of PDFs, i.e. $$L(\theta; x_1,x_2, ..., x_n) = \prod_{i=1}^n f(x_i\mid \theta) $$ However, in survival analysis with ...
1
vote
0answers
25 views

Estimation of unbiased estimator for population variance in case of poisson distribution

sample variance is normally biased estimator for population variance. but in case of poisson distribution sample variance is unbiased for population variance. how do you prove this?
1
vote
0answers
30 views

How to bound $E \left[\left(E[Z^2\mid Y] \right)^2\right]- 2E \left[ |E[Z\mid Y]| \sqrt{E[Z^2\mid Y]} \right]$

I am looking for an upper bound on the following quantity \begin{align*} A=E \left[\left(E[Z^2\mid Y] \right)^2\right]- 2E \left[ |E[Z\mid Y]| \sqrt{E[Z^2\mid Y]} \right] \end{align*} where $Z$ is ...
0
votes
2answers
45 views

Estimation, bias, and mean square error

Let $X$ be a continuous random variable with pdf $f(x) =\frac{1}{2}(1+ \theta x)$, for $-1 < x < 1$, and $-1 < \theta < 1$ (a) Show that $E(X) = \frac{\theta}{3}$. (b) Given a random ...
1
vote
0answers
21 views

Proof of differentialbility in mean square calculus?

let $x_t$ be a mean squared Riemann integrable over $[a, t]$ for every $t\in[a,b]$. Then $y_t=\int\limits_a^t x_\tau d\tau\ $ is mean squared continuous on $[a, b]$. Furthermore, if $x_t $ is mean ...
0
votes
1answer
42 views

Bayes estimator under squared error loss

Consider one random variable X from the Bernoulli distribution with parameter θ. Let p, the prior density, be equal to 6θ(1 − θ), for θ ∈ (0, 1). Under squared error loss, L(t, θ) = (t − θ)$^2$, the ...
2
votes
0answers
26 views

MLE of two-dimensional distribution

Let $X_1, ..., X_n$ be a random sample from a continuous distribution with pdf $$f_{\theta,\kappa}(x) = \frac{\kappa\theta^\kappa}{x^{\kappa+1}}, x\geq \theta, \theta > 0, \kappa > 0.$$ How to ...
1
vote
2answers
272 views

Maximum a Posteriori (MAP) Estimator of Exponential Random Variable with Uniform Prior

What would be the Maximum a Posteriori (MAP) estimator for $ \lambda $ for IID $ \left\{ {x}_{i} \right\}_{i = 1}^{N} $ where $ {x}_{i} \sim \exp \left( \lambda \right), \; \lambda \sim U \left[ ...
2
votes
0answers
34 views

Conditional distribution [closed]

I am trying to figure out the derivation of Kalman filter based on Bayesian estimator. As we know, the assumption of Gauss-Markov model is used, then, the conditional distribution p(x(t)|Y(t-1))can be ...
1
vote
0answers
207 views

How to calculate Fisher Information (FI) matrix for Multivariate Normal Distribution (MN)

Below is the gradient (score) of the MN log likelihood function L for n=1 observation. I originally attempted to calculate the Hessian matrix but ran into difficulty calculating 2nd order derivatives ...
4
votes
2answers
130 views

Estimating the “step size” of a grid

Suppose one is given a set of $M$ points distributed on a "grid", i.e: $$x_i = x_0 + \alpha n_i + \epsilon_i, \quad n_i\in\mathbb{Z}$$ This might like something like this: $\quad\ ...
3
votes
2answers
48 views

Difficult to understand difference between the estimates on E(X) and V(X) and the estimates on variance and std.dev. on lambda-hat

I'm having a very hard time to separate estimates on population values versus estimates on sample values. I'm struggling with this exercise (not homework, self-study for my exam in introductionary ...
1
vote
0answers
22 views

bias reduction when the bias depends on the true parameter

Let's say we estimate a parameter, $\theta$, by $\hat{\theta}$. For this estimator we have the following property that $$\hat{\theta}\to_{p}\theta+f(\theta)$$ where $\to_{p}$ denotes convergence in ...
2
votes
1answer
55 views

How to prove that the maximum likelihood estimator of $\theta$ is aysmptotically unbiased and cosistent

In a class we looked at this example: Let $X_1,...,X_n\sim U(0,\theta)$. Then the maximum likelihood function is $\mathcal{L}(\theta) = \begin{cases} \dfrac{1}{\theta^{n}} & \text{if } ...
0
votes
1answer
64 views

Maximum likelihood estimator and confidence interval

Let $\theta$ be an unknown constant. Let $W_1,…,W_n$ be independent exponential random variables each with parameter $1$. Let $X_i=θ+W_i$. First, I need to find $\hat\theta _{ML}(x_1,\ldots ,x_ n)$. ...
1
vote
0answers
49 views

MLE of variance for a spherical Gaussian

I am trying to implement the X-Means clustering algorithm. In it, the authors use the BIC to determine which model fits the data best. It is explained here: ...