Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

1
vote
1answer
22 views

For $X_1,X_2, \cdots,X_n$ ~ Poisson($\lambda$), find UMVU estimator for $\lambda^k$ ($k=1,2$,…)

I have some questions about this problem as I'm reviewing for a qual. Our TA provided us with a solution, but I don't understand what is going on: So it looks like they are trying to find an ...
3
votes
1answer
706 views

Maximum likelihood estimators, hypergeometric and binomial

I'm trying to solve a two part problem. The set up is as follows: consider a bag with $\theta$ red marbles and $7-\theta$ blue marbles, with $\theta$ being unknown. Let $x$ denote the number of red ...
0
votes
0answers
27 views

Expected value of maximum likelihood estimator of a Bernoulli random variable

While reading the text from Keith H. Thompson on the Estimation of the Proportion of Vectors in a Natural Population of Insects, I came across the following part where I don't understand everything. ...
1
vote
0answers
14 views

expected 1 norm of a normal vector

Given a normal vector $X$ such that $\mathbb{E}(X)=0$ and $Cov(X)=Id$, is it possible to get an expression for $$\mathbb{E}(\|A X\|_1)$$ where $A$ is a given matrix. I know that in dimension 1, we ...
0
votes
0answers
8 views

If a family of densities is not complete then is it necessary that there isn't any MVUE?

The question is about the truth of this statement: "If the family $\{f(x;\theta):\theta\in\Omega\}$ is not complete, then there doesn't exist any MVUE" MVUE is an abbreviation for "Minimum Variance ...
1
vote
0answers
17 views

The Cramer-Rao Lower Bound proof

Let $X_1, . . . , X_n$ be i.i.d. with density function $f (x|θ)$. Let $T = t (X_1, . . . , X_n)$ be an unbiased estimate of $θ$. Then, under smoothness assumptions on $f (x|θ)$, $$Var(T) >= ...
0
votes
1answer
42 views

Improving data gaussianity using neural networks

I wanted to know if there is a way to use neural networks (deep neural networks or autoencoders) for a data gaussianization. I wonder how could the output data distribution be monitored and ...
1
vote
0answers
29 views

How to Derive Relationship to the Gain Constant?

I want to implement a formula but I'm having issues understanding some of the components which make it up. The premise of the equation is to use a modified version of the Kalman filter that ...
4
votes
1answer
2k views

How to estimate variances for Kalman filter from real sensor measurements without underestimating process noise.

As the title says, I want to estimate the variances needed for a Kalman filter from real sensor measurements only. For example we can take a temperature sensor, but the solution shall be as ...
0
votes
0answers
36 views

Kalman Filter with State Constrained to a Surface

I have a state that represents a direction vector condtrained to the surface of a unit sphere . In the update step of a Kalman filter, the state estimate is the sum of two values, like this ...
1
vote
1answer
58 views

How to evaluate the goodness of Fit of parameters obtained from EM algorithm

I have a set of observations $\mathcal{Y} = {Y_1, \ldots, Y_T}$. I am running EM algorithm to fit the observations to the following Hidden Markov Model $$A = [a_{ij}]_{N \times N}, a_{ij} = P(X_{k+1} ...
2
votes
1answer
285 views

Bias, SE and MSE of Uniform Distribution

Let $X_1,\ldots,X_n$ be an i.i.d. sequence of Uniform $(\mu,2\mu)$ and let an estimator be $\hat{\mu} = \frac{1}{2} \max\{X_1,\ldots,X_n\}$. Find the bias, SE, and MSE of this estimator. Hint: Let ...
0
votes
1answer
17 views

An issue related to the expectation maximization algorithm for a coin toss experiment

I just read a very nicely written introduction paper for the expectation maximisation algorithm published in Nature biotechnology by Do and Batzoglou ...
0
votes
2answers
27 views

Optimal estimation of the fusion of two measurements

Suppose I have a sensor measuring a quantity $\text R$. For example the sensor could be a radar estimating the range of a target. We can write: $$R(t)=r(t)+\nu_0(t)$$ where $r(t)$ is the real range ...
0
votes
0answers
6 views

Finding the norm of estimation error asymptotically

Let $\theta \in \mathbb{R}^p$ be such that it has uniform distribution on the set of standard unit vectors $\{\tau e_1,\ldots,\tau e_p\}$, for $\tau=\sqrt{(2-\varepsilon)\log p}, \varepsilon>0$. ...
2
votes
1answer
35 views

Maximum Likelihood of single observation

I'm stumped on this problem... I keep getting an undefined answer of having to solve -20 = 0. The likelihood function I get is $e^{-20\alpha}$. So I have $y_i=$ $ \begin{cases} 1& ...
0
votes
0answers
9 views

Distribution of sample minimum after bivariate selection (double truncation)

Let $X$ and $Y$ be two RVs with joint distribution $$ (X,Y)\sim \text{Normal}(\mu,\Sigma) $$ Suppose that there is selection on $X$ and $Y$, such that we observe a vector of realisations of $X$, ...
0
votes
0answers
23 views

Number of measurements for least squares and relation to maximum likelihood

I have a simple overdetermined system of equations: $$ y = Xc + e $$ $y, e \in \mathbb{R}^n$, $c \in \mathbb{R}^m$, $X \in \mathbb{R}^{n \times m}$, $e \sim \mathcal{N}(o,\sigma^2)$, $n>>m$ ...
0
votes
1answer
31 views

What is an estimator?

If $p_y$ is a probability function for a density, which depends on the value of $y$ (for example, $y$ might be the mean in the poisson distribution). Assuming that $y$ is random -- i.e. unknown -- ...
0
votes
1answer
41 views

Prove the consistency of Gamma distribution estimators

Given $X$ a random variable in a Gamma distribution, $f(x ; \alpha,\beta)$, and: $E(X) = \alpha \beta$ $Var(X) = \alpha \beta^2$ $\hat \alpha = $$\bar X \over \beta$ $\hat \beta = $$\frac {n \bar ...
1
vote
0answers
34 views

Estimating a sparse vector: Mean squared error when support known

I was reading this paper ("How well can we estimate a sparse vector?" by Candès and Davenport, link: http://arxiv.org/pdf/1104.5246v5.pdf). They consider the problem of estimating a $k$-sparse vector ...
1
vote
1answer
35 views

How to find a MVUE for a certain function of a parameter

The following is one of the exercises from my course in statistics Let $X_1, \ldots, X_n$ be a random sample from a Poisson distribution with parameter $\theta > 0$. Find the MVUE for ...
2
votes
0answers
25 views

Finding minimum energy graph, subject to constraints

I imagine there's a known algorithm for this, but am not totally sure what to search for, and so my search didn't turn up much. Basically, I have have a set of $N$ nodes $\hat x_i $ in a graph $\hat ...
0
votes
0answers
15 views

Intuition behind sampling distributions – specific case

I'm still trying to understand the basics of understanding the intuition of sampling distributions and calculating the sampling distributions of common estimators. For example, I understand the ...
2
votes
1answer
442 views

Numerical calculation of fisher information

I am trying to obtain numerically the fisher information. Given a likelihood function $$ f(X,\theta),$$ with $X \in [0,1]$. The fisher information is given by $$ ...
0
votes
0answers
49 views

Fitting discrete distribution with normal-inverse gaussian weights

I regard a rather curious discrete distribution $X$ on $(1,2,...)$. Its weights are given by $P(X=i)=P_{NIG}(i)-P_{NIG}(i-1)$ where $P_{NIG}$ denotes the cumulative distribution function of the ...
2
votes
1answer
43 views

How many data points are “enough” for linear regression?

I have data points $(x_t,y_t)$ generated from $y_t = a + b x_t + \epsilon$ where $\epsilon$ is gaussian error term with zero mean and unknown variance. I want to estimate coefficients $a$ and $b$ but ...
0
votes
0answers
13 views

Determining wave algorithm based on sine wave

I have some data that I've noticed conforms to a sine wave and I want to approximate it as closely as I can. In the graph, the blue line is the data I want to model as closely as possible. From ...
1
vote
0answers
21 views

Can I bound $P[R > x + \epsilon]$ independently of R?

I have this probability distribution: $P[\Theta < \varphi] = \frac{\varphi}{\pi}$ for $\phi \in [0,\pi]$. Now I have $n$ samples of $D = R\Theta$ i.i.d. ($R>0$) and I want to estimate $R$ as ...
1
vote
2answers
35 views

Asymptotically unbiased estimator for 1/p in Bernouilli distribution?

Suppose I have a sample of $n$ independent stochastic variables, each Bernouilli distributed with parameter $p$ (you may assume $0 < p <1$). I was wondering if there exist (asymptotically) ...
2
votes
2answers
124 views

Variance of sample mean (problems with proof)

Assuming that I have $\{x_1,\ldots, x_N\}$ - an iid (independent identically distributed) sample size $N$ of observations of random variable $\xi$ with unknown mean $m_1$, variance (second central ...
1
vote
0answers
17 views

How to use MLE method for non-distribution function?

I understand that maximum likelihood estimation (MLE) method is normally used with distribution function. However is there anyway around I can do to use MLE for a function which is not distribution ...
0
votes
0answers
9 views

Fisher's information better than covariance matrix in estimators

I know Fishers information matrix is the inverse of covariance matrix. But why is it better to use fishers information matrix instead of covariance matrix in the case of distributed sensor networks?
0
votes
0answers
25 views

Maximum Likelihood Estimator of exponential of L2 norm

given the observed data $x = (x_1; x_2; \cdots; x_n)^T$ , the likelihood function p(x; $\theta$) can be charaterized as $$p(x; \theta) = \alpha(x) e^{ ||x - \hat{\theta}||_2} $$ where $\hat{\theta} ...
0
votes
0answers
44 views

maximum likelihood estimation of exponential and polynomial components model

I tried to find the maximum likelihood estimator and MMSE of the non linear model but I got stuck. Can you help me to explain it? The output of a system can be modeled using a combination of ...
0
votes
0answers
39 views

Compound Poisson process estimation

Let the stock price $S_t$ follows the following equation: \begin{equation} d\log S_t = \sigma _t dW_t, \end{equation} where $W_t$ is a Wiener process and \begin{equation} \sigma _t^2 = \sigma ^2 \exp ...
0
votes
0answers
49 views

Nonparametric Estimation of the Hazard Ratio

I am not sure if this is appropriate here, but I'm hoping someone would be able to help. The following is an excerpt from the following paper "Nonparametric Estimation of the Hazard Ratio" by ...
1
vote
0answers
46 views

Taylor of $\ln(f(exp(x))))$?

Let $ f(x) = \sum a_n x^n$ Such that The $a_n$ are real and $f(a),f ' (a) , f " (a) > 0 $ for any real $a > 0$. Let $ \ln(f(exp(x))) = \sum b_n x^n $. Let $c_n = a_n - b_n$. For a given $f$ ...
0
votes
1answer
49 views

Proving that a statistic is not sufficient (uniform case).

Let $X=(X_1,...,X_n)$ be i.i.d. $U(0,\theta)$. How to show that $$\frac{2}{n}\sum_{i=1}^{n}X_i$$ is not a sufficient statistic? I have already proven that $\max_{i=1,...,n}X_i$ is a sufficient ...
0
votes
1answer
34 views

Help me understand this matrix derivative (for the LS estimation proof)

I'm trying to understand this proof of LS estimation, but I've never studied matrix calculus. I've managed to find a couple of identities on the web and and I see how to get the first part of the ...
1
vote
2answers
79 views

Is there an iterative way to evaluate least squares estimation?

Suppose to have a set of data $\{y_i, u_i\}_{i=1}^m$, where $y_i \in \mathbb{R}$ and $u_i \in \mathbb{R}^n$. The claim is that $$y_i = u_i^\top \theta + \varepsilon$$ where $\theta \in ...
1
vote
0answers
20 views

Neyman Pearson rule but not a Bayes rule

Consider a binary hypothesis testing problem of $P_0$ vs. $P_1$ under uniform costs. Let $r(\delta,\pi)$ denote the risk line for any decision rule $\delta$ and prior $\pi$, i.e, $r(\delta,\pi)=\pi ...
0
votes
2answers
26 views

Show that $\hat{\mu}$ has minimal variance

So two independent analyses of a content in a water sample have been made using two different methods, both without systematical errors but with different standard deviations. Method $B$ is assumed to ...
0
votes
0answers
27 views

How could an estimator be biased but consistent according to mathematical definition?

According to the definition, an estimator can be biased, if $E_{\theta}[\hat{\theta}]\ne\theta$, with $\theta$ as parameter for a distribution we want to get from samples. While the estimator can be ...
0
votes
0answers
13 views

Estimation of binomial probabilities $f(r)$ over $r \in [0,\frac{1}{2}]$

I want to fit a (decreasing) univariate function, \begin{equation} f(r), \end{equation} over $r \in [0,\frac{1}{2}]$ to a series ($r =\frac{1}{100}, \frac{2}{100}, \frac{3}{100} ,\ldots,\frac{1}{2}$) ...
0
votes
1answer
36 views

IMPROVED - Proving that a statistics is not sufficient (Gaussian case).

Let $X=(X_1,...,X_n)$ be i.i.d. $N(0,\sigma^2)$. How to show that $$\frac{2}{n}\sum_{i=1}^{n}X_i$$ is not a sufficient statistic? I have already proven that $\max_{i=1,...,n}X_i$ is a sufficient ...
2
votes
2answers
40 views

Is “non-random parameter estimation” the same thing as maximum likelihood estimation?

In one book and a few papers, mostly on navigational tracking, I have found reference to the method of "non-random parameter estimation" but this term is not on the Wikipedia and not in a lot of ...
0
votes
0answers
13 views

How to define a likelihood function for an EM algorithm

Assuming $A$ a set of vectors from a normal distribution, and $X$ a projection matrix and $B$ a set of projected vectors of $A$ using $X$: $B=A*X$ Using an EM approach and by initializing X from ...
1
vote
1answer
33 views

How do you show that the estimator for the covariance matrix is unbiased?

So according to Wikipedia (Here) the sample covariance matrix is an unbiased estimator of the covariance matrix, but how do I prove this mathematically?
1
vote
1answer
15 views

Does the correlation between the measurement noise influence the result of the LS?

Consider a linear measurement process with some noise: $$y=Hx+v$$ with $$v \sim \mathcal{N}(0,\Sigma)$$ the covariance matrix $\Sigma$ is not a diagonal matrix. As we know, using LS, the $\hat{x}$ is ...