Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

2
votes
0answers
25 views

Find an estimator by using the method of moment

Let $X$ be a discrete random variable with density function: $$p(x;\theta)=\left(\frac{\theta}{2}\right)^{\lvert x\rvert}(1-\theta)^{1-\lvert x\rvert}$$ where $x\in\{-1,0,1\}$ and $\theta \in[0,1]...
4
votes
2answers
3k views

How to estimate variances for Kalman filter from real sensor measurements without underestimating process noise.

As the title says, I want to estimate the variances needed for a Kalman filter from real sensor measurements only. For example we can take a temperature sensor, but the solution shall be as ...
3
votes
1answer
717 views

Maximum likelihood estimators, hypergeometric and binomial

I'm trying to solve a two part problem. The set up is as follows: consider a bag with $\theta$ red marbles and $7-\theta$ blue marbles, with $\theta$ being unknown. Let $x$ denote the number of red ...
2
votes
1answer
25 views

Definition of Bias for Baysian Estimation

I know for parameter estimation the estimator $ \hat{\theta}(X)$ of $\theta$ based on observation $\theta$ is said to be unbiased if \begin{align} E[ \hat{\theta}(X)]=\theta, \ \forall \theta. \end{...
1
vote
1answer
23 views

Show that MLE estimator convergences in probability to actual parameter

For iid stochastic variables $X_1, ..., X_n$ whose distribution is defined by 2 parameters, I have found the MLE estimators. They are $\hat{\mu} = \sum x_i/n$, and $\hat{\lambda}$ given by $$ \frac{...
0
votes
0answers
38 views

Bayesian Estimation: calculating an integral

I am reading a book on Bayesian filtering and I have a question regarding calculating transition density $p(X_t|X_{t-1})$. My question is how the term $p(X_t|X_{t-1}, V_{t}=v)$ is converted to the ...
1
vote
1answer
29 views

'Bounds' on the Covariance Matrix

We define covariance of random vector ${\bf X}$ as \begin{align} Cov({\bf X})=E \left[ \left( {\bf X}-E[{\bf X}] \right) \left( {\bf X}-E[{\bf X}] \right)^T \right]. \end{align} In the scalar case ...
0
votes
0answers
8 views

A question about Estimation problem in digital communication setup.

I originally asked this problem here http://dsp.stackexchange.com/questions/31503/estimation-problem-for-m-ary-pam-transmission-over-awgn-channel-problem I would appreciate if someone can take a ...
2
votes
2answers
23 views

Trying to find the MLE of $\tau$

Let $\tau = \int x \,dF(x),$ and I want to find the MLE of $\tau$ given $X_1,\ldots,X_n \sim \mathrm{Uniform}(a,b).$ I am not entirely sure, but I would imagine that $\tau = \int x \, dF(x) = \int_a^b ...
0
votes
1answer
21 views

Which test statistic is better for testing population mean?

Consider the following situation: a stochastic variable $X: (\Omega, \mathcal{F}) \to (\mathbb{R}, \mathcal{R})$ is known to be normally distributed with some mean $\mu$ and some variance $\sigma^2$. ...
1
vote
1answer
26 views

For $X_1,X_2, \cdots,X_n$ ~ Poisson($\lambda$), find UMVU estimator for $\lambda^k$ ($k=1,2$,…)

I have some questions about this problem as I'm reviewing for a qual. Our TA provided us with a solution, but I don't understand what is going on: So it looks like they are trying to find an ...
0
votes
0answers
31 views

Expected value of maximum likelihood estimator of a Bernoulli random variable

While reading the text from Keith H. Thompson on the Estimation of the Proportion of Vectors in a Natural Population of Insects, I came across the following part where I don't understand everything. ...
1
vote
0answers
15 views

expected 1 norm of a normal vector

Given a normal vector $X$ such that $\mathbb{E}(X)=0$ and $Cov(X)=Id$, is it possible to get an expression for $$\mathbb{E}(\|A X\|_1)$$ where $A$ is a given matrix. I know that in dimension 1, we ...
0
votes
0answers
10 views

If a family of densities is not complete then is it necessary that there isn't any MVUE?

The question is about the truth of this statement: "If the family $\{f(x;\theta):\theta\in\Omega\}$ is not complete, then there doesn't exist any MVUE" MVUE is an abbreviation for "Minimum Variance ...
1
vote
0answers
19 views

The Cramer-Rao Lower Bound proof

Let $X_1, . . . , X_n$ be i.i.d. with density function $f (x|θ)$. Let $T = t (X_1, . . . , X_n)$ be an unbiased estimate of $θ$. Then, under smoothness assumptions on $f (x|θ)$, $$Var(T) >= \frac{1}...
0
votes
1answer
42 views

Improving data gaussianity using neural networks

I wanted to know if there is a way to use neural networks (deep neural networks or autoencoders) for a data gaussianization. I wonder how could the output data distribution be monitored and ...
1
vote
0answers
29 views

How to Derive Relationship to the Gain Constant?

I want to implement a formula but I'm having issues understanding some of the components which make it up. The premise of the equation is to use a modified version of the Kalman filter that ...
0
votes
0answers
38 views

Kalman Filter with State Constrained to a Surface

I have a state that represents a direction vector condtrained to the surface of a unit sphere . In the update step of a Kalman filter, the state estimate is the sum of two values, like this $$\hat{x_{...
1
vote
1answer
58 views

How to evaluate the goodness of Fit of parameters obtained from EM algorithm

I have a set of observations $\mathcal{Y} = {Y_1, \ldots, Y_T}$. I am running EM algorithm to fit the observations to the following Hidden Markov Model $$A = [a_{ij}]_{N \times N}, a_{ij} = P(X_{k+1} ...
2
votes
1answer
320 views

Bias, SE and MSE of Uniform Distribution

Let $X_1,\ldots,X_n$ be an i.i.d. sequence of Uniform $(\mu,2\mu)$ and let an estimator be $\hat{\mu} = \frac{1}{2} \max\{X_1,\ldots,X_n\}$. Find the bias, SE, and MSE of this estimator. Hint: Let $...
0
votes
1answer
17 views

An issue related to the expectation maximization algorithm for a coin toss experiment

I just read a very nicely written introduction paper for the expectation maximisation algorithm published in Nature biotechnology by Do and Batzoglou (http://www.nature.com/nbt/journal/v26/n8/full/...
0
votes
2answers
28 views

Optimal estimation of the fusion of two measurements

Suppose I have a sensor measuring a quantity $\text R$. For example the sensor could be a radar estimating the range of a target. We can write: $$R(t)=r(t)+\nu_0(t)$$ where $r(t)$ is the real range ...
0
votes
0answers
6 views

Finding the norm of estimation error asymptotically

Let $\theta \in \mathbb{R}^p$ be such that it has uniform distribution on the set of standard unit vectors $\{\tau e_1,\ldots,\tau e_p\}$, for $\tau=\sqrt{(2-\varepsilon)\log p}, \varepsilon>0$. ...
2
votes
1answer
44 views

Maximum Likelihood of single observation

I'm stumped on this problem... I keep getting an undefined answer of having to solve -20 = 0. The likelihood function I get is $e^{-20\alpha}$. So I have $y_i=$ $ \begin{cases} 1& w/...
0
votes
0answers
10 views

Distribution of sample minimum after bivariate selection (double truncation)

Let $X$ and $Y$ be two RVs with joint distribution $$ (X,Y)\sim \text{Normal}(\mu,\Sigma) $$ Suppose that there is selection on $X$ and $Y$, such that we observe a vector of realisations of $X$, $\{...
0
votes
0answers
25 views

Number of measurements for least squares and relation to maximum likelihood

I have a simple overdetermined system of equations: $$ y = Xc + e $$ $y, e \in \mathbb{R}^n$, $c \in \mathbb{R}^m$, $X \in \mathbb{R}^{n \times m}$, $e \sim \mathcal{N}(o,\sigma^2)$, $n>>m$ ...
0
votes
1answer
31 views

What is an estimator?

If $p_y$ is a probability function for a density, which depends on the value of $y$ (for example, $y$ might be the mean in the poisson distribution). Assuming that $y$ is random -- i.e. unknown -- ...
0
votes
1answer
42 views

Prove the consistency of Gamma distribution estimators

Given $X$ a random variable in a Gamma distribution, $f(x ; \alpha,\beta)$, and: $E(X) = \alpha \beta$ $Var(X) = \alpha \beta^2$ $\hat \alpha = $$\bar X \over \beta$ $\hat \beta = $$\frac {n \bar ...
1
vote
0answers
56 views

Estimating a sparse vector: Mean squared error when support known

I was reading this paper ("How well can we estimate a sparse vector?" by Candès and Davenport, link: http://arxiv.org/pdf/1104.5246v5.pdf). They consider the problem of estimating a $k$-sparse vector $...
1
vote
1answer
41 views

How to find a MVUE for a certain function of a parameter

The following is one of the exercises from my course in statistics Let $X_1, \ldots, X_n$ be a random sample from a Poisson distribution with parameter $\theta > 0$. Find the MVUE for $q(\theta)...
2
votes
0answers
26 views

Finding minimum energy graph, subject to constraints

I imagine there's a known algorithm for this, but am not totally sure what to search for, and so my search didn't turn up much. Basically, I have have a set of $N$ nodes $\hat x_i $ in a graph $\hat ...
0
votes
0answers
15 views

Intuition behind sampling distributions – specific case

I'm still trying to understand the basics of understanding the intuition of sampling distributions and calculating the sampling distributions of common estimators. For example, I understand the ...
2
votes
1answer
456 views

Numerical calculation of fisher information

I am trying to obtain numerically the fisher information. Given a likelihood function $$ f(X,\theta),$$ with $X \in [0,1]$. The fisher information is given by $$ \mathbb{I}(\theta)=\mathbb{E}\left[\...
0
votes
0answers
51 views

Fitting discrete distribution with normal-inverse gaussian weights

I regard a rather curious discrete distribution $X$ on $(1,2,...)$. Its weights are given by $P(X=i)=P_{NIG}(i)-P_{NIG}(i-1)$ where $P_{NIG}$ denotes the cumulative distribution function of the Normal-...
2
votes
1answer
46 views

How many data points are “enough” for linear regression?

I have data points $(x_t,y_t)$ generated from $y_t = a + b x_t + \epsilon$ where $\epsilon$ is gaussian error term with zero mean and unknown variance. I want to estimate coefficients $a$ and $b$ but ...
0
votes
0answers
15 views

Determining wave algorithm based on sine wave

I have some data that I've noticed conforms to a sine wave and I want to approximate it as closely as I can. In the graph, the blue line is the data I want to model as closely as possible. From ...
1
vote
0answers
21 views

Can I bound $P[R > x + \epsilon]$ independently of R?

I have this probability distribution: $P[\Theta < \varphi] = \frac{\varphi}{\pi}$ for $\phi \in [0,\pi]$. Now I have $n$ samples of $D = R\Theta$ i.i.d. ($R>0$) and I want to estimate $R$ as $\...
1
vote
2answers
36 views

Asymptotically unbiased estimator for 1/p in Bernouilli distribution?

Suppose I have a sample of $n$ independent stochastic variables, each Bernouilli distributed with parameter $p$ (you may assume $0 < p <1$). I was wondering if there exist (asymptotically) ...
2
votes
2answers
128 views

Variance of sample mean (problems with proof)

Assuming that I have $\{x_1,\ldots, x_N\}$ - an iid (independent identically distributed) sample size $N$ of observations of random variable $\xi$ with unknown mean $m_1$, variance (second central ...
1
vote
0answers
17 views

How to use MLE method for non-distribution function?

I understand that maximum likelihood estimation (MLE) method is normally used with distribution function. However is there anyway around I can do to use MLE for a function which is not distribution ...
0
votes
0answers
10 views

Fisher's information better than covariance matrix in estimators

I know Fishers information matrix is the inverse of covariance matrix. But why is it better to use fishers information matrix instead of covariance matrix in the case of distributed sensor networks?
0
votes
0answers
26 views

Maximum Likelihood Estimator of exponential of L2 norm

given the observed data $x = (x_1; x_2; \cdots; x_n)^T$ , the likelihood function p(x; $\theta$) can be charaterized as $$p(x; \theta) = \alpha(x) e^{ ||x - \hat{\theta}||_2} $$ where $\hat{\theta} =...
0
votes
0answers
45 views

maximum likelihood estimation of exponential and polynomial components model

I tried to find the maximum likelihood estimator and MMSE of the non linear model but I got stuck. Can you help me to explain it? The output of a system can be modeled using a combination of ...
0
votes
0answers
41 views

Compound Poisson process estimation

Let the stock price $S_t$ follows the following equation: \begin{equation} d\log S_t = \sigma _t dW_t, \end{equation} where $W_t$ is a Wiener process and \begin{equation} \sigma _t^2 = \sigma ^2 \exp ...
0
votes
0answers
49 views

Nonparametric Estimation of the Hazard Ratio

I am not sure if this is appropriate here, but I'm hoping someone would be able to help. The following is an excerpt from the following paper "Nonparametric Estimation of the Hazard Ratio" by H.T....
1
vote
0answers
46 views

Taylor of $\ln(f(exp(x))))$?

Let $ f(x) = \sum a_n x^n$ Such that The $a_n$ are real and $f(a),f ' (a) , f " (a) > 0 $ for any real $a > 0$. Let $ \ln(f(exp(x))) = \sum b_n x^n $. Let $c_n = a_n - b_n$. For a given $f$ ...
0
votes
1answer
50 views

Proving that a statistic is not sufficient (uniform case).

Let $X=(X_1,...,X_n)$ be i.i.d. $U(0,\theta)$. How to show that $$\frac{2}{n}\sum_{i=1}^{n}X_i$$ is not a sufficient statistic? I have already proven that $\max_{i=1,...,n}X_i$ is a sufficient ...
0
votes
1answer
35 views

Help me understand this matrix derivative (for the LS estimation proof)

I'm trying to understand this proof of LS estimation, but I've never studied matrix calculus. I've managed to find a couple of identities on the web and and I see how to get the first part of the ...
1
vote
2answers
85 views

Is there an iterative way to evaluate least squares estimation?

Suppose to have a set of data $\{y_i, u_i\}_{i=1}^m$, where $y_i \in \mathbb{R}$ and $u_i \in \mathbb{R}^n$. The claim is that $$y_i = u_i^\top \theta + \varepsilon$$ where $\theta \in \mathbb{R}^n$...
1
vote
0answers
26 views

Neyman Pearson rule but not a Bayes rule

Consider a binary hypothesis testing problem of $P_0$ vs. $P_1$ under uniform costs. Let $r(\delta,\pi)$ denote the risk line for any decision rule $\delta$ and prior $\pi$, i.e, $r(\delta,\pi)=\pi ...