Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

0
votes
1answer
45 views

Bayesian Example

Ex. suppose that $x=2$ denotes the number of successes in $n=5$ independent trials with probability $θ$ of success, that is $x$ has a binomial distribution with the parameters $n=5$ and $ θ$. ...
2
votes
0answers
27 views

Find an estimator by using the method of moment

Let $X$ be a discrete random variable with density function: $$p(x;\theta)=\left(\frac{\theta}{2}\right)^{\lvert x\rvert}(1-\theta)^{1-\lvert x\rvert}$$ where $x\in\{-1,0,1\}$ and $\theta \in[0,1]...
1
vote
1answer
23 views

Show that MLE estimator convergences in probability to actual parameter

For iid stochastic variables $X_1, ..., X_n$ whose distribution is defined by 2 parameters, I have found the MLE estimators. They are $\hat{\mu} = \sum x_i/n$, and $\hat{\lambda}$ given by $$ \frac{...
0
votes
0answers
39 views

Bayesian Estimation: calculating an integral

I am reading a book on Bayesian filtering and I have a question regarding calculating transition density $p(X_t|X_{t-1})$. My question is how the term $p(X_t|X_{t-1}, V_{t}=v)$ is converted to the ...
1
vote
1answer
30 views

'Bounds' on the Covariance Matrix

We define covariance of random vector ${\bf X}$ as \begin{align} Cov({\bf X})=E \left[ \left( {\bf X}-E[{\bf X}] \right) \left( {\bf X}-E[{\bf X}] \right)^T \right]. \end{align} In the scalar case ...
0
votes
0answers
8 views

A question about Estimation problem in digital communication setup.

I originally asked this problem here http://dsp.stackexchange.com/questions/31503/estimation-problem-for-m-ary-pam-transmission-over-awgn-channel-problem I would appreciate if someone can take a ...
2
votes
2answers
25 views

Trying to find the MLE of $\tau$

Let $\tau = \int x \,dF(x),$ and I want to find the MLE of $\tau$ given $X_1,\ldots,X_n \sim \mathrm{Uniform}(a,b).$ I am not entirely sure, but I would imagine that $\tau = \int x \, dF(x) = \int_a^b ...
0
votes
1answer
21 views

Which test statistic is better for testing population mean?

Consider the following situation: a stochastic variable $X: (\Omega, \mathcal{F}) \to (\mathbb{R}, \mathcal{R})$ is known to be normally distributed with some mean $\mu$ and some variance $\sigma^2$. ...
1
vote
1answer
28 views

For $X_1,X_2, \cdots,X_n$ ~ Poisson($\lambda$), find UMVU estimator for $\lambda^k$ ($k=1,2$,…)

I have some questions about this problem as I'm reviewing for a qual. Our TA provided us with a solution, but I don't understand what is going on: So it looks like they are trying to find an ...
0
votes
0answers
32 views

Expected value of maximum likelihood estimator of a Bernoulli random variable

While reading the text from Keith H. Thompson on the Estimation of the Proportion of Vectors in a Natural Population of Insects, I came across the following part where I don't understand everything. ...
1
vote
0answers
15 views

expected 1 norm of a normal vector

Given a normal vector $X$ such that $\mathbb{E}(X)=0$ and $Cov(X)=Id$, is it possible to get an expression for $$\mathbb{E}(\|A X\|_1)$$ where $A$ is a given matrix. I know that in dimension 1, we ...
0
votes
0answers
10 views

If a family of densities is not complete then is it necessary that there isn't any MVUE?

The question is about the truth of this statement: "If the family $\{f(x;\theta):\theta\in\Omega\}$ is not complete, then there doesn't exist any MVUE" MVUE is an abbreviation for "Minimum Variance ...
1
vote
0answers
19 views

The Cramer-Rao Lower Bound proof

Let $X_1, . . . , X_n$ be i.i.d. with density function $f (x|θ)$. Let $T = t (X_1, . . . , X_n)$ be an unbiased estimate of $θ$. Then, under smoothness assumptions on $f (x|θ)$, $$Var(T) >= \frac{1}...
0
votes
2answers
29 views

Optimal estimation of the fusion of two measurements

Suppose I have a sensor measuring a quantity $\text R$. For example the sensor could be a radar estimating the range of a target. We can write: $$R(t)=r(t)+\nu_0(t)$$ where $r(t)$ is the real range ...
0
votes
1answer
17 views

An issue related to the expectation maximization algorithm for a coin toss experiment

I just read a very nicely written introduction paper for the expectation maximisation algorithm published in Nature biotechnology by Do and Batzoglou (http://www.nature.com/nbt/journal/v26/n8/full/...
0
votes
0answers
6 views

Finding the norm of estimation error asymptotically

Let $\theta \in \mathbb{R}^p$ be such that it has uniform distribution on the set of standard unit vectors $\{\tau e_1,\ldots,\tau e_p\}$, for $\tau=\sqrt{(2-\varepsilon)\log p}, \varepsilon>0$. ...
2
votes
1answer
45 views

Maximum Likelihood of single observation

I'm stumped on this problem... I keep getting an undefined answer of having to solve -20 = 0. The likelihood function I get is $e^{-20\alpha}$. So I have $y_i=$ $ \begin{cases} 1& w/...
0
votes
0answers
10 views

Distribution of sample minimum after bivariate selection (double truncation)

Let $X$ and $Y$ be two RVs with joint distribution $$ (X,Y)\sim \text{Normal}(\mu,\Sigma) $$ Suppose that there is selection on $X$ and $Y$, such that we observe a vector of realisations of $X$, $\{...
0
votes
0answers
26 views

Number of measurements for least squares and relation to maximum likelihood

I have a simple overdetermined system of equations: $$ y = Xc + e $$ $y, e \in \mathbb{R}^n$, $c \in \mathbb{R}^m$, $X \in \mathbb{R}^{n \times m}$, $e \sim \mathcal{N}(o,\sigma^2)$, $n>>m$ ...
0
votes
1answer
31 views

What is an estimator?

If $p_y$ is a probability function for a density, which depends on the value of $y$ (for example, $y$ might be the mean in the poisson distribution). Assuming that $y$ is random -- i.e. unknown -- ...
0
votes
1answer
42 views

Prove the consistency of Gamma distribution estimators

Given $X$ a random variable in a Gamma distribution, $f(x ; \alpha,\beta)$, and: $E(X) = \alpha \beta$ $Var(X) = \alpha \beta^2$ $\hat \alpha = $$\bar X \over \beta$ $\hat \beta = $$\frac {n \bar ...
1
vote
0answers
65 views

Estimating a sparse vector: Mean squared error when support known

I was reading this paper ("How well can we estimate a sparse vector?" by Candès and Davenport, link: http://arxiv.org/pdf/1104.5246v5.pdf). They consider the problem of estimating a $k$-sparse vector $...
1
vote
1answer
44 views

How to find a MVUE for a certain function of a parameter

The following is one of the exercises from my course in statistics Let $X_1, \ldots, X_n$ be a random sample from a Poisson distribution with parameter $\theta > 0$. Find the MVUE for $q(\theta)...
2
votes
0answers
26 views

Finding minimum energy graph, subject to constraints

I imagine there's a known algorithm for this, but am not totally sure what to search for, and so my search didn't turn up much. Basically, I have have a set of $N$ nodes $\hat x_i $ in a graph $\hat ...
0
votes
0answers
52 views

Fitting discrete distribution with normal-inverse gaussian weights

I regard a rather curious discrete distribution $X$ on $(1,2,...)$. Its weights are given by $P(X=i)=P_{NIG}(i)-P_{NIG}(i-1)$ where $P_{NIG}$ denotes the cumulative distribution function of the Normal-...
2
votes
1answer
46 views

How many data points are “enough” for linear regression?

I have data points $(x_t,y_t)$ generated from $y_t = a + b x_t + \epsilon$ where $\epsilon$ is gaussian error term with zero mean and unknown variance. I want to estimate coefficients $a$ and $b$ but ...
0
votes
0answers
15 views

Determining wave algorithm based on sine wave

I have some data that I've noticed conforms to a sine wave and I want to approximate it as closely as I can. In the graph, the blue line is the data I want to model as closely as possible. From ...
1
vote
2answers
36 views

Asymptotically unbiased estimator for 1/p in Bernouilli distribution?

Suppose I have a sample of $n$ independent stochastic variables, each Bernouilli distributed with parameter $p$ (you may assume $0 < p <1$). I was wondering if there exist (asymptotically) ...
0
votes
0answers
17 views

Intuition behind sampling distributions – specific case

I'm still trying to understand the basics of understanding the intuition of sampling distributions and calculating the sampling distributions of common estimators. For example, I understand the ...
1
vote
0answers
21 views

Can I bound $P[R > x + \epsilon]$ independently of R?

I have this probability distribution: $P[\Theta < \varphi] = \frac{\varphi}{\pi}$ for $\phi \in [0,\pi]$. Now I have $n$ samples of $D = R\Theta$ i.i.d. ($R>0$) and I want to estimate $R$ as $\...
1
vote
0answers
17 views

How to use MLE method for non-distribution function?

I understand that maximum likelihood estimation (MLE) method is normally used with distribution function. However is there anyway around I can do to use MLE for a function which is not distribution ...
0
votes
0answers
11 views

Fisher's information better than covariance matrix in estimators

I know Fishers information matrix is the inverse of covariance matrix. But why is it better to use fishers information matrix instead of covariance matrix in the case of distributed sensor networks?
0
votes
0answers
30 views

Maximum Likelihood Estimator of exponential of L2 norm

given the observed data $x = (x_1; x_2; \cdots; x_n)^T$ , the likelihood function p(x; $\theta$) can be charaterized as $$p(x; \theta) = \alpha(x) e^{ ||x - \hat{\theta}||_2} $$ where $\hat{\theta} =...
0
votes
0answers
45 views

maximum likelihood estimation of exponential and polynomial components model

I tried to find the maximum likelihood estimator and MMSE of the non linear model but I got stuck. Can you help me to explain it? The output of a system can be modeled using a combination of ...
0
votes
0answers
41 views

Compound Poisson process estimation

Let the stock price $S_t$ follows the following equation: \begin{equation} d\log S_t = \sigma _t dW_t, \end{equation} where $W_t$ is a Wiener process and \begin{equation} \sigma _t^2 = \sigma ^2 \exp ...
0
votes
0answers
49 views

Nonparametric Estimation of the Hazard Ratio

I am not sure if this is appropriate here, but I'm hoping someone would be able to help. The following is an excerpt from the following paper "Nonparametric Estimation of the Hazard Ratio" by H.T....
1
vote
0answers
46 views

Taylor of $\ln(f(exp(x))))$?

Let $ f(x) = \sum a_n x^n$ Such that The $a_n$ are real and $f(a),f ' (a) , f " (a) > 0 $ for any real $a > 0$. Let $ \ln(f(exp(x))) = \sum b_n x^n $. Let $c_n = a_n - b_n$. For a given $f$ ...
0
votes
1answer
35 views

Help me understand this matrix derivative (for the LS estimation proof)

I'm trying to understand this proof of LS estimation, but I've never studied matrix calculus. I've managed to find a couple of identities on the web and and I see how to get the first part of the ...
1
vote
2answers
86 views

Is there an iterative way to evaluate least squares estimation?

Suppose to have a set of data $\{y_i, u_i\}_{i=1}^m$, where $y_i \in \mathbb{R}$ and $u_i \in \mathbb{R}^n$. The claim is that $$y_i = u_i^\top \theta + \varepsilon$$ where $\theta \in \mathbb{R}^n$...
1
vote
0answers
27 views

Neyman Pearson rule but not a Bayes rule

Consider a binary hypothesis testing problem of $P_0$ vs. $P_1$ under uniform costs. Let $r(\delta,\pi)$ denote the risk line for any decision rule $\delta$ and prior $\pi$, i.e, $r(\delta,\pi)=\pi ...
0
votes
0answers
28 views

How could an estimator be biased but consistent according to mathematical definition?

According to the definition, an estimator can be biased, if $E_{\theta}[\hat{\theta}]\ne\theta$, with $\theta$ as parameter for a distribution we want to get from samples. While the estimator can be ...
0
votes
0answers
13 views

Estimation of binomial probabilities $f(r)$ over $r \in [0,\frac{1}{2}]$

I want to fit a (decreasing) univariate function, \begin{equation} f(r), \end{equation} over $r \in [0,\frac{1}{2}]$ to a series ($r =\frac{1}{100}, \frac{2}{100}, \frac{3}{100} ,\ldots,\frac{1}{2}$) ...
0
votes
1answer
50 views

Proving that a statistic is not sufficient (uniform case).

Let $X=(X_1,...,X_n)$ be i.i.d. $U(0,\theta)$. How to show that $$\frac{2}{n}\sum_{i=1}^{n}X_i$$ is not a sufficient statistic? I have already proven that $\max_{i=1,...,n}X_i$ is a sufficient ...
0
votes
1answer
37 views

IMPROVED - Proving that a statistics is not sufficient (Gaussian case).

Let $X=(X_1,...,X_n)$ be i.i.d. $N(0,\sigma^2)$. How to show that $$\frac{2}{n}\sum_{i=1}^{n}X_i$$ is not a sufficient statistic? I have already proven that $\max_{i=1,...,n}X_i$ is a sufficient ...
0
votes
2answers
26 views

Show that $\hat{\mu}$ has minimal variance

So two independent analyses of a content in a water sample have been made using two different methods, both without systematical errors but with different standard deviations. Method $B$ is assumed to ...
2
votes
2answers
45 views

Is “non-random parameter estimation” the same thing as maximum likelihood estimation?

In one book and a few papers, mostly on navigational tracking, I have found reference to the method of "non-random parameter estimation" but this term is not on the Wikipedia and not in a lot of ...
0
votes
0answers
13 views

How to define a likelihood function for an EM algorithm

Assuming $A$ a set of vectors from a normal distribution, and $X$ a projection matrix and $B$ a set of projected vectors of $A$ using $X$: $B=A*X$ Using an EM approach and by initializing X from ...
0
votes
0answers
20 views

Checking if estimators are sufficient

For an i.i.d. sample of random variables Xi distributed according to a normal distribution, known variance. I found a sufficient statistic—the sample mean. How do I check if other statistic like (...
0
votes
2answers
53 views

check if estimation is unbiased?

Assume we that we calculate the expected value of some measurements $x=\dfrac {x_1 + x_2 + x_3 + x_4} 4$. what if we dont include $x_3$ and $x_4$, but instead we use $x_2$ as $x_3$ and $x_4$. Then We ...
0
votes
0answers
29 views

Error Covariance of Minimum-Variance Estimate

I'm working my way through Luenberger's "Optimization by Vector Space Methods". On chapter 4, "Least-Squares Estimation", Section 4.5., Theorem 1, Luenberger shows that given a measurement setup of ...