Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

0
votes
2answers
25 views

Optimal estimation of the fusion of two measurements

Suppose I have a sensor measuring a quantity $\text R$. For example the sensor could be a radar estimating the range of a target. We can write: $$R(t)=r(t)+\nu_0(t)$$ where $r(t)$ is the real range ...
0
votes
1answer
16 views

An issue related to the expectation maximization algorithm for a coin toss experiment

I just read a very nicely written introduction paper for the expectation maximisation algorithm published in Nature biotechnology by Do and Batzoglou ...
0
votes
0answers
6 views

Finding the norm of estimation error asymptotically

Let $\theta \in \mathbb{R}^p$ be such that it has uniform distribution on the set of standard unit vectors $\{\tau e_1,\ldots,\tau e_p\}$, for $\tau=\sqrt{(2-\varepsilon)\log p}, \varepsilon>0$. ...
2
votes
1answer
28 views

Maximum Likelihood of single observation

I'm stumped on this problem... I keep getting an undefined answer of having to solve -20 = 0. The likelihood function I get is $e^{-20\alpha}$. So I have $y_i=$ $ \begin{cases} 1& ...
0
votes
0answers
8 views

Distribution of sample minimum after bivariate selection (double truncation)

Let $X$ and $Y$ be two RVs with joint distribution $$ (X,Y)\sim \text{Normal}(\mu,\Sigma) $$ Suppose that there is selection on $X$ and $Y$, such that we observe a vector of realisations of $X$, ...
0
votes
0answers
22 views

Number of measurements for least squares and relation to maximum likelihood

I have a simple overdetermined system of equations: $$ y = Xc + e $$ $y, e \in \mathbb{R}^n$, $c \in \mathbb{R}^m$, $X \in \mathbb{R}^{n \times m}$, $e \sim \mathcal{N}(o,\sigma^2)$, $n>>m$ ...
0
votes
1answer
29 views

What is an estimator?

If $p_y$ is a probability function for a density, which depends on the value of $y$ (for example, $y$ might be the mean in the poisson distribution). Assuming that $y$ is random -- i.e. unknown -- ...
0
votes
1answer
40 views

Prove the consistency of Gamma distribution estimators

Given $X$ a random variable in a Gamma distribution, $f(x ; \alpha,\beta)$, and: $E(X) = \alpha \beta$ $Var(X) = \alpha \beta^2$ $\hat \alpha = $$\bar X \over \beta$ $\hat \beta = $$\frac {n \bar ...
1
vote
0answers
32 views

Estimating a sparse vector: Mean squared error when support known

I was reading this paper ("How well can we estimate a sparse vector?" by Candès and Davenport, link: http://arxiv.org/pdf/1104.5246v5.pdf). They consider the problem of estimating a $k$-sparse vector ...
0
votes
0answers
23 views

Relation of sufficent statistic to random mechanism in constuction of a randomized estimator?

After reading a about randomized estimators and sufficent statistics, the question weather the random mechanism is determined uniquely by the statistic struck me, but I cant seem to get an answer to ...
1
vote
1answer
28 views

How to find a MVUE for a certain function of a parameter

The following is one of the exercises from my course in statistics Let $X_1, \ldots, X_n$ be a random sample from a Poisson distribution with parameter $\theta > 0$. Find the MVUE for ...
2
votes
0answers
25 views

Finding minimum energy graph, subject to constraints

I imagine there's a known algorithm for this, but am not totally sure what to search for, and so my search didn't turn up much. Basically, I have have a set of $N$ nodes $\hat x_i $ in a graph $\hat ...
0
votes
0answers
22 views

Other method instead of using OLS estimation

enter image description here I get that the resulting beta* is the same as the estimated beta 1. But I think something is wrong. If the researcher is true, then why do we learn the OLS estimation, ...
0
votes
0answers
48 views

Fitting discrete distribution with normal-inverse gaussian weights

I regard a rather curious discrete distribution $X$ on $(1,2,...)$. Its weights are given by $P(X=i)=P_{NIG}(i)-P_{NIG}(i-1)$ where $P_{NIG}$ denotes the cumulative distribution function of the ...
2
votes
1answer
42 views

How many data points are “enough” for linear regression?

I have data points $(x_t,y_t)$ generated from $y_t = a + b x_t + \epsilon$ where $\epsilon$ is gaussian error term with zero mean and unknown variance. I want to estimate coefficients $a$ and $b$ but ...
0
votes
0answers
13 views

Determining wave algorithm based on sine wave

I have some data that I've noticed conforms to a sine wave and I want to approximate it as closely as I can. In the graph, the blue line is the data I want to model as closely as possible. From ...
1
vote
2answers
34 views

Asymptotically unbiased estimator for 1/p in Bernouilli distribution?

Suppose I have a sample of $n$ independent stochastic variables, each Bernouilli distributed with parameter $p$ (you may assume $0 < p <1$). I was wondering if there exist (asymptotically) ...
0
votes
0answers
15 views

Intuition behind sampling distributions – specific case

I'm still trying to understand the basics of understanding the intuition of sampling distributions and calculating the sampling distributions of common estimators. For example, I understand the ...
1
vote
0answers
20 views

Can I bound $P[R > x + \epsilon]$ independently of R?

I have this probability distribution: $P[\Theta < \varphi] = \frac{\varphi}{\pi}$ for $\phi \in [0,\pi]$. Now I have $n$ samples of $D = R\Theta$ i.i.d. ($R>0$) and I want to estimate $R$ as ...
1
vote
0answers
17 views

How to use MLE method for non-distribution function?

I understand that maximum likelihood estimation (MLE) method is normally used with distribution function. However is there anyway around I can do to use MLE for a function which is not distribution ...
0
votes
0answers
9 views

Fisher's information better than covariance matrix in estimators

I know Fishers information matrix is the inverse of covariance matrix. But why is it better to use fishers information matrix instead of covariance matrix in the case of distributed sensor networks?
0
votes
0answers
23 views

Maximum Likelihood Estimator of exponential of L2 norm

given the observed data $x = (x_1; x_2; \cdots; x_n)^T$ , the likelihood function p(x; $\theta$) can be charaterized as $$p(x; \theta) = \alpha(x) e^{ ||x - \hat{\theta}||_2} $$ where $\hat{\theta} ...
0
votes
0answers
43 views

maximum likelihood estimation of exponential and polynomial components model

I tried to find the maximum likelihood estimator and MMSE of the non linear model but I got stuck. Can you help me to explain it? The output of a system can be modeled using a combination of ...
0
votes
0answers
38 views

Compound Poisson process estimation

Let the stock price $S_t$ follows the following equation: \begin{equation} d\log S_t = \sigma _t dW_t, \end{equation} where $W_t$ is a Wiener process and \begin{equation} \sigma _t^2 = \sigma ^2 \exp ...
0
votes
0answers
49 views

Nonparametric Estimation of the Hazard Ratio

I am not sure if this is appropriate here, but I'm hoping someone would be able to help. The following is an excerpt from the following paper "Nonparametric Estimation of the Hazard Ratio" by ...
1
vote
0answers
46 views

Taylor of $\ln(f(exp(x))))$?

Let $ f(x) = \sum a_n x^n$ Such that The $a_n$ are real and $f(a),f ' (a) , f " (a) > 0 $ for any real $a > 0$. Let $ \ln(f(exp(x))) = \sum b_n x^n $. Let $c_n = a_n - b_n$. For a given $f$ ...
0
votes
1answer
32 views

Help me understand this matrix derivative (for the LS estimation proof)

I'm trying to understand this proof of LS estimation, but I've never studied matrix calculus. I've managed to find a couple of identities on the web and and I see how to get the first part of the ...
1
vote
2answers
32 views

Is there an iterative way to evaluate least squares estimation?

Suppose to have a set of data $\{y_i, u_i\}_{i=1}^m$, where $y_i \in \mathbb{R}$ and $u_i \in \mathbb{R}^n$. The claim is that $$y_i = u_i^\top \theta + \varepsilon$$ where $\theta \in ...
1
vote
0answers
20 views

Neyman Pearson rule but not a Bayes rule

Consider a binary hypothesis testing problem of $P_0$ vs. $P_1$ under uniform costs. Let $r(\delta,\pi)$ denote the risk line for any decision rule $\delta$ and prior $\pi$, i.e, $r(\delta,\pi)=\pi ...
0
votes
0answers
27 views

How could an estimator be biased but consistent according to mathematical definition?

According to the definition, an estimator can be biased, if $E_{\theta}[\hat{\theta}]\ne\theta$, with $\theta$ as parameter for a distribution we want to get from samples. While the estimator can be ...
0
votes
0answers
13 views

Estimation of binomial probabilities $f(r)$ over $r \in [0,\frac{1}{2}]$

I want to fit a (decreasing) univariate function, \begin{equation} f(r), \end{equation} over $r \in [0,\frac{1}{2}]$ to a series ($r =\frac{1}{100}, \frac{2}{100}, \frac{3}{100} ,\ldots,\frac{1}{2}$) ...
0
votes
1answer
42 views

Proving that a statistic is not sufficient (uniform case).

Let $X=(X_1,...,X_n)$ be i.i.d. $U(0,\theta)$. How to show that $$\frac{2}{n}\sum_{i=1}^{n}X_i$$ is not a sufficient statistic? I have already proven that $\max_{i=1,...,n}X_i$ is a sufficient ...
0
votes
1answer
35 views

IMPROVED - Proving that a statistics is not sufficient (Gaussian case).

Let $X=(X_1,...,X_n)$ be i.i.d. $N(0,\sigma^2)$. How to show that $$\frac{2}{n}\sum_{i=1}^{n}X_i$$ is not a sufficient statistic? I have already proven that $\max_{i=1,...,n}X_i$ is a sufficient ...
0
votes
2answers
26 views

Show that $\hat{\mu}$ has minimal variance

So two independent analyses of a content in a water sample have been made using two different methods, both without systematical errors but with different standard deviations. Method $B$ is assumed to ...
2
votes
2answers
37 views

Is “non-random parameter estimation” the same thing as maximum likelihood estimation?

In one book and a few papers, mostly on navigational tracking, I have found reference to the method of "non-random parameter estimation" but this term is not on the Wikipedia and not in a lot of ...
0
votes
0answers
13 views

How to define a likelihood function for an EM algorithm

Assuming $A$ a set of vectors from a normal distribution, and $X$ a projection matrix and $B$ a set of projected vectors of $A$ using $X$: $B=A*X$ Using an EM approach and by initializing X from ...
0
votes
0answers
18 views

Checking if estimators are sufficient

For an i.i.d. sample of random variables Xi distributed according to a normal distribution, known variance. I found a sufficient statistic—the sample mean. How do I check if other statistic like ...
0
votes
2answers
52 views

check if estimation is unbiased?

Assume we that we calculate the expected value of some measurements $x=\dfrac {x_1 + x_2 + x_3 + x_4} 4$. what if we dont include $x_3$ and $x_4$, but instead we use $x_2$ as $x_3$ and $x_4$. Then We ...
0
votes
0answers
24 views

Error Covariance of Minimum-Variance Estimate

I'm working my way through Luenberger's "Optimization by Vector Space Methods". On chapter 4, "Least-Squares Estimation", Section 4.5., Theorem 1, Luenberger shows that given a measurement setup of ...
0
votes
0answers
18 views

Unbiased estimation of parameter with singular matrix.

Given sample $N_p(A\theta,Q)$, where $\theta, Q$ - unknown. A - known $q*p$ matrix, $rankA = q, q<p$. The question is: how can I find unbiased estimation $\hat\theta$ of $\theta$? It seems easy ...
0
votes
0answers
25 views

Cluster sampling: Compare efficiencies

A company operates from 12 branches, and the numbers of cars, $N_i$ and means $\bar{X}_i$ and variances $S_i^2$ of miles driven last year for each brand, are as follows Branch: $N_i$; $\bar{X}_i$ ; ...
0
votes
0answers
27 views

Simple Random Sampling: Find the variance

I have trouble answering this simple question. There is a total of 280 trees. The assessed total yield is at 432,6 tons. 25 trees are picked at random and their timber yields are accurately ...
1
vote
0answers
31 views

Higher Order Estimation Errors

I well know estimation measure is the so called minimum mean square error (MMSE) defined as: \begin{align} E[|W-\hat{W}(V)|^2] \end{align} where $W$ is a random variable (that we want to estimate) and ...
2
votes
1answer
55 views

What is the concentration result of the entropy?

Let $X_1, X_2, \ldots, X_n$ be i.i.d. binary variables with $Pr(X_i=1)=p$ and $Pr(X_i=0)=1-p$. The famous result about $p$ is $$Pr\left(\left|\frac{1}{n}\sum_{i=1}^n ...
0
votes
1answer
34 views

Maximum Likelihood Estimator of $\theta$

I have the following question I tried to answer I got answer that same like this answer Is this true answer? (Note that: in the question $0<p<\frac{1}{2}$, but in this answer ...
1
vote
1answer
15 views

Does the correlation between the measurement noise influence the result of the LS?

Consider a linear measurement process with some noise: $$y=Hx+v$$ with $$v \sim \mathcal{N}(0,\Sigma)$$ the covariance matrix $\Sigma$ is not a diagonal matrix. As we know, using LS, the $\hat{x}$ is ...
1
vote
1answer
32 views

How do I compute a realization of h(x) given its PDF and covariance?

Desription of problem I've added a picture of what I want to compute. In the nomenclature of the picture, I want to compute a realization of y(x) given the known distributions and constants. Let's ...
1
vote
1answer
30 views

How do you show that the estimator for the covariance matrix is unbiased?

So according to Wikipedia (Here) the sample covariance matrix is an unbiased estimator of the covariance matrix, but how do I prove this mathematically?
0
votes
0answers
9 views

Estimate function given PDF and covariance

Let's say $h(x)$, random variable, represents the height of a surface, with x being the usual x-axis. The probability distribution function is: $P(h) = Ke^{-\frac{h^2}{2s^2}}$ is Gaussian, where $K$ ...
0
votes
0answers
41 views

Minimum-variance and minimum-divergence estimator

Given a parametric family of distributions $\{P_\theta \colon \theta \in \Theta\}$ and a sample $X \sim P_\theta$, an estimator $T^\star(X)$ of the parameter $\theta$ is said to be a uniformly ...