Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.
11
votes
2answers
542 views
You see a route 14 bus on the moon. What is the most likely number of bus routes on the moon?
This question was asked on a forum and while many argued that the answer is 14 (since the probability of you seeing bus 14 is maximum in this case), I argued against it that they were working ...
7
votes
1answer
1k views
Minimum variance unbiased estimator for scale parameter of a certain gamma distribution
Let $X_1, X_2, ..., X_n$ be a random sample from a distribution with p.d.f.,
$$f(x;\theta)=\theta^2xe^{-x\theta} ; 0<x<\infty, \theta>0$$ Obtain minimum variance unbiased estimator of ...
6
votes
2answers
180 views
Intuitive explanation of a definition of the Fisher information
I'm studying statistics. When I read the textbook about Fisher Information, I couldn't understand why the Fisher Information is defined like this:
$$I(\theta)=E_\theta\left[-\frac{\partial^2 ...
6
votes
2answers
119 views
Why should Gaussian noise have fractal dimension of 1.5?
In a paper I'm trying to understand, the following time series is generated as "simulated data":
$$Y(i)=\sum_{j=1}^{1000+i}Z(j) \:\:\: ; \:\:\: (i=1,2,...,N)$$
where $Z(j)$ is a Gaussian noise with ...
5
votes
3answers
257 views
Estimating population size
Let's suppose there are $n$ real numbers $a_0 < ... < a_n$ uniformly selected from interval [0, 1). If one knows $k$ numbers on consecutive positions $a_i < ... < a_{i+k-1}$ how good is ...
5
votes
1answer
139 views
Finding the joint distribution of $X_{1:n}$ and $\overline{X}$
I need to show that, given a random sample of independent variables $X_1, ... , X_n$, each following a distribution EXP($\theta$,$\eta$), that is, ...
4
votes
1answer
127 views
Showing that $X_{1:n}$ is sufficient for $\eta$, by factorization
I'm asked to show that $X_{1:n}$ (the minimum order statistic) is sufficient for $\eta$, in the case of a random sample $(X_1, ... , X_n)$ where $X_i\sim EXP(1,\eta$) (this is the two-parameter ...
3
votes
2answers
2k views
Difference between logarithm of an expectation value and expectation value of a logarithm
Assuming I have a always positive random variable $X$, $X \in \mathbb{R}$, $X > 0$. Then I am now interested in the difference between the following two expectation values:
$E \left[ \ln X ...
3
votes
3answers
421 views
Estimating the Gamma function to high precision efficiently?
I know there are several approximations of the Gamma function that provide decent approximations of this function.
I was wondering, how can I efficiently estimate specific values of the Gamma ...
3
votes
2answers
252 views
How can I compare two Markov processes?
There is a discrete-time irreductible Markov process with $r$ possible states. $k$ observations were performed. At each observation a state of process was determined.
$T_0 = \lbrace 0,1,\dots ...
3
votes
2answers
149 views
estimate the perimeter of the island
I'm assigned a task involving solving a problem that can be described as follows: Suppose I'm driving a car around a lake. In the lake there is an island of irregular shape. I have a GPS with me in ...
3
votes
1answer
179 views
Unstable linear inverse problem: which “dampening” Tikhonov matrix should I use?
A linear inverse problem is given by:
$\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$
where d: observed data, A: theory operator, m: unknown model and e: error.
The Least Square Error (LSE) model ...
3
votes
1answer
99 views
Solving perturbed polynomial equations
Rather than asking the most general question possible, I will frame it in terms of what I believe is an illustrative example.
Let $\epsilon>0$ be a small parameter, let $a,b>0$ and $x\in ...
3
votes
2answers
382 views
biased Maximum Likelihood estimation
Given $N$ points ($x_k$, k from $1$ to $N$) generated from a normal distribution (1-dimensional case) with known mean $\mu$, the Maximum Likelihood estimation of the variance is ...
3
votes
1answer
133 views
MLE of the mean of a heteroscedastic Gaussian time series
Suppose we observe $Y_i\sim \mathcal{N}(\theta_0 + \theta_1 x_i, \sigma_i^2)$, with $x_i$ and $\sigma_i^2$ known for all $i = 1,\ldots,n$ and $Y_1,\ldots,Y_n$ independent. Assume $\theta_0$ is ...
3
votes
0answers
91 views
Estimate number of distinct items
I have a large array of $n$ integers, some of which may be repeated, and I want to estimate how many distinct integers are in the array. Say the number of distinct integers is $N$. I can sample with ...
2
votes
2answers
285 views
Inverse problem from pdes
A linear inverse problem is given by:
$\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$
where d: observed data, A: theory operator, m: unknown model and e: error.
To minimize the effect of the noise; ...
2
votes
2answers
134 views
Is $\frac{m-1}{x}$ an unbiased estimator of $\theta$ for given pdf?
Let $X$ be a continuous random variable with pdf,
$$f(x;\theta)=\frac {\theta^m.x^{m-1}e^{-\theta x}} {(m-1)!} ; x\geq0, \theta>0$$
Is $\frac{m-1}{x}$ an unbiased estimator of $\theta$ for given ...
2
votes
1answer
133 views
Maximum Likelihood Estimation
For iid random variables from a distribution with p.d.f.
$$f(x;\theta_1,\theta_2)=\frac{1}{\theta_2}\exp\bigg(-\frac{(x-\theta_1)}{\theta_2}\bigg), \quad x>\theta_1, ...
2
votes
2answers
78 views
Have spd $(A^TA)$ and $(B^TB)$, need $A^TB$.
Given two symmetric positive definite matrices $(A^TA)$ and $(B^TB)$ I need to compute $A^TB$.
$A$ and $B$ are not given directly.
$(A^TA)$ and $(B^TB)$ have the same dimensions. $A$ and $B$ are ...
2
votes
1answer
755 views
Prove the sample variance is an unbiased estimator
I'm trying to prove that the sample variance is an unbiased estimator.
I know that I need to find the expected value of the sample variance estimator $$\sum_i\frac{(M_i - \bar{M})^2}{n-1}$$ but I get ...
2
votes
2answers
433 views
Some expectation values for a Gamma distribution
Assuming I have a Gamma distributed random Variable $x \sim Gamma( \alpha, \beta )$. Now I like to have the following two expectation values (integrals):
$E \left[ x \ln x \right]$
$E \left[ \ln ...
2
votes
2answers
77 views
Expected value of a max
We have a roulette with the circumference $a$. We spin the roulette 10 times and we measure 10 distances, $x_1,\ldots,x_{10}$, from a predefined zero-point. We can assume that those distances are ...
2
votes
0answers
46 views
Cramer-Rao bounds for noncentral $\chi^2$ distribution parameters estimates
In many practical problems I came across the necessity to estimate the parameters of noncentral $\chi^2$ distribution (for example the degrees of freedom or non-centrality parameter etc.). Found some ...
2
votes
1answer
279 views
How do I find the MLE of $\theta$ when x is dependent on $\theta$?
Let $X_{1},X_{2},...,X_{n}$ represent a random sample from a distribution with pdf:
$f(x; \theta)=e^{-(x-\theta)}, \theta \le x<\infty, -\infty<\theta<\infty$ | zero elsewhere
I need to ...
2
votes
0answers
135 views
Fisher Information and minimum variance estimators
I am trying to understand what can be proved about minimum variance estimators. I have changed the question to make it more specific.
Let us assume we have some finite set $S$ of elements and we just ...
2
votes
1answer
43 views
Parameter optimization in probabilistic models
Task: Suppose we model a variable $y = Wx + \mu$ as a linear transformation of $x$ plus some Gaussian noise $\mu\sim\mathcal N(0,\sigma I)$. Our aim is to minimize the estimation error of $x$ given ...
2
votes
1answer
156 views
Determine whether a statistic is sufficient, given the probability density
Let $X_1, X_2, \dots, X_n$ be a sample of i.i.d. random variables, with density $$f_\theta=\frac{2}{3\theta}\left(1-\frac{x}{3\theta}\right) $$ for $0 < x < 3\theta$. And
$f_\theta=0$ if $ x ...
2
votes
0answers
76 views
Likelihood Function of Random Process
Given the following data:
$$
x(t) = A + \omega(t)
$$
where $ \omega(t) $ is an AWGN with zero mean, what would be likelihood function $p(x(t);A)$?
I know it could be proven to be:
$$
p(x;A) = C ...
1
vote
2answers
89 views
Show that estimates are unbiased
The following is a problem in my book that I don't really understand:
We take a random sample: $x_1,x_2,\ldots,x_n$ from a population that is $N(μ,σ)$ where $\mu$ and $\sigma$ are unknown.
We build ...
1
vote
1answer
71 views
Sufficient Estimators and Generalized Likelihood Ratios
If you can make the assumption that a sufficient statistic exists for some parameter - let's call it $\theta$.
How would you show that the critical region of a likelihood ratio test will depend on ...
1
vote
2answers
371 views
Question about unbiased estimator
$X_1,X_2,\ldots,X_n$ are iid random variables $B(1,\theta)$ where $0< \theta<1$. Let $w = 1$ if $\sum_i X_i = n$ and $0$ otherwise. What is the best unbiased estimator of $\theta^2$.
Attempt:
...
1
vote
1answer
70 views
Rao-Blackwell unbiased estimator geometric distribution
I'm looking at review questions and having trouble with this one!
Let $X_1,\ldots,X_n$ be i.i.d. geometric R.V.s with the pmf: $(1-p)^{x-1}p$, for $x=1,2,\ldots$ and $0<p<1$.
I need to use ...
1
vote
2answers
60 views
Filter to obtain MMSE of data from Gaussian vector
Data sampled at two time instances giving bivariate Gaussian vector $X=(X_1,X_2)^T$ with
$f(x_1,x_2)=\exp(-(x_1^2+1.8x_1x_2+x_2^2)/0.38)/2\pi \sqrt{0.19}$
Data measured in noisy environment with ...
1
vote
1answer
162 views
How to find parameters from an equation
the question could be "stupid" but i don't know if it is feasible or not, please don't kill me :)
EDIT WITH NEW FORMULAS!
I have an equation like this: (unfortunatly in my first Q&A i cannot ...
1
vote
3answers
527 views
How to estimate failure probability from count until first failure?
What would be the formula to estimate the rate of failure of some test as a percentage chance of failure from the number of runs of the test until the first failure was seen?
For example, considering ...
1
vote
1answer
80 views
The distribution when combining two samples together?
Suppose $X\sim N(0,{\sigma}^2)$ and $Y\sim N(0,{2\sigma}^2)$ . $X_1, ..., X_m$ are the samples from $X$ and $Y_1, ..., Y_n$ are the samples from $Y$. And then combine two samples as a new sample ...
1
vote
1answer
135 views
Using the MSE criterion, which is a better estimator for $\Theta^2$?
Question: Let $T_1$ and $T_2$ be independent unbiased estimators of a parameter $\Theta$.
Assume that $\operatorname{Var}(T_2) = \operatorname{Var}(T_1)$.
Using the MSE critertion, define which is a ...
1
vote
1answer
49 views
Proving that the sum of Good-Turing estimators is $1$
I want to know how to go about proving that the Good-Turing estimator has a total probability of $1$. I have seen this proof (page 2) but I found unclear the first step:
$$\sum_j \theta[j] = \sum_r ...
1
vote
2answers
122 views
Fast variance calculation
Suppose to have a sequence $X$ of $m$ samples and for each $i^{th}$ sample you want to calculate a local mean $\mu_{X}(i)$ and a local variance $\sigma^2_{X}(i)$ estimation over $n \ll m$ samples of ...
1
vote
1answer
171 views
Gauss-Markov estimator properties
Consider a linear model
$$
y = Ab+n,
$$
where $b \in \mathbb{R^m}$ is a parameter to be estimated, $n \in \mathbb{R^{n}}$ is a noise with mean $\mathbb{E}n = m_{n}$ and with covariation matrix ...
1
vote
1answer
90 views
How to find out the control function of a cosine wave?
I have a system which is sampling at 100Hz. There is only one input for the system. The output is similar to cosine waveforms with varying frequency. I have no clue how to find out the exact formula ...
1
vote
1answer
116 views
Maximum Likelihood optimal threshold
I have a decision (detection) problem trying to decide between symbols ${0,2}$. I have the two probability density functions: $$
f(z|s=0) =
\begin{cases}
0.25z + 0.5, & -2\le\ z <0 \\
...
1
vote
1answer
128 views
Likelihood function estimating two different means
I completely understand how to find the likelihood functions of simple pdfs. However, how would you attempt to find the likelihood function of a pdf with a negative exponential function with two ...
1
vote
1answer
31 views
Improving related estimates
There are three underlying quantities $x$, $y$, and $a$, where $x$ and $y$ are vectors, and $a$ is a scalar. They are related by $x = ay$.
We get noisy observations, $x_0,y_0$. We want to find $a$, ...
1
vote
1answer
57 views
Maximum likelihood estimators, hypergeometric and binomial
I'm trying to solve a two part problem. The set up is as follows: consider a bag with $\theta$ red marbles and $7-\theta$ blue marbles, with $\theta$ being unknown. Let $x$ denote the number of red ...
1
vote
0answers
17 views
Is there a way to estimate the range of fitting coefficients from only the data?
Considering an approximation $f$ for a set of $N$ data points $(x,y)$ using, for example, $M$ radial basis functions at arbitrary sites in the domain
$f_i = \sum_{j=1} ^M c_j\phi(||x_i-x_j||)$
where ...
1
vote
0answers
29 views
Hypothesis testing problem of Normal distributions.
Consider the following Hypothesis Testing problem:
Hypothesis $H_0$ : $X \sim N(\mu_0, \sigma_0)$. Mean $\mu_0$ is known but only upper and lower bounds on $\sigma_0$ are known.
Hypothesis $H_1$ : ...
1
vote
0answers
33 views
Cramer-Rao bound for $\chi^2$ distribution parameter estimates.
I've stuck in unpleasant problem with noncentral $\chi^2$ distribution.
I work with random variables, distributed as $\chi^2_{\nu}(\lambda)$, where $\nu$ is the degree of freedom and $\lambda$ is ...
1
vote
1answer
53 views
biasedness/unbiasedness of an MLE.
To show whether an MLE I just found is biased/unbiased, would I need to find the expectation of the answer? Plus would I do this by integrating $\text{MLE} \cdot \text{pdf}$.
My MLE is $ ...
