Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

1
vote
1answer
37 views

On the finite expectation of a function

Let $y_i$, $i=1,\dots,n$, be independent Gaussian rv's of mean $\theta_i$ and variance $\sigma^2$ and let $\mathbf{y}:=[y_1,\dots,y_n]^\top$. Consider the function $f\colon \mathbb{R}^n\to ...
2
votes
0answers
23 views

Risk in density estimation: grasping the definition

When generalizing estimators to an entire function what is the space in which we perform the integral to obtain the expected value (with respect to this function)? For example, when estimating ...
1
vote
1answer
57 views

When to use Central Limit Theorem or Cramers Theorem

In for example this paper the authors say The central limit theorem provides an estimate of the probability \begin{align} P\left( \frac{\sum_{i=1}^n X_i - n\mu}{\sigma \sqrt{n}} > x \right) ...
2
votes
2answers
61 views

Convergence Rate of Sample Average Estimator

Let $X_1, X_2,\cdots$ be i.i.d. random variables with $E(X_1) = \mu, Var(X_1) = σ^2> 0$ and let $\bar{X}_n = {X_1 + X_2 + \cdots + X_n \over n}$ be the sample average estimator. Is there a way to ...
1
vote
0answers
27 views

asymptotic unbiasedness of weibull mle

It's known that the MLEs of the two-parameter Weibull distribution scale and shape parameters are not available in a closed form. It is, however, known that they do exist, are unique, and moreover, ...
-1
votes
1answer
51 views

Cumulative distribution function of exponentials

I have the cumulative distribution function $F(x)=(1-e^{-x})\mathbb{1}_{x≥0}$ and want to write the CDF to $F(\frac{x-\mu}{\sigma})$. I have derived ...
0
votes
0answers
63 views

Second derivatives of MLE to multivariate normal distribution

I want to calculate the second derivatives to the MLE's $\hat{\mu}$ and $\hat{\Sigma}$ to confirm that the extremums indeed are maximums to the multivariate normal distribution $$ ...
0
votes
0answers
9 views

Find the spectral density of white noise $\omega$~(0,2)

I am find the spectral density of a white noise given by $\omega$ ~ (0,2). Could you help me to find it? Thank all.
1
vote
2answers
51 views

Variance of sample mean (problems with proof)

Assuming that I have $\{x_1,\ldots, x_N\}$ - an iid (independent identically distributed) sample size $N$ of observations of random variable $\xi$ with unknown mean $m_1$, variance (second central ...
1
vote
0answers
21 views

Estimation of Linear Projection

Given a linear system: $Y=AX+W$ Where: $X$ is the input signal of size $N \times K$ $Y$ is the output signal of size $M \times K$ $A$ is a projection of size $M\times N$; with $M >> N$ $W$ ...
1
vote
1answer
56 views

asymptotic normality and unbiasedness of mle

Suppose $\hat{\theta}_n$ is the MLE for some parameter $\theta$. Suppose also that the MLE is such that the Cramer regularity conditions are fulfilled, and $\hat{\theta}_n$ is asymptotically normal ...
0
votes
1answer
89 views

Sample size required to estimate population proportion with given precision

It plans to conduct a study on the percentage of homeowners who have at least two TVs. What should be the sample size if we want to ensure that $95\%$ of estimation error is less than $0.01$? ...
1
vote
2answers
32 views

How to periodically estimate states of a LTI if the output is measured irregularly?

How can I periodically estimate the states of a discrete linear time-invariant system in the form $$\vec{x}(k+1)=\textbf{A}\vec{x}(k)+\textbf{B}\vec{u}(k)$$ ...
1
vote
1answer
46 views

Linear Regression with independent but non-identical noise

If I have this linear regression equation: $$y=X\beta+\epsilon $$ ($x$ and $\beta$ are vectors) The likelihood function can be written as $$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi ...
1
vote
1answer
41 views

Maximum Likelihood Question

The aim is to find the maximum likelihood estimator for theta. $f(x)$ is given and we can assume that $1\le x\le-1$. I have completed the steps seen in the image, however I am having difficulty ...
1
vote
0answers
27 views

Unbiased estimator for maximum

Assume $n$ independent random variables with unknown distributions $\{X_1,X_2,...,X_n\}$. Multiple "samples" or observations for each of these variables are given (not necessarily with the same ...
0
votes
0answers
7 views

Least square estimator: $N( \beta x_i, \sigma^2)$

Let $ Y_1,...,Y_n$ be i.i.d $N(\beta x_i, \sigma^2) $ with known $ x_i's$. It is asked to find the Mean Squared Estimator for $\beta.$ I didn't understandmuch about this method of pbtaining an ...
0
votes
0answers
14 views

Estimate time it takes the minimum mean cycle cancelling algorithm to converge

This particular algorithm solves the circulation problem, equivalent to the minimum-capacitated flow. My question rather than only from this particular algorithm, but for combinatorial solutions in ...
2
votes
0answers
146 views

Polluting an image with Gaussian anisotropic noise and estimate the covariance matrix

Assume that we have a $d\times d$ grey-scale image represented as a vector $$ \mathbf{x}=(x_1,\ldots,x_D)^T\in[0,255]^D, $$ where $D=d\times d$. We would like to import some noise concerning the ...
0
votes
1answer
30 views

$|p- \dfrac xn|>|q- \dfrac xn|$ $\implies$ $p^x(1-p)^{n-x}<q^x(1-q)^{n-x}$?

If $p,q \in (0,1)$ , and $ n \in \mathbb N$ be given and $x$ be given integer between $0$ and $n$ such that $|p- \dfrac xn|>|q- \dfrac xn|$ , then is it true that ...
4
votes
0answers
246 views

A late-diverging “approximating solution” for a system of functional equations

Peace be upon you, At the end of this question, I have shown that how computing MLE on an i.i.d Beta distributed data, results in the following system \begin{align*} &\begin{cases} ...
0
votes
0answers
17 views

Asymptotic coincidence of the MAP and MMSE estimator

In many works, simulations show that as number of samples increases, the mean-square-error (MSE) of the MAP estimator attains the minimum MSE. Where can I find a theoretical proof to these empirical ...
0
votes
1answer
33 views

Does consistent estimators have in-variance property?

If $(T_n)$ is a sequence of consistent estimators of a parameter $\theta$ ( i.e. for every $ \epsilon >0$ , $\lim_{n \to \infty} P [ \space |T_n -\theta|< \epsilon ]=1$ ) , then is it true that ...
1
vote
1answer
261 views

Maximum likelihood estimate vs likelihood ratio tests?

Can someone explain to me the intuition behind why we need likelihood ratio tests. From my understanding, they make use of maximum likelihood estimators over different parameters space and they are a ...
1
vote
0answers
43 views

Working with the sum of two independent random variables, and estimating a parameter

A network source sends a sequence of zeros and ones, $X_1, X_2, ...$ with $X_i$(iid) Bernoulli with $p = P(X_i = 1), 0 < p < 1$. Due to disturbances the received sequence is $Y_1, Y_2, ...$ ...
0
votes
2answers
47 views

Biased MLE estimate of mean (expectation)

Please give an example of p.m.f. or p.d.f. , the maximum likely-hood estimate of whose mean (expectation) is a biased estimator . Thanks
0
votes
0answers
33 views

What is the problem with this model parameter estimation algorithm?

In a statistical model with parameters $\theta$ and unobserved laten variables $Z$, the model likelihood is $$L(\theta;X)=Pr(X|\theta)=\sum_ZPr(X,Z|\theta)$$ The standard way to estimate $\theta$ ...
0
votes
1answer
83 views

estimation problem for two-parameter weibull distribution

Suppose the two-parameter Weibull distribution is given by the pdf $$ f(x;a,b) = \left(\frac{x}{a}\right)^b\frac{b}{a}\exp\left\{-\left(\frac{x}{a}\right)^b\right\}, $$ where $x,a,b>0$. I am ...
0
votes
0answers
22 views

Optimal combination of multiple estimates of a random variable

For the following estimation problem: y = hx + n, x is the sent data, y is the observation (received data), h is a scaling factor (known), n is an AWGN random variable with zero mean ...
1
vote
1answer
36 views

Shapiro-Wilk test

I am trying to determine if a given sample comes from a Normal distribution. For that purpose I want to perform a Shapiro-Wilk test in the way stated on wikipedia. My concern comes with the vector ...
3
votes
1answer
75 views

Finding an efficient estimator for $\theta$ in $U[0, \theta]$ in terms of the sample maximum

This question appeared in a past exam paper, in the form: Let $X = (X_1\dotsc X_n)\in\mathbb{R}^n$ be an i.i.d. sample from $U[0, \theta], \theta>0$ Apply Rao-Blackwell's theorem to the unbiased ...
1
vote
2answers
340 views

how to calculate vehicle speed using mathematics and Image processing?

i am doing my project in image processing.using segmentation i have detected the moving part(i.e the car) in the video successfully. But now i want to calculate speed of vehicle. in the above ...
2
votes
0answers
48 views

paramter estimation (maximum likelihood) of a mixture density

I have this mixture distribution $f(x) =w \cdot \mathcal{LN}(\mu_1,\sigma) + (1-w)\cdot \mathcal{LN}(\mu_2,\sigma) $ where $\mathcal{LN}(\mu,\sigma)$ is a lognormal distribution. I now have random ...
2
votes
1answer
40 views

Bayes - dual estimation of parameter value and parameter growth

I am trying to find an bayesian approach to the following problem: Image a bucket with 100 white balls and an unknown number of red balls During each year, one can take a sample with replacement of ...
1
vote
0answers
96 views

Expectation of $\cos(\|X\|)$ where $X \sim \mathcal{N}(\mu,\Sigma)$

Do: $$ \int_{-\infty}^\infty \int_{-\infty}^\infty \cos\left(\sqrt{x^2+y^2}\right) e^{-\frac{1}{2}\left[\frac{(x-\mu_x)^2}{\sigma_x^2} + ...
1
vote
1answer
84 views

Order related to Empirical distribution function and Normal distribution

Let $X_1,\dots,X_n$ are i.i.d with distribution function $F$. Let $\hat F_n$ be its empirical distribution function, i.e., $$ \hat F_n(x)=\frac1n\sum_{i=1}^n1_{\{X_\le x\}}(x) $$ where $1_A(x)$ is the ...
2
votes
1answer
72 views

How to compute the unique MLE from an Exponential Family of Distributions?

Let $$ f(x;\theta)=\frac{1}{\pi} \frac{e^{\theta x}\cos(\theta \pi/2)}{\cosh(x)}, x\in{\mathbb{R}} $$ be a family of densities and which is clearly exponential family. Then what is the Maximum ...
0
votes
2answers
50 views

MAP Estimator with Laplacian Noise

I need to calculate the MAP estimator of $ x $ in the following case: $$ \left [ \begin{matrix} {y}_{1}\\ {y}_{2} \end{matrix} \right ] = \left [ \begin{matrix} x\\ x \end{matrix} \right ] + ...
1
vote
1answer
94 views

Conditional expectation and Rao Blackwell

Consider a family of densitites $f(x,\theta)=\frac{\exp(-\sqrt{x})}{\theta}$. Let $X_1$ be a single observation from this family. I have shown that $\sqrt{X_1}/2$ is an unbiased estimator. Now ...
3
votes
1answer
157 views

Is a probability density function necessarily a $L^2$ function?

If a nonnegative continuous real valued function $f$ is integrable over $\mathbb{R}$ with $$\int_\mathbb{R} f\,\mathrm{d}x = 1,$$ does it hold true $$\int_\mathbb{R} f^2 \,\mathrm{d}x<\infty?$$ ...
2
votes
0answers
63 views

Alternatives to Fisher information

The Fisher information matrix is defined as the following: $$\mathcal{I}(\theta)=E[(\frac{\partial \log f(x;\theta)}{\partial \theta})^2]=-E[\frac{\partial^2 \log f(x;\theta)}{\partial \theta ...
1
vote
2answers
53 views

how to estimate the phase parameter of a complex function

There is a complex serie: $f(t_n)=\alpha_n+\beta_n i$, for $n = 1,...,N$,$t_n,\alpha_n$ and $\beta_n$ are known.When we have know that $f(t)$ has the following form: $$f(t)=Ae^{-iBt}$$ with unknown ...
0
votes
1answer
28 views

How to compute MAP estimate of y?

Suppose that a scalar random variable y is of the form $y=z+v$, where the pdf of $v$ is $p_{v}(t)=\frac{t}{2}$ on the interval $[0,2]$, and the pdf of $z$ is $p_{z}(t)=2t$on the interval $[0,1]$. Both ...
0
votes
1answer
154 views

questions on bias of estimator

a) Let $X_{1},...,X_{n}$ be i.i.d Uniform$[0,\theta]$. Show that estimator $\beta(X)=max(X_{1},..,X_{n})$ is a biased estimator for $\theta$.Find an unbiased estimator, based on $\theta$. My attempt: ...
1
vote
1answer
96 views

statistics inequality

Let $\theta$ be a discrete pararmeter and $\gamma_{n}$ be an estimator. Prove that for any $c>0$ we have that $$\text{E}[(\gamma_n-\theta)^2] \ge\Pr[|\gamma_n-\theta|>c]\cdot c^2$$
1
vote
0answers
27 views

ML estimate of sum of guassian variables?

consider the sum $z=x_{1}+...+x_{k}$, where the scalar variables $x_{i}$ are statistically independent and Gaussian, each having the same mean $0$ and variance $\sigma^2_{x}.$ how can I construct the ...
0
votes
1answer
253 views

log likelihood function of a cauchy distribution

What is the log likelihood function of a random varible x with cauchy distribution (0,1)? I've tried to work it out. I think its $\log (1+x)^2$. Is that correct?
1
vote
1answer
83 views

When is a minimum distance decoder also a maximum likelihood decoder?

It is well known that if we have a binary symmetric channel with crossover probability $\epsilon<0.5$ and we send a word $x$ through it, the most likely word is the one with minimum hamming ...
0
votes
1answer
31 views

Given 50 IID normals, find the exact SE for the estimate of $\sigma^2$?

Given 50 I.I.D Normal distributions random variables $X_i$, the Maximum Likelihood estimator for $\sigma^2$ is $\hat{\sigma}^2$, as proven in my lecture notes. Find the EXACT SE. My Attempt: ...
1
vote
0answers
37 views

Convergence rate of an estimator

Say we are interested in estimating some unknown real scalar parameter $\alpha$ using data. Suppose the estimator $\widehat \alpha_N$ of $\alpha$ using the data is consistent. I want to know what it ...