Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

4
votes
0answers
247 views

A late-diverging “approximating solution” for a system of functional equations

Peace be upon you, At the end of this question, I have shown that how computing MLE on an i.i.d Beta distributed data, results in the following system \begin{align*} &\begin{cases} ...
3
votes
0answers
49 views

Trying to show convergence (in probability) of integrals using Taylor expansion

I've been working for a long time now on how to prove a proposition given in a paper about the asymptotic normality of POT-quantile estimators. Hope somebody can help me out. Proposition (i) Let ...
3
votes
0answers
121 views

Estimate number of distinct items

I have a large array of $n$ integers, some of which may be repeated, and I want to estimate how many distinct integers are in the array. Say the number of distinct integers is $N$. I can sample with ...
2
votes
0answers
73 views

Near-Application of Cauchy-Schwarz Inequality

I have the following situation: I have two estimators of $\alpha$, both via maximum likelihood of the density: $$ f(x,y\mid \alpha,\beta) = f(y \mid x,\alpha,\beta)f(x \mid \alpha) $$ One uses only ...
2
votes
0answers
578 views

How to estimate variances for Kalman filter from real sensor measurements without underestimating process noise.

As the title says, I want to estimate the variances needed for a Kalman filter from real sensor measurements only. For example we can take a temperature sensor, but the solution shall be as ...
2
votes
0answers
53 views

Simulation Velocity of a harmonic oscillator system

I am write a simulation for get true Velocity of a harmonic oscillator system as Where P=[p1 p2;p2 p3] can find using Rung-Kutta Integration method with P(0)=[1 0; 0 1] This is code to find p Now, ...
2
votes
0answers
41 views

Risk in density estimation: grasping the definition

When generalizing estimators to an entire function what is the space in which we perform the integral to obtain the expected value (with respect to this function)? For example, when estimating ...
2
votes
0answers
149 views

Polluting an image with Gaussian anisotropic noise and estimate the covariance matrix

Assume that we have a $d\times d$ grey-scale image represented as a vector $$ \mathbf{x}=(x_1,\ldots,x_D)^T\in[0,255]^D, $$ where $D=d\times d$. We would like to import some noise concerning the ...
2
votes
0answers
48 views

paramter estimation (maximum likelihood) of a mixture density

I have this mixture distribution $f(x) =w \cdot \mathcal{LN}(\mu_1,\sigma) + (1-w)\cdot \mathcal{LN}(\mu_2,\sigma) $ where $\mathcal{LN}(\mu,\sigma)$ is a lognormal distribution. I now have random ...
2
votes
0answers
64 views

Alternatives to Fisher information

The Fisher information matrix is defined as the following: $$\mathcal{I}(\theta)=E[(\frac{\partial \log f(x;\theta)}{\partial \theta})^2]=-E[\frac{\partial^2 \log f(x;\theta)}{\partial \theta ...
2
votes
0answers
79 views

Identification of real functions

this my second question, so I'm still new... thanks in advance for any help! Basically, I'm looking for some references and tools to study the following problem. Consider the following function ...
2
votes
0answers
63 views

Unbiased estimator with conditional expectation.

Suppose that $X$ has a binomial distribution with parameter $N=1$ and $p=1/2$. Y, which is independent of $X$, has a normal distribution with mean $\mu$ and variance 1. Consider the estimator $\mu$ of ...
2
votes
0answers
51 views

What are the properties of median-unbiased estimators?

On Wikipedia it says that " A median-unbiased estimator minimizes the risk with respect to the absolute-deviation loss function, as observed by Laplace." How to prove this? Note that I asked on Cross ...
2
votes
0answers
207 views

Fisher Information and minimum variance estimators

I am trying to understand what can be proved about minimum variance estimators. I have changed the question to make it more specific. Let us assume we have some finite set $S$ of elements and we just ...
2
votes
0answers
97 views

Likelihood Function of Random Process

Given the following data: $$ x(t) = A + \omega(t) $$ where $ \omega(t) $ is an AWGN with zero mean, what would be likelihood function $p(x(t);A)$? I know it could be proven to be: $$ p(x;A) = C ...
1
vote
0answers
15 views

MLE of two-dimensional distribution

Let $X_1, ..., X_n$ be a random sample from a continuous distribution with pdf $$f_{\theta,\kappa}(x) = \frac{\kappa\theta^\kappa}{x^{\kappa+1}}, x\geq \theta, \theta > 0, \kappa > 0.$$ How to ...
1
vote
0answers
22 views

How to calculate Fisher Information (FI) matrix for Multivariate Normal Distribution (MN)

Below is the gradient (score) of the MN log likelihood function L for n=1 observation. I originally attempted to calculate the Hessian matrix but ran into difficulty calculating 2nd order derivatives ...
1
vote
0answers
26 views

How to estimate minimum of a random variable

Let $X_i, i =1,\ldots,N$ be vectors of random varaibles. Each of them has $m$ components representing dimension, $X_{ij}, j=1,2,\ldots,m$. Specific values $x_{ij}$ are observations or data. From, ...
1
vote
0answers
16 views

bias reduction when the bias depends on the true parameter

Let's say we estimate a parameter, $\theta$, by $\hat{\theta}$. For this estimator we have the following property that $$\hat{\theta}\to_{p}\theta+f(\theta)$$ where $\to_{p}$ denotes convergence in ...
1
vote
0answers
21 views

MLE of variance for a spherical Gaussian

I am trying to implement the X-Means clustering algorithm. In it, the authors use the BIC to determine which model fits the data best. It is explained here: ...
1
vote
0answers
31 views

Hammersley–Chapman–Robbins bound for Rice distribution

I am trying to evaluate the Hammersley–Chapman–Robbins bound for the variance of an unbiased estimate $\hat{\alpha}$ of $\alpha$ (for a given $\sigma$) for the Rice distribution: $$p(x|\alpha,\sigma) ...
1
vote
0answers
18 views

Fisher Expected Information for a Gaussian Process model

Suppose I have a two dimensional Gaussian process model (GP), defined by a squared exponential correlation function s.t: $$R(x_{i},x_{j}) = \exp\left(-\frac{|x_{i} - x_{j}|^2}{2}\right).$$ I am ...
1
vote
0answers
15 views

Rao-Blackwell improvement for a nonrandomized estimator

Context: please consider a parametric statistical model $(\mathcal{Y},\{P_\theta:\theta\in\Theta\})$ and suppose that we are estimating $g(\theta)$. Associated with this is the set of decisions ...
1
vote
0answers
27 views

asymptotic unbiasedness of weibull mle

It's known that the MLEs of the two-parameter Weibull distribution scale and shape parameters are not available in a closed form. It is, however, known that they do exist, are unique, and moreover, ...
1
vote
0answers
21 views

Estimation of Linear Projection

Given a linear system: $Y=AX+W$ Where: $X$ is the input signal of size $N \times K$ $Y$ is the output signal of size $M \times K$ $A$ is a projection of size $M\times N$; with $M >> N$ $W$ ...
1
vote
0answers
31 views

Unbiased estimator for maximum

Assume $n$ independent random variables with unknown distributions $\{X_1,X_2,...,X_n\}$. Multiple "samples" or observations for each of these variables are given (not necessarily with the same ...
1
vote
0answers
48 views

Working with the sum of two independent random variables, and estimating a parameter

A network source sends a sequence of zeros and ones, $X_1, X_2, ...$ with $X_i$(iid) Bernoulli with $p = P(X_i = 1), 0 < p < 1$. Due to disturbances the received sequence is $Y_1, Y_2, ...$ ...
1
vote
0answers
96 views

Expectation of $\cos(\|X\|)$ where $X \sim \mathcal{N}(\mu,\Sigma)$

Do: $$ \int_{-\infty}^\infty \int_{-\infty}^\infty \cos\left(\sqrt{x^2+y^2}\right) e^{-\frac{1}{2}\left[\frac{(x-\mu_x)^2}{\sigma_x^2} + ...
1
vote
0answers
27 views

ML estimate of sum of guassian variables?

consider the sum $z=x_{1}+...+x_{k}$, where the scalar variables $x_{i}$ are statistically independent and Gaussian, each having the same mean $0$ and variance $\sigma^2_{x}.$ how can I construct the ...
1
vote
0answers
39 views

Convergence rate of an estimator

Say we are interested in estimating some unknown real scalar parameter $\alpha$ using data. Suppose the estimator $\widehat \alpha_N$ of $\alpha$ using the data is consistent. I want to know what it ...
1
vote
0answers
88 views

Estimation of a Ito's semi-martingale linear functional

Could someone check my solution for the following problem please? Or maybe propose a smarter/shorter solution. Consider a stochastic process $X=(X_t)_{t \in [0,1]}$ defined in a filtred ...
1
vote
0answers
81 views

Time series (stochastic process) estimating parameters using characteristic function

I have a time series of assets ${A_1, A_2, ..., A_n}$, which is described by a sophisticated distribution having the following characteristic function: $\phi(u; t;\theta)$, where $\theta$ is a vector ...
1
vote
0answers
24 views

Making sense of an equation

I'm wanting to impliment a formula but I'm having issues understanding some of the components that make it up. The premise of the equation is to use a modified version of Kalman filter that estimates ...
1
vote
0answers
27 views

A question on Stochastic Approximation

I have just started learning stochastic approximation methods, so the question I'm going to ask may be a trivial one in this field, but I need to know this seriousely. I know, that if $g(x,\xi)$ is a ...
1
vote
0answers
25 views

ML Estimation for number of animals in a park. Hypothesis Testing.

A park of area $S=10 000 km^2$ was surveyed for bears, and out of $n$ disjoint regions of equal area $s=1km^2$, there were $n_k$ regions with $k=0,1,....,N$ bears. On each of these regions, the amount ...
1
vote
0answers
29 views

Is there a way to estimate the range of fitting coefficients from only the data?

Considering an approximation $f$ for a set of $N$ data points $(x,y)$ using, for example, $M$ radial basis functions at arbitrary sites in the domain $f_i = \sum_{j=1} ^M c_j\phi(||x_i-x_j||)$ where ...
1
vote
0answers
90 views

Hypothesis testing problem of Normal distributions.

Consider the following Hypothesis Testing problem: Hypothesis $H_0$ : $X \sim N(\mu_0, \sigma_0)$. Mean $\mu_0$ is known but only upper and lower bounds on $\sigma_0$ are known. Hypothesis $H_1$ : ...
1
vote
0answers
79 views

Estimating the number of observations from a set of samples

I repeatedly measure a value $S_n$ which is the sum of a set of $n$ hidden inputs. The goal is to identify the number of hidden inputs. All of the hidden inputs are driven by an experimenter ...
1
vote
0answers
72 views

Showing that statistic is unbiased

Let $X $ be observed data. Let $\hat{\theta}(X)$ be an unbiased estimate of $\theta$ and let T be a sucient statistic for $\theta$. Define the new estimator $\hat\theta^{*}$ of $\theta$, $$ ...
1
vote
0answers
52 views

Estimate the size of a set given random sub sets.

Assuming there is a set $S$ that you are given subsets of, $s_1, s_2, ..., s_n$, estimate $|S|$ (and a confidence interval if possible) making as few assumptions as possible. I'm not going to quibble ...
1
vote
0answers
125 views

Worst-case error related to Cramer-Rao bound

I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple definition of "reconstruction accuracy" which doesn't use any ...
1
vote
0answers
106 views

Maximum Likelihood Estimator of SNR for a Known Signal Superimposed in AWGN

I would like to evaluate the Maximum Likelihood Estimator for the SNR of a given signal: $ x(t) = as(t-\tau) + n(t) $ Under the following assumptions (This is the model of Radar Signal): The input ...
1
vote
0answers
32 views

MLE estimation of parameters, converting normalized observations to integers and back

I am fitting a model's parameters to grouped data by maximizing the likelihood equation: $L(\theta)=N!\prod_{i=1}^{G}\frac{p_i(\theta)^{n_i}}{n_i!}$ $\theta$ is the vector of parameters. $n_i$ is ...
0
votes
0answers
33 views

Most efficient estimator

$X_1,...X_n$ is a random sample of size $n$ from a population with mean $\mu$ and variance $\sigma^2$.There are three estimators for $\mu$:  $\hat\mu _1=\frac{x_1+x_2}{2}$ $\hat\mu ...
0
votes
0answers
20 views

Variance estimation of a diffusion process

The framework of this question is a 1 dimensional diffusion process, defined ny the following equation: $dx_t=adt+bdw_t$ Where $w_t$ is a standard berownian motion and and $a$ is a constant drif ...
0
votes
0answers
4 views

A mix between the Horvitz-Thompson and ordinary estimator

I have asked this question on mathoverflow, but got no answer. Here I have corrected some mistakes and wish to hear any ideas that may bring at least numerical result: The data I have two samples: ...
0
votes
0answers
19 views

Second partial derivative of a minimum function

I am reading a book on detection and estimation theory, and the author had this to say in the derivation of the white noise process from the Wiener process: We can formally obtain the covariance ...
0
votes
0answers
6 views

Function of efficient estimator

Say I have an efficient estimator $\theta$, call it $\hat{\theta}$. If I wanted to estimate another quantity, call it $\delta = g(\theta)$, is there any result which allows me to say that ...
0
votes
0answers
22 views

Expectation of inverse of a symmetric matrix with gaussian elements

Is there any way to calculate: \begin{equation} \mathbb{E} \; ( H^{T}H )^{-1} \end{equation} assuming that the entries of the matrix $H$ are gaussian random variables with unknown means but same ...
0
votes
0answers
18 views

ML estimator for correlated random vectors

Suppose, we have two random vectors in $\mathbb{R}^2 $ denoted by $(X_1,Y_1)$ and $(X_2,Y_2)$ . The individual random variables can only take value in $\lbrace 0, 1 \rbrace$. We get samples of the ...