Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

2
votes
1answer
36 views

Prove that $\int k(w)o(h^2w^2)dw=o(h^2)$ for $\int k(w)dw=1$

Suppose that $k$ is nonnegative real-valued function satisfying $$ \int k(w)dw=1,\quad\int wk(w)dw=0,\quad\int w^2k(w)dw=\kappa_2<\infty.\tag{$\star$} $$ (The limits of the integrals are all ...
1
vote
0answers
15 views

How to estimate variances for Kalman filter from real sensor measurements without underestimating process noise.

As the title says, I want to estimate the variances needed for a Kalman filter from real sensor measurements only. For example we can take a temperature sensor, but the solution shall be as ...
0
votes
0answers
21 views

Description length in model coding

In class, our professor posted the following: We will discretize $\theta$ (some model) into $1/\sqrt{n}$ distinct values. Intuitive argument: with N data points, our estimation error for $\hat ...
0
votes
1answer
21 views

Maximum likelihood estimator for general multinomial

Let $(X_1,\ldots,X_r)\sim\text{multinomial}(n,(p_1,\ldots,p_r))$, where $p_r=1-p_1-\cdots-p_{r-1}$. The random likelihood is $Ap_1^{X_1}\ldots p_r^{X_r}$, for some non-zero $A$. The random ...
1
vote
0answers
17 views

Help in understanding derivation of density function and expectation maximization

I am unable to understand how the density function is derived in this paper Semiblind System Identification with Symbolic Chaotic Sequences The Authors have ...
0
votes
1answer
12 views

Computing likelihood for data corrupted by zero mean noise

The following statement is from a text on Statistical Estimation. I am trying to figure out how the likelihood function was arrived at. By definition of likelihood, ...
0
votes
1answer
14 views

Maximum Likelihood (ML) estimation when 1 estimator is dependent on the other.

Let $\mathcal{L}(\theta_1,\theta_2)$ be the log-likelihood function. If I manage to find an estimator for $\theta_1$, as $\hat{\theta}_1=g(\theta_2,data)$. Then, if I want to find a ML estimator for ...
0
votes
0answers
12 views

How can I find the nonlinear and linear MS estimates of y in terms of x and the resulting MS errors?

If $y=x^3$, find the nonlinear and linear MS estimates of $y$ in terms of $x$ and the resulting MS errors? This is what I got for the nonlinear MS estimation: Since $e=E\{[y-C(x)]^2\}$, $C(x)=x^3$ and ...
0
votes
0answers
9 views

how can I find the resulting MS error with linear estimation?

The problem says: If $\eta_x=\eta_y=0, \sigma_x=\sigma_y=4$ and $\hat{y}=0.2x$ (linear estimate), find $E\{(y-\hat{y})^2\}$. I am doing this, based on Papulis formulas for homogeneous linear estimate: ...
0
votes
0answers
12 views

How Can I show that a=A in this linear MS estimation problem?

How can I show that if the constants A,B and a are such that $E\{[y-(Ax+B)]^2\}$ and $E {\{[(y-\eta_y)-a(x-\eta_x)]^2 \}}$ are minimum, then $a=A$. I am trying to use this: $e=e_m$ is minimum if ...
3
votes
0answers
47 views

Trying to show convergence (in probability) of integrals using Taylor expansion

I've been working for a long time now on how to prove a proposition given in a paper about the asymptotic normality of POT-quantile estimators. Hope somebody can help me out. Proposition (i) Let ...
2
votes
0answers
30 views

Simulation Velocity of a harmonic oscillator system

I am write a simulation for get true Velocity of a harmonic oscillator system as Where P=[p1 p2;p2 p3] can find using Rung-Kutta Integration method with P(0)=[1 0; 0 1] This is code to find p Now, ...
0
votes
0answers
13 views

Simulation Position of a harmonic oscillator system

I am write a simulation for get true Postion of a harmonic oscillator system as Now, I want to write matlab code to get the true postion z of the system. However, ...
0
votes
1answer
26 views

A Estimation about Hölder condition

Let $p:[0,\inf) \to \mathbb{R}$ be a contionous function such that $p(0)=0$ Fix $a>1/2 , k$ is a positive integer $>\frac{1}{a-\frac{1}{2}}$. Suppose for all $n \in \mathbb{N}$ and $\lambda ...
0
votes
1answer
14 views

Improving data gaussianity using neural networks

I wanted to know if there is a way to use neural networks (deep neural networks or autoencoders) for a data gaussianization. I wonder how could the output data distribution be monitored and ...
0
votes
0answers
7 views

Sensitivity analysis of paramaters and input variables

I am trying to perform a sensitivity analysis of an optimization problem $f(x,\alpha)= \min_{ Q} {g(x,\alpha , Q)}$ where $x$ is an input variable for our function, and $\alpha $ is a parameter. ...
1
vote
1answer
28 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
1
vote
1answer
32 views

On the finite expectation of a function

Let $y_i$, $i=1,\dots,n$, be independent Gaussian rv's of mean $\theta_i$ and variance $\sigma^2$ and let $\mathbf{y}:=[y_1,\dots,y_n]^\top$. Consider the function $f\colon \mathbb{R}^n\to ...
2
votes
0answers
17 views

Risk in density estimation: grasping the definition

When generalizing estimators to an entire function what is the space in which we perform the integral to obtain the expected value (with respect to this function)? For example, when estimating ...
0
votes
0answers
20 views

Find the MLE of mu/sigma and its expected value?

The problem: $X_1, \ldots, X_n$ be random sample from a normal distribution $N(\mu, \sigma^2).$ Find the MLE of $\mu/\sigma^2$ and its expected value. My solution: Since $\hat{\mu} = \bar{x}$ and ...
1
vote
1answer
43 views

When to use Central Limit Theorem or Cramers Theorem

In for example this paper the authors say The central limit theorem provides an estimate of the probability \begin{align} P\left( \frac{\sum_{i=1}^n X_i - n\mu}{\sigma \sqrt{n}} > x \right) ...
2
votes
2answers
46 views

Convergence Rate of Sample Average Estimator

Let $X_1, X_2,\cdots$ be i.i.d. random variables with $E(X_1) = \mu, Var(X_1) = σ^2> 0$ and let $\bar{X}_n = {X_1 + X_2 + \cdots + X_n \over n}$ be the sample average estimator. Is there a way to ...
1
vote
0answers
23 views

asymptotic unbiasedness of weibull mle

It's known that the MLEs of the two-parameter Weibull distribution scale and shape parameters are not available in a closed form. It is, however, known that they do exist, are unique, and moreover, ...
-1
votes
1answer
37 views

Cumulative distribution function of exponentials

I have the cumulative distribution function $F(x)=(1-e^{-x})\mathbb{1}_{x≥0}$ and want to write the CDF to $F(\frac{x-\mu}{\sigma})$. I have derived ...
0
votes
0answers
34 views

Second derivatives of MLE to multivariate normal distribution

I want to calculate the second derivatives to the MLE's $\hat{\mu}$ and $\hat{\Sigma}$ to confirm that the extremums indeed are maximums to the multivariate normal distribution $$ ...
0
votes
0answers
6 views

Find the spectral density of white noise $\omega$~(0,2)

I am find the spectral density of a white noise given by $\omega$ ~ (0,2). Could you help me to find it? Thank all.
1
vote
2answers
24 views

Variance of sample mean (problems with proof)

Assuming that I have $\{x_1,\ldots, x_N\}$ - an iid (independent identically distributed) sample size $N$ of observations of random variable $\xi$ with unknown mean $m_1$, variance (second central ...
1
vote
0answers
19 views

Estimation of Linear Projection

Given a linear system: $Y=AX+W$ Where: $X$ is the input signal of size $N \times K$ $Y$ is the output signal of size $M \times K$ $A$ is a projection of size $M\times N$; with $M >> N$ $W$ ...
1
vote
1answer
36 views

asymptotic normality and unbiasedness of mle

Suppose $\hat{\theta}_n$ is the MLE for some parameter $\theta$. Suppose also that the MLE is such that the Cramer regularity conditions are fulfilled, and $\hat{\theta}_n$ is asymptotically normal ...
0
votes
1answer
53 views

Sample size required to estimate population proportion with given precision

It plans to conduct a study on the percentage of homeowners who have at least two TVs. What should be the sample size if we want to ensure that $95\%$ of estimation error is less than $0.01$? ...
1
vote
2answers
28 views

How to periodically estimate states of a LTI if the output is measured irregularly?

How can I periodically estimate the states of a discrete linear time-invariant system in the form $$\vec{x}(k+1)=\textbf{A}\vec{x}(k)+\textbf{B}\vec{u}(k)$$ ...
1
vote
1answer
38 views

Linear Regression with independent but non-identical noise

If I have this linear regression equation: $$y=X\beta+\epsilon $$ ($x$ and $\beta$ are vectors) The likelihood function can be written as $$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi ...
1
vote
1answer
33 views

Maximum Likelihood Question

The aim is to find the maximum likelihood estimator for theta. $f(x)$ is given and we can assume that $1\le x\le-1$. I have completed the steps seen in the image, however I am having difficulty ...
0
votes
0answers
46 views

Determine a distribution of a gaussian stochastic at different time

I would like to determine the autocorrelation function of a Gaussian stochastic. Let see my problem So my solution is The distribution of $y=x(t_1)-x(t_2)$ is also a Gaussian stochastic with ...
1
vote
0answers
16 views

Unbiased estimator for maximum

Assume $n$ independent random variables with unknown distributions $\{X_1,X_2,...,X_n\}$. Multiple "samples" or observations for each of these variables are given (not necessarily with the same ...
0
votes
0answers
4 views

Least square estimator: $N( \beta x_i, \sigma^2)$

Let $ Y_1,...,Y_n$ be i.i.d $N(\beta x_i, \sigma^2) $ with known $ x_i's$. It is asked to find the Mean Squared Estimator for $\beta.$ I didn't understandmuch about this method of pbtaining an ...
0
votes
0answers
13 views

Estimate time it takes the minimum mean cycle cancelling algorithm to converge

This particular algorithm solves the circulation problem, equivalent to the minimum-capacitated flow. My question rather than only from this particular algorithm, but for combinatorial solutions in ...
0
votes
0answers
39 views

Estimators of the binomial distribution

This is a follow-up of this previous question and elaborates on the answer I received there. $\def\b{\begin{pmatrix}}\def\e{\end{pmatrix}}\def\a{\alpha}X_1,X_2,\dots, X_n$ are a random sample from ...
2
votes
0answers
143 views

Polluting an image with Gaussian anisotropic noise and estimate the covariance matrix

Assume that we have a $d\times d$ grey-scale image represented as a vector $$ \mathbf{x}=(x_1,\ldots,x_D)^T\in[0,255]^D, $$ where $D=d\times d$. We would like to import some noise concerning the ...
0
votes
1answer
27 views

$|p- \dfrac xn|>|q- \dfrac xn|$ $\implies$ $p^x(1-p)^{n-x}<q^x(1-q)^{n-x}$?

If $p,q \in (0,1)$ , and $ n \in \mathbb N$ be given and $x$ be given integer between $0$ and $n$ such that $|p- \dfrac xn|>|q- \dfrac xn|$ , then is it true that ...
4
votes
0answers
238 views

A late-diverging “approximating solution” for a system of functional equations

Peace be upon you, At the end of this question, I have shown that how computing MLE on an i.i.d Beta distributed data, results in the following system \begin{align*} &\begin{cases} ...
0
votes
0answers
9 views

Asymptotic coincidence of the MAP and MMSE estimator

In many works, simulations show that as number of samples increases, the mean-square-error (MSE) of the MAP estimator attains the minimum MSE. Where can I find a theoretical proof to these empirical ...
0
votes
1answer
31 views

Does consistent estimators have in-variance property?

If $(T_n)$ is a sequence of consistent estimators of a parameter $\theta$ ( i.e. for every $ \epsilon >0$ , $\lim_{n \to \infty} P [ \space |T_n -\theta|< \epsilon ]=1$ ) , then is it true that ...
1
vote
1answer
121 views

Maximum likelihood estimate vs likelihood ratio tests?

Can someone explain to me the intuition behind why we need likelihood ratio tests. From my understanding, they make use of maximum likelihood estimators over different parameters space and they are a ...
1
vote
0answers
36 views

Working with the sum of two independent random variables, and estimating a parameter

A network source sends a sequence of zeros and ones, $X_1, X_2, ...$ with $X_i$(iid) Bernoulli with $p = P(X_i = 1), 0 < p < 1$. Due to disturbances the received sequence is $Y_1, Y_2, ...$ ...
0
votes
2answers
43 views

Biased MLE estimate of mean (expectation)

Please give an example of p.m.f. or p.d.f. , the maximum likely-hood estimate of whose mean (expectation) is a biased estimator . Thanks
0
votes
0answers
28 views

What is the problem with this model parameter estimation algorithm?

In a statistical model with parameters $\theta$ and unobserved laten variables $Z$, the model likelihood is $$L(\theta;X)=Pr(X|\theta)=\sum_ZPr(X,Z|\theta)$$ The standard way to estimate $\theta$ ...
0
votes
1answer
55 views

estimation problem for two-parameter weibull distribution

Suppose the two-parameter Weibull distribution is given by the pdf $$ f(x;a,b) = \left(\frac{x}{a}\right)^b\frac{b}{a}\exp\left\{-\left(\frac{x}{a}\right)^b\right\}, $$ where $x,a,b>0$. I am ...
0
votes
0answers
17 views

Optimal combination of multiple estimates of a random variable

For the following estimation problem: y = hx + n, x is the sent data, y is the observation (received data), h is a scaling factor (known), n is an AWGN random variable with zero mean ...
1
vote
1answer
31 views

Shapiro-Wilk test

I am trying to determine if a given sample comes from a Normal distribution. For that purpose I want to perform a Shapiro-Wilk test in the way stated on wikipedia. My concern comes with the vector ...