Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

learn more… | top users | synonyms

2
votes
1answer
1k views

How do I find the MLE of $\theta$ when x is dependent on $\theta$?

Let $X_{1},X_{2},...,X_{n}$ represent a random sample from a distribution with pdf: $f(x; \theta)=e^{-(x-\theta)}, \theta \le x<\infty, -\infty<\theta<\infty$ | zero elsewhere I need to ...
0
votes
1answer
25 views

How to obtain estimate of covariance matrix that will be guarantee to be semi-positive define?

How to obtain estimate of covariance matrix that will be guarantee to be semi-positive define ? (Is CrossValidated better place for this question ?)
8
votes
2answers
367 views

Why should Gaussian noise have fractal dimension of 1.5?

In a paper I'm trying to understand, the following time series is generated as "simulated data": $$Y(i)=\sum_{j=1}^{1000+i}Z(j) \:\:\: ; \:\:\: (i=1,2,...,N)$$ where $Z(j)$ is a Gaussian noise with ...
1
vote
1answer
283 views

The distribution when combining two samples together?

Suppose $X\sim N(0,{\sigma}^2)$ and $Y\sim N(0,{2\sigma}^2)$ . $X_1, ..., X_m$ are the samples from $X$ and $Y_1, ..., Y_n$ are the samples from $Y$. And then combine two samples as a new sample ...
14
votes
3answers
3k views

Intuitive explanation of a definition of the Fisher information

I'm studying statistics. When I read the textbook about Fisher Information, I couldn't understand why the Fisher Information is defined like this: $$I(\theta)=E_\theta\left[-\frac{\partial^2 ...
2
votes
2answers
331 views

Expected value of a max

We have a roulette with the circumference $a$. We spin the roulette 10 times and we measure 10 distances, $x_1,\ldots,x_{10}$, from a predefined zero-point. We can assume that those distances are ...
1
vote
2answers
432 views

Show that estimates are unbiased

The following is a problem in my book that I don't really understand: We take a random sample: $x_1,x_2,\ldots,x_n$ from a population that is $N(μ,σ)$ where $\mu$ and $\sigma$ are unknown. We build ...
0
votes
1answer
288 views

how can I get minimum error probability for this decision problem?

I have the decision problem for 4 hypotheses as follows: $$H_j: Y_k=N_k-s_{jk},\ k=1,2,\ldots,n;\ j=0,1,2,3.$$ where signals are $s_{jk}=E_0\sin(w_cT(k-1)+(j+\frac{1}{2})\frac{\pi}{2}).$ $$$$ In ...
3
votes
0answers
110 views

Estimate number of distinct items

I have a large array of $n$ integers, some of which may be repeated, and I want to estimate how many distinct integers are in the array. Say the number of distinct integers is $N$. I can sample with ...
2
votes
0answers
190 views

Fisher Information and minimum variance estimators

I am trying to understand what can be proved about minimum variance estimators. I have changed the question to make it more specific. Let us assume we have some finite set $S$ of elements and we just ...
1
vote
1answer
79 views

biasedness/unbiasedness of an MLE.

To show whether an MLE I just found is biased/unbiased, would I need to find the expectation of the answer? Plus would I do this by integrating $\text{MLE} \cdot \text{pdf}$. My MLE is $ ...
0
votes
1answer
57 views

Several Unbiased Estimators

If I have some data set $ D={X_1,...X_N} $ and have an esitmator be "pick the first point" $X_1$, how can I show that this estimator is unbiased? I also have to show why its highly undesirable, and I ...
2
votes
1answer
54 views

Parameter optimization in probabilistic models

Task: Suppose we model a variable $y = Wx + \mu$ as a linear transformation of $x$ plus some Gaussian noise $\mu\sim\mathcal N(0,\sigma I)$. Our aim is to minimize the estimation error of $x$ given ...
1
vote
0answers
74 views

Estimating the number of observations from a set of samples

I repeatedly measure a value $S_n$ which is the sum of a set of $n$ hidden inputs. The goal is to identify the number of hidden inputs. All of the hidden inputs are driven by an experimenter ...
0
votes
0answers
47 views

Identification of parameters problem

I always struggle to get the true essence of identification in econometrics. I know that we state that a parameter (say $\hat{\theta}$) can be identified if by simply looking at its (joint) ...
1
vote
0answers
68 views

Showing that statistic is unbiased

Let $X $ be observed data. Let $\hat{\theta}(X)$ be an unbiased estimate of $\theta$ and let T be a sucient statistic for $\theta$. Define the new estimator $\hat\theta^{*}$ of $\theta$, $$ ...
1
vote
0answers
49 views

Estimate the size of a set given random sub sets.

Assuming there is a set $S$ that you are given subsets of, $s_1, s_2, ..., s_n$, estimate $|S|$ (and a confidence interval if possible) making as few assumptions as possible. I'm not going to quibble ...
3
votes
1answer
357 views

Determine whether a statistic is sufficient, given the probability density

Let $X_1, X_2, \dots, X_n$ be a sample of i.i.d. random variables, with density $$f_\theta=\frac{2}{3\theta}\left(1-\frac{x}{3\theta}\right) $$ for $0 < x < 3\theta$. And $f_\theta=0$ if $ x ...
1
vote
1answer
238 views

Using the MSE criterion, which is a better estimator for $\Theta^2$?

Question: Let $T_1$ and $T_2$ be independent unbiased estimators of a parameter $\Theta$. Assume that $\operatorname{Var}(T_2) = \operatorname{Var}(T_1)$. Using the MSE critertion, define which is a ...
1
vote
1answer
80 views

Proving that the sum of Good-Turing estimators is $1$

I want to know how to go about proving that the Good-Turing estimator has a total probability of $1$. I have seen this proof (page 2) but I found unclear the first step: $$\sum_j \theta[j] = \sum_r ...
1
vote
2answers
541 views

Fast variance calculation

Suppose to have a sequence $X$ of $m$ samples and for each $i^{th}$ sample you want to calculate a local mean $\mu_{X}(i)$ and a local variance $\sigma^2_{X}(i)$ estimation over $n \ll m$ samples of ...
0
votes
2answers
707 views

Exponential Distribution Maximum Likelihood

I found the following question in a past exam paper and I would like to ask how to solve it as I can't find anything in the notes related to it: ...
1
vote
1answer
150 views

Sufficient Estimators and Generalized Likelihood Ratios

If you can make the assumption that a sufficient statistic exists for some parameter - let's call it $\theta$. How would you show that the critical region of a likelihood ratio test will depend on ...
0
votes
1answer
115 views

Proof of convergence of a sum of mean-consistent estimators

After a few weeks off I am back at my self-study of Measure-Theoretic probability. As always, I thank the community for any detail and answers they can provide as I try to work myself through these ...
0
votes
1answer
167 views

Estimation Theory - Maximum Likelihood Estimation

The below homework question comes from Larsen and Marx, 4th edition. Is the maximum likelihood estimator for $\sigma^{2}$ in a normal pdf, where both $\mu$ and >$\sigma^{2}$ are unknown, ...
3
votes
2answers
910 views

Inverse problem from pdes

A linear inverse problem is given by: $\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$ where d: observed data, A: theory operator, m: unknown model and e: error. To minimize the effect of the noise; ...
4
votes
3answers
1k views

Estimating the Gamma function to high precision efficiently?

I know there are several approximations of the Gamma function that provide decent approximations of this function. I was wondering, how can I efficiently estimate specific values of the Gamma ...
3
votes
1answer
550 views

Unstable linear inverse problem: which “dampening” Tikhonov matrix should I use?

A linear inverse problem is given by: $\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$ where d: observed data, A: theory operator, m: unknown model and e: error. The Least Square Error (LSE) model ...
0
votes
1answer
127 views

How to match a discrete distribution to a continuous distribution in information theoretic sense?

Let $$ S \sim N(\mu, \sigma^2) $$ be a normally distributed random variable with known $\mu$ and $\sigma^2$. Suppose, we observe $$ X = \begin{cases} T & \text{if $S \ge 0$}, \\ -T & ...
3
votes
1answer
187 views

Solving perturbed polynomial equations

Rather than asking the most general question possible, I will frame it in terms of what I believe is an illustrative example. Let $\epsilon>0$ be a small parameter, let $a,b>0$ and $x\in ...
1
vote
0answers
119 views

Worst-case error related to Cramer-Rao bound

I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple definition of "reconstruction accuracy" which doesn't use any ...
2
votes
2answers
86 views

Have spd $(A^TA)$ and $(B^TB)$, need $A^TB$.

Given two symmetric positive definite matrices $(A^TA)$ and $(B^TB)$ I need to compute $A^TB$. $A$ and $B$ are not given directly. $(A^TA)$ and $(B^TB)$ have the same dimensions. $A$ and $B$ are ...
1
vote
1answer
322 views

Gauss-Markov estimator properties

Consider a linear model $$ y = Ab+n, $$ where $b \in \mathbb{R^m}$ is a parameter to be estimated, $n \in \mathbb{R^{n}}$ is a noise with mean $\mathbb{E}n = m_{n}$ and with covariation matrix ...
1
vote
1answer
99 views

How to find out the control function of a cosine wave?

I have a system which is sampling at 100Hz. There is only one input for the system. The output is similar to cosine waveforms with varying frequency. I have no clue how to find out the exact formula ...
3
votes
3answers
4k views

Prove the sample variance is an unbiased estimator

I'm trying to prove that the sample variance is an unbiased estimator. I know that I need to find the expected value of the sample variance estimator $$\sum_i\frac{(M_i - \bar{M})^2}{n-1}$$ but I get ...
2
votes
1answer
227 views

Parameter estimation for a distribution by minimizing its conditional entropy

Let $X$ be a discrete random variable with Laplacian distribution with mean $0$ and scale $\lambda$, as $$ p(X) = \frac{1}{2\lambda} \exp\left(-\frac{|x|}{2\lambda}\right), \\ X \in ...
0
votes
1answer
46 views

Estimating a function given a noisy sequence of its output

I am new to this forum. Please forgive me if this question is elementary, but I am somewhat lost and could use a little guidance. Suppose I have an unknown function $f(i)=x_i$. I have a sequence of ...
2
votes
1answer
232 views

Maximum Likelihood Estimation

For iid random variables from a distribution with p.d.f. $$f(x;\theta_1,\theta_2)=\frac{1}{\theta_2}\exp\bigg(-\frac{(x-\theta_1)}{\theta_2}\bigg), \quad x>\theta_1, ...
0
votes
3answers
128 views

Fair selection of most popular items among separate voting sets

This is a practical problem that arose in real life, which I believe creates interesting mathematical questions. There is a festival of small plays lasting 8 weeks. Each week 10 short plays are ...
1
vote
1answer
317 views

Maximum Likelihood optimal threshold

I have a decision (detection) problem trying to decide between symbols ${0,2}$. I have the two probability density functions: $$ f(z|s=0) = \begin{cases} 0.25z + 0.5, & -2\le\ z <0 \\ ...
3
votes
2answers
371 views

How can I compare two Markov processes?

There is a discrete-time irreductible Markov process with $r$ possible states. $k$ observations were performed. At each observation a state of process was determined. $T_0 = \lbrace 0,1,\dots ...
4
votes
2answers
342 views

estimate the perimeter of the island

I'm assigned a task involving solving a problem that can be described as follows: Suppose I'm driving a car around a lake. In the lake there is an island of irregular shape. I have a GPS with me in ...
0
votes
1answer
137 views

Estimation of discrete random variable

Suppose you have a discrete random variable $X_1$ with known probability mass function. I guess that choosing a variable drawn from the same pmf would be the best way to guess $X_1$ assuming all ...
1
vote
1answer
144 views

The fundamental gaussian identities of bayesian estimation

In bayesian estimation, when the model and plant noise is hold , the optimal estimator is Kalman filter. but I am wondering is there any literature that could prove the following gaussian identities? ...
2
votes
0answers
93 views

Likelihood Function of Random Process

Given the following data: $$ x(t) = A + \omega(t) $$ where $ \omega(t) $ is an AWGN with zero mean, what would be likelihood function $p(x(t);A)$? I know it could be proven to be: $$ p(x;A) = C ...
1
vote
2answers
461 views

Question about unbiased estimator

$X_1,X_2,\ldots,X_n$ are iid random variables $B(1,\theta)$ where $0< \theta<1$. Let $w = 1$ if $\sum_i X_i = n$ and $0$ otherwise. What is the best unbiased estimator of $\theta^2$. Attempt: ...
1
vote
2answers
84 views

Filter to obtain MMSE of data from Gaussian vector

Data sampled at two time instances giving bivariate Gaussian vector $X=(X_1,X_2)^T$ with $f(x_1,x_2)=\exp(-(x_1^2+1.8x_1x_2+x_2^2)/0.38)/2\pi \sqrt{0.19}$ Data measured in noisy environment with ...
3
votes
2answers
1k views

biased Maximum Likelihood estimation

Given $N$ points ($x_k$, k from $1$ to $N$) generated from a normal distribution (1-dimensional case) with known mean $\mu$, the Maximum Likelihood estimation of the variance is ...
1
vote
0answers
102 views

Maximum Likelihood Estimator of SNR for a Known Signal Superimposed in AWGN

I would like to evaluate the Maximum Likelihood Estimator for the SNR of a given signal: $ x(t) = as(t-\tau) + n(t) $ Under the following assumptions (This is the model of Radar Signal): The input ...
1
vote
0answers
29 views

MLE estimation of parameters, converting normalized observations to integers and back

I am fitting a model's parameters to grouped data by maximizing the likelihood equation: $L(\theta)=N!\prod_{i=1}^{G}\frac{p_i(\theta)^{n_i}}{n_i!}$ $\theta$ is the vector of parameters. $n_i$ is ...