Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.
0
votes
1answer
108 views
Estimation Theory - Maximum Likelihood Estimation
The below homework question comes from Larsen and Marx, 4th edition.
Is the maximum likelihood estimator for $\sigma^{2}$ in a normal pdf, where both $\mu$ and >$\sigma^{2}$ are unknown, ...
2
votes
2answers
285 views
Inverse problem from pdes
A linear inverse problem is given by:
$\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$
where d: observed data, A: theory operator, m: unknown model and e: error.
To minimize the effect of the noise; ...
3
votes
3answers
421 views
Estimating the Gamma function to high precision efficiently?
I know there are several approximations of the Gamma function that provide decent approximations of this function.
I was wondering, how can I efficiently estimate specific values of the Gamma ...
3
votes
1answer
179 views
Unstable linear inverse problem: which “dampening” Tikhonov matrix should I use?
A linear inverse problem is given by:
$\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$
where d: observed data, A: theory operator, m: unknown model and e: error.
The Least Square Error (LSE) model ...
0
votes
1answer
70 views
How to match a discrete distribution to a continuous distribution in information theoretic sense?
Let
$$
S \sim N(\mu, \sigma^2)
$$
be a normally distributed random variable with known $\mu$ and $\sigma^2$. Suppose, we observe
$$
X = \begin{cases} T & \text{if $S \ge 0$}, \\ -T & ...
3
votes
1answer
99 views
Solving perturbed polynomial equations
Rather than asking the most general question possible, I will frame it in terms of what I believe is an illustrative example.
Let $\epsilon>0$ be a small parameter, let $a,b>0$ and $x\in ...
1
vote
0answers
83 views
Worst-case error related to Cramer-Rao bound
I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple definition of "reconstruction accuracy" which doesn't use any ...
2
votes
2answers
78 views
Have spd $(A^TA)$ and $(B^TB)$, need $A^TB$.
Given two symmetric positive definite matrices $(A^TA)$ and $(B^TB)$ I need to compute $A^TB$.
$A$ and $B$ are not given directly.
$(A^TA)$ and $(B^TB)$ have the same dimensions. $A$ and $B$ are ...
1
vote
1answer
171 views
Gauss-Markov estimator properties
Consider a linear model
$$
y = Ab+n,
$$
where $b \in \mathbb{R^m}$ is a parameter to be estimated, $n \in \mathbb{R^{n}}$ is a noise with mean $\mathbb{E}n = m_{n}$ and with covariation matrix ...
1
vote
1answer
90 views
How to find out the control function of a cosine wave?
I have a system which is sampling at 100Hz. There is only one input for the system. The output is similar to cosine waveforms with varying frequency. I have no clue how to find out the exact formula ...
2
votes
1answer
755 views
Prove the sample variance is an unbiased estimator
I'm trying to prove that the sample variance is an unbiased estimator.
I know that I need to find the expected value of the sample variance estimator $$\sum_i\frac{(M_i - \bar{M})^2}{n-1}$$ but I get ...
1
vote
0answers
99 views
Parameter estimation for a distribution by minimizing its conditional entropy
Let $X$ be a discrete random variable with Laplacian distribution with mean $0$ and scale $\lambda$, as
$$
p(X) = \frac{1}{2\lambda} \exp\left(-\frac{|x|}{2\lambda}\right), \\
X \in ...
0
votes
1answer
44 views
Estimating a function given a noisy sequence of its output
I am new to this forum. Please forgive me if this question is elementary, but I am somewhat lost and could use a little guidance.
Suppose I have an unknown function $f(i)=x_i$. I have a sequence of ...
2
votes
1answer
133 views
Maximum Likelihood Estimation
For iid random variables from a distribution with p.d.f.
$$f(x;\theta_1,\theta_2)=\frac{1}{\theta_2}\exp\bigg(-\frac{(x-\theta_1)}{\theta_2}\bigg), \quad x>\theta_1, ...
0
votes
3answers
116 views
Fair selection of most popular items among separate voting sets
This is a practical problem that arose in real life, which I believe creates interesting mathematical questions.
There is a festival of small plays lasting 8 weeks. Each week 10 short plays are ...
1
vote
1answer
116 views
Maximum Likelihood optimal threshold
I have a decision (detection) problem trying to decide between symbols ${0,2}$. I have the two probability density functions: $$
f(z|s=0) =
\begin{cases}
0.25z + 0.5, & -2\le\ z <0 \\
...
3
votes
2answers
252 views
How can I compare two Markov processes?
There is a discrete-time irreductible Markov process with $r$ possible states. $k$ observations were performed. At each observation a state of process was determined.
$T_0 = \lbrace 0,1,\dots ...
3
votes
2answers
149 views
estimate the perimeter of the island
I'm assigned a task involving solving a problem that can be described as follows: Suppose I'm driving a car around a lake. In the lake there is an island of irregular shape. I have a GPS with me in ...
0
votes
1answer
105 views
Estimation of discrete random variable
Suppose you have a discrete random variable $X_1$ with known probability mass function. I guess that choosing a variable drawn from the same pmf would be the best way to guess $X_1$ assuming all ...
1
vote
0answers
109 views
The fundamental gaussian identities of bayesian estimation
In bayesian estimation, when the model and plant noise is hold , the optimal estimator is Kalman filter. but I am wondering is there any literature that could prove the following gaussian identities?
...
2
votes
0answers
76 views
Likelihood Function of Random Process
Given the following data:
$$
x(t) = A + \omega(t)
$$
where $ \omega(t) $ is an AWGN with zero mean, what would be likelihood function $p(x(t);A)$?
I know it could be proven to be:
$$
p(x;A) = C ...
1
vote
2answers
371 views
Question about unbiased estimator
$X_1,X_2,\ldots,X_n$ are iid random variables $B(1,\theta)$ where $0< \theta<1$. Let $w = 1$ if $\sum_i X_i = n$ and $0$ otherwise. What is the best unbiased estimator of $\theta^2$.
Attempt:
...
1
vote
2answers
60 views
Filter to obtain MMSE of data from Gaussian vector
Data sampled at two time instances giving bivariate Gaussian vector $X=(X_1,X_2)^T$ with
$f(x_1,x_2)=\exp(-(x_1^2+1.8x_1x_2+x_2^2)/0.38)/2\pi \sqrt{0.19}$
Data measured in noisy environment with ...
3
votes
2answers
382 views
biased Maximum Likelihood estimation
Given $N$ points ($x_k$, k from $1$ to $N$) generated from a normal distribution (1-dimensional case) with known mean $\mu$, the Maximum Likelihood estimation of the variance is ...
0
votes
0answers
79 views
Maximum Likelihood Estimator of SNR for a Known Signal Superimposed in AWGN
I would like to evaluate the Maximum Likelihood Estimator for the SNR of a given signal:
$ x(t) = as(t-\tau) + n(t) $
Under the following assumptions (This is the model of Radar Signal):
The input ...
1
vote
0answers
27 views
MLE estimation of parameters, converting normalized observations to integers and back
I am fitting a model's parameters to grouped data by maximizing the likelihood equation:
$L(\theta)=N!\prod_{i=1}^{G}\frac{p_i(\theta)^{n_i}}{n_i!}$
$\theta$ is the vector of parameters. $n_i$ is ...
11
votes
2answers
542 views
You see a route 14 bus on the moon. What is the most likely number of bus routes on the moon?
This question was asked on a forum and while many argued that the answer is 14 (since the probability of you seeing bus 14 is maximum in this case), I argued against it that they were working ...
1
vote
1answer
162 views
How to find parameters from an equation
the question could be "stupid" but i don't know if it is feasible or not, please don't kill me :)
EDIT WITH NEW FORMULAS!
I have an equation like this: (unfortunatly in my first Q&A i cannot ...
5
votes
1answer
139 views
Finding the joint distribution of $X_{1:n}$ and $\overline{X}$
I need to show that, given a random sample of independent variables $X_1, ... , X_n$, each following a distribution EXP($\theta$,$\eta$), that is, ...
4
votes
1answer
127 views
Showing that $X_{1:n}$ is sufficient for $\eta$, by factorization
I'm asked to show that $X_{1:n}$ (the minimum order statistic) is sufficient for $\eta$, in the case of a random sample $(X_1, ... , X_n)$ where $X_i\sim EXP(1,\eta$) (this is the two-parameter ...
2
votes
2answers
134 views
Is $\frac{m-1}{x}$ an unbiased estimator of $\theta$ for given pdf?
Let $X$ be a continuous random variable with pdf,
$$f(x;\theta)=\frac {\theta^m.x^{m-1}e^{-\theta x}} {(m-1)!} ; x\geq0, \theta>0$$
Is $\frac{m-1}{x}$ an unbiased estimator of $\theta$ for given ...
3
votes
1answer
133 views
MLE of the mean of a heteroscedastic Gaussian time series
Suppose we observe $Y_i\sim \mathcal{N}(\theta_0 + \theta_1 x_i, \sigma_i^2)$, with $x_i$ and $\sigma_i^2$ known for all $i = 1,\ldots,n$ and $Y_1,\ldots,Y_n$ independent. Assume $\theta_0$ is ...
0
votes
1answer
220 views
Intuitive explanation of Fisher Information and Cramer-Rao bound [closed]
I am not comfortable with Fisher information, what it measures and how and how is it helpful. Also it's relationship with the Cramer-Rao bound is not apparent to me.
Can someone please give an ...
7
votes
1answer
1k views
Minimum variance unbiased estimator for scale parameter of a certain gamma distribution
Let $X_1, X_2, ..., X_n$ be a random sample from a distribution with p.d.f.,
$$f(x;\theta)=\theta^2xe^{-x\theta} ; 0<x<\infty, \theta>0$$ Obtain minimum variance unbiased estimator of ...
2
votes
2answers
433 views
Some expectation values for a Gamma distribution
Assuming I have a Gamma distributed random Variable $x \sim Gamma( \alpha, \beta )$. Now I like to have the following two expectation values (integrals):
$E \left[ x \ln x \right]$
$E \left[ \ln ...
3
votes
2answers
2k views
Difference between logarithm of an expectation value and expectation value of a logarithm
Assuming I have a always positive random variable $X$, $X \in \mathbb{R}$, $X > 0$. Then I am now interested in the difference between the following two expectation values:
$E \left[ \ln X ...
5
votes
3answers
257 views
Estimating population size
Let's suppose there are $n$ real numbers $a_0 < ... < a_n$ uniformly selected from interval [0, 1). If one knows $k$ numbers on consecutive positions $a_i < ... < a_{i+k-1}$ how good is ...
1
vote
3answers
527 views
How to estimate failure probability from count until first failure?
What would be the formula to estimate the rate of failure of some test as a percentage chance of failure from the number of runs of the test until the first failure was seen?
For example, considering ...
1
vote
1answer
128 views
Likelihood function estimating two different means
I completely understand how to find the likelihood functions of simple pdfs. However, how would you attempt to find the likelihood function of a pdf with a negative exponential function with two ...
0
votes
0answers
405 views
Maximum likelihood estimation,
Given a problem, we are asked to use a statistical method to come up with a conclusion. e.g. XYZ drug decreases heart diseases. (null hypothesis)
After searching on internet, I got a list of these ...
