Questions tagged [estimation-theory]
Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.
526
questions
1
vote
1
answer
38
views
Difference between MVB and UMVU estimators
I am trying to understand the difference between the UMVUE (uniformly minimum-variance unbiased estimator, also known as minimum-variance unbiased estimator (MVUE)) and the MVBE (minimum variance ...
- 327
3
votes
1
answer
33
views
Statistical estimator of expected value of the gradient of an unknown function
Fix a probability space $(\Omega, \mathcal{A}, \Bbb P),$ a continuously differentiable function $f:\Bbb R^n \rightarrow \Bbb R,$ and a random vector $X: \Omega \rightarrow \Bbb R^n.$ Furthermore, we ...
- 1,775
1
vote
1
answer
63
views
Find UMVUE of ${\rm e}^{-\theta \tau}$ in which $\theta$ is parameter of ${\rm Exp}(\theta)$
Suppose $X_1,... ,X_n\ {\rm i.i.d.\sim Exp}(\theta)$, i.e. $X_i \sim f(x) = \theta{\rm e}^{-\theta x}I_{(x>0)}$. Find UMVUE of ${\rm e}^{-\theta \tau}$ in which $\tau > 0$ is given.
Hint: ${\rm ...
- 83
0
votes
0
answers
45
views
Find UMVUE of $\sigma$ and $3\sigma^2$ in which $\sigma$ is parameters of $N(0, \sigma^2)$.
Assume $X_1,...,X_n$ is sample of $N(0,\sigma^2), \sigma > 0$, find
(1) complete and sufficient statistic of $\sigma^2$,
(2) UMVUE(uniformly minimum variance unbiased estimation) of $\sigma$ and $...
- 83
-1
votes
0
answers
25
views
unbiased estimator for supremum of expectation
Can we get an unbiased estimator for $$ \sup_{\theta} E_{X\sim P} [\max\{\theta-X, 0\}] \quad ? $$
Let $X_1$ be a sample from $P$. I tried using the $1$-sample estimator: $$ \sup_\theta \max\{\theta-...
- 1,666
1
vote
0
answers
21
views
How to prove multivariate Bayesian Cramér-Rao inequality?
I came cross multivariate Bayesian Cramér-Rao inequality as follow recently, but I don't know how to prove it.
Let $\{f(\cdot ; \theta): \theta \in \Theta\}$ be a family of probability density ...
- 405
-3
votes
0
answers
22
views
proof that a random vector is normal [closed]
X vector is a signal vector combined of a uniform distribution and a reyleigh distribution.
$$ x=a*e^(jΩ)
\begin{pmatrix}
1 \\
e^jw \\
.\\
.\\
e^j(N-1)w
\end{pmatrix}
$$
which a has a reyleigh ...
1
vote
0
answers
45
views
Complete statistics and Sufficient statistics
I was aolving the following problem:
Find the statistics of the following density that is complete:
$$\text{Let } y_k \text{ be i.i.d random variables with the density: }\\f(y_k;x)=(x-1)y_k^{-x}\...
- 47
0
votes
0
answers
30
views
Ranking estimators according to mean absolute deviation, with estimated ground truth
Looking for simple proof (or counterexample) for the following claim.
Setup: Let $\{\hat{\theta}_A, \hat{\theta}_B, \hat{\theta}\}$ be random variables such that $\text{Cov}(\hat{\theta}_A, \hat{\...
0
votes
0
answers
31
views
Transformation of random variables for sum of two random variables
Given
$r = \ln(a) + n$
where $a$ and $n$ are independent random variables with pdf(probability distribution function) $P_{n}(N) = e^(N)u(N)$ and $P_a(A) = \frac{1}{2}A(u(A) - u(A-2))$ respectively. ...
- 23
0
votes
0
answers
20
views
Log-Likelihood with probability density function
I'm working on an assignment and cannot figure out where I make a mistake:
Calculate the Log-Likelihood with the probability density function $p(x;\mu,\sigma) = \frac{1}{\sqrt{2\pi\sigma^{2}}}\, \text{...
0
votes
0
answers
24
views
New condition of an unbiasedness
The usual unbiasedness condition of an estimand $g(\theta)$ is this $$E_\theta[\delta(X)]=g(\theta).$$
Here $g(\theta)$ is a real valued function over $\Omega$ whose
value is to be estimated. This is ...
- 3,578
1
vote
0
answers
25
views
Incredibly low standard errors
I am currently estimating the parameters of an interest rate model by means of a maximum likelihood estimation in combination with the iterated extended Kalman filter, and I obtain incredibly low ...
- 11
0
votes
0
answers
5
views
Interpolating noisy measurements taking into account derivative bounds
For context, I have a target moving in space which I am able to locate using vision sensors. I end up obtaining $M$ noisy samples of the trajectory $[x(t), y(t)]$ for some time instants $\{t_k\}$. ...
- 1,553
2
votes
0
answers
39
views
Consistently estimate the covariance matrix with weakly correlated observations
Suppose there are T k-dimensional observations following the generating process:
$Y_t = \mu + \epsilon_t$, where $\mu$ is the mean and $\epsilon$ is a weak stationary error with zero mean and time-...
- 61
1
vote
0
answers
19
views
How to apply MCMC to a exponential complex data-set
I am a beginner in probability, estimation and statistics. I can understand the physics behind the problem really well, but when it comes to estimation using some statistical algorithms I am very ...
- 91
1
vote
0
answers
19
views
Correlating two matrices $A, B$ with stochastic dependency structure imposed by cross-validation
Consider a labelled data set
$$D = \{(x_1, y_1),...,(x_n, y_n)\} $$
on which we want to evaluate a machine learning algorithm using $k$-fold cross validation with $m$ different random seeds. This ...
- 1,624
0
votes
0
answers
48
views
what is the bias and variance of this LS estimator?
I want to estimate the variables $a$ and $b$ ($\theta = \left[ {\begin{array}{*{20}{c}}
a\\
b
\end{array}} \right]$) in the nonlinear model:
$$y\left( t \right) = au\left( t \right) + b\exp (u(t)) + ...
- 83
0
votes
0
answers
27
views
Under or over determined linear system vs inconsistent linear system
We know the following:
If there are more number of observations than that of unknowns, the linear system term as an over-determined system.
If there are less number of observations than that of ...
- 53
0
votes
0
answers
31
views
Estimating parameters using 0-1 matrix system of equations
I have the following set of $n$ equations:
$ \hspace{4cm}\boxed{y^{(1)}=h_1x_1^{(1)}+ h_2x_2^{(1)} +\ldots+ h_kx_k^{(1)} +w_1\\ y^{(2)}=h_1x_1^{(2)}+ h_2x_2^{(2)} +\ldots+ h_kx_k^{(2)}+w_2\\ \vdots \\...
- 55
4
votes
0
answers
86
views
Convergence of estimator defined by supremum over measurable sets
Let $X \in L_1$ be a positive random variable on the probability space $([0,1], \mathcal B, P)$, where $\mathcal B$ is the Borel $\sigma$ algebra on $[0,1]$. Consider
$$\phi(A) = E[X\mid A] \cdot I\...
- 263
0
votes
0
answers
28
views
How to compute covariance of random variable after applying random quaternion?
Let $\mathbf{R}(q)$ denote the rotation matrix $\mathbf{R}$ corresponding to the unit quaternion $q = [w, x, y, z]^T$. Now, consider the following model:
$$
\begin{align}
\mathbf{y} = \mathbf{R}(q)(\...
- 1,299
0
votes
1
answer
175
views
Pareto distribution: exponential family
I am struggling to find the expression that shows that some transformation of Y belongs to the exponential family. Were Y has the Pareto distribution. I am given the following pdf:
$$f(y;\theta)=\frac{...
- 57
0
votes
0
answers
11
views
Covariance of fused poses. Should it be normalised by the number of poses?
I came across this paper from T. Barfoot and P. Furgale:
"Associating Uncertainty With Three-Dimensional Poses for Use in Estimation Problems"
Link: http://ncfrn.mcgill.ca/members/pubs/...
0
votes
1
answer
24
views
Minimum mean square estimation with 3 random variable
I am trying to find the solution for part "a", I have try finding the expectaion
$E[x|y=y] = \int x f(x|y=y) dx = \int x
\frac{f(y=y|x=x)f(x))}{f(y=y)}$.
With y|x ~ Normal (2x,1). Yet it ...
- 37
0
votes
0
answers
20
views
Estimate random variables given estimated function values
Let $X_1$ and $X_2$ be two positive independent random vectors. Let random variable $Y_1$ depend only on $X_1$; and random variable $Y_2$ depend only on $X_2$. If I have the following means square ...
- 11
0
votes
0
answers
15
views
Can weak measurement principles of Quantum mechanics be used in similar questions in classical estimation problems?
This is a surface level question and I don't want to go into detail.
Imagine an algorithm which when used with a sensor output gives the statistical moments of a variable in nature (for example mean ...
- 91
0
votes
0
answers
7
views
Distributing a fixed number of noisy samples of a function to best estimate that function
I want to learn more about how to best spatially distribute noisy samples of a function to reconstruct the function itself. The name of this field of research or some references to papers or books ...
1
vote
0
answers
43
views
Minimal sufficient statistics for two unknown parameters
I have a problem for finding a sufficient and a minimal sufficient statistics for the next density: let $X_1,...,X_n$ a sample with $$f_X(x;\alpha,\theta)=\frac{\alpha x^{\alpha-1}}{\theta^{\alpha}}I_{...
- 7,419
2
votes
1
answer
25
views
Assessing bias and consistency of modified OLS estimator
Given the (multivariate) linear regression model $\mathbf y=\mathbf X \mathbf\beta_0 + \epsilon$ and $\mathbb E[\epsilon|\mathbf X]=0$ for $\beta_0 \in \mathbb R^k$, determine if the following ...
- 143
1
vote
1
answer
73
views
UMVUE over a discrete distribution
The problem is: let $X$ a random variable such that $P(X=x)=\left\{\begin{array}{cl}2p(1-p)&\mbox{if }x=-1\\p^x(1-p)^{3-x}&\mbox{if }x\in\{0,1,2,3\}\end{array}\right.$
Find, if there exist, an ...
- 7,419
1
vote
0
answers
17
views
Covariance matrix from asynchronously observed time series
Let $(X_t) = (X^1_t, \cdots, X^d_t)$ be a time series in $\mathbb{R}^d$ with
covariance matrix $C_t := \big(\mathrm{Cov}[X^i_t, X^j_t]\big)_{i,j}$.
Suppose that $X\equiv(X_t)$ satisfies some ...
- 319
2
votes
1
answer
77
views
Is every estimator a sufficient statistic?
It is clear that not any sufficient statistic (s.s.) makes a good estimator (since a monotonic transform of a s.s. is still a s.s.). But is a "good" estimator of the parameter always a s.s.?...
- 7,896
0
votes
0
answers
29
views
How to find priori information in Cramer Rao Lower Bound when the pdf of the parameter is unkown
In order to compute a Bayesian Cramer Rao Lower bound, we need to find
the prior information of a random parameter $\theta$ which is a complex
scalar parameter. The pdf of $\theta$ is $f(\cdot)$.
...
1
vote
0
answers
157
views
How can the sample covariance matrix be biased and unbiased?
The Wikipedia entry on covariance estimation states that the sample covariance matrix (SCM), viewed in $\mathbb{R}^{p \times p}$ is an unbiased and efficient estimator, but w hen viewed intrinsically ...
- 397
1
vote
0
answers
43
views
MSE minimization
Let $ X $ be a $p$-dimensional random vector, $X^{T} = \left (x_{1}, \ldots, x_{p} \right) $, with $ E(X)=0$ and $Variance(X)=V $. Suppose you are interested in minimizing
$$
E\left[\left(x_{1}-\sum_{...
- 85
0
votes
0
answers
20
views
Maximum Likelihood Estimator for $r$ with $r^n$ in noise
$x[n]=r^n+w[n]$ for $n=0,1,\ldots,N-1$ where $w[n]$ is WGN with variance $\sigma^2$. I need to estimate $r$.
No efficient estimator exists (that I can discover), so I am looking to build a maximum ...
- 43
3
votes
1
answer
564
views
What is the difference between MVUE and UMVUE
I was going through the minimum variance unbiased estimators and I am confused about the concept of MVUE and UMVUE. Is the unbiased estimator whose variance attaining CRLB a UMVUE or MVUE? I referred ...
0
votes
0
answers
27
views
Population minimizer for mixture of two regressions
Consider the generative model where,
\begin{align*}
Z &\sim \mathrm{Bernoulli}\left( \frac{1}{2}\right) \\
Y &= \begin{cases}
\pmb{\beta}_{1,*}^T \mathbf{X} + \eta &~~\text{given}~~ Z = 0\\...
- 434
0
votes
0
answers
15
views
Use probability theory to estimate the successors of an event out of a countable finite sample space?
Scenario:
There exists to two façades in a system: façade zero, and façade one.
When façade zero builds a LinearDataStructure, that object contains elements of ...
- 13
2
votes
2
answers
76
views
Can someone explain how to find the t-distribution for estimating the difference in means of two normal populations when variance is not known?
See the question referenced. How do I find the t-part? If my confidence level is $95\%$ what do I do with that number? I know what to do to find the inverse distribution in other estimations but this ...
- 39
2
votes
0
answers
47
views
Are there invariant sets for the covariance of the Kalman filter?
Consider the standard Kalman filter as in here. I'm interested in understanding how $\mathbf{P}_{k|k}$ evolves, when $\mathbf{F}_k, \mathbf{Q}_k, \mathbf{R}_k$ change in $k$. I wonder if $\mathbf{F}_k,...
- 1,553
0
votes
0
answers
32
views
Similarity transform for matrix product with determinant equal to zero.
Context:
Consider matrices $A \in \mathbb{R}^{m \times n}$ and $B \in \mathbb{R}^{m \times m}$ where $B$ is invertible. Let where $A$ have $k$ zero columns, so $A$ has the following form:
$$A = \begin{...
- 1,299
2
votes
0
answers
51
views
Is there a way to find UMVUE without just guessing?
We have $(X_1,...,X_n)$ random sample of distribution $N(m,1)$. I need to find UMVUE of $g(m)=e^m$.
The natural guess was that the unbiased estimator which I need to find may be something like $e^T$, ...
- 23
0
votes
0
answers
6
views
unbiased estimate
suppose we have a uniform distribution such that $u\in \left[0,\:\frac{1}{\phi }\right]$ from which a single observation is $x\left(0\right)$.
We want to prove that an unbiased estimator $\phi'\:=\:h\...
- 137
0
votes
0
answers
54
views
Alternative distribution of likelihood ratio test statistic
I am wondering if there is an existing theorem for the distribution of a likelihood ratio test statistic under the alternative. By Wilks theorem, we know the distribution under the null. However, ...
- 139
0
votes
1
answer
111
views
Finding UMVUE, given complete and sufficient statistic
Let $X_1,X_2,…,X_n(n≥3) $ be a random sample from Poisson(𝜃), where$ θ∈(0,∞)$ is unknown and let
$T=\sum _{i=1}^n X_i$
The UMVUE of $ e^{−2\theta}*\theta^3$ is
Since UMVMUE is a Complete Sufficient ...
user827016
1
vote
1
answer
63
views
Concluding that MLEs for exponential distribution parameters are biased and then unbiasing them
I have the i.i.d. exponential random variables $X_1, \dots, X_n$ with the density functions
$$f(x; \sigma, \tau)=
\begin{cases}
\dfrac{1}{\sigma} e^{-(x - \tau)/\sigma} &\text{if}\, x\geq \tau\\
...
- 4,056
1
vote
1
answer
44
views
Estimating a bivariate moment generating function [closed]
Assume there are many identical and independent sample pairs e.g. $(X_1, Y_1), (X_2, Y_2), (X_3, Y_3), \dots, (X_n, Y_n)$.
How do you consistently estimate the following function $M(t_1, t_2)$, such ...
- 13
0
votes
1
answer
38
views
Applicability of different formulas for Fisher's information
Question: Let $X_1, ..., X_n \sim \text{Exp}(\lambda)$ where $E(X) = \lambda$. Find Fisher's information of $\lambda$.
From the description, $f_X(x) = \lambda^{-1}\exp(-\lambda^{-1}x)$
I first tried ...
- 414