Questions tagged [estimation-theory]
Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.
461
questions
0
votes
0answers
26 views
Energy estimate for nonlinear term
Let $ \Omega$ be open and bounded. Consider that $u$ is smooth solution of
\begin{equation}
\frac{\partial u}{\partial t}=(u\cdot \nabla)u~~~in~\Omega\times[0,T]\\
u=0~on~~~\partial \Omega\times[0,T]\\...
0
votes
0answers
7 views
Gelbs Solution of Nonhomogeneous Linear System of Differential Equations
I am struggling with Gelbs derivation of the discretization of a differential equation of the form:
$$ \dot{x}(t) = F(t)x(t) + L(t)u(t)$$
Here F and L are matrices, while u represents a ...
0
votes
0answers
18 views
Estimation of statistics of an iid sequence using Kalman filtering
I am given the sequence $v_0, v_1, \ldots$ of i.i.d. Gaussian random variables $\sim \mathcal{N}(\mu,\sigma^2)$ for some unknown parameter $\mu$ ($\sigma$ is known) which I need to estimate given the ...
1
vote
1answer
27 views
Neyman Pearson Lemma — Optimizing for Detecting the Null Hypothesis
So for my Neyman-Pearson test, I have already calculated the sufficient statistic $T$. It is distributed as following:
$$ T_{|H_{0}} = \text{Rayleigh}(\sigma)$$
$$ T_{|H_{1}} = \text{Rice}(A, \sigma)$...
0
votes
1answer
40 views
minimum mean square error of $X_{1}$ given $S_{n}=\sum_{i=1}^{n}X_{i}$ when $p=0.5$ and $X_{1},X_{2},\ldots$ are bernoulli iid
I need help with this one:
Let $X_{1},X_{2},\ldots$ be bernoulli i.i.d random variables with $$\forall i:\mathbb{P}\left\{ X_{i}=1\right\} =\mathbb{P}\left\{ X_{i}=0\right\} =\frac{1}{2}$$
define $S_{...
0
votes
0answers
4 views
Creation of ROC curve of logistic regression classifier with rejection
For a logistic regression classifier, I create a roc curve by variation of the threshold on the output probability.
Question: can I create an additional ROC curve with 5% rejection rate based on the ...
0
votes
0answers
10 views
Calculating Total Error In $g(x,y,z) = \frac{x-y}{z}$ Given Errors In Variables $x,y,z$
So I have a function given by:
$$ g(x,y,z) = \frac{x-y}{z} $$
where I can only estimate the values of $x,y,z$. I assume that these estimates are uncorrelated. From the Cramer-Rao lower bound, I know ...
2
votes
1answer
84 views
A math inequality cannot prove
I try to solve a inequality, I use R to run simulation with different $\alpha>0$ and a sequence of $n$, the simulation all show the LHS is smaller than RHS, but I cannot prove it analytically, can ...
0
votes
0answers
16 views
Variance of the MLE estimator of a function of bernoulli parameter
I have a bernoulli RV $X$ defined as below:
$pr(X=1)=1-(1-p)^{k}$
$pr(X=0)=(1-p)^{k}$
Assume I have $N$ independent observations on the RV $X$. I am interested in finding the MLE estimator of $k$ ...
0
votes
0answers
19 views
Graphical models for regression problem
I am studying about the Gaussian graphical model (GGM). I have a $N\times D$ matrix X of my observations. The structure of the network has been found by using the graphical lasso method. It means I ...
0
votes
1answer
40 views
Estimating indicator of normal distribution
Given selection of i.i.d's ${X_1 \dots X_n} \sim \mathscr{N}(\theta, 1)$ ($\theta$ is unknown), how to express probability of $X_i > 0$?
(and express it without normal distrbution' cdf which ...
2
votes
0answers
23 views
Estimating the value of $\sigma$ for Brownian motion
Let $X_t=\sigma W_t$ be a stochastic process, where $W_t$ is the Wiener process and $\sigma$ is an unknown parameter.
I want a formula to estimate the value of $\sigma$ (which could not be found in ...
0
votes
0answers
18 views
Computing MMSE estimator in recursive state dynamics
Consider the $p\times p$ matrix $\pmb{A}$ and suppose $\theta_n, n\ge 0$ are $p$-dimensional vectors we want to estimate according to MMSE criterion.
We have the recursive relation $\theta_n = \pmb{...
0
votes
0answers
18 views
Can we use an LQG controller if the system is not observable?
I understand an LQG controller is essentially an LQE + LQR. So, if I understand properly, we basically apply the gain estimated from LQR to our state estimate from LQE (e.g., a Kalman filter).
Does ...
1
vote
0answers
33 views
Prove an estimator is biased
I want to prove that an estimator in biased.
For a big n, like $10^{13}$ we define a distribution of values $x_i \in (0,1]$ for $i \in [n]$. We want to estimate with some error $\epsilon$ the value ...
0
votes
1answer
31 views
Estimating the probability of success in the Bernoulli scheme
I'm reading 'Probability' by Shiryayev, in particular the paragraph 'Estimating the probability of success in the Bernoulli scheme'.
The author wants to determine if the estimator (estimator of the ...
0
votes
0answers
31 views
How to find the NP decision Rule for false alarm and the power of that decision
Suppose we observe a random variable Y given by:
Y=N+ÓØ Ī»
where ÓØ is either 0 or 1, Ī» is a fixed number between 0 and 2, and where N is a random variable that has a uniform density (-1,1). We wish to ...
1
vote
1answer
51 views
Can I determine unbiased estimators of a product of expectations?
Let $n$ be any positive natural number.
I was wondering if for any function $g\colon\{0,1\}^n \to \{0,1\}$ it is always possible to determine a second function $f\colon \{0,1\}^n \to \mathbb{R}$ such ...
1
vote
1answer
41 views
Independence of MLEs for mean and covariance matrix of multivariate normal distribution
I am looking for information on/the proof of the following theorem. Any help would be great, a book recommendation with the theorem in it would be even better.
Let $X_i \sim \mathcal{N}_N(\mu, \Sigma)...
0
votes
2answers
44 views
Show that estimator is unbiased
I'm working on a problem about the estimator $T(X_1,...,X_n)=\overline{X}^2-\frac{1}{n}\overline{X}$ where $X_1,...,X_n$ is an i.i.d. sample from the poisson distribution with paramater $\lambda$. I ...
0
votes
0answers
100 views
How do I evaluate this derivative in a well-known maximum likelihood derivation?
I have been trying for some considerable time to derive a very widely used result in the field of (wireless) direction-finding, namely the Maximum Likelihood direction estimate (based on a simple, ...
0
votes
0answers
36 views
Robustness of a model to learnt parameters
There is a recent push to study how sensitive a model is to small changes in its input. This has also been studied from an adversarial point of view: e.g what is the smallest input perturbation that ...
0
votes
1answer
29 views
Estimator for number of success in M bernoulli trials
I have M bernoulli trials each of which is independent and with probability of successes given by $\left[p_{1}, p_{2}, \dots, p_{M}\right].$
I am trying to find an estimate for the number of ...
1
vote
3answers
54 views
Estimate the error if $P_2 = 1-\frac{x^2}{2}$ is used to estimate the value of $cos(x)$ at $x = 0.6$
This is how I tried to work through the problem:
$ |Error| < $
$ P_2 = 1-x^2/2 $
$n=0$ ā£ā£ $ Ę(x) = cox(x) $
$n=1$ ā£ā£ $ Ę^{'}(x) = -sin(x) $
$n=2$ ā£ā£ $ Ę^{''}(x) = -cox(x) $
$n=3$ ā£ā£ $ Ę^{'''...
2
votes
1answer
41 views
Finding the optimal weights to place on $n$ estimators to create a weighted-average that minimizes the expected squared error
Consider $n$ independent random variables; $X_1, X_2, ... X_n$, each of which are estimators of a criterion $Y$.
$X_1, X_2, ... X_n$ may have different (known) variances, and may have different (...
0
votes
2answers
58 views
What does level of confidence mean in statistics?
The thing I am talking about is the interval estimate of the population mean. Let $X\sim N(\mu,\sigma^2)$ where $\mu$ is unknown but $\sigma$ is known. To estimate $\mu$, we perform $n$ experiments ...
0
votes
0answers
14 views
Admissibility of Bayes Estimators
Are there any easy/clear examples where a Bayes estimator is inadmissible? Of course it has to be non-unique, because uniqueness implies admissibility.
Do we have to have an improper prior? Or are ...
0
votes
1answer
25 views
Estimating Scalar Value from Multiple Observations
I'm working on a project in which I get data from ~15 same sensors located differently around the object and I want to estimate the state of the the object I'm intersted by using multiple observations....
1
vote
0answers
39 views
Kiefer bound on variance of unbiased estimator
I am trying to work through the very short paper entitled "On Minimium Variance Estimators" by J. Kiefer (1952). In this work he gives two explicit examples on calculating the lower bound on the ...
1
vote
1answer
82 views
Find a Maximum Likelihood Estimator for $\theta$
I am trying to solve the following problem:
$X$ is a random variable with logistic logarithmic distribution:
$$
f(x,\theta) = 3x^2\theta(1 + \theta x^3)^{-2} ; x\in \mathbb{R}
$$
$(x_1, ...
0
votes
0answers
27 views
Question regarding linear regression weighting matrix
Consider the linear regression model
$$b = Xy + e, \quad E[e] = 0, \quad E[ee'] = V$$
Assume that the matrix $X$ has linearly independent columns. It is well known that the minimum variance affine ...
0
votes
0answers
12 views
Multivariate Density Estimation Problem
It is difficult to carry out multivariate density estimation due to the "curse of dimensionality". A prominent example for this is that the optimal convergence rate for the minimax risk $\inf_{\hat f}\...
0
votes
0answers
23 views
How to evaluate Cramer-Rao lower bound and Fisher information matrix for multivariate?
I am having trouble finding practical explanations of the multivariate Fisher information matrix and Cramer-Rao lower bound. The definitions seem circular, so I must be missing something. Wikipedia ...
0
votes
1answer
69 views
Can the MMSE estimator be just interpreted as Tikhonov regularization?
Given an ordinary least squares problem with the normal setup (error Gaussian i.i.d. with $\sigma^2$, $N$ observations and $L$ unknowns):
$$
\mathbf{y} = \mathbf{X}\mathbf{b} + \mathbf{e}
$$
The ...
0
votes
0answers
25 views
Differentiability almost everywhere vs differentiability everywhere
We want to estimate $F(\theta)$, where $F$ is the CDF of $Y$, and $\theta$ is an unknown number belonging to the support of $Y$.
In the first step we estimate $\hat \theta$, which is $\sqrt n$ ...
0
votes
1answer
35 views
Deconvolution with respect to a particular function
Let $\mathcal L, \mathcal L^*: \Theta \times \mathcal A \to \mathbb R$ be functions. When can $\mathcal L$ be expressed as the convolution of $\mathcal L^*$ with some third function $U$? That is, when ...
0
votes
1answer
35 views
Confusion in MAP estimation
Consider the situation $r = a+n$, where $n \sim \mathcal{N}(0,\sigma_n^2)$. I am having confusion with respect the computation of $p_{a|r}(A)$ for the above scenario.
Option 1, $r = a+n \implies a = r ...
0
votes
0answers
30 views
Use MGFs to show convergence in distribution of sum of negative binomial random variables to chi square.
I am working on the following problem for self-study:
Let $X_1,...,X_n$ be iid negative binomial(r,p). Assume r is known, and we are interested in computing a confidence interval for p.
An ...
0
votes
0answers
13 views
Show that an estimator is not minimax.
I am working on the following problem for self-study:
Let $\hat{\theta}$ be an unbiased estimator.
Under squared error loss, show that the estimator $c*\hat{\theta}$ is not minimax unless $sup_{\...
1
vote
0answers
44 views
How to estimate the input vector norm of inner product with small space cost?
Given a public vector $A$=$(a_1, a_2, \cdots, a_n) \in \mathbb{Z}^n_q$ and a unknown input vector $B$=$(a_1, a_2, \cdots, a_n)$, which could be sampled from either $\mathbb{Z}^n_2$ or $\mathbb{Z}^n_q$,...
0
votes
0answers
41 views
Unique Maximum Likelihood Estimator which is not sufficient
I'm looking for an example for a statistical model for which Maximum Likelihood Estimator $\hat{\theta}(\vec{Y})$ for $\theta$ exists, and satisfies the following conditions:
It is unique. ...
0
votes
1answer
30 views
Estimate variance in a nested study design
I have a "mother" bacteria. I cloned it $R$ times and measured a variable $P$ on these $R$ clones and recorded the mean $\bar p$ and the variance $V$ among these clones.
Then, I made $M$ mutants ($M$ ...
0
votes
0answers
20 views
Show Consistency for every component
For $j=1,...,k$ let $t_{n,j}:\Omega_n \rightarrow \mathbb{R}$ be an estimator for $h_j(\theta) \in \mathbb{R}$.
Show that $t_n(X)=(t_{n,1}(X),...,t_{n,k}(X))$ is a consistent estimator of $h(\theta)=(...
4
votes
1answer
80 views
What adaptive controller can be used in embedded system with low RAM?
This is not a question for data science, hardware or programming languages. This is a more practical question about adaptive control for embedded systems, but still a math question.
I have tried to ...
0
votes
1answer
56 views
If $X_i \sim U(\theta-\frac{1}{2};\theta+\frac{1}{2})$, show that $[X_{(1)},X_{(n)}]$ is a confidence interval
Let $X_1,...X_n$ random sample from
$f(x;\theta)=I_{[\theta-\frac{1}{2};\theta+\frac{1}{2}]}(x)$.
a) Show that $[X_{(1)},X_{(n)}]$ is a confidence interval for $\theta$.
b) Compute the ...
1
vote
1answer
106 views
Finding shortest Confidence Interval for an Exponential Distribution
Let $X$ such that $f_{X}(x\mid\theta) = \theta e^{-\theta x} I_{(0, \infty)}(x)$, where $\theta > 0$. If $[X, 2X]$ is a confidence interval for $\frac{1}{\theta}$:
a)Find the confidence ...
0
votes
1answer
74 views
Finding aconfidence interval for $\theta$ of the uniform distribution on $(0, \frac{1}{\theta})$
Suppose $X_1,\ldots, X_n$ are i.i.d. random variables $Uniform (0, 1/ \theta)$. Find a 95% confidence interval for $\theta$.
What I tried:
$f_{X} (x) = \frac{1}{1/\theta}=\theta, F_{X} (x) = \frac{...
0
votes
1answer
67 views
Finding confidence interval for $\frac{kx^{k-1}}{\theta^k}$
Let $X_1,\ldots, X_n$ are i.i.d. random variables such that: $$f(x;\theta)=\frac{kx^{k-1}}{\theta^k}, x\in (0,\theta)$$ where $\theta \gt 0 $ and $k$ is a positive integer. Find a $100(1-\alpha)% $% ...
0
votes
1answer
43 views
Text book recommendations for statistical estimation theory -specifically MLEs and confidence intervals
I am looking for a textbook on statistical estimation theory. In particular I am interested in a book that explains MLEs and confidence intervals. Preferably accompanied by exercises.
The book should ...
3
votes
1answer
597 views
Mean squared error for vectors
I know that when we compare estimators $\hat{b_1}$ and $\hat{b_2}$ to an unknown parameter $\beta$, in classical statistics an estimator $\hat{b_1}$ is said to be "better" than $\hat{b_2}$ if:
$$ ...
