Questions tagged [estimation-theory]

Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data that has a random component.

Filter by
Sorted by
Tagged with
0
votes
0answers
26 views

Energy estimate for nonlinear term

Let $ \Omega$ be open and bounded. Consider that $u$ is smooth solution of \begin{equation} \frac{\partial u}{\partial t}=(u\cdot \nabla)u~~~in~\Omega\times[0,T]\\ u=0~on~~~\partial \Omega\times[0,T]\\...
0
votes
0answers
7 views

Gelbs Solution of Nonhomogeneous Linear System of Differential Equations

I am struggling with Gelbs derivation of the discretization of a differential equation of the form: $$ \dot{x}(t) = F(t)x(t) + L(t)u(t)$$ Here F and L are matrices, while u represents a ...
0
votes
0answers
18 views

Estimation of statistics of an iid sequence using Kalman filtering

I am given the sequence $v_0, v_1, \ldots$ of i.i.d. Gaussian random variables $\sim \mathcal{N}(\mu,\sigma^2)$ for some unknown parameter $\mu$ ($\sigma$ is known) which I need to estimate given the ...
1
vote
1answer
27 views

Neyman Pearson Lemma — Optimizing for Detecting the Null Hypothesis

So for my Neyman-Pearson test, I have already calculated the sufficient statistic $T$. It is distributed as following: $$ T_{|H_{0}} = \text{Rayleigh}(\sigma)$$ $$ T_{|H_{1}} = \text{Rice}(A, \sigma)$...
0
votes
1answer
40 views

minimum mean square error of $X_{1}$ given $S_{n}=\sum_{i=1}^{n}X_{i}$ when $p=0.5$ and $X_{1},X_{2},\ldots$ are bernoulli iid

I need help with this one: Let $X_{1},X_{2},\ldots$ be bernoulli i.i.d random variables with $$\forall i:\mathbb{P}\left\{ X_{i}=1\right\} =\mathbb{P}\left\{ X_{i}=0\right\} =\frac{1}{2}$$ define $S_{...
0
votes
0answers
4 views

Creation of ROC curve of logistic regression classifier with rejection

For a logistic regression classifier, I create a roc curve by variation of the threshold on the output probability. Question: can I create an additional ROC curve with 5% rejection rate based on the ...
0
votes
0answers
10 views

Calculating Total Error In $g(x,y,z) = \frac{x-y}{z}$ Given Errors In Variables $x,y,z$

So I have a function given by: $$ g(x,y,z) = \frac{x-y}{z} $$ where I can only estimate the values of $x,y,z$. I assume that these estimates are uncorrelated. From the Cramer-Rao lower bound, I know ...
2
votes
1answer
84 views

A math inequality cannot prove

I try to solve a inequality, I use R to run simulation with different $\alpha>0$ and a sequence of $n$, the simulation all show the LHS is smaller than RHS, but I cannot prove it analytically, can ...
0
votes
0answers
16 views

Variance of the MLE estimator of a function of bernoulli parameter

I have a bernoulli RV $X$ defined as below: $pr(X=1)=1-(1-p)^{k}$ $pr(X=0)=(1-p)^{k}$ Assume I have $N$ independent observations on the RV $X$. I am interested in finding the MLE estimator of $k$ ...
0
votes
0answers
19 views

Graphical models for regression problem

I am studying about the Gaussian graphical model (GGM). I have a $N\times D$ matrix X of my observations. The structure of the network has been found by using the graphical lasso method. It means I ...
0
votes
1answer
40 views

Estimating indicator of normal distribution

Given selection of i.i.d's ${X_1 \dots X_n} \sim \mathscr{N}(\theta, 1)$ ($\theta$ is unknown), how to express probability of $X_i > 0$? (and express it without normal distrbution' cdf which ...
2
votes
0answers
23 views

Estimating the value of $\sigma$ for Brownian motion

Let $X_t=\sigma W_t$ be a stochastic process, where $W_t$ is the Wiener process and $\sigma$ is an unknown parameter. I want a formula to estimate the value of $\sigma$ (which could not be found in ...
0
votes
0answers
18 views

Computing MMSE estimator in recursive state dynamics

Consider the $p\times p$ matrix $\pmb{A}$ and suppose $\theta_n, n\ge 0$ are $p$-dimensional vectors we want to estimate according to MMSE criterion. We have the recursive relation $\theta_n = \pmb{...
0
votes
0answers
18 views

Can we use an LQG controller if the system is not observable?

I understand an LQG controller is essentially an LQE + LQR. So, if I understand properly, we basically apply the gain estimated from LQR to our state estimate from LQE (e.g., a Kalman filter). Does ...
1
vote
0answers
33 views

Prove an estimator is biased

I want to prove that an estimator in biased. For a big n, like $10^{13}$ we define a distribution of values $x_i \in (0,1]$ for $i \in [n]$. We want to estimate with some error $\epsilon$ the value ...
0
votes
1answer
31 views

Estimating the probability of success in the Bernoulli scheme

I'm reading 'Probability' by Shiryayev, in particular the paragraph 'Estimating the probability of success in the Bernoulli scheme'. The author wants to determine if the estimator (estimator of the ...
0
votes
0answers
31 views

How to find the NP decision Rule for false alarm and the power of that decision

Suppose we observe a random variable Y given by: Y=N+Ө λ where Ө is either 0 or 1, λ is a fixed number between 0 and 2, and where N is a random variable that has a uniform density (-1,1). We wish to ...
1
vote
1answer
51 views

Can I determine unbiased estimators of a product of expectations?

Let $n$ be any positive natural number. I was wondering if for any function $g\colon\{0,1\}^n \to \{0,1\}$ it is always possible to determine a second function $f\colon \{0,1\}^n \to \mathbb{R}$ such ...
1
vote
1answer
41 views

Independence of MLEs for mean and covariance matrix of multivariate normal distribution

I am looking for information on/the proof of the following theorem. Any help would be great, a book recommendation with the theorem in it would be even better. Let $X_i \sim \mathcal{N}_N(\mu, \Sigma)...
0
votes
2answers
44 views

Show that estimator is unbiased

I'm working on a problem about the estimator $T(X_1,...,X_n)=\overline{X}^2-\frac{1}{n}\overline{X}$ where $X_1,...,X_n$ is an i.i.d. sample from the poisson distribution with paramater $\lambda$. I ...
0
votes
0answers
100 views

How do I evaluate this derivative in a well-known maximum likelihood derivation?

I have been trying for some considerable time to derive a very widely used result in the field of (wireless) direction-finding, namely the Maximum Likelihood direction estimate (based on a simple, ...
0
votes
0answers
36 views

Robustness of a model to learnt parameters

There is a recent push to study how sensitive a model is to small changes in its input. This has also been studied from an adversarial point of view: e.g what is the smallest input perturbation that ...
0
votes
1answer
29 views

Estimator for number of success in M bernoulli trials

I have M bernoulli trials each of which is independent and with probability of successes given by $\left[p_{1}, p_{2}, \dots, p_{M}\right].$ I am trying to find an estimate for the number of ...
1
vote
3answers
54 views

Estimate the error if $P_2 = 1-\frac{x^2}{2}$ is used to estimate the value of $cos(x)$ at $x = 0.6$

This is how I tried to work through the problem: $ |Error| < $ $ P_2 = 1-x^2/2 $ $n=0$ ␣␣ $ ƒ(x) = cox(x) $ $n=1$ ␣␣ $ ƒ^{'}(x) = -sin(x) $ $n=2$ ␣␣ $ ƒ^{''}(x) = -cox(x) $ $n=3$ ␣␣ $ ƒ^{'''...
2
votes
1answer
41 views

Finding the optimal weights to place on $n$ estimators to create a weighted-average that minimizes the expected squared error

Consider $n$ independent random variables; $X_1, X_2, ... X_n$, each of which are estimators of a criterion $Y$. $X_1, X_2, ... X_n$ may have different (known) variances, and may have different (...
0
votes
2answers
58 views

What does level of confidence mean in statistics?

The thing I am talking about is the interval estimate of the population mean. Let $X\sim N(\mu,\sigma^2)$ where $\mu$ is unknown but $\sigma$ is known. To estimate $\mu$, we perform $n$ experiments ...
0
votes
0answers
14 views

Admissibility of Bayes Estimators

Are there any easy/clear examples where a Bayes estimator is inadmissible? Of course it has to be non-unique, because uniqueness implies admissibility. Do we have to have an improper prior? Or are ...
0
votes
1answer
25 views

Estimating Scalar Value from Multiple Observations

I'm working on a project in which I get data from ~15 same sensors located differently around the object and I want to estimate the state of the the object I'm intersted by using multiple observations....
1
vote
0answers
39 views

Kiefer bound on variance of unbiased estimator

I am trying to work through the very short paper entitled "On Minimium Variance Estimators" by J. Kiefer (1952). In this work he gives two explicit examples on calculating the lower bound on the ...
1
vote
1answer
82 views

Find a Maximum Likelihood Estimator for $\theta$

I am trying to solve the following problem: $X$ is a random variable with logistic logarithmic distribution: $$ f(x,\theta) = 3x^2\theta(1 + \theta x^3)^{-2} ; x\in \mathbb{R} $$ $(x_1, ...
0
votes
0answers
27 views

Question regarding linear regression weighting matrix

Consider the linear regression model $$b = Xy + e, \quad E[e] = 0, \quad E[ee'] = V$$ Assume that the matrix $X$ has linearly independent columns. It is well known that the minimum variance affine ...
0
votes
0answers
12 views

Multivariate Density Estimation Problem

It is difficult to carry out multivariate density estimation due to the "curse of dimensionality". A prominent example for this is that the optimal convergence rate for the minimax risk $\inf_{\hat f}\...
0
votes
0answers
23 views

How to evaluate Cramer-Rao lower bound and Fisher information matrix for multivariate?

I am having trouble finding practical explanations of the multivariate Fisher information matrix and Cramer-Rao lower bound. The definitions seem circular, so I must be missing something. Wikipedia ...
0
votes
1answer
69 views

Can the MMSE estimator be just interpreted as Tikhonov regularization?

Given an ordinary least squares problem with the normal setup (error Gaussian i.i.d. with $\sigma^2$, $N$ observations and $L$ unknowns): $$ \mathbf{y} = \mathbf{X}\mathbf{b} + \mathbf{e} $$ The ...
0
votes
0answers
25 views

Differentiability almost everywhere vs differentiability everywhere

We want to estimate $F(\theta)$, where $F$ is the CDF of $Y$, and $\theta$ is an unknown number belonging to the support of $Y$. In the first step we estimate $\hat \theta$, which is $\sqrt n$ ...
0
votes
1answer
35 views

Deconvolution with respect to a particular function

Let $\mathcal L, \mathcal L^*: \Theta \times \mathcal A \to \mathbb R$ be functions. When can $\mathcal L$ be expressed as the convolution of $\mathcal L^*$ with some third function $U$? That is, when ...
0
votes
1answer
35 views

Confusion in MAP estimation

Consider the situation $r = a+n$, where $n \sim \mathcal{N}(0,\sigma_n^2)$. I am having confusion with respect the computation of $p_{a|r}(A)$ for the above scenario. Option 1, $r = a+n \implies a = r ...
0
votes
0answers
30 views

Use MGFs to show convergence in distribution of sum of negative binomial random variables to chi square.

I am working on the following problem for self-study: Let $X_1,...,X_n$ be iid negative binomial(r,p). Assume r is known, and we are interested in computing a confidence interval for p. An ...
0
votes
0answers
13 views

Show that an estimator is not minimax.

I am working on the following problem for self-study: Let $\hat{\theta}$ be an unbiased estimator. Under squared error loss, show that the estimator $c*\hat{\theta}$ is not minimax unless $sup_{\...
1
vote
0answers
44 views

How to estimate the input vector norm of inner product with small space cost?

Given a public vector $A$=$(a_1, a_2, \cdots, a_n) \in \mathbb{Z}^n_q$ and a unknown input vector $B$=$(a_1, a_2, \cdots, a_n)$, which could be sampled from either $\mathbb{Z}^n_2$ or $\mathbb{Z}^n_q$,...
0
votes
0answers
41 views

Unique Maximum Likelihood Estimator which is not sufficient

I'm looking for an example for a statistical model for which Maximum Likelihood Estimator $\hat{\theta}(\vec{Y})$ for $\theta$ exists, and satisfies the following conditions: It is unique. ...
0
votes
1answer
30 views

Estimate variance in a nested study design

I have a "mother" bacteria. I cloned it $R$ times and measured a variable $P$ on these $R$ clones and recorded the mean $\bar p$ and the variance $V$ among these clones. Then, I made $M$ mutants ($M$ ...
0
votes
0answers
20 views

Show Consistency for every component

For $j=1,...,k$ let $t_{n,j}:\Omega_n \rightarrow \mathbb{R}$ be an estimator for $h_j(\theta) \in \mathbb{R}$. Show that $t_n(X)=(t_{n,1}(X),...,t_{n,k}(X))$ is a consistent estimator of $h(\theta)=(...
4
votes
1answer
80 views

What adaptive controller can be used in embedded system with low RAM?

This is not a question for data science, hardware or programming languages. This is a more practical question about adaptive control for embedded systems, but still a math question. I have tried to ...
0
votes
1answer
56 views

If $X_i \sim U(\theta-\frac{1}{2};\theta+\frac{1}{2})$, show that $[X_{(1)},X_{(n)}]$ is a confidence interval

Let $X_1,...X_n$ random sample from $f(x;\theta)=I_{[\theta-\frac{1}{2};\theta+\frac{1}{2}]}(x)$. a) Show that $[X_{(1)},X_{(n)}]$ is a confidence interval for $\theta$. b) Compute the ...
1
vote
1answer
106 views

Finding shortest Confidence Interval for an Exponential Distribution

Let $X$ such that $f_{X}(x\mid\theta) = \theta e^{-\theta x} I_{(0, \infty)}(x)$, where $\theta > 0$. If $[X, 2X]$ is a confidence interval for $\frac{1}{\theta}$: a)Find the confidence ...
0
votes
1answer
74 views

Finding aconfidence interval for $\theta$ of the uniform distribution on $(0, \frac{1}{\theta})$

Suppose $X_1,\ldots, X_n$ are i.i.d. random variables $Uniform (0, 1/ \theta)$. Find a 95% confidence interval for $\theta$. What I tried: $f_{X} (x) = \frac{1}{1/\theta}=\theta, F_{X} (x) = \frac{...
0
votes
1answer
67 views

Finding confidence interval for $\frac{kx^{k-1}}{\theta^k}$

Let $X_1,\ldots, X_n$ are i.i.d. random variables such that: $$f(x;\theta)=\frac{kx^{k-1}}{\theta^k}, x\in (0,\theta)$$ where $\theta \gt 0 $ and $k$ is a positive integer. Find a $100(1-\alpha)% $% ...
0
votes
1answer
43 views

Text book recommendations for statistical estimation theory -specifically MLEs and confidence intervals

I am looking for a textbook on statistical estimation theory. In particular I am interested in a book that explains MLEs and confidence intervals. Preferably accompanied by exercises. The book should ...
3
votes
1answer
597 views

Mean squared error for vectors

I know that when we compare estimators $\hat{b_1}$ and $\hat{b_2}$ to an unknown parameter $\beta$, in classical statistics an estimator $\hat{b_1}$ is said to be "better" than $\hat{b_2}$ if: $$ ...

1
2 3 4 5
10