Questions tagged [parameter-estimation]
Questions about parameter estimation. Estimation theory is a branch of statistics that deals with estimating the values of parameters based on measured/empirical data that has a random component. (Def: http://en.m.wikipedia.org/wiki/Estimation_theory)
719
questions with no upvoted or accepted answers
10
votes
1
answer
289
views
Estimating Parameter - What is the qualitative difference between MLE fitting and Least Squares CDF fitting?
Given a parametric pdf $f(x;\lambda)$ and a set of data $\{ x_k \}_{k=1}^n$, here are two ways of formulating a problem of selecting an optimal parameter vector $\lambda^*$ to fit to the data. The ...
- 102k
6
votes
0
answers
227
views
Is the Fisher-Information even continuous in a regular statistical model?
Definition (Regular Model [1, p. 203]).
A standard statistical model $\big( X, \mathcal F, (\mathbb P_{\vartheta})_{\vartheta \in \Theta}\big)$, where $\Theta \subset \mathbb R$ is an open interval, $...
- 4,840
6
votes
0
answers
851
views
Expectation of inverse of a symmetric matrix with gaussian elements
Is there any way to calculate:
\begin{equation}
\mathbb{E} \; ( H^{T}H )^{-1}
\end{equation}
assuming that the entries of the matrix $H$ are gaussian random variables with unknown means but same ...
- 331
6
votes
1
answer
249
views
Estimate number of distinct items
I have a large array of $n$ integers, some of which may be repeated, and I want to estimate how many distinct integers are in the array. Say the number of distinct integers is $N$. I can sample with ...
- 69
6
votes
1
answer
8k
views
What is the Fisher information for the parameter $\theta$ in a uniform distribution with likelihood $f(X,\theta)=\frac1\theta 1\{0\le x\le\theta\}$?
If X is U[$0$,$\theta$], then the likelihood is given by $f(X,\theta) = \dfrac{1}{\theta}\mathbb{1}\{0\leq x \leq \theta\}$. The definition of Fisher information is $I(\theta) = \mathbb{E} \left[ \...
- 81
5
votes
0
answers
96
views
Minimax Estimator for Normal Random Vector
Question. Suppose $Y_i \sim N(\mu_1, 1)$. Let $Y := (Y_1, Y_2)$, and $T_y = (Y_1, 0)$. Denote $\Theta$ as the space of all estimators $\mu := (\mu_1, \mu_2)$. Is it necessarily true that $\hat{\mu}$ ...
- 311
5
votes
0
answers
221
views
Intuitive explanation of requirement for achieving the Cramer Rao Lower Bound
this question relates to the requirement for achieving CRLB.
I know that for a random sample $Y_1, \ldots, Y_n$, an estimator $U$ of $g(\theta)$ is MVUE (i.e. it is unbiased and also $\operatorname{...
- 329
4
votes
1
answer
218
views
Estimating the value of $\sigma$ for Brownian motion
Let $X_t=\sigma W_t$ be a stochastic process, where $W_t$ is the Wiener process and $\sigma$ is an unknown parameter.
I want a formula to estimate the value of $\sigma$ (which could not be found in ...
- 7,452
4
votes
0
answers
114
views
Estimating two means of bivariate gaussian
Consider a bivariate gaussian distribution, with parameters $\mu_1$ and $\mu_2$ for the two unknown means, and $\sigma_1$, $\sigma_2$ and $\rho$ for the known covariance matrix,
\begin{align}
\...
- 833
4
votes
0
answers
215
views
Maximum estimator in upper Chernoff bound
I have the following exercise about Chernoff bounds:
Let $X_{1}, X_{2}, \dots, X_{n}$ be independent, identically distributed (i.i.d) random variables with distribution $N(0,\sigma^{2})$. Show that ...
- 41
4
votes
0
answers
934
views
Maximum Likelihood Estimator (MLE ) for the Gaussian-noise simple linear regression model
I am reviewing the method of maximum likelihood and I was looking at the Gaussian-noise simple linear regression model at this link. I understand up to equation (3) on page 2. I assumed it followed ...
- 1,091
4
votes
0
answers
287
views
How to discretize a certain integral
I would like to discretize the following integral operator:
$$\frac{1}{s^2}\sum_{j=1}^N\mu_j\int d\mathbf{x}d\mathbf{x}'f(\mathbf{x})f(\mathbf{x'}) \left(x_j + x'_j - 2\mu_j\right)\hat{a}^\dagger(\...
- 403
4
votes
1
answer
2k
views
ML estimator of an double exponential distribution
Im trying to figure out the ML estimator of $$f_X(x)=\frac{1}{2\beta}\exp\left(-\frac{|x|}{\beta}\right)$$ as well as the variance of this estimator.
So far I have
$$L(\beta;x)=\prod_{i=1}^n\frac{1}{...
- 79
4
votes
0
answers
63
views
Estimating the parameters of stochastic asset price models using Matlab
I am simulating asset prices using different existing stochastic models, as well as my own proposed stochastic models. I would like to estimate the parameters of each model using the historical spot ...
- 41
4
votes
0
answers
192
views
Properties of an MLE based on likelihood constructed from both PDF and CDF
For continuous RV the likelihood function is (typically) given by a product of PDFs, i.e.
$$L(\theta; x_1,x_2, ..., x_n) = \prod_{i=1}^n f(x_i\mid \theta) $$
However, in survival analysis with ...
- 583
4
votes
0
answers
3k
views
Holt's linear trend method and optimal smoothing parameters
I am learning about time series and forecasting and I stumbled across exponential smoothing and other derived methods. In an exponential smoothing model, each prediction is given by a level equation ...
- 781
4
votes
0
answers
314
views
A late-diverging "approximating solution" for a system of functional equations
Peace be upon you,
At the end of this question, I have shown that how computing MLE on an i.i.d Beta distributed data, results in the following system
\begin{align*}
&\begin{cases}
\psi(\alpha)-\...
- 401
4
votes
0
answers
798
views
Convergence rate of an estimator
Say we are interested in estimating some unknown real scalar parameter $\alpha$ using data. Suppose the estimator $\widehat \alpha_N$ of $\alpha$ using the data is consistent. I want to know what it ...
- 2,368
3
votes
0
answers
93
views
Unbiased estimator of a complex function
Let $X_1, X_2 \cdots X_N$ be random variables, which follow a Gaussian distribution.
\begin{equation}
X \sim N(\mu, \sigma^2)
\end{equation}
Let the parameters $\mu$ and $\sigma^2$ be unknown.
I know ...
- 31
3
votes
0
answers
100
views
HMM, reverse engineering the transition matrix
I fitted a 2-states-HMM model last week, and generate a bunch of 1s and 0s, but I forgot to store its parameters (transition matrix). Now, I only got these 1s and 0s, how do I backward/reverse-...
- 159
3
votes
0
answers
106
views
Fisher Information and Cramér-Rao lower bound problem
Suppose $X_1,...,X_n$ are random samples from $N(\mu, \sigma^2)$, where both $\mu$ and $\sigma \gt 0$ are unknown, and let $\theta = \sigma^p$ for some $p \gt 0$. I want to find the Fisher Information ...
- 183
3
votes
0
answers
119
views
Convergence of estimator defined by supremum over measurable sets
Let $X \in L_1$ be a positive random variable on the probability space $([0,1], \mathcal B, P)$, where $\mathcal B$ is the Borel $\sigma$ algebra on $[0,1]$. Consider
$$\phi(A) = E[X\mid A] \cdot I\...
- 325
3
votes
0
answers
155
views
UMVUE for some function $\tau(\theta)$ when $f(x;\theta)=\frac{\ln(\theta)}{\theta -1} \theta^x$
I need to find an UMVUE for some function $\tau(\theta)$ for $X_1, \dots, X_n$ random variables with $f(x;\theta)=\frac{\ln(\theta)}{\theta -1} \theta^x$ where $x\in (0,1)$.
I know that $\sum_{i=1}^{...
- 408
3
votes
0
answers
44
views
Question about the extrinsic information in turbo based equalizer
In light of the question given in
why only extrinsic information is passed in turbo decoding/equalization, why not a posteriori information?
if we have different observation provided to two linear ...
- 159
3
votes
0
answers
27
views
Best representation/model of a 3D object from multiple observations
I have a sensor that detects the corners of a 3D object. There are 9 corners for that rigid object.
The goal is to create the most accurate representation of that 3D object, which can be defined as a ...
3
votes
0
answers
63
views
Question about confidence intervals for the ratio $\frac{\sigma^2_x}{\sigma^2_y}$
Hello I have already one problem that stipulates:
Consider two independent random samples $\mathsf{X_1,X_2,\ldots,X_n}$ and $\mathsf{Y_1,Y_2,\ldots,Y_m}$ from the respective normal distribution $N(\...
- 774
3
votes
0
answers
133
views
Equality for the Hill Estimator
Let us define the Hill estimator by
\begin{align}
\widehat{\gamma}_H:=\frac{\int_{X_{n-k,n}}^\infty \log u-\log X_{n-k,n} \: dF_n(u)}{1-F_n(X_{n-k,n})}=\frac n k \int_{X_{n-k,n}}^\infty \log u-\log ...
- 307
3
votes
0
answers
117
views
Finding minimum energy graph, subject to constraints
I imagine there's a known algorithm for this, but am not totally sure what to search for, and so my search didn't turn up much.
Basically, I have have a set of $N$ nodes $\hat x_i $ in a graph $\hat ...
- 141
3
votes
0
answers
294
views
A priori Estimates of PDE (burgers equation)
How can I find a-priori estimates for u, $u_x$ and $u_{xx}$ which do not depend on time? The two independent PDEs that I would like to find these estimates for are provided below:
$u_t + u^2u_x - \...
- 31
3
votes
0
answers
37
views
Maximizing the Parameters of a Function
I am currently in an assistant student research position (I am working directly with a, for sake of privacy, Dr. R, on developmental Toxicity Dose-Response Modeling simulation).
I am currently using R ...
- 153
3
votes
1
answer
924
views
MLE of variance for a spherical Gaussian
I am trying to implement the X-Means clustering algorithm. In it, the authors use the BIC to determine which model fits the data best. It is explained here: http://www.cs.cmu.edu/~dpelleg/download/...
- 511
3
votes
1
answer
104
views
ML estimation for Weibull
What are the maximum likelihood estimators of $\eta$ and $\beta$ ($\eta>0$ and $\beta>0$) for an i.i.d. sample of size $n$ from the following density: $f(x_i)=\frac{\beta x_i^{\beta-1} }{\eta ^ {...
- 43
3
votes
0
answers
115
views
Trying to show convergence (in probability) of integrals using Taylor expansion
I've been working for a long time now on how to prove a proposition given in a
paper about the asymptotic normality of POT-quantile estimators. Hope somebody
can help me out.
Proposition
(i) Let $...
- 71
3
votes
2
answers
103
views
Estimating time in harmonic signal
I hope someone can help me with the following problem:
Assume a periodic signal of the form
$$\begin{align}
s(t) &= \sum\limits_{p=1}^P \sin(p\Omega_0t)\\
&= \sum\limits_{p=1}^P \sin(\...
- 39
3
votes
1
answer
78
views
Parameter optimization in probabilistic models
Task: Suppose we model a variable $y = Wx + \mu$ as a linear transformation of $x$ plus some Gaussian noise $\mu\sim\mathcal N(0,\sigma I)$. Our aim is to minimize the estimation error of $x$ given $y$...
- 261
2
votes
0
answers
41
views
Minimum variance unbiased estimator for $\mu$ in Normal location model with known but random variance
Consider observing $X \mid \sigma \sim N(\mu, \sigma^2)$ and $\sigma \sim F$ for some known distribution $F$ supported on the positive reals. We observe a single draw $(X, \sigma)$. An estimator $T(X, ...
- 275
2
votes
0
answers
38
views
How to estimate the best variance-proxy of a sub-Gaussian distribution from data?
Suppose we have $N$ independently identically distributed (i.i.d.) samples $X_1,\cdots,X_N$ generated from a sub-Gaussian random variable $X \sim \mathbb{P}$. Then by definition there exists the ...
- 21
2
votes
0
answers
27
views
Analysis of Gaussian mixture probability densities and Gaussian mixture approximations
I am looking for advice on the analysis of Gaussian mixture densities and the approximation of non-parameterizable densities via Gaussian mixture models. Say, for instance, I have some target Bayesian ...
- 21
2
votes
0
answers
10
views
Does "independent variables X and dependent variable Y are jointly gaussian" means "the residual term has 0 conditional mean"?
I saw the following statement in my lecture note:
"The data generation process is $y = x+\epsilon$, whereas in the regression we run y on x so the regression model is $y = \beta_{OLS} x+e$, the ...
- 87
2
votes
0
answers
52
views
Show biasedness of estimator of discrete uniform distribution
Let be $X:\Omega\to[0,1,2,3,\dots, \theta]$ a discrete random variable which obeys a uniform distribution. We don't know $\theta$ and estimate it by the maximum of the sample $(x_1,x_2,\dots,x_n)$, ...
- 4,493
2
votes
0
answers
19
views
Estimating/Tuning a Coefficient from an Objective Function so Optimal Solution Reflects Data
I am working on a problem that is a modified version of a two-knapsack knapsack problem. I am able to find the optimal choices by using Gurobi. However, I would like to estimate a coefficient that is ...
- 21
2
votes
0
answers
67
views
Likelihood of the parameter or the data?
Why do some references say "likelihood of the data" instead of "likelihood of the parameter"?
I learned at the university that the terminology should be probability of the data and ...
- 1,217
2
votes
0
answers
24
views
How to give a high-probability uniform estimation of a potential having access to noisy pointwise estimates of the associated vector field?
As in the title, our goal is to estimate uniformly and with high probability (and up to a constant) a potential having access to noisy pointwise estimates of the associated vector field (i.e., the ...
- 5,725
2
votes
0
answers
50
views
Can we leverage a priori information about monotonicity of parameters to improve estimation?
Suppose $(X_t)_{t \in\mathbb{N}}$ and $(Y_t)_{t \in \mathbb{N}}$ are two independent sequences of i.i.d. Bernoulli random variables, the first one of parameter $x \in [0,1]$ and the second one of ...
- 5,725
2
votes
0
answers
47
views
Why does the unbiased statistic in this example be MVUE immediately?
I am reading "Introduction to Mathematical Statistics" to familiarize myself with sufficient statistics. I got stuck by an example in Sec. 7.3 of the book. Below are page 427 and 428 that ...
- 31
2
votes
0
answers
11
views
Is it possible that the correlation between $\hat{b}$ and $\hat{c}$ can be negative multiple linear regression?
Given the following linear regression model as following, with two explanatory variables $x_1$ and $x_2$ and response $y$
$$y_i=a+bx_{i1}+cx_{i2}+\epsilon_{i}$$
We say that $\hat{a}, \hat{b}, \hat{c}$ ...
- 425
2
votes
0
answers
112
views
Boltzmann distribution - estimator
I have a discrete random variable $X$, with possible values $\{E_k\}$ , following a Boltzmann distribution, with probability mass function given by:
$$ p_k = p(X=E_k) = e^{-\beta E_k}/Z $$
where $Z = \...
- 436
2
votes
0
answers
52
views
Consistently estimate the covariance matrix with weakly correlated observations
Suppose there are T k-dimensional observations following the generating process:
$Y_t = \mu + \epsilon_t$, where $\mu$ is the mean and $\epsilon$ is a weak stationary error with zero mean and time-...
- 91
2
votes
0
answers
74
views
Large sample properties of classical estimator for scale parameter
I've also post this question on Stats Stackexchange as advised in the comment.
Suppose $X=(X_1,X_2,\ldots,X_n)$ are non-negative and have a joint probability density
$$\frac{1}{\sigma^n}f\bigl(\frac{x}...
- 1,394
2
votes
0
answers
228
views
How to optimize the parameter values?
Let us assume that there is the number of infectious individuals, $I(t)$, where $t=1, 2, \ldots, T$ is time. We want to take into account the restrictive measures that affect the infection probability....
- 1,231