Questions tagged [fisher-information]

For question about fisher information that appears in mathematical statistics.

Filter by
Sorted by
Tagged with
-2 votes
0 answers
17 views

Fisher Information for Beta Distribution when beta=1 [closed]

Let {X1, X2, . . . , Xn} is a random sample coming from Beta(α, β = 1), how can I calculate the Fisher Information I(α)?
antttttt's user avatar
1 vote
0 answers
37 views

What am I doing wrong when finding the Fisher information of a binomial distribution? [closed]

I am trying to find the Fisher information of a binomial distribution where $n=2$ and $n=\theta$. I have the log-likelihood function as $$n\ln2 + \sum^{n}_{x=1}x_i\ln \theta + (2n-\sum^{n}_{x=1}x_i)(...
Peverel Shipley's user avatar
0 votes
0 answers
19 views

Fisher Information and Parameter Space

I am reviewing Fisher information and saw that one of the requirements is that the distribution of the data, say $f(x|\theta)$, involves a parameter $\theta$ that is unknown but lies within a given ...
kpr62's user avatar
  • 571
0 votes
2 answers
78 views

Deriving the Fisher information matrix for a reparameterised gamma distribution

Let $X \sim \mathrm{Gamma}(\alpha, \theta),$ where $$f(x) = \frac {x^{\alpha - 1} e^{-\frac x \theta}} {\theta^{\alpha}\Gamma(\alpha)}.$$ The log-likelihood function can be shown to be $$l(\alpha, \...
Ethan Mark's user avatar
  • 2,167
0 votes
0 answers
99 views

Intuition for vector calculus

In my statistics class, I was introduced to Fisher Information. As it comes from the Taylor Expansion in vector form, I wanted to know terms were ordered in a certain way - whether it was just to make ...
Jackanap3s's user avatar
0 votes
0 answers
27 views

Differential inequality with KL-divergence and covariance

Let $p_t$ and $q_t$ be two families of probability densities on $\mathbb{R}^d$ indexed by time $t\geq 0$. Does the following differential inequality imply that the KL-divergence is identically zero? $$...
Vasily Ilin's user avatar
2 votes
0 answers
37 views

Fisher information with known moments

I have a sequence $X^n$ of length $n$, where each $X_i$ takes a value from a finite set with probability vector $\mathbf{p} = [p_1, \ldots, p_K]^T$, i.e., $X_i \in [K]$, where $p_{X_i}(k) = p_k, k = 1,...
Abas's user avatar
  • 355
0 votes
0 answers
11 views

Difference between Likelihood Estimation and CRLB Estimation for Cooperative Radar

I do not know if this question fits this stack but I do not know if there's other place where I can ask. The question is about the difference between the cooperative/collaborative radar system when ...
Loco Citato's user avatar
1 vote
0 answers
34 views

About Calculate fisher information of normal distribution [closed]

Suppose $X_1, \ldots, X_n$ are iid $\mathrm{N}(0, \exp (2 \gamma))$; that is, the density of $X_i$ is $$ (2 \pi)^{-1 / 2} e^{-\gamma} \exp \left(-x^2 e^{-2 \gamma} / 2\right) . $$ I want to calculate ...
trivial_fish's user avatar
0 votes
0 answers
32 views

Derive Cramer-Rao lower bound for $Var(\hat{\theta})$ given that $\mathbb{E}[\hat{\theta}U]=1$

I am trying to derive the Cramer-Rao lower bound for $Var(\hat{\theta})$ given that we already know $\mathbb{E}[U]=0$, $Var(U)=I(\theta)$ and $\mathbb{E}[\hat{\theta}U]=1$. I am struggling with using ...
Lucas's user avatar
  • 1
0 votes
0 answers
28 views

How do I find the confidence interval for the MLE of a Bradley Terry Model?

I am trying to find out how to find the confidence interval of the Bradley Terry Model. I have the log likelihood equation, and I know I need to use the Fisher Information which is the negative ...
John Smith's user avatar
2 votes
1 answer
353 views

Fisher Information Matrix for Weibull Distribution...

I wish to find the Fisher Information Matrix for the Weibull Distribution... I face two difficulties, I can't find any sufficient guide in internet to lead me to derive the Fisher Information Matrix.....
Gambit's user avatar
  • 285
1 vote
0 answers
50 views

Fisher Information Matrix in machine learning

In these weeks I am reading some machine learning papers dealing with Fisher information theory. Given a parameter set $\Theta \in \Bbb R^d$, I have always defined the Fisher information of a ...
aleand's user avatar
  • 31
1 vote
1 answer
48 views

likelihood and fisher information of two dependent variables

Let $\{X, Y\}$ be two sets of variables that depend on the parameter $\theta$. Let $z_1 = f_1(X,Y), z_2 = f_2(X,Y)$ be two variables constructed from $\{X, Y\}$. The functions $f_i$'s are known. $I_{...
lost-neutrino's user avatar
0 votes
0 answers
62 views

How to show that Fischer Matrix can be obtained from Hessian of Logistic Loss Function

I am solving a progressive question where I need to prove several things. Given, $$ f(\theta) = \frac{1}{m}\sum_{i=1}^m\log(1 + exp(-y_ix_i^T\theta))\text{ , }\sigma(s) = \frac{1}{1 + exp(-s)} $$ Here ...
Hokkyokusei's user avatar
1 vote
0 answers
62 views

What is Fisher Information defined on?

The definition of Fisher Information on wikipedia is: \begin{equation} I(\theta) = E_\theta \left[\left(\dfrac{\partial}{\partial\theta}\ln p\left(X;\theta\right)\right)^2\right] \end{equation} Inside ...
Rolling Island's user avatar
2 votes
1 answer
98 views

A decomposition of Fisher Information: is $I_{\mathbf X,T(\mathbf X)}=I_{\mathbf X}$ true?

Suppose $\mathbf X=(X_1,\dots,X_N)$ where $X_n\sim f_X(\theta)$ are iid random variables parameterized in terms of $\theta$. Furthermore, denote the Fisher information by $$ I_{\mathbf X}(\theta)=-\...
Aaron Hendrickson's user avatar
0 votes
0 answers
44 views

Computational Formula for Geodesic Distance on a Statistical Manifold

I am working on a problem where I need to compute the geodesic distance between two points on a statistical manifold. suppose we have two datasets $X$ and $Y$, I map them to PDF using the Kernel ...
Andyale's user avatar
  • 95
0 votes
0 answers
75 views

Intuition of Wasserstein and Information geometry geodesics

Two important geometries that can be given to the space of multivariate Gaussian distributions are given by the Wasserstein distance and by the Fisher metric (ie. Information geometry). Although there'...
dherrera's user avatar
  • 212
1 vote
1 answer
92 views

Asymptotic confidence interval using MLE and Fisher Information

We have observed x1, x2, ..., xn, independent samples from a Poisson distribution with an unknown mean λ > 0. Let $z_{1-α/2}$ denote the $1-\frac{α}{2}$ quantile of the standard normal distribution....
Miłosz 's user avatar
0 votes
0 answers
69 views

Interpretation of skewness tensor in information geometry

In many textbooks about the information geometry (for instance, "Information Geometry with Applications" by Amari), authors emphasize that statistical manifold is described by tripple $\{\...
Artem Alexandrov's user avatar
5 votes
0 answers
132 views

What is known about pairs of "physical" SDEs and "statistical" SDEs?

Background: Recall that a Langevin motion on a Riemannian manifold $(M, g)$ in $\mathbb{R}^D$ can be written down as the solution to the SDE in a local chart $U\subset \mathbb{R}^d$ (open) $$dY_t = [\...
Nap D. Lover's user avatar
  • 1,209
3 votes
2 answers
253 views

Energy minimization doesn't seem to yield a geodesic

I'm minimizing (through optimization using gradient descent) the energy $E(\gamma)=\int_{t_1}^{t_2} g_{\alpha\beta}(\gamma^{\alpha})'(\gamma^{\beta})'\operatorname{d}\!t$ of a curve using the inner ...
Tarantula's user avatar
  • 196
0 votes
0 answers
29 views

why should conditioning on a random variable subtract information from it?

The defintion of a sufficient statistic is as follows: A statistic $t = T(X)$ is sufficient for underlying parameter $θ$ precisely if the conditional probability distribution of the data $X$, given ...
abhishek's user avatar
  • 901
0 votes
1 answer
66 views

Expected value of a Poisson process in an expression for Fisher's Information?

I was hoping someone could help explain a step? I thought I got it, but reading it today, it's not making sense, and I'm wondering if I'm missing something after all. They go from Eq.1 to Eq.2 (given ...
JHS's user avatar
  • 3
1 vote
1 answer
269 views

How to calculate Fisher Information of exponential family w.r.t. mean parameterization in maximum likelihood estimation?

We have the exponential family: $$ f_\mathbf{X}(x;\theta) = h(x)\exp\{\langle\theta, T(x)\rangle-A(\theta)\} $$ where the parameter vector $\theta$ is often referred to as canonical parameter or ...
Jason Ye's user avatar
0 votes
0 answers
17 views

Riccati transformation about fisher-information

my question is about, how to find the Substitute term f(x)=sqrt(g(x)/g(0)), with g(0)=1 I(μ) =∫ (g′(x))^2/g(x)) dx. (Fisher-Information) How did he finde f(x) ? thanks everyone
Konvergenz's user avatar
0 votes
0 answers
15 views

Estimating one of the means of a bivariate Gaussian when the two means are unknown

Suppose we want to estimate a single mean $\mu_1$ of a bivariate Gaussian, whose covariance matrix is known, but the means $\mu_1$ and $\mu_2$ are unknown. Let $N$ be the number of joint samples. If ...
Daniel S.'s user avatar
  • 833
0 votes
1 answer
169 views

Fisher information of normal distribution where the mean is separated into two components

Say we have a normally distributed variable $X \sim \mathcal{N}(\mu, \sigma^2)$. The Fisher information for $\mu$ is $\mathcal{I}(\mu) = \frac{1}{\sigma^2}$. But if the variable is distributed as $X \...
J. Doe's user avatar
  • 249
0 votes
0 answers
97 views

finding fisher information $e^{\theta-xe^{\theta}}$

given a random sample with the following density $$e^{\theta-xe^{\theta}}$$ Find maximum likelihood estimator for $\theta$ e specify the asymptotic distribution. I useded $\lambda = e^{\theta}$ ...
V013's user avatar
  • 127
0 votes
0 answers
26 views

Minimizing Fisher information with absolute moment constraints

Hello dear community, I apologize for my english, but would like to ask how to show something like this? the fisher-information is defined the convexity is shown a hyperplane is created here we ...
Konvergenz's user avatar
0 votes
1 answer
71 views

Discrete Fisher Information

I have a discrete data distribution: (1,1,2,2,1,1). I wish to estimate its Fisher information. How should I proceed? FI is normally inversely proportional to the variance of the data. The variance of ...
nivedita's user avatar
0 votes
0 answers
18 views

Fisher test to aggregate p-values in a simulation (from Wilcoxon rank-sum tests )- interpret correctly null-hypothesis

I got contradictory results between p-values in single trials, obtained with Wilcoxon test, and Fisher's p-value, obtained from a simulation over 1000 trials. I am confused about correctly ...
user305883's user avatar
1 vote
0 answers
96 views

Fisher Information

In the Fisher information, do we consider joint probability distribution. I have referred Thomas M. Cover's Elements of Information theory. The authors mention that - $$FI(\theta) = E_{\theta}({\frac{\...
nivedita's user avatar
0 votes
0 answers
27 views

Confused about Fisher Information matrix derivation for the unknown parameter vector in MIMO Radar System

I am studying the MIMO Radar System from the book MIMO Radar Signal Processing. The derivation about FIM for widely separated antennas system in Chapter 9 (9A.6) makes me confused, since I have no ...
Loco Citato's user avatar
0 votes
1 answer
46 views

Fisher Information of a Function

Fisher Information is defined as $I_{\lambda}=E[(\frac{\partial f(y,x)}{\partial \lambda})^2]$. I want to show that $I_{g(\lambda)} = I_{\lambda}(g'(\lambda))^2$. All I managed to do is $I_{g(\lambda)}...
You1234's user avatar
  • 301
0 votes
0 answers
21 views

Tail bound version of Cramer-Rao

The Cramer-Rao bound is well-known: for an unbiased estimator $\hat{\theta}$, $$\text{var}(\hat\theta(X_1,...,X_n)) \geq\frac{1}{n}\mathcal{I}(\theta)^{-1}),$$ where $ X_1,...,X_n\sim P_\theta$ i.i.d.,...
RS.'s user avatar
  • 117
1 vote
1 answer
280 views

Why is only the Fischer information a Riemannian metric?

I recently discovered that the Fischer information induces a Riemannian metric on statistical manifolds. On discrete probability distributions the Fischer information is given by $$\mathcal I_{j,k}(\...
AccidentalTaylorExpansion's user avatar
1 vote
0 answers
51 views

Local existence of solution of a PDE in the context of manifolds

Let $X = (x^1, x^2, \dots, x^n)$ and $Y = (y^1,y^2,\dots,y^n)$ be coordinate systems onto a smooth manifold $M$. Denote by $\partial_i$ the i-th coordinate field of $Y$. Now, I'm trying to solve the ...
Gabriel's user avatar
  • 21
1 vote
0 answers
84 views

Equivalence between CRLB and uncertainty propagation formula

I am looking for a link between the "uncertainty propagation formula" and the Cramér-Rao lower bound (CRLB) of a function of normally distributed independent variables (not necessarily ...
Mammouth's user avatar
  • 123
1 vote
0 answers
75 views

Fisher information for log-convex densities

Suppose there is a density $f(x; \theta)$ over $X\subseteq\mathbb{R}_{+}$ with parameter $\theta\in \Theta \subseteq \mathbb{R}_{+}$. Both sets are bounded. Suppose as well that $f(x; \theta)$ is ...
Caio Lorecchio's user avatar
0 votes
2 answers
74 views

Finding the Fisher information given the density

For $X_1,...,X_n$ i.i.d. continuously distributed random variables with Lebesgue density $f_{\theta}(x)=\theta(\theta+1)x^{\theta-1}(1-x)\mathbf{1}_{(0,1)}(x)$ (Note: $\mathbf{1}_{(0,1)}(x)$ is an ...
Eric L.'s user avatar
  • 119
1 vote
1 answer
456 views

OLS MLE and Cramer-Rao for Basic Linear Regression

In general I know Maximum Likelihood Estimation of parameters should have a variance bounded by the Cramer-Rao Bound (i.e variance of estimated parameter should go to $E [I(\theta \vert x)^{-1}]$. For ...
Wannabe Chemist's user avatar
0 votes
1 answer
189 views

Deriving Wald Statistic for Gamma Sample with known Alpha parameter.

Let $X_1, X_2, ..., X_n$ be a random sample from a $Gamma(\alpha, \beta)$ population. Assume $\alpha$ is known and $\beta$ is unknown. Consider testing $H_0: \beta = \beta_0$. I'm trying to derive a ...
Pierre's user avatar
  • 85
0 votes
1 answer
29 views

"Amount of information about parameters in random variables" means what?

I can't figure out what a question is asking, nor what an answer would look like. Given $2$ iid normal random variables $X_1$ and $X_2$, "find the amount of information about $\mu$ and $\sigma ^2$...
RobertTheTutor's user avatar
2 votes
1 answer
38 views

$\frac{\partial \text{score}(x; \lambda)}{\partial \lambda}$.

Apparently I forgot how to do take a derivative.. This is the score of some Poisson distribution random variables: $\text{score}(x; \lambda) = \frac{x}{\lambda} - 1$. Now I want to take the derivative ...
Kev's user avatar
  • 157
2 votes
1 answer
115 views

How to calculate the Cramer-Rao lower bound

I am having trouble calculating the Cramer-Rao lower bound for an unbiased estimator $\theta$ for the following function: $$f(x|\theta) = \theta^{2} x e^{-\theta x} , x>0$$ I have already ...
EBecauseWhyNot's user avatar
2 votes
1 answer
261 views

Construct a confidence interval for $\theta$

Let $X_{1}, \cdots, X_{n}$ be a random c.i.i.d sample such as, given $\theta$, $X_{1} \sim \mathcal{N}(0,\theta)$. Construct a confidence interval for $\theta$ using asymptotic results. This question ...
Frrr's user avatar
  • 31
2 votes
1 answer
671 views

Fisher information of poisson distributed random variable

Let's consider a printer queue. We know that the expected number of printer jobs almost obeys a Poisson distribution, so $P_{\vartheta}(X=k)=e^{-\vartheta}\frac{\vartheta^k}{k!}$, where $\vartheta\in]...
Philipp's user avatar
  • 4,493
0 votes
1 answer
99 views

Calculate Fisher information of $n$ independent random variables

Let be $X:=\left(X_1,\dots, X_n\right)$ a vector of independent random variables and $P_X(\theta)=f(X_1,\theta)\cdot f(X_2,\theta)\dots \cdot f(X_n,\theta)$ their probability distribution. We assume ...
Philipp's user avatar
  • 4,493