Questions tagged [fisher-information]

For question about fisher information that appears in mathematical statistics.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Statistics with Fisher formalism : loss or gain of information with data cross-correlations

I am currently working on Fisher's formalism which is part of a more general theory, that of information. My problem applies to estimating cosmological parameters from input data with the Fisher ...
0
votes
0answers
35 views

Fisher Information in Statistical Mechanics

I am studying the canonical ensemble and it seems to me there is an analogy between derivatives of the partition function, which can extract energy momenta for the system and Fisher score /information....
0
votes
1answer
45 views

Fisher Information of log-normal distribution

I have the pdf of a log-normal distribution $$ f(y;\theta)= \frac {1}{y\sqrt{2\pi\theta}}\exp \left(-{\frac {(\log y)^2}{ 2\theta}}\right)$$ for $y>0$ and $\theta>0$ and $f(y; \theta) = 0$ ...
0
votes
1answer
45 views

Sign of a Fisher Information changes depending on the formula I use

I am trying to compute the Fisher Information for θ in the following scenario: ...
0
votes
1answer
86 views

Fisher information matrix for normal distribution

The below is captured from my lecture note, for the third column of first and second row and for the third row of the first and second column, is it because the summation of $x_i - \alpha - Bz_i$ ...
3
votes
1answer
101 views

MLE of simultaneous exponential distributions

Given the $X_i\sim \text{exp}({\theta})$ and $Y_i\sim \text{exp}(\frac{1}{\theta})$, where $X_i$ and $Y_i$ are indpendent, with the same $\theta>0$. I have to find the MLE and its distribution. I ...
0
votes
0answers
40 views

Connection between two definitions of Fisher information.

In statistics, the Fisher information is commonly defined as the covariance matrix$\operatorname{Cov}_\theta X$ of the random vector $X$, with $X_i = \frac{\partial}{\partial \theta_i } \left(\log(f(X,...
0
votes
0answers
17 views

The Fisher infromation for a Weibull distribution (2 parameteres)

I have an iid sample $Y_1, Y_2, ..., Y_n$ of a Weibull$(\alpha, \beta)$ with the density given by $f(y)=\beta \alpha y^{\alpha-1} exp(-\beta y^\alpha)$, with $\alpha, \beta>0$ and $\theta=(\alpha,\...
0
votes
0answers
15 views

Matrix Derivative of Fisher Discriminant Analysis

Let $Z_c \in \mathbb{R}^{D\times N_c}$ is column matrix which includes mapped data of class C, and $\alpha_c = \frac{1}{N_c}$ where $D:$ dimension, $N_c:$ data number of class C. $\mathbf{i}:$ One ...
1
vote
0answers
27 views

How is the replicator dynamic a gradient flow of the Fisher information metric?

I am trying to understand how the replicator dynamic can be derived as a gradient flow of the Fisher information metric (aka Shahshahani metric). I have a question about understanding a particular ...
0
votes
1answer
496 views

Fisher information of normal distribution with unknown mean and variance?

I am asked to find the fisher information contained in $X_1 \sim N(\theta_1, \theta_2)$ (ie: two unknown parameters, only one observation). How would I find the Fisher information here? I know that ...
0
votes
0answers
41 views

Finding Fisher information

Let $X$ distribution belongs for the family $\mathcal{P}\{P_{\theta}, \theta \in \Theta \}$. We need to find Fisher information $I(\theta)$ according $n$ simple sample, when $P_{\theta}$ is $N(\mu,\...
0
votes
0answers
19 views

Instead of computing Fisher information, why don't we just evaluate the second derivative of log(L) at the mle?

So I (sort of) understand that Fisher information is $I(\theta) = -E_{\theta}[\frac{d^2}{d\theta^2} log f(x|\theta)]$, but what I'm confused by is why we bother taking the expectation with respect to $...
1
vote
0answers
16 views

Standard deviation of k-th class, corresponding to j-th feature

I'm trying to calculate Fisher's score for a feature from a data set. The fisher score for the $j$-th feature: $$F(x^{j}) = \frac{\sum^c_{k=1} n_k(\mu^j_k - \mu^j)^2{}} {\sum^c_{k=1}n_k(\sigma^j_k)^2}...
0
votes
0answers
26 views

Fisher's formalism - which conditions to get equivalence between Fisher matrix and inverse of covariance matrix

I am currently studying Fisher's formalism as part of parameter estimation. From this documentation : They that Fisher matrix is ​​the inverse matrix of the covariance matrix. Initially, one builds ...
0
votes
1answer
46 views

How to prove alternative form of Fisher information

Assuming the FI regularity conditions hold. The Fisher information matrix $I(\theta;X)$ about $\theta$ based on $X$ is defined as the matrix with elements $$ I_{i,j}(\theta;X)=Cov_{\theta}\Big(\frac{\...
1
vote
0answers
14 views

Why can the best Fisher's linear discriminant vector be solved by $w_{lda}=S_W^{-1}(m_2-m_1)$?

Why can the best Fisher's linear discriminant vector be solved by $w_{lda}=S_W^{-1}(m_2-m_1)$? Background: https://www.cs.ccu.edu.tw/~wylin/publications/ieee_smc.pdf pages 11-12.
0
votes
0answers
17 views

How to derive the optimal bayesian solution to a model of two normal distributed populations

In the "Introduction" section of the paper Support-Vector Networks, it mentioned Fisher's solution to a model of two normal distributed populations: My questions are: How to derive equation (1)? I ...
0
votes
1answer
54 views

Exponential map on the Fisher manifold for exponential family distribution

So, I don't really understand too well Diff. Geometry and Manifolds currently. Hence, I've started studying it and it is very interesting. However, atm I just need to understand how to compute the ...
0
votes
0answers
9 views

What/when does one need $E$ in expected Fisher information for?

What/when does one need $E$ in expected Fisher information for? Since I read an example which merely calculated the second derivatives, put a minus on them and then wrote them in matrix form. It ...
0
votes
1answer
32 views

What lemma for product of derivatives equals the n-derivative?

Why must one require regularity in order for Fisher information to be $E\bigg( \frac{\partial^2 l(\theta, X)}{\partial \theta^2} \bigg)$? Rather than $E\bigg( \frac{\partial l(\theta, X)}{\partial \...
0
votes
0answers
24 views

How does one usually evaluate the expected value of observed Fisher information?

How does one usually evaluate the expected value of observed Fisher information? That is what does $$\mathcal{I}(\theta)=-E\left[\frac{\partial^2}{\partial\theta^2}l(X,\theta)\mid\theta\right]$$ ...
0
votes
0answers
29 views

Prove information inequality without deriviatives

Suppose we have a random variable $X$, where the support of its pdf $f(x | \theta)$ does not depend on $\theta$. Let $\hat\theta(X)$ be a statistic with finite expectation $\psi(\theta)$. Show ...
3
votes
1answer
79 views

Comparing Fisher Information of sample to that of statistic

Let $X_1,...,X_n$ be Bernoulli($p$) where $p$ is unknown, and $n>2$, and let $T=X_1+X_2$. My task is to calculate the information about $p$ in the entire sample and compare it to the information ...
0
votes
0answers
9 views

Asymptotically biased maximum likelihood estimator

I have a model such that a single experiment $\boldsymbol{y}$ driven by an unknown parameter vector $\boldsymbol{\theta} \in \mathbb{R}^4$ consists of $N$ binary observations $\boldsymbol{y} = (y_1, ...
0
votes
1answer
62 views

Derivative of quadratic form involving inverse of matrix-valued function of a scalar: $\frac{\partial}{\partial \gamma} y^T V^{-1}(\gamma) y$

Suppose we have a non-singular matrix-valued function of a scalar defined element-wise $$V:\mathbb{R}\rightarrow \mathbb{R}^{n\times n}\qquad V(\gamma)=\left\{v_{ij}(\gamma) \right\}$$ and a map $$Q:\...
0
votes
0answers
85 views

Derive the asymptotic distribution of maximum likelihood estimator

Suppose that $X_1,...,X_n$ is a random sample from a distribution with pdf : $$ f(x,\theta)= \frac{\theta^3}{2}x^2e^{-\theta x} \space for \space 0,x,\infty$$ I found that $l(\theta)=-nlog(2)+3nlog(\...
1
vote
0answers
17 views

How can one define the quantum interferometric power in the case of a multiparametric system using the quantum Fisher information matrix?

please, was the quantum interferometric power defined in the case of a multiparametric system? I know that in the case of a single parameter, the quantum interferometric power is defined as the ...
1
vote
1answer
95 views

Second Derivative with respect to a Matrix

I have a question regarding (second order) derivative with respect to a matrix. I encounter this question because I am calculating Fisher Information, but I guess the context is not very relevant in ...
1
vote
0answers
784 views

Fisher information of reparametrized Gamma Distribution

I'm trying to solve the following problem: Let $X_1,...,X_n$ be iid from $\Gamma(\alpha,\beta)$ distribution with density $f(x)=\frac{1}{\Gamma(\alpha)\beta^\alpha}x^{\alpha-1}e^{-\frac{x}{\beta}}$....
0
votes
1answer
94 views

Flatness of a statistical manifold with Fisher information metric

Let $\mathcal{M} = \{p_\theta := p(\cdot | \theta), \theta \in \Theta\}$ be a statistical manifold with Fisher information metric: $$g_{{jk}}(\theta )=\operatorname {E} \left[\left({\frac {\partial }{\...
0
votes
1answer
39 views

Fidning the Cramer Rao inequality using Fisher Information

$$I(\theta) = E\left[\left(\sum^{n}_{i=1} \frac{\partial}{ \partial\theta}\ln\{f(X_i;\theta\} \right)^2\right]$$ Is the definition of the Fisher criteria and in the text it says that because $X_i$'...
0
votes
0answers
65 views

Fisher information $I(\theta,X)$,where X=(Y,Z)

Let $Y$ be the number of patients (out of n) who survive for one year after an operation. Let $Z$ be the number of patients who survive for 5 years. Let $\theta=(p,q)$ and suppose that we model $Y\sim ...
0
votes
2answers
108 views

Fisher Information Matrix

I am currently taking a module in predictive analytics and I have come across the Fisher Information Matrix. Can somebody explain why this is so important, its use and why we need to calculate it. ...
0
votes
1answer
49 views

Find parameter transformation of probability distribution such that the transformed parameters are orthogonal

I'm looking for a parameter transformation of a probability distribution such that the resulting parameters are orthogonal. That is, the off-diagonal elements of the Fisher Information matrix of the ...
2
votes
1answer
58 views

Compactness of the Gaussian random variable distribution as a statistical manifold?

This question is related to Topology of statistical manifolds, but it was never answered to its fullest. To be concrete, let's just work with the univariate Gaussian distribution $\mathcal{N}(\mu, v)...
2
votes
0answers
273 views

Fisher information of sufficient statistic

Why is it true that if $X \sim f_{\theta}(x) $ (let's assume for simplicty that theta is one dimensional) is some random variable and $T(X)$ a sufficient statistic then $I_{X}(\theta)$ (Fisher ...
0
votes
1answer
81 views

Fisher convergence as sample goes to infinity.

I was wondering how does the Fisher law behave when $n => \inf.$. $F_{q, n-p} = \frac{SSE_0-SSE/q}{SSE/(n-p)}$ I expect the test statistics goes lower as n goes up but does the pdf of the law ...
0
votes
1answer
104 views

Fisher Information for a misspecified model

Up until a few days ago I was thinking that the following two forms of the Fisher Information are "always" equivalent: $$(1) \quad \mathcal{I(\theta)}= E_\theta [\frac{\partial \log \ell(y;\theta)}{\...
3
votes
1answer
893 views

What is the Fisher information for a Uniform distribution?

If X is U[$0$,$\theta$], then the likelihood is given by $f(X,\theta) = \dfrac{1}{\theta}\mathbb{1}\{0\leq x \leq \theta\}$. The definition of Fisher information is $I(\theta) = \mathbb{E} \left[ \...
1
vote
1answer
177 views

Relationship between Fisher information and KL-divergence

Let $p(x,\theta)$ be a density function and the KL-divergence be given by $$K\left(\theta,\theta^{'}\right)=\int \log \left( \frac{p(x,\theta)}{p(x,\theta^{'})} \right) p(x,\theta)dx$$ Let the fisher ...
0
votes
1answer
96 views

Why does $\int(\theta-\hat{\theta})\exp\left\{-\frac{n}{2}(\theta-\hat{\theta})^TJ(\hat{\theta})(\theta-\hat{\theta})\right\}\,d\theta=0$?

I'm reading about the derivation of Bayesian information criterion (page 216) and in the proof, it is given as fact that: $$\int(\theta-\hat{\theta})\exp\left\{-\frac{n}{2}(\theta-\hat{\theta})^TJ(...
1
vote
1answer
487 views

Can Fisher information be zero?

I have been looking at this problem and attempted calculating the Fisher information. I got zero, so I become unsure and wanted to my answer checked. $X_1,X_2, ..., X_n$ are i.i.d. with density $\...
1
vote
1answer
88 views

Fisher information is non-increasing under well-behaved transformations

I'm looking for a proof of the following fact: Under assumptions of regularity by Cramer–Rao, if $T(X_1,\ldots,X_n)$ is an statistic whose induced distribution satisfies also de Cramer–Rao ...
1
vote
0answers
77 views

How can the covariance of 1xn score function be computed for fisher information

Fisher Information can be calculated using the covariance of score function. For logistic regression the log-likelihood is taken to obtain the score function which is ...
0
votes
1answer
235 views

Intuition on fisher information on $n$ observations and its relationship with one observation

For a sample with a set of iid random variables, $X_1,X_2,\ldots,X_n$ with common parametric family $f(x;\theta)$, its Fisher information is defined to be: $$I_n(\theta):= -\mathbb{E}\left(\frac{\...
1
vote
0answers
47 views

Updating weight based on fisher scoring algorithm [wikipedia]

My understanding on weight update for fisher is based on wikipedia description. The fisher information is said as the expectation of hessian. Can someone explain how is the expection for each entry of ...
0
votes
1answer
65 views

Fisher information and moment generating functions

I'm new on Mathematics Stack Exchange, and it's awesome to be a new member of this awesome community. Please correct me if I have made any errors in terms of following the rules for asking questions ...
2
votes
0answers
130 views

Coordinate independent definition of Fisher metric on statistical manifolds

Is there a manifestly coordinate independent definition of the Fisher metric? I was reading Methods of Information Geometry by Amari and Nagaoka and Information Geometry and Its Applications by Amari, ...