Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [covariance]

Questions about covariance, a measure of (linear) association between two random variables.

28
votes
4answers
9k views

Weak Law of Large Numbers for Dependent Random Variables with Bounded Covariance

I'm currently stuck on the following problem which involves proving the weak law of large numbers for a sequence of dependent but identically distributed random variables. Here's the full statement: ...
22
votes
2answers
22k views

What does Determinant of Covariance Matrix give?

I am representing my 3d data in convariance matrix. I just want to know what the determinant of Convariance Matrix gives. If the determinant is positive, zero, negative, high positive, high negative. ...
17
votes
4answers
10k views

When does the inverse of a covariance matrix exist?

We know that a square matrix is a covariance matrix of some random vector if and only if it is symmetric and positive semi-definite (see Covariance matrix). We also know that every symmetric positive ...
10
votes
1answer
115 views

Constructing a probability measure on the Hypercube with given moments

Let $H = [-1, 1]^d$ be the $d$-dimensional hypercube, and let $\mu \in \text{int} H$. Under these conditions, I can explicitly construct a tractable probability measure $P$, supported on on $H$, ...
9
votes
1answer
3k views

How to tell is a matrix is a covariance matrix?

How can we know that these matrices are valid covariance matrices? $$ C= \begin{pmatrix} 1 & -1 & 2 \\ -1 & 2 & -1 \\ 2 & -1 & 1 \\ ...
8
votes
1answer
2k views

Uncorrelated successive differences of martingale

I read somewhere that given a martingale ${X_n}$, the successive differences of the martingale series are uncorrelated, namely $X_i −X_{i−1}$ is uncorrelated with $X_j −X_{j−1}$ for $i \neq j$. I ...
7
votes
2answers
7k views

unbiased estimate of the covariance

How can I prove that $$ \frac 1 {n-1} \sum_{i=1}^n (X_i - \bar X)(Y_i-\bar Y) $$ is an unbiased estimate of the covariance $\operatorname{Cov}(X, Y)$ where $\bar X = \dfrac 1 n \sum_{i=1}^n X_i$ and $\...
7
votes
1answer
425 views

“we note that the matrix Σ can be taken to be symmetric, without loss of generality”

I'm reading the book Pattern Recognition and Machine Learning by Christopher Bishop, and on page 80, with regard to the multivariate gaussian distribution: $$ \mathcal{N}(\mathbf{x} | \boldsymbol{\mu}...
7
votes
0answers
3k views

Is a symmetric positive definite matrix always diagonally dominant?

A Hermitian diagonally dominant matrix $A$ with real non-negative diagonal entries is positive semidefinite. Is it possible to have a Hermitian matrix be positive semidefinite/definite and not be ...
6
votes
3answers
96 views

Prove that $U=Y - E[Y|X]$ and $X$ are uncorrelated

Let $U = Y - E[Y|X]$. How can I prove that $U$ and $X$ are not correlated? I've been doing a lot of things but when I calculate $\text{cov}(U,X)$ I finish with $EXY - EXEY$ and not $0$ which would be ...
6
votes
1answer
319 views

Variance of Z = max(X,Y) where X Y are jointly bivariate normal

I have a question about the bivariate normal r.v.'s Given $X, Y \sim \operatorname{Normal}(0,1)$ with correlation coefficient $\rho$. Let $Z=\max(X,Y)$. Show that $\operatorname E Z^2=1$. My ...
6
votes
2answers
177 views

Covariance of $X^2$ and $X^3$ when $X$ is exponentially distributed

Here is my work.... $\begin{align*} Cov(Y,Z) &= E(YZ) - E(Y)E(Z)\\ &= E(X^2\cdot X^3) - E(X^2)E(X^3)\\ &= E(X^5) - E(X^2)E(X^3) \end{align*}$ And we know $E(X^n) = \frac{n!}{\lambda^n}$...
6
votes
1answer
266 views

Multivariate normal density function of function of random variable

Let $X_1,\dots,X_n$ be i.i.d random variables and $g$ be a symmetric function such that $$g(X_i,X_j)\sim N(\mu,\sigma^2)$$ for all $1\le i<j\le n$. I wish to know the density function of the joint ...
6
votes
1answer
461 views

Largest eigenvalue of a Hermitian matrix

I have two Toeplitz positive semi-definite Hermitian matrices $\mathbf{R}_1, \mathbf{R}_2 \in \mathbb{C}^{M \times M}$. They are in fact covariance matrices satisfing the following conditions: (1) ${\...
5
votes
2answers
510 views

Covariance $X$ and $e^{X}$

How can I prove that covariance of random variable $X$ and $e^{X}$ is non-negative regardless distribution of $X$. I assume it is true.
5
votes
1answer
3k views

What does rotational invariance mean in statistics?

What does rotational invariance mean in statistics? The property that the normal distribution satisfies for independent normal distributed $X_i$, $\Sigma_i X_i$ is also normal with variance $\Sigma_i ...
5
votes
3answers
2k views

Dependence and second Borel-Cantelli lemma.

I'll put the problem and then I'll explain my problem. Problem: Let ${A_n}$ be events such as $\operatorname{Cov}(I_{A_i},I_{A_j})=E[I_{A_i}I_{A_j}]-E[I_{A_i}]E[I_{A_j}]\leq 0,\ \forall i\neq j\tag{...
5
votes
2answers
202 views

Compute $E(X\mid X+Y)$ if $(X,Y)$ is centered normal with known covariance matrix [closed]

The random variable $(X,Y)$ has a two dimensional normal distribution with mean $(0,0)$ and covariance matrix $\begin{pmatrix} 4&2 \\ 2&2 \end{pmatrix}$. Find $E(X\mid X+Y)$. I am ...
5
votes
1answer
2k views

Norm of covariance and precision matrices: is there any meaning?

Let $\Sigma$ be a covariance matrix of some distribution. Then $\Sigma^{-1}$ is the precision matrix. Question: Does $\|\Sigma\|$ or $\|\Sigma^{-1}\|$ have any meaning (for any norm, though I ask in ...
5
votes
1answer
859 views

Bound the variance of the product of two random varables.

For two random variables $X$ and $Y$ show that the following inequality holds $$\mathrm{Var}(XY)\leq 2\|Y\|_{\infty}^{2}\mathrm{Var}(X)+2\|X\|_{\infty}^{2}\mathrm{Var}(Y).$$ Well first I tried to ...
5
votes
2answers
225 views

What linear transformations preserve these conditions?

Main Question Let's define $\Gamma(n)$ as the set of real antisymmetric matrices of size $n$ ($n$ is an even Integer), fulfilling: $$ \forall \gamma\in \Gamma(n) \Rightarrow \gamma^2=-\mathbb I_n$$ ...
5
votes
2answers
160 views

For two random variables $X_1 + X_2$ and $\min(X_1,X_2)$ find the joint-distribution and the covariance

Let $X_1,X_2$ be independent random variables. Moreover $X_1,X_2$ are discrete uniform distributed({$1,...,N$}) We define: $A:= X_1+X_2$ $B:= \min(X_1,X_2)$ Find joint-distribution of $A$ and $B$...
5
votes
1answer
639 views

Understanding the definition of the covariance operator

Let $\mathbb H$ be an arbitrary separable Hilbert space. The covariance operator $C:\mathbb H\to\mathbb H$ between two $\mathbb H$-valued zero mean random elements $X$ and $Y$ with $\operatorname E\|X\...
5
votes
1answer
3k views

3-sigma Ellipse, why axis length scales with square root of eigenvalues of covariance-matrix

This is my first post on math.stackexchange and i am not a mathematician, but i took some undergrad math courses and some grad mathematical modelling courses, so i come with a basic understanding of ...
5
votes
1answer
8k views

Prove that Cov(X,Y)=Cov(X,E[Y|X])

I've been working on this problem for 3 hours now, and my complete lack of progress is getting disheartening. I've looked up definitions, proofs, and have even seen a solution for this particular ...
5
votes
1answer
70 views

A question about a generalization of covariance

Suppose, $H$ is a Hilbert space over $\mathbb{R}$. Suppose, $X$ and $Y$ are random vectors in $H$. Let’s define Hilbert expectation of a random vector $X$ in a Hilbert space $H$ as a vector $v \in H$, ...
5
votes
1answer
169 views

If $X,Y$ are positively correlated, are $f(X),f(Y)$ also positively correlated for a positive increasing $f$?

Suppose that $X$ and $Y$ are positive and square-integrable random variables, such that $X$ and $Y$ are positively correlated, i.e., $\mathbb{E}[XY] - \mathbb{E}[X]\mathbb{E}[Y] \geq 0$. Let $f: \...
5
votes
1answer
163 views

Given a set of 3 orthogonal vectors, how can I find a minimum volume enclosing ellipsoid expressed in the Cartesian coordinate frame?

Generalized Problem Given values to start the problem: A 3D orthonormal coordinate frame (we'll call it the 'V' coordinate frame) that is rotated from the global coordinate system (we'll call it 'G') ...
5
votes
2answers
409 views

Why does the variance formula has a square term?

I was reading about variance from Head First Statistics : And then - Q. I find the reasoning a little absurd. Wouldn't just taking the absolute distance suffice if cancelling out of the terms was ...
5
votes
0answers
186 views

computing the inversion of a matrix which is the sum of a Kronecker product and an identity matrix

I'd like to evaluate a single entry $s_{ik}$ of the $\mathbf{S}$ matrix, using Markov chain Monte Carlo approach. The posterior of $\mathbf{S}$ has a Gaussian likelihood with a covariance matrix $$\...
5
votes
1answer
84 views

Finding two tangential vectorfields with Lie-Bracket equal zero

I've been dealing with the following problem for a while and meanwhile I have no idea how to move on. Maybe one of you can help me? :) Let $M = \mathbb{R}^2\subset \mathbb{R}^3$ a manifold. I need ...
5
votes
0answers
162 views

Applying PCA on covariance matrix in order to generate a new random variable.

Let $\mathbf{x}$ be a random $n\times1$ real vector, $\mathbf{x}\in\Bbb{R}^n$, which is distributed normally with mean $\bar{\mathbf{x}}$ and covariance matrix $\Sigma_x\in\Bbb{R}^{n\times n}$, i.e. $\...
4
votes
1answer
89 views

Is this statement true: $\mathsf{Cov}(X,Y)\geq 0$, $\mathsf{P}(Y>0)=1$, show that $\mathsf{Cov}\Big(X,\dfrac{1}{Y}\Big)\leq 0$

I was trying to prove a problem and I got stuck at a point. The problem leads to a point where I have to show: Let, $X$ and $Y$ are two r.v. If $\mathsf{Cov}(X,Y)\geq 0$ and $\mathsf{P}(Y>0)=1$, ...
4
votes
1answer
454 views

Cholesky decomposition and variance

Well, I've been reading about simulating correlated data and I've come across Cholesky decomposition. Everything seemed clear until I found a couple of posts on this site and Cross-Validated that ...
4
votes
1answer
41 views

Understanding Variance-Covariance Matrix

Suppose data set is expressed by the matrix $X \in\mathbb R^{n \times d}$ where $n =$ Number of samples and $d =$ dimension/features of each sample Then what does $\operatorname{Cov}(X) \in\mathbb R^...
4
votes
1answer
40 views

Random variables and co-variance, Statistics 318

For the given example in the book John E. Freund's Mathematical Statistics with Applications, 8th edition, by Miller and Miller. ISBN: 9780321807090 I've highlighted using colors what numbers ...
4
votes
2answers
661 views

Covariance matrix of uniform distribution over the Sierpinski triangle

Let $(X_1, X_2)$ be uniform over the unit Sierpinski triangle (represented in Cartesian coordinates). What is its covariance matrix? This is a question I saw in a jobs ad. I would love some leads on ...
4
votes
1answer
2k views

Covariance Matrix of mean-centered Random Variables

I read here that for n x d data matrix X, where X is mean-centered, V = $X^{T}*X$ is its covariance matrix. Why is that? As I understand the element $V_{i,j}$ of the covariance matrix is defined by $...
4
votes
1answer
190 views

efficient way to invert a Matrix plus a diagonal one

Let $\Sigma$ be a $n \times n$ matrix, $V$ a $2 \times 2$ matrix and $I_{2 n}$ the identity matrix on dimension $2n \times 2n$. Both $\Sigma$ and $V$ are covariance matrices, thus real, symmetric and ...
4
votes
1answer
207 views

How to prove that $\mathrm{var}(X-E(X|Y)) \leq \mathrm{var}(X)$?

I tried to solve this exercise but got stuck: Assume we have the random variables $X$ and $Y$ where $E(X) = 0$. How can we prove the following inequality $\operatorname{Var} (X-E(X|Y)) \leq\...
4
votes
1answer
2k views

What's the best way to think about the covariance matrix?

Let $X$ be a random vector with covariance matrix $\Sigma$. People often describe $\Sigma$ in terms of its components: $\Sigma_{ij}$ is the covariance of the $i$th and $j$th components of $X$. But ...
4
votes
1answer
128 views

perfectly correlated processes

I am really stuck in this question: Let $\{S_t\}$ and $\{S'_t\}$ be two stochastic processes, satisfying \begin{equation} dS_t = S_t ( \sigma_t \,dB_t + r_t \,dt), \quad dS'_t = S'_t (\sigma'_t \,...
4
votes
1answer
93 views

Covariance of two expectation estimators that used different numbers of samples

Say I have two estimates of the mean of two functions: $$Q^1_{N_1}=\frac{1}{N_1}\sum_{i=1}^{N_1}f^1(X_i), \quad Q^2_{N_2}=\frac{1}{N_2}\sum_{i=1}^{N_2}f^2(X_i),$$ where each sample $X_i$ is identical ...
4
votes
1answer
262 views

Why do polynomial regressions have larger variance at the end?

In reading the book "An Introduction to Statistical Learning with Applications in R", I came across this graph: It shows that the point-wise variance is larger at the ends of the regression curve. ...
4
votes
1answer
155 views

Which matrices are covariances matrices?

Let $V$ be a matrix. What conditions should we require so that we can find a random vector $X = (X_1, \dots, X_n)$ so that $V = Var(X)$? Of course necessary conditions are: All the elements on ...
4
votes
0answers
102 views

The variance of a covariance

The expectation over $x$ of a covariance between variables $A$ and $B$ (where the distribution of $B$ varies according to $x$) is equal to the covariance of $A$ with the expectation of $B$ over $x$: $...
4
votes
1answer
2k views

Degrees of Freedom in Covariance: Intuition?

If we say $\operatorname{Var}(x)$ has $n-1$ degrees of freedom which are lost after we estimate $\operatorname{Var}(x)$, this matches how $n-1$ observations are now constrained to be sufficiently ...
4
votes
0answers
90 views

Deriving $Cov(X,Y,Z)$, is it even a thing?

So I am trying to derive a nice general formula for $Cov(X,Y,Z)$ and $Corr(X,Y,Z)$, I defined it as such $$ Cov(X,Y,Z) = E[(X-E[X])(Y-E[Y])(Z-E[Z])] $$ $$ Corr(X,Y,Z) = \frac {Cov(X,Y,Z)} {\sqrt{Var[...
4
votes
0answers
336 views

What is ${\rm cov}(e_i, \hat y_i)$ in simple linear regression?

The model is $y_i = \beta_0 + \beta_1x_i + \epsilon_i$ What is ${\rm cov}(e_i, \hat y_i)$? What is ${\rm cov}(\epsilon_i, \hat \beta_1)$? What is ${\rm cov}(e_i, \epsilon_i)$? For 1, I am writing $...
3
votes
2answers
635 views

Why is covariance something to care about?

The reasoning behind why one might be interested in variance is quite intuitive to me but covariance is not. What information do I attain from covariance? Perhaps an example could help me. In my book ...