Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [covariance]

Questions about covariance, a measure of (linear) association between two random variables.

28
votes
4answers
9k views

Weak Law of Large Numbers for Dependent Random Variables with Bounded Covariance

I'm currently stuck on the following problem which involves proving the weak law of large numbers for a sequence of dependent but identically distributed random variables. Here's the full statement: ...
5
votes
3answers
2k views

Dependence and second Borel-Cantelli lemma.

I'll put the problem and then I'll explain my problem. Problem: Let ${A_n}$ be events such as $\operatorname{Cov}(I_{A_i},I_{A_j})=E[I_{A_i}I_{A_j}]-E[I_{A_i}]E[I_{A_j}]\leq 0,\ \forall i\neq j\tag{...
22
votes
2answers
22k views

What does Determinant of Covariance Matrix give?

I am representing my 3d data in convariance matrix. I just want to know what the determinant of Convariance Matrix gives. If the determinant is positive, zero, negative, high positive, high negative. ...
5
votes
1answer
3k views

What does rotational invariance mean in statistics?

What does rotational invariance mean in statistics? The property that the normal distribution satisfies for independent normal distributed $X_i$, $\Sigma_i X_i$ is also normal with variance $\Sigma_i ...
7
votes
2answers
7k views

unbiased estimate of the covariance

How can I prove that $$ \frac 1 {n-1} \sum_{i=1}^n (X_i - \bar X)(Y_i-\bar Y) $$ is an unbiased estimate of the covariance $\operatorname{Cov}(X, Y)$ where $\bar X = \dfrac 1 n \sum_{i=1}^n X_i$ and $\...
4
votes
2answers
661 views

Covariance matrix of uniform distribution over the Sierpinski triangle

Let $(X_1, X_2)$ be uniform over the unit Sierpinski triangle (represented in Cartesian coordinates). What is its covariance matrix? This is a question I saw in a jobs ad. I would love some leads on ...
2
votes
1answer
56 views

Why is finding $M$ eigenvectors on smaller matrix valid?

I am following this article on face recognition. In "calculating eigenfaces" section, the authors present a solution for the problem of calculating a very big matrix: Let $A_{N^2\times M}$ be an $M$ ...
1
vote
1answer
1k views

Proof of Hoeffding's Covariance Identity

Let $X,Y$ be random variables such that $\text{Cov}\left(X,Y\right)$ is well defined, let $F\left(x,y\right)$ be the joint-CDF of $X,Y$ and let $F_{X}\left(x\right),F_{Y}\left(y\right)$ be ...
0
votes
1answer
692 views

Equivalence of two formulas for variance and covariance

I know two formulas for variance: $$\operatorname{variance}(f) = \operatorname{expectation}((f(x) - \operatorname{expectation}(f^2(x)) \\ = \operatorname{expectation}(f(x)^2) - \operatorname{...
6
votes
1answer
266 views

Multivariate normal density function of function of random variable

Let $X_1,\dots,X_n$ be i.i.d random variables and $g$ be a symmetric function such that $$g(X_i,X_j)\sim N(\mu,\sigma^2)$$ for all $1\le i<j\le n$. I wish to know the density function of the joint ...
1
vote
1answer
212 views

Difference between Covariant derivative notations

I try to understand the difference between the classical definition of covariant derivative : $$\nabla_{i}V^{j}=\partial_{i}V^{j}+V^{k}\Gamma_{ik}^{j}\quad\quad(1)$$ (where index $i$ represents ...
0
votes
3answers
132 views

Uncorrelating random variables.

I was reading this answer, and the first sentence seemed more intuitive at first than after thinking through it: If $\pmatrix{X\\ Y}$ is bivariate normal with mean $\pmatrix{0\\0}$ and covariance ...
6
votes
1answer
319 views

Variance of Z = max(X,Y) where X Y are jointly bivariate normal

I have a question about the bivariate normal r.v.'s Given $X, Y \sim \operatorname{Normal}(0,1)$ with correlation coefficient $\rho$. Let $Z=\max(X,Y)$. Show that $\operatorname E Z^2=1$. My ...
6
votes
3answers
96 views

Prove that $U=Y - E[Y|X]$ and $X$ are uncorrelated

Let $U = Y - E[Y|X]$. How can I prove that $U$ and $X$ are not correlated? I've been doing a lot of things but when I calculate $\text{cov}(U,X)$ I finish with $EXY - EXEY$ and not $0$ which would be ...
3
votes
1answer
962 views

Weak/strong law of large numbers for dependent variables with bounded covariance

Let $(X_i)_{i\in\mathbb{N}}$ be a sequence of $L^2$ random variables with expected value $m$ for all $n$. Let $S_n=\sum_{i=1}^n X_i$ and $|\mathrm{Cov}(X_i,X_j)|\leq\epsilon_{|i-j|}$ for finite, non-...
7
votes
0answers
3k views

Is a symmetric positive definite matrix always diagonally dominant?

A Hermitian diagonally dominant matrix $A$ with real non-negative diagonal entries is positive semidefinite. Is it possible to have a Hermitian matrix be positive semidefinite/definite and not be ...
0
votes
0answers
356 views

A question on conditional expectation leading to zero covariance and vice versa

In my probability class I was tackled with this seemingly weird question involving conditional expectation: Let X,Y be two random variables (it is not mentioned whether or not they are discrete or ...
5
votes
1answer
859 views

Bound the variance of the product of two random varables.

For two random variables $X$ and $Y$ show that the following inequality holds $$\mathrm{Var}(XY)\leq 2\|Y\|_{\infty}^{2}\mathrm{Var}(X)+2\|X\|_{\infty}^{2}\mathrm{Var}(Y).$$ Well first I tried to ...
4
votes
1answer
2k views

What's the best way to think about the covariance matrix?

Let $X$ be a random vector with covariance matrix $\Sigma$. People often describe $\Sigma$ in terms of its components: $\Sigma_{ij}$ is the covariance of the $i$th and $j$th components of $X$. But ...
2
votes
0answers
389 views

Gradient is covariant or contravariant?

I read somewhere people write gradient in covariant form because of their proposes. I think gradient expanded in covariant basis i , j , k so by invariance nature of vectors, component of gradient ...
2
votes
3answers
44 views

Covariance for stochastic variables

if $X$ and $Y$ are stochastic variables with $\operatorname{Var}(X)=1.34$ and $\operatorname{Cov}(X,Y) = 0.64$, find $\operatorname{Cov}(2X, 3X+2Y)$. No ideas on this one, as I don't see any way of ...
1
vote
2answers
114 views

Why does the identity $\mathbb{E}(X) = \mathbb{E}\left(\int \mathbb{1}_{u \leq X}du\right)$ hold?

I'm reading on Hoeffding's covariance identity, the proof of which is neatly covered here, or, in a similar manner, in this MSE post, but I can't seem to fully understand the trick/property used there....
1
vote
0answers
62 views

How to prove inverse direction for correlation coefficient?

To show: If |Cor(X,Y)| = 1, then there exists a, b ∈ R s.t Y = bX + a. Any ideas or hints to proceed? Basically, I've to prove that if the absolute value of correlation b/w two random variables is 1, ...
1
vote
0answers
41 views

How to optimize a singular covariance-weighted residual?

Definitions: $$v(x)\equiv\{g_1(x),g_2(x),\ldots,g_n(x)\}^T$$ $$C\equiv \operatorname{cov}(v)=\langle vv^T \rangle -\langle v\rangle \langle v^T \rangle =\int f(x)v(x)v(x)^T \, dx-\int f(x)v(x) \, dx ...
1
vote
0answers
167 views

Joint Density and Covariance between Two Random Variables with the same Mean and Variance

This seems like a deceptively simple question, (and it perhaps is and I am missing something) but I could not find anything on this. Q1) Are there any general results / relationships to get the ...
1
vote
2answers
279 views

How to find the variance of $U= X-2Y+4Z$? & The Co-variance of $U=X-2Y+4Z$ and $V = 3X-Y-Z$

EDIT If the random variables $X,Y, Z$ have the expected, $$\text{ means: }\mu_{x}=2 \qquad \qquad \mu_{y}=-3 \qquad \qquad \mu_{z} = 4$$ $$ \text{variances: }\sigma_{x}^{2}=3 \qquad \...
1
vote
0answers
242 views

Minimum / Maximum and other Advanced Properties of the Covariance of Two Random Variables [closed]

Are there any advanced results established regarding the behavior of the Covariance of two random variables other than the bounds on the correlation and independence when it is zero etc. which are ...
1
vote
0answers
53 views

Calculation of covariances $cov(x_i^{2},x_j)$ and $cov(x_i^{2},x_j^{2})$ for multinomial distribution

We know that,$ \ \ \ \ cov(x_i,x_j)=-n \ x_i \ x_j$. It can be proven in this manner: We know, $Var(x_i+x_j)=cov((x_i+x_j),(x_i+x_j))$ Now, $cov((x_i+x_j),(x_i+x_j))=cov(x_i,x_i)+2 \ cov(x_i,...
1
vote
1answer
29 views

W and b for LMMSE using covariance(XY)

I would like to calculate the $W_{LMMSE}$ and $b_{LMMSE}$ for X which is a uniform random variable between $-\pi/2$ and $\pi/2$ and $Y=\sin(X)$. I have the following info: $\Sigma_{XY} = 2/\pi$ $\...
1
vote
0answers
89 views

Why is the covariance a measure of how much two random variables change together?

In descriptive statistics, the empirical covariance of a point cloud $\left\{(x_i,y_i) : 1\le i\le n \right\}$ is defined as $$\operatorname{cov}(x,y):=\frac{1}{n}\sum_{i=1}^n\left(x_i-\overline{x}\...
-1
votes
1answer
46 views

Finding variance using the matrix method [closed]

If the random variables $X,Y$ and $Z$ have means: $$\bar{x}=2 \;\;\; \bar{y}=-3 \;\;\; \bar{z}=4$$ and variances $$\operatorname{Var}(x)=1 \;\;\; \operatorname{Var}(Y)=5 \;\;\; \operatorname{Var}(Z)=2$...
5
votes
1answer
3k views

3-sigma Ellipse, why axis length scales with square root of eigenvalues of covariance-matrix

This is my first post on math.stackexchange and i am not a mathematician, but i took some undergrad math courses and some grad mathematical modelling courses, so i come with a basic understanding of ...
3
votes
1answer
141 views

Minimum Covariance Between Bernoulli Variables

Suppose $X_1,...,X_n \sim Ber(\frac{1}{2})$, and that $COV(X_i,X_j) = COV(X_k,X_l)$ for $k\neq l,j\neq i$. How small can the covariance be? My attempt: We know that $COV(X_i,X_j) = E(X_1X_2)-E(X_1)...
3
votes
0answers
68 views

Is the off-diagonal part of a covariance matrix, $M = \Sigma -\operatorname{ diag}(\Sigma)$ studied?

If $\Sigma$ is a real, symmetric, positive semidefinite matrix (a covariance matrix), then we can construct $M = \Sigma - \operatorname{diag}(\Sigma)$, where we essentially take the covariance matrix ...
2
votes
1answer
62 views

Covariance between centered and scaled normal entries of a random vector

From this post and following a tip by @Ian in the comments. If $X_1,\dots,X_n \sim \text{ i.i.d. } N(\mu,\sigma^2)$ with $\displaystyle \bar X= \frac{\sum_{i=1}^n X_i}{n},$ the covariance of the ...
2
votes
1answer
106 views

Gaussian kernels for arbitrary metric spaces

Let $(I,d)$ be an arbitrary (pseudo-)metric space. Define the function $$c(i,i') := \exp\big( - d(i,i')^2 / 2 \big)$$ Is $c$ necessarily nonnegative-definite, hence a kernel function?
2
votes
1answer
81 views

Covariance of number of 1's and the sum of all results of $n$ fair die tosses

A die is tossed $n$ times. Find the covariance of the number of one's and the sum of all results. I started by defining a random variable $X_i$ as $ i=1,\ldots,n$, $X_i=1$ if at the $i$'th toss ...
2
votes
1answer
327 views

Question about creating $2\times 2$ covariance matrix with call option?

I'm completely stuck on how to do this problem. How can you go about calculating the variance of $Y$ and the covariance between $X$ and $Y$? I'm not sure how to use the information given to solve this ...
1
vote
2answers
585 views

Variance of combination of random variables (not independent)

I am doing this question. You have random variables X, Y and Z which have means 1,2,3 respectively and variances 2,4,6 respectively. Also, Cov(X,Y) = Cov(Y,Z) = 1 and Cov(X,Z) = 0. I have to find the ...
1
vote
1answer
2k views

Is a covariance matrix full rank?

I am reading a paper [1] and saw something confusing to me. Is a covariance matrix full rank? Let $\textbf{s}(t)$ be a $d\times1$ complex-valued column vector containing signals from $d$ different ...
1
vote
1answer
840 views

proof of positive semi-definiteness of the precision matrix (inverse of the covariance matrix)

I would like to know how to prove that the inverse of a covariance matrix $\Sigma^{-1}=\Omega$ is positive semi-definite too. My second question can we prove that for any matrix $A\in\cal{M}_{nm}$ we ...
1
vote
0answers
60 views

Why the Ito isometry implies this equality? [duplicate]

If $${\rm Cov}[dW_t,dB_t]=\rho \, dt$$ then why $\mathbb{Cov} \left( \int_0^t \sigma_{1}(s) \mathrm{d} W_s, \int_0^t \sigma_{2}(u) \mathrm{d} B_u \right)$ $\stackrel{\text{Ito isometry}}{=} \...
1
vote
1answer
615 views

What is the derivative of the determinant of a symmetric positive definite matrix?

According to matrix cookbook $$\frac{\partial \det Y}{\partial x} = \det (Y) \text{Tr}\left( Y^{-1} \frac{\partial Y}{\partial x}\right) $$ Now assume $Y$ is a variance covariance matrix $\Sigma$, ...
1
vote
1answer
163 views

The distance distribution from the mean for an n-dimensional normal(Gaussian) distribution

Let's say we have an n-dimensional normal distribution with identity covariance matrix and 0 mean. When we draw random points in this distribution, how do I get the distribution of the distance from ...
1
vote
0answers
691 views

Covariance of Stochastic Differential Equation

What is the general expression for the covariance $cov \left[ X_s X_t \right]$ of a stochastic process given by \begin{equation} dX_t = f(X_t,t)dt + g(X_t,t) dW_t \end{equation} for some general (...
1
vote
1answer
198 views

Co-variance of dependent binomial random variables.

We roll a dice n times , let X (random variable) be the number of times we get 2 and Y- the number of times we get 3. I know that $X,Y$~$bin(n,\frac{1}{6})$, I need to find Cov(X,Y). I thought about ...
1
vote
1answer
84 views

Why is Covariance defined as…

Why is Covariance defined as: $\operatorname{cov}(X,Y) = E[(X-E[X])(Y-E[Y])]$ My book simply states this identity but doesn't explain how it is derived. I know that covariance can also be written as:...
1
vote
1answer
73 views

Prove $\operatorname{Cov}(\overline{X_n}, X_j - \overline{X_n}) = 0$ for independent normally distributed random variables

My homework states the following problem: Let $X_1, \dots, X_n$ be independent $N(\mu, \sigma^2)$ distributed random variables, $\overline{X_n}$ be the sample mean and $S_n^2$ the empirical ...
0
votes
1answer
722 views

Find the covariance of a brownian motion.

Given a standard Brownian motion $\{W_t\}_{t\geq0}$, find the value of $\operatorname{Cov}\left(W_t,W_s\right)$. Is there a way to simplify $$\operatorname{Cov}\left(W_t,W_s\right)?$$
0
votes
2answers
51 views

$X$ a random variable; if $E[(b_i, X)^2] \lt \infty$ for basis $b_i$ of $(V, (\cdot, \cdot))$, then $\text{Cov}(X)$ exists.

Let $V$ be a finite dimensional vectors space with inner product $(\cdot, \cdot)$, and $X$ a random variable in $V$. If $E[(b_i, X)^2] \lt \infty$ for all $i = 1\dots n$, then $\text{Cov}(X)$ exists. ...