Questions tagged [covariance]

Questions about covariance, a measure of (linear) association between two random variables.

4
votes
0answers
94 views

Deriving $Cov(X,Y,Z)$, is it even a thing?

So I am trying to derive a nice general formula for $Cov(X,Y,Z)$ and $Corr(X,Y,Z)$, I defined it as such $$ Cov(X,Y,Z) = E[(X-E[X])(Y-E[Y])(Z-E[Z])] $$ $$ Corr(X,Y,Z) = \frac {Cov(X,Y,Z)} {\sqrt{Var[...
3
votes
1answer
5k views

Why are the eigenvalues of a covariance matrix equal to the variance of its eigenvectors?

This assertion came up in a Deep Learning course I am taking. I understand intuitively that the eigenvector with the largest eigenvalue will be the direction in which the most variance occurs. I ...
3
votes
2answers
646 views

Why is covariance something to care about?

The reasoning behind why one might be interested in variance is quite intuitive to me but covariance is not. What information do I attain from covariance? Perhaps an example could help me. In my book ...
3
votes
1answer
923 views

Mutual or pairwise independence needed? Variance of a sum.

This is a simple question: Do we need mutual independence or only pairwise independence in order to state that $$\mathrm{Var}\left[\sum_{i=1}^n X_i\right] = \sum_{i=1}^n \mathrm{Var}\left[X_i\right]?...
3
votes
1answer
335 views

Singular correlation matrix implies linear dependence

Given a random vector $X = \begin{bmatrix} X_1 \\ \vdots \\ X_n\end{bmatrix}$ Suppose that $R = \mathbb{E}[XX^T]$ is the correlation matrix of the random vector $X$ We claim that if $R$ is ...
3
votes
3answers
94 views

Finding covariance for the joint density function $f(y_1, y_2) = 3 y_1$ with $0 \leq y_2 \leq y_1 \leq 1$ and $0$ otherwise

The problem asks to find the covariance. The joint density function is defined as, $$ f(y_1,y_2)= \begin{cases} 3y_1, & \text{for $0 \leq y_2 \leq y_1 \leq 1$}, \\ 0, & \text{...
3
votes
1answer
54 views

Are Covariance Operators based on square integrable stochastic Processes semi-positive definite?

Given a $\textbf{square integrable stochastic process}$ $X$ with $E\left(X\left(t\right)\right)=0$ $\forall t $ the $\textbf{Covariance Operator}$ is defined by \begin{align} C_X: L^2 \rightarrow L^...
3
votes
2answers
360 views

Why does the Eigen decomposition of the covariance matrix of a point cloud give its orientation?

I am working with point clouds and one of the problems I had to face was to find the orientation of a given cluster. Most algorithms I have found suggest that one must first calculate the centroid of ...
3
votes
1answer
336 views

Conditional Variance of the sum of two variables

$\newcommand{\v}{\operatorname{Var}}\newcommand{\c}{\operatorname{Cov}}$I have a simple question that, after thinking for a while, got me confused and I cannot figure it out. Does the following ...
3
votes
2answers
174 views

Question about the relation between Expectation and Covariance

I have a question regarding the relationship between the Expectation $E(X)$ and Covariance $Cov(X, Y)$. For reference, Wolfram MathWorld defines Expectation for a single discrete random variable as: $...
3
votes
1answer
144 views

Minimum Covariance Between Bernoulli Variables

Suppose $X_1,...,X_n \sim Ber(\frac{1}{2})$, and that $COV(X_i,X_j) = COV(X_k,X_l)$ for $k\neq l,j\neq i$. How small can the covariance be? My attempt: We know that $COV(X_i,X_j) = E(X_1X_2)-E(X_1)...
3
votes
1answer
77 views

How to calculuate $E(XY)$?

Here's a question: Tossing a fair cube 6 times. Let $X$ = number of tosses gave 1, and let $Y$ = number of tosses gave 2. Calculate $Cov(X, Y)$. I know that $Cov(X, Y) = E(XY) - E(X)E(Y)...
3
votes
2answers
313 views

Variance of a function of independent random variables

Suppose I have two discrete independant random variables $X$ and $Y$, and that I'm interested in the expected value of the random variable $W$, where: $$ W= \text{sign}(X-Y). $$ So, W is 1 if $X>Y$,...
3
votes
1answer
817 views

How do covariance matrix $C = AA^T$ justify the actual $A^TA$ in PCA?

For computational efficiency in PCA, evaluation of covariance matrix is done as $C = AA^T$ instead of original $C=A^TA$. How can that be justified? What steps are needed then to recover the original ...
3
votes
1answer
118 views

Optimize $M$ such that $MM^T$ is the “smallest” (in a positive semi-definite sense)

I want to solve the following problem: \begin{alignat}{} \underset{M}{\text{minimize}} \quad & MM^T \\ \text{subject to} \quad & MF=I. \end{alignat} where by minimize $MM^T$ I mean to find $...
3
votes
1answer
76 views

An Intuitive Understanding of Covariance

I'm trying to intuitively understand what it means for two random variables to have non-zero covariance. At the moment, I imagine that the two random variables (which have non-zero covariance) both ...
3
votes
1answer
170 views

Broadcasted subtraction notation in mathematics

Consider the calculation of the (uncorrected) sample covariance as an example. Given a set of $N$ vector observations $\mathbf { X } = \{ \mathbf { x } _ 1, \dots, \mathbf { x } _ N \}$, the sample ...
3
votes
1answer
771 views

why is E(XX') the covariance matrix

In regression people often refer to the $X'X$ term as the estimate of $E(xx')$ which makes sense by LLN. However, people also refer to $E(xx')$ as the covariance matrix of $x$ which only seems to make ...
3
votes
1answer
108 views

What is the correct definition of correlation?

According to [2008, Grewal M S, Andrews A P, sec. 3.4] the correlation of two vector-valued process $\vec{x}(t)$ and $\vec{y}(t)$ is defined by $$ \text{corr}[\vec{x}(t_1),\vec{y}(t_2)]=E\langle \vec{...
3
votes
2answers
562 views

What is the inverse of the covariance matrix generated by the exponential covariance function?

I'm trying to analytically find the inverse of the covariance matrix generated by the exponential covariance function (also known as Ornstein-Uhlenbeck kernel) in $\mathbb{R}$, that is $K_{ij} = k(...
3
votes
1answer
101 views

Independence of random variables and covariance in the limit.

Consider two sequences of random variables $(X_n)$ and $(Y_n)$ which converge in distribution to $X$ and $Y$ respectively, where $X$ and $Y$ are independent, but each pair $(X_n, Y_n)$ is not ...
3
votes
1answer
151 views

Calculating the covariance of two random variables

I have 4 random variables: $X\sim Pois(6)$ $Y \sim Geom (\frac{1}{4})$ $Z=6X-Y$ $U=2X-1$ What is the covariance of X and Y if Cov(Z,U)=0? What I did: $Cov(X,Y)=E(XY)-E(X)E(Y)$, I know $E(X)$ ...
3
votes
1answer
3k views

Expected value and covariance of compound Poisson process

$Y_1,Y_2,...$ are independent random variables with a distribution identical to that of $Y$. $N(t)$ is a poisson process with parameter $\lambda$. $$X(t)=\sum\limits_{n=1}^{N(t)}Y_n$$ Find the ...
3
votes
1answer
269 views

Help with covariance of two random variables.

A deck of 52 cards is shuffled and a bridge hand of 13 cards is dealt out. Let X and Y denote, respectively, the number of aces and the number of spades in the hand. (a) Show that X and Y are ...
3
votes
1answer
282 views

roll a dice two times. Calculate covariance $\mathbb{C}ov(X,Y)$ . Calculate correlation coefficient.

Here is the situation: We a roll a fair dice $2$ times. ( independent ) $a)$ $X$ denotes the number of the first throw; $Y$ denotes the sum of the two throws. Calculate $\mathbb{C}ov(X,Y)$. ...
3
votes
1answer
871 views

Converting contravariant (or covariant) tensors to mixed tensors

Sirs, As a physicist studying General Relativity and Quantum Field Theory, I feel like I have my head wrapped around upstairs/downstairs notation fairly well. However, my questions is as follows: ...
3
votes
1answer
112 views

Find all solutions of $f$ such that $ \operatorname{Cov}(f(x),x)=c $ where $x \sim N(\mu,\sigma^2)$ and $c$ does not depend on $(\mu,\sigma^2)$

Find all solutions of $f$ such that $$ \operatorname{Cov}(f(x),x)=c$$ where $x \sim N(\mu,\sigma^2)$ and $c$ does not depend on $(\mu,\sigma^2)$ I think the only solution is $f$ is constant (almost ...
3
votes
1answer
154 views

How can the variance of data be represented as this product of the principal direction and the covariance matrix?

I am coming from reading the selected answer to this question. I have a question about the following bit: It's not hard to show that if the covariance matrix of the original data points $x_i$ ...
3
votes
1answer
176 views

Covariance of 1-D random process is $n\times n$!!!!

I'm reading a tutorial on stochastic processes. There is an example in the tutorial as follows: General Moving Average random process given as $X[n]=\frac{(U[n]+U[n-1])}{2}$ where $E[U[n]]=\mu$ ...
3
votes
1answer
112 views

Matrix Dimension's effect on Positive Definiteness

Recently I have been working on a project involving stock returns and portfolio efficiency. What I thought was a coding error, turned out to be me not understanding matrix math very well. Long ...
3
votes
1answer
119 views

Distribution of Brownian Motion help

If $X = \frac{B_1 - B_3 + B_2}{\sqrt{2}}$ Where $B_t$ is brownian motion at time $t$. And I want to find the the distribution of $X$, how would I do so? $E[X] = 0$ is fairly straight forward. For ...
3
votes
1answer
969 views

Weak/strong law of large numbers for dependent variables with bounded covariance

Let $(X_i)_{i\in\mathbb{N}}$ be a sequence of $L^2$ random variables with expected value $m$ for all $n$. Let $S_n=\sum_{i=1}^n X_i$ and $|\mathrm{Cov}(X_i,X_j)|\leq\epsilon_{|i-j|}$ for finite, non-...
3
votes
1answer
973 views

Removing balls from an urn in pairs: expected number of pairs in which both are red

I am sorry that I cannot make the title more clear. The following is from Sheldon M. Ross: Introduction to Probability Models (11th Edition). I am able to reach the desired answer but actually I don't ...
3
votes
1answer
585 views

Estimate large covariance matrix using few samples.

Let $\mathbf{x}$ be a random vector in $\Bbb{R}^n$, such that $\mathbf{x}\sim N(\bar{\mathbf{x}}, \Sigma)$. $N$ observations of $\mathbf{x}$ are available, say $\{\mathbf{x}_i, i=1,\ldots,N\}$. The ...
3
votes
1answer
1k views

Covariance Matrix of various $x,y,z$ (cartesian coordinates)

I have around 1000 values of gps receiver positions as follows: I have to calculate the covariance matrix with all these values. All of the following values represent a SINGLE POINT. How can I get ...
3
votes
0answers
67 views

Fourier transform and covariant/contravariant vectors

A Fourier transform of a function of $x$ is related by the following equation: $$φ'(p) = \int_{}^{}φ(x) e^{ix·p} \mathrm{d}x.$$ Let's say that $x$ is a contravariant vector. Does it follow that $p$ ...
3
votes
0answers
70 views

Is the off-diagonal part of a covariance matrix, $M = \Sigma -\operatorname{ diag}(\Sigma)$ studied?

If $\Sigma$ is a real, symmetric, positive semidefinite matrix (a covariance matrix), then we can construct $M = \Sigma - \operatorname{diag}(\Sigma)$, where we essentially take the covariance matrix ...
3
votes
0answers
122 views

Explicitly proving that the Hamiltonian is Lorentz covariant

I want to show explicitly that the Hamiltonian $$ H = -\Omega V + \int d\tilde{\textbf{k}}\ \omega (a^\dagger(\textbf{k}) a(\textbf{k}) + b(\textbf{k}) b^\dagger(\textbf{k}) ) $$ is Lorentz ...
3
votes
2answers
452 views

Covariance matrix $P$ for an Extended Kalman Filter not symmetric

I am trying to implement an Extended Kalman Filter (EKF) and it is becoming harder than I thought. I have one question. I noticed that the covariance matrix which should get updated over each ...
3
votes
0answers
79 views

Are autocovariance operators trace class?

Suppose that $\{X_k\}_{k\in\mathbb Z}$ is a weakly stationary sequence of random elements with values in a complex separable Hilbert space $\mathbb H$ and let us define the sequence of autocovariance ...
3
votes
0answers
111 views

AR(p) Covariance Matrix

So I have seen specific derivations of covariance matrices for AR(2) and AR(1) processes. However, I have not seen one for a general AR(p) process. Suppose I know the coefficients of a given AR(p) ...
3
votes
0answers
1k views

Why is the range of covariance infinity?

I don't understand why the range of covariance is $+\infty,-\infty$ $$\operatorname{Cov} (X,Y)= \Bbb E\left[(X-\mu_x)(Y-\mu_y)\right] .$$ Can explain why this is true? All I can think of is if one ...
3
votes
0answers
26 views

covariance and expectional in proccess

Show that the process $X=(W_{\sqrt{t}}I_{(1,2)}(t))_{t \ge 0} \in \mathcal{L}_3^2$. ($W$- Wiener) Additionally calculate, for $t,s \in [1,2]$, $EX_t$ and $Cov(X_t,X_s)$ I have no idea how to start ...
3
votes
1answer
76 views

co-variance between a sample from normal distribution and the sample mean?

$X_1$ is a sample from a normal distribution with mean$=\mu$ and variance $= 1$. The joint distribution of $X_1$ and the sample mean is bivariate normal. I need to find the conditional distribution of ...
3
votes
0answers
3k views

Compute the Var(x1+x2+x3) and other Variances.

The problem said; Let X1, x2, X3 be independent and identically distributed random variables each with mean 0 and variance 1. Below I state the work I did so far, I need help specificaly in point b....
3
votes
0answers
33 views

Linear space transform transformation based on covariance?

I have a linear space of n dimensions with non-overlapping groups characterized by different variation (different covariance matrices). Is there a way to deform non-linearly the space according to an ...
3
votes
0answers
134 views

Tensor notation generally

I'm pretty new to tensors in differential geometry and I have a basic question about the notation used. In general a vector field $X$ can be expressed as $$X=\sum_{i=1}^n X^i \partial_i,$$ where $X^...
3
votes
0answers
129 views

Finite Moments of complicated Stochastic Differential Equation

Suppose I have a SDE of the form: $$dx_i = x_i\left(b_i-\sum_{j=1}^n a_{ij}x_j\right) \,dt + \sigma_i x_i \, d\eta(t)$$ where $\eta$ solves the Ornstein-Uhlenbeck process: $$d\eta(t) = \lambda \eta(...
3
votes
0answers
155 views

sampling from a multivariate guassian: intuition behind using cholesky decomposition

I'm trying to understand how sampling from a multivariate gaussian works and why the cholesky decomposition is a way to do it. Let's say we have a 25 dimensional multivariate with a 25x25 covariance ...
3
votes
0answers
815 views

How does 2D kriging interpolation work?

I have a grid of points Example ...