Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [covariance]

Questions about covariance, a measure of (linear) association between two random variables.

1
vote
0answers
12 views

Simplifying an equation with covariances of random vectors

Let $I=\begin{pmatrix}I_1\\\vdots\\ I_n \end{pmatrix}$ and $X=\begin{pmatrix}X_1\\\vdots\\ X_p \end{pmatrix}$ be two random vectors and $\Omega_I$ a random variable. I am looking for A such that: $$\...
5
votes
3answers
50 views

Prove that $U=Y - E[Y|X]$ and $X$ are uncorrelated

Let $U = Y - E[Y|X]$. How can I prove that $U$ and $X$ are not correlated? I've been doing a lot of things but when I calculate $\text{cov}(U,X)$ I finish with $EXY - EXEY$ and not $0$ which would be ...
0
votes
1answer
14 views

equal covariance matrices

Let's pretend that there exist generators which generate datasets, is it safe to assume that the 2 datasets were generated by the same generator if they present equal covariance structures? In other ...
1
vote
1answer
15 views

Kalman Filter applied to linear discrete time process and interpretation of the estimated covariance matrix

I want to have a deeper understanding of the discrete time Kalman Filter. As a part of this I have modeled a forced, damped, mass spring system numerically in the Jupyter Notebook available here: ...
0
votes
0answers
8 views

How to Modify Measurement-Noise in Kalman Filter from 2D Const-Velocity to 2D Const-Acceleration

After extending a Kalman Filter from 2D Linear Velocity (code) to 2D Constant Acceleration, I realized the State-Predictions have the Y-Position pinned to roughly zero. As you can see, the yellow-...
0
votes
0answers
27 views

Showing $\text{cov} (x_d-x_c, x_b-x_a)=0$

Suppose that $(x_k, \mathcal{D}_k), k=1,...,n,$ is a martingale define on finite probability space. Show that $$\text{cov} (x_d-x_c, x_b-x_a)=0$$ for arbitrary integers $a<b<c<d$. How can I ...
0
votes
1answer
22 views

Calculation of variance of complicated random variable ( white noise discretization )

so I have been doing some state estimation and in one part of my work it is necessary to discretize a continous time differential equation with white noise. I understood the discretization process for ...
1
vote
1answer
19 views

$V(X|Y)=\Sigma_{XX}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX}$

We know that the conditional variance of a multivariate normal vector $(X,Y)$ is equal to the Schur complement: $$V(X|Y)=\Sigma_{XX}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX}$$ However, $\Sigma_{XX}-\...
0
votes
0answers
7 views

Autocovariance function of $ARMA(3, 1)$ process

The ACF of a causal ARMA(p,q) process is given by the following general homogeneous equation: $$ \gamma(h) - \sum_{j = 1}^p\phi_j\gamma(h-j) = 0, \quad h \geq \max(p, q+1) $$ with initial conditions ...
0
votes
1answer
16 views

Question for covariance stationary process

Given a random variable Y with characteristic function C(w) = E[exp(iwy)] . Let the random process X(t) be defined as X(t)=cos(wt+y). Show that the process X(t) is covariance stationary if C(1)-C(2)=...
0
votes
1answer
32 views

Mean vector and covariance matrix

I am given a home work for one subject, but my probability theory course is just started, so I dont have enough information. Could someone help me with that? Given: $$\begin{equation} p_\underline x(...
0
votes
0answers
20 views

covariant derivative and metric compatibility

The requirement that parallel transport preserve the length of vectors is equivalent to requiring that $ \nabla_Z ( g(X,Y) )$ vanish for all vector fields $Z$ and all vector fields $X,Y$ that are ...
0
votes
1answer
28 views

Does a zero conditional expectation imply pairwise covariance is 0?

Suppose in econometrics, $$ y = \beta_{0} + \beta_{1}x_{1} + \beta_{2}x_{2} + ... + \beta_{k}x_{k} + u$$ In Gujarati's book, it says that the following equation (1) $$ E[u | x_{1}, x_{2},..., x_{k}] = ...
0
votes
1answer
18 views

Covariance in and input-output filter [Stationary Stochastic Processes]

The weakly stationary processes $X_t$, $\;t=0$, $\,\pm$$1$, $\,\pm$$2$,$\,\ldots$ and $Y_t$,$\;$ $t=0$, $\,\pm$$1$, $\,\pm$$2$,$\ldots$ are input and output of a linear filter according to $...
1
vote
0answers
26 views

Why two vectors' covariance is the dot product of these two vectors

I am trying to understand the OLS property that SST(Total sum of squares) = SSE (explained sum of squares) + SSR (residual sum of squares). One of the steps is to prove that sample covariance of ...
3
votes
1answer
29 views

Understanding Variance-Covariance Matrix

Suppose data set is expressed by the matrix $X \in\mathbb R^{n \times d}$ where $n =$ Number of samples and $d =$ dimension/features of each sample Then what does $\operatorname{Cov}(X) \in\mathbb R^...
0
votes
0answers
8 views

Finding an expression for the autocovariance $\gamma_k$ of a stochastic process Xt

Find an expression for the autocovariance function of the stochastic process {Xt} for general values of q1, q2, ${\alpha_i}$ and ${\beta_i}$. Where ${X_t}$ = ${\sum_{i=0}^a \alpha_i\varepsilon_i}$ + $...
0
votes
0answers
31 views

Point-wise scaling of stochastic processes on $\mathbb{R}$

Consider a $n$-dimensional random vector $\boldsymbol{X}$ with covariance matrix $\mathbf{\Sigma} = (\sigma_{ij})$. We may apply a element-wise scaling to $\boldsymbol{X}$ by multiplying it with a ...
0
votes
1answer
29 views

A question about variance-covariance matrix.

If $X$ is a vector of random variables such that no element of $X$ is a linear combination of the remaining elements[i.e. there do not exist $a(\neq0)$ and b such that $a'X=b$ for all values of $X=x$],...
1
vote
0answers
16 views

Connection between $\operatorname{Var}(M^n v)$ and largest eigenvalue of $M$

In a proof I am trying to understand, the following is stated: $ M$ is a non-random matrix with eigenvalues $\lambda_i$, $v$ is a random vector, $n$ is a scalar, $\operatorname{Var}(M^n v) \ge \max(|...
1
vote
0answers
34 views

What is the unbiased estimator of covariance matrix of N-dimensional random variable?

Suppose $x$ is a random vector in $\mathbb{R}^n$ which is distributed according to $D$. What is the unbiased estimator of covariance matrix of an N-dimensional random variable? When $y$ is a i.i.d. ...
1
vote
1answer
31 views

What is the relation between $\sum_{i=1}^N x_ix_i^T$ and the covariance matrix?

Suppose $x$ is a random vector in $\mathbb{R}^n$ which is distributed according to $D$. Assume $x_i$ is a sample. What is $\sum_{i=1}^N x_ix_i^T$? How can I relate this to covariance of data $...
1
vote
0answers
12 views

The maximum expected deviation from the sample average matrix?

I have reached to $\mathbb{E}[\|x_tx_t^T - G_t\|^2_F]$, and I need an upperbound for it in terms of probabilistic characteristics of $x_t$ where: $x_t$ is a random vector in $\mathbb{R}^n$ drawn from ...
0
votes
0answers
21 views

Variance of ratio of mean value of functions of a random variable

The problem: I have a random process in which the outcomes of the real valued random variable takes the independent values $x_1, x_2, ...,x_n$. Then I defined $Q$ as $$ Q = \frac{n^{-1}\sum^n f(x_i)...
0
votes
1answer
14 views

Can off-diagonals be nonzero for covariance matrix after PCA?

I have some data for which I found the covariance matrix for: $$\Sigma = \begin{bmatrix}3.33 & −1.00 & 3.33 & 33.00 \\ −1.00 & 1.58 & −1.92 & −13.92 \\ 3.33 & −1.92 & ...
-1
votes
1answer
22 views

Find the covariance of two jointly distributed RV's when you don't know the joint pmf

I have two correlated random variables, X and Y, and I am trying to find their covariance. I know: their individual expected values $\langle X \rangle$ and $\langle Y \rangle$ and their individual ...
1
vote
1answer
42 views

Explaining $\sum_{i=1}^n\sum_{j=1}^nCov(Y_i,B_j)=\sum_{i=1}^n\sum_{j=1,j\neq i}^nCov(Y_i,B_j)+\sum_{i=1}^nCov(Y_i,B_i)$

Sorry for the non-specific title, I was unsure of how to word this. I can't wrap my head around this calculation where they split the summation $$\sum_{i=1}^n \sum_{j=1}^n \operatorname{Cov}(Y_i,B_j) ...
1
vote
0answers
42 views

Computing covariance matrix given partial elements

I have a random vector $\mathbf{X}$ consisting of three random variables, $\mathbf{X} = [X_1 X_2 X_3]^T$. I am given a partial covariance matrix C for $\mathbf{X}$, which is $$ C=\begin{bmatrix} 6 &...
0
votes
0answers
29 views

Eigenvalues of PCA transformed data are the same as original data, why?

We have an arbitrary Matrix $A^{n×m}$ with $m$ measurements and $n$ samples To compute the PCA transform of $A$ we compute the eigenvectors for the covariance matrix of $A$ $COV(A)\ x_A = \lambda \ ...
2
votes
0answers
27 views

Expectation of the product of three normal variables

Let $(X_1, X_2, X_3)\sim N(\mu,\Sigma)$ be a three-dimensional random variable where each coordinates are dependent (i.e. $\Sigma$ has non-zero values outside of its diagonal) I want to know how to ...
0
votes
0answers
27 views

What is $cov (x+y, x+y+z+w) $?

I need your feedback whether I am doing this right. So... $cov(x+y,x+y+z+w) = cov(x,x+y+z+w) + cov(y,x+y+z+w)=$ $cov(x,x) + cov(x,y) + cov(x,z) + cov(x,w) + cov(x,y) + cov(y,y) + cov (y,z) + cov(y,...
0
votes
2answers
32 views

Covariance and Law of Large numbers

Say I am taking the average value of the product of two dependent random variables $X$ and $Y$ sampled an infinite amont of times. That is I am computing $\lim_{n \rightarrow \infty} E \left[ \sum_{i=...
0
votes
1answer
16 views

Compute mean and covariance matrix of $\bar{X}$ from a simple random sample

Given $\{X_\alpha , \alpha =1,...N\}$ a simple random sample obtained from any p-dimensional distribution with mean $\mu$ and covariance matrix $\Sigma$, compute the mean and the covariance matrix of $...
0
votes
0answers
28 views

Terminological conflict in tensor calculus

In the wiki article (I know, not alaways the MOST trusted source, and I’m also not sure how to add the link) for covariance and contravariance of vectors, near the beginning a paragraph starts by ...
1
vote
1answer
40 views

Compute the $Cov(X,Y)$ of the two random variables

Let $X,Y$ be absolutely continuous random variables. $X$ is $Uniform[0,12]$ and $f_{Y|X}(y|x)=\frac{1}{x}$, for $y\in [0,x]$ and $0$ otherwise. We are asked to find $Cov(X,Y)$. $Cov(X,Y)=E(XY)-E(X)E(...
1
vote
0answers
32 views

Is there a shortcut to computing $\text{Cov}(X, Y)$ if we have $\mathbb{E}[X]$, $\mathbb{E}[Y]$ and $\mathbb{E}[Y\mid X]$?

Is there a shortcut to computing $\text{Cov}(X, Y)$ if we have $\mathbb{E}[X]$, $\mathbb{E}[Y]$ and $\mathbb{E}[Y\mid X]$? I have the joint PDF $$f(x, y) = \begin{cases} \lambda e^{-\lambda x}/x &...
1
vote
0answers
19 views

What effects the estimation of a sample covariance matrix?

If I have some data recorded at a sampling rate, $F_s$, from $N$ different sensors attached to some hypothetical experiment. Each signal lasts $T$ seconds. I subtract each of the means from each of ...
0
votes
1answer
24 views

Correlation and Covariance on Standardized X

I am stuck on the following problem: Let $Z_X$ be the standardized $X$, $Z_X=(X-\mu_X)/\sigma_X$, and let $Z_Y$ be the standardized $Y$, $Z_Y=(Y-\mu_Y)/\sigma_Y$. Show that $Corr(X,Y)=Cov(Z_X,Z_Y)=E(...
0
votes
1answer
33 views

Calculate covariance and correlation

$X$ is to $Ber(1/2)$ and $Y$ is to $N(0,1)$. Assume $X$ is indep to $Y$. We define $Z=X+Y, W=X-Y$. Find $Cov(Z,W)$ and $Corr(Z,W)$. Firstly, we know some information: $E(X)=1/2, E(Y)=0$ $Var(X)=1/...
0
votes
2answers
33 views

Let $X$ be the number of heads showing, and let $Y$ be the number of tails showing. Compute $Cov(X, Y)$ and $Corr(X, Y)$.

Suppose you flip four fair coins. Let $X$ be the number of heads showing, and let $Y$ be the number of tails showing. Compute $Cov(X, Y)$ and $Corr(X, Y)$. Although it is not stated, it is clear ...
2
votes
1answer
41 views

Finding the probability of two random variables being equal to 1

Question: A die is thrown $n+2$ times. After each throw a '$+$' is recorded for $4$, $5$, or $6$ and '$-$' for $1$,$2$, or $3$, the signs forming an ordered sequence. To each, except the first and ...
0
votes
0answers
38 views

Expression for Covariance- summation over duplicates

Suppose I have a random variable: $$ Y_{it}=A_{it}+B_{it} $$ which is made up of two random variables $A$ and $B.$ $i$ is the unit of observation that runs through $i=1....N$ for each time period $t=...
3
votes
1answer
113 views

Optimize $M$ such that $MM^T$ is the “smallest” (in a positive semi-definite sense)

I want to solve the following problem: \begin{alignat}{} \underset{M}{\text{minimize}} \quad & MM^T \\ \text{subject to} \quad & MF=I. \end{alignat} where by minimize $MM^T$ I mean to find $...
0
votes
0answers
9 views

Covariance, how to deduce from linear regression

This is mainly concerning machine learning and linear regression, but I think my question still is mathrelated and for that reason I post my question here. I have a linear regression looking like ...
-1
votes
1answer
39 views

sample covariance matrix

Suppose two covariance function estimators, with the same formula except for a coefficient. Then make two sample covariance matrix(SCM) from each of the functions. Why should these matrices differ in ...
0
votes
1answer
36 views

Why is the number of non zero eigenvalues equal to $x^T \Sigma^{-1} x$

I've been reading this code and I found that the number of non-zero eigenvalues of the estimated covariance is equal to $x_i^T \Sigma^{-1} x_i$. I want to know how to arrive at this result. Some ...
0
votes
1answer
74 views

Making Random variable from other Random variables but keep them independent

I am trying to solve the following question. Random Variable X has the following PMF: P(X = 0) = 0.5 P(X = 1) = 0.5 We define another random variable U = XZ. Construct random variable Z , ...
0
votes
2answers
30 views

Confusion in Relationship between regression line slope and covariance

In simple linear regression model between RVs $(X,Y)$, the slope $\hat\beta_1$ is given as $$ \hat\beta_1 = \dfrac{\sum_i^N(x-\overline{x})(y - \overline{y})}{\sum_i^N(x - \overline{x})^2} \tag{1} $$ ...
0
votes
2answers
24 views

Finding covariance function

Let $X(t) = A_0 +A_1t+A_2t^2$, where $A_i's$ are uncorrelated random variables with mean $0$ and variance $1$. Find the mean function and covariance function of $X(t)$. I do not have any clue on how ...
1
vote
0answers
37 views

Updating Cholesky decomposition when deleting one row and one and column.

I found the answer of updating Updating Cholesky decomposition when deleting one row and one and column on Cholesky decomposition when deleting one row and one and column. Is there any generalisation ...