0
votes
0answers
13 views

Common or distinct covariance matrices for mahalanobis distance classifier

I'm very new to pattern recognition so apologies in advance for a newbies question. I've been experimenting with mahalanobis distance to class mean based classifiers to predict diagnoses from ...
0
votes
1answer
44 views

Deriving equation in vector notation

I had some trouble deriving an equation from the book 'Elements of statistical Learning' p. 108 equation 4.9. This heavily relies on linear algebra, so I was wondering how the author came to his final ...
0
votes
0answers
14 views

Comparision between Hamming distance and cosine similarities?

I want to check the similarities between binary vectors of different length and I am using cosine similarities and hamming distances for calculations. These are of length 1000 elements(0 and 1). ...
0
votes
0answers
30 views

Tool for differentiating a matrix and solving a matrix equation (in closed-form)?

Is there any software or online tool that can analytically solve a complex matrix equation, such as the following one in closed form (not numerically)? $$\frac{\partial \; ...
2
votes
0answers
11 views

Showing Hat matrix equal specific values

Consider a one way layout model $y_{ij}$ = $\mu_i + e_{ij}$ (1 $\leq$ i $\leq$ a, 1 $\leq$ j $\leq$ $n_i$) where a = 3 and $n_1$ = 2, $n_2$ = 3, $n_3$ = 4. Show that the hat matrix for this design ...
0
votes
0answers
20 views

Is there a bound on largest eigenvalue for covariance matrix of discrete random variable?

I have a random variable $Z=(Z_1,\ldots,Z_p)$. Each component can take values in {-1,0,1}. Is there a way to bound the largest eigenvalue of Cov(Z)? Actually, I have a latent multinormal variable ...
4
votes
0answers
67 views

Expectation of the absolut value of the determinant of a random matrix

Let $A$ be a random matrix of size $m\times m$ with integer entries $-n\ldots n$. Each value should have the same probability. What is the expectation of the random variable $$X := |\det A|$$ Can ...
2
votes
1answer
30 views

Markov chains by hand

If I have a starting point: $A_T=[0,1]$ at $T=1$ and a one step transition matrix of: $B=\left[ \begin{align} &\frac34 & \frac14& \\& \frac1{20}& \frac {19}{20} &\end{align} ...
0
votes
1answer
27 views

What does iid with zero mean +- 1 entries imply?

I'm looking at a statement in a paper and do not understand what it means: It says : let X be a matrix with i.i.d zero mean with +- 1 entries. What does this matrix look like and what are reasons ...
3
votes
2answers
147 views

Relationship between the singular value decomposition (SVD) and the principal component analysis (PCA). A radical result(?)

I was wondering if I could get a mathematical description of the relationship between the singular value decomposition (SVD) and the principal component analysis (PCA). To be more specific I have ...
0
votes
1answer
27 views

Show that the entries of a matrix are:

For a regression model $y=\beta x$ (note there is no intercept term), show that entries of the matrix $\bf{H} = \bf{X}[\bf{X'}\bf{X}]^{-1}\bf{X'}$ are $h_{ij} = ...
0
votes
0answers
12 views

Derivation of a fixed effects estimator

I've come across parts of a derivation of an estimator in a paper i don't understand. The log likelihood function is where $Y_i=(Y_{i1},...,Y_{iT})'$, and $X_i$ is a $T\times k$ matrix ...
1
vote
1answer
22 views

Solving for a ridge penalty given a fitted model

This is kind of embarrassing; I once knew this stuff, and I've forgotten it. I've got a fitted ridge regression: $$ \hat\beta = \left(X'X+\lambda\right)^{-1}X'y $$ X is n by k y is n by 1 ...
0
votes
0answers
25 views

Condition number of covariance matrix

I am interested in generating a covariance matrix of dimension say 100. I managed to get a correlation matrix with finite condition number. To construct a covariance matrix I need to have standard ...
2
votes
2answers
78 views

what's the relationship of $A*A^T$ and $A^T*A$

For a $m \times n$ matrix $A$, what's the relationship of $A*A^T$ and $A^T*A$? The background of this question is that if we see the row of $A$ as observations and column as variables, $A*A^T$ is the ...
0
votes
0answers
26 views

Multivariate Linear Regression for a System of Linear Equations

I have a system of linear equations in the form of $A\vec{x}=\vec{y}$, where $A$ is an $n\times n$ matrix and $\vec{x}$ and $\vec{y}$ are $n\times 1$ matrices. Suppose $\vec{x}$ and $\vec{y}$ ...
1
vote
1answer
51 views

Inverse of a triangular matrix in a statistical problem

Can any one give to me idea how to solve this problem? Find the inverse of the triangular matrix T, where $ T =\left[ \begin{array}{ccc} I & J & J \\ 0 & I & J \\ 0 & 0 & I ...
0
votes
0answers
11 views

covariance matrix for posteriori estimation

Assume $X, Y$ are two $n$-dimensional random vectors, matrix $U$ is column-orthgonal with dimension $n \times q$ ($q<n$), i.e. $U^{T}U=I$, do we have the following relation: $$cov(U^{T}X|X+Y) = ...
2
votes
2answers
100 views

connection between PCA and linear regression

Is there a formal link between linear regression and PCA? The goal of PCA is to decompose a matrix into a linear combination of variables that contain most of the information in the matrix. Suppose ...
1
vote
1answer
33 views

Meaning of off-diagonal multivariate covariance matrices

My terminology might be a bit sloppy. I apologize in advance. I'm reading on multivariate probabilistic distributions, particularly on Gaussian normal distribution (in the context of probabilistic ...
0
votes
0answers
36 views

Pareto distribution and matrix

I am wondering if there are any bounds are known on the eigenvalues of random matrix whose entries are with Pareto distribution? Thank you.
1
vote
2answers
81 views

Distribution of sum of jointly normal random variables with given covariance matrix

Assume that $(X_1, X_2, X_3)$ are jointly normal random variables with the mean vector $(a,b,c)$ and the covariance matrix: $$\left( \begin{array}{ccc} \sigma_1^2 & \alpha & \beta \\ \alpha ...
0
votes
1answer
26 views

Calculating Partitioned Matrices from subs

Say you have a matrix $A$ which is of size $P\times P$ and a number $Q < P$ can be used to take a partition of said matrix, where: $A_1$ is the upper-left sub matrix, with dimension $Q\times Q$, ...
-1
votes
1answer
78 views

Why is $(x'x)^{-1}x' = x(x'x)^{-1}$

If $(AB)'=B'A'$ then $(x'x)^{-1}x'$ should be equal to $x((x'x)^{-1})'$ . However most econometrics textbooks say that this is equal to $x(x'x)^{-1}$ . What happened to the transpose of $(x'x)^{-1}$? ...
0
votes
1answer
47 views

Finding estimates of a Linear Regression Equation - R

I'm new to Statistics and R. I'm currently looking through a book called "Discovering Statistics using R". Although the book implies you don't need any statistical background, some of the content ...
1
vote
1answer
15 views

how can I write a regression model in matrix form?

The question is $$y_{ij} = \mu + \nu d_{2j} + \beta x'_{ij} +\varepsilon_{ij} \text{ where }i= 1 ,\ldots, n,\ j= 1,2$$ So I am trying to write this model in matrix form. I know how to put ...
0
votes
0answers
84 views

Minimizing the Kullber-Leibler divergence between two multivariate normal distributions

Take two zero-mean multivariate normal distributions: $p=\mathcal{N}(\mathbf{0},\boldsymbol\Sigma)$ and $q=\mathcal{N}\left(\mathbf{0},\left(\mathbf{A}^{T} \boldsymbol\Omega ...
1
vote
2answers
474 views

Expectation Operator on a Matrix

Kind of embarrassing, but I'm completely blanking on what applying the expectation operator to a matrix means, and I can't find a simple explanation anywhere, or an example of how to carry out the ...
1
vote
0answers
96 views

mean and variance of gaussian vector times a matrix times gaussian vector.

I have a vector e of size (N $\times$ 1) and a matrix A of size (N $\times$ M). I need to find the mean and variance of M = $ e^H A A^H e $ where $^H$ represents the complex conjugate transpose. ...
2
votes
5answers
107 views

I am going to learn these Mathematics Topics. I need advice and suggestions please .

I am really horrible when it comes to maths since I never had any maths background in my High school. I am fairly good at programming ( C++ and Java) but without mathematics I cant advance in any ...
2
votes
0answers
135 views

Cauchy Schwarz inequality for random vectors

If $X$ and $Y$ are random scalars, then Cauchy-Schwarz says that $$| \mathrm{Cov}(X,Y) | \le \mathrm{Var}(X)^{1/2}\mathrm{Var}(Y)^{1/2}.$$ If $X$ , $Y \in \mathrm{R}^n$ are random vectors, is there a ...
1
vote
0answers
27 views

What may I infer from knowing that some data set's covariance matrix is singular?

I know what a matrix being singular means. I know what a covariance matrix is. What I'd like to know, though, is what I can infer about a data set if I know that its covariance matrix is singular. ...
0
votes
0answers
23 views

How to extract covariance (error) matrix of independent from covariance matrix of dependent parameters?

Suppose I have a covariance (error) matrix C_xyz for system with 3 parameters: x, y, z. Then it appears that ...
0
votes
1answer
83 views

Least Square with homogeneous solution!

I've read somewhere that: $x=A^+b+(I-A^+A)Z$ is a solution for $Ax=b$ ,when is doesn't have a particular solution. where $A^+$ indicates the pseudo-inverse and $Z$ is an arbitrary vector!!! I know ...
0
votes
0answers
30 views

Multiplication of transpose?

We are learning about the multiple linear model where the variance-covariance matrix of $e$ is the n by n matrix, in a applied linear regression class (in statistics) and I am confused about a matrix ...
0
votes
1answer
29 views

statistics and financial ratios

Currently i am trying to derive the volatility of a financial ratio. I have calculated the volatility (standard deviation) of both the denominator and numerator however I am running into trouble ...
0
votes
1answer
44 views

Formulating regression model in matrix notation

The observations $y_1, y_2, y_3$ were taken on the random variables $Y_1, Y_2, Y_3$ where $Y_1=\theta+e_1$ $Y_2=2\theta - \phi+e_2$ $Y_3=\theta +2 \phi+e_3$ and $E(e_i)=0, var(e_i)=\sigma^2 ...
2
votes
1answer
4k views

How to calculate the covariance matrix

I tried searching a lot on the net and got the following sources: Source One Source Two The first source seems to be incorrect cause when I calculate it using matlab it comes to be different from ...
1
vote
2answers
275 views

How can I show rank $( AB)$ = rank $( A)$?

Can you help me how to show rank $( AB)$ = rank $( A)$ iff null$(A)$ $\cap $ range$(B) = \{0\} $? I can understand that rank $(AB)$ would be no greater than rank A. But not sure how to show this ...
0
votes
1answer
33 views

matrix spectral representation

can you give me any idea how to show this? Show that $A \ge I$ implies $A^{-1} \le I$, and use this result to deduce that $A\ge B\ge0$ implies $0\le A^{-1} \le B^{-1}$. I think this question is ...
1
vote
0answers
48 views

Redundancies in covariance matrix

We know that covariance matrix is symmetrical. I have a vague intuition that there may be some other redundancies beyond that. For example, if A is correlated to B and B is correlated to C then A and ...
0
votes
1answer
109 views

Covariance matrix always positive semidefinite?

I actually was perusing here right now to see if anything could explain a result I have been getting - of a covariance matrix which has a negative eigenvalue, yet the correlation matrix does not - but ...
2
votes
1answer
218 views

standard deviation calculation using covariance?

i require a formula to calculate the standard deviation using variances of three or more variables (lets call them a,b,c) and the covariances between them. To complicate matters more i only need a ...
0
votes
1answer
45 views

Mysterious failure to generate independent set of random variables

After getting my answer in this: Making a well conditioned orthonormal basis I am running into a problem which I do not understand. I have n dependent gaussian random variables that are related by a ...
0
votes
1answer
39 views

Making a well conditioned orthonormal basis

Ok, so I have n dependent gaussian random variables that are related by a known n x n covariance matrix. What I would like to do, is produce a linear transformation to turn them into n independent ...
0
votes
2answers
281 views

Is every symmetric positive semi-definite matrix a covariance of some multivariate distribution?

One can easily prove that every covariance matrix is positive semi-definite. I come across many claims that the converse is also true; that is, Every symmetric positive semi-definite matrix is a ...
2
votes
1answer
47 views

Unbiased estimate of cross-product for unbiased vector

Let $g$ be an unbiased estimate of a vector $G$. Can $g$ be used to find an unbiased estimate of the cross product $GG'$? I'm stuck because naively using $gg'$ is a biased estimator, with the ...
1
vote
0answers
14 views

How to find the best fit when you have a set of ideal ratios, but some of those are below a minimum?

Say you have a set of ideal ratios, whose sum = 1. For example, input = [0.2, 0.2, 0.3, 0.3] But suppose that there is a rule stating that every ratio should be ...
6
votes
1answer
1k views

sum of squares of dependent gaussian random variables

Ok, so the Chi-Squared distribution with n degrees of freedom is the sum of the squares of n independent gaussian random variables. The trouble is, my gaussian random variables are not independent. ...
1
vote
1answer
90 views

Can the diagonal elements of a precision matrix be 0

I have this confusion that why the diagonal elements of the precision matrix cannot be 0? Any suggestions will be much appreciated