0
votes
1answer
26 views

Product rule type formula for $\nabla \cdot (M(x)v(x))$ where $M(x)$ is a matrix and $v(x)$ is a vector?

Let $M(x)$ be a $n\times n$ matrix with each element depending on $x$ a variable on $\mathbb{R}^n$. Let $v(x)$ be a vector. Is there a nice product rule formula for $\nabla \cdot (M(x)v(x))$?
3
votes
3answers
70 views

Vector multiplication. Difference between scaler and dot product?

We just started a new class where the first topic is briefly talking about vectors and vector multiplication. All tying this into neural networks. I am a bit behind with the understanding of what the ...
1
vote
3answers
72 views

Differentiating a non-linear functional with respect to a vector

I have the functional: $$F=v^T\times A \times v$$ Where $A$ is a function of $v$. The non-linear system of equations necessary to find $v$ is obtained doing: $$\frac{\partial F}{\partial v}=0$$ ...
4
votes
2answers
64 views

What have Vectors and Matrices got to do with each other?

In my undergraduate course work I learnt Vectors (as in those in vector space with magnitude and direction) separately from Matrices - an $n \times m$ array of numbers. However, after sitting in for a ...
2
votes
1answer
56 views

To show the inequality $\|A\|\geq\max\{\|u_1\|,\ldots,\|u_q\|,\|\vec{v_1}\|,\ldots,\|\vec{v_q}\|\}$

Let $A\in$ $\mathbb{C}^{p\times q}$ with column $u_1,\ldots,u_q$ and rows $\vec{v_1},\ldots,\vec{v_p}$. show that $$\|A\|\geq\max\{\|u_1\|,\ldots,\|u_q\|,\|\vec{v_1}\|,\ldots,\|\vec{v_q}\|\}$$ and ...
0
votes
1answer
223 views

How would one use matrices to find a normal unit vector?

A recent class assignment involved finding a unit vector perpendicular to a plane, given two unit vectors to start with. The solution given involved using the cross product; I was wondering if such a ...
2
votes
1answer
102 views

Why does this equation converge to 1?

The following simple equation takes in an N-length (real) vector, and spits out a (real) number between 0 and 1. (I believe this means that it is a transformation mapping $\mathfrak{R}^N \rightarrow ...
3
votes
1answer
105 views

Is this vector derivative correct?

I want to comprehend the derivative of the cost function in linear regression involving Ridge regularization, the equation is: $$L^{\text{Ridge}}(\beta) = \sum_{i=1}^n (y_i - \phi(x_i)^T\beta)^2 + ...
1
vote
1answer
157 views

Gradient vector function using sum and scalar

Could someone take a look on my attempt to compute the gradient for: $$f(x) = \lambda \sum_{x = 1}^n g(x_i)$$ Where $x \in \mathbb{R^d}$, $\lambda \in \mathbb{R}$ and $$g(x_i) = \begin{cases} x_i - ...
1
vote
1answer
100 views

Vector derivative with inner function

I want to compute the gradient for the following function: $$L(\beta) = \sum_{i=1}^n (y_i - \phi(x_i)^T \cdot \beta)^2 + \sum_{j = 1}^k l(\beta_j)$$ where $l(\beta_j) = \begin{cases} \beta_j - ...
1
vote
1answer
421 views

Minimizing L1 Regularization

I have given a high dimensional input $x \in \mathbb{R}^m$ where $m$ is a big number. Linear regression can be applied, but in generel it is expected, that a lot of these dimensions are actually ...
1
vote
1answer
46 views

Vector derivative with power of two in it

I want to compute the gradient of the following function with respect to $\beta$ $$L(\beta) = \sum_{i=1}^n (y_i - \phi(x_i)^T \cdot \beta)^2$$ Where $\beta$, $y_i$ and $x_i$ are vectors. The ...
1
vote
2answers
535 views

Log-likelihood gradient and Hessian

Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions: $f(x) = x^T \beta$ $p(x) = ...
8
votes
2answers
5k views

What is the proof that covariance matrices are always semi-definite?

Suppose that we have two different discreet signal vectors of $N^{th}$ dimension, namely $\textbf{x}[i]$ and $\textbf{y}[i]$, each one having a total of $M$ set of samples/vectors. $\textbf{x}[m] = ...
1
vote
1answer
53 views

What does $Az=y$ and $A^Tx=0$ imply about the relationship between $x$ and $y$?

Suppose A is a $m \times n$ matrix and the vectors $x$ and $y$ are such that $Az=y$ for some vector $z$ and $A^T x=0$. Which one is correct? $x^Ty=0$ $||x||_2=||y||_2$ $||x||_2 < ||y||_2$ $x=ay$ ...
0
votes
2answers
59 views

All the matrices that are orthogonal and have $q_1,q_2$

"Determine all the orthogonal matrices $Q=[q_1,q_2,q_3]$ that have as the first two columns the vectors $q_1=\frac{1}{\sqrt{6}}(-1,2,-1)^T, \ q_2=\frac{1}{\sqrt{3}}(1,1,1)^T$". I used the ...
3
votes
2answers
246 views

Differentiating a function with respect to a vector

I need to differentiate the function $u$ shown below with respect to a vector $\psi$: ($a, c$ and $f$ are constants) $$u(\psi) =\left[\begin{array}{cccc} a & f & 0 & 0\\ c & a & f ...
0
votes
0answers
72 views

clever solution to decomposition of linear products?

There may be a better name for this class of problem, and if so feel free to edit! Imagine a matrix consisting of the following columns: daily return, $\alpha_t$, $factor^1_t$, $factor^2_t$, ... and ...
1
vote
1answer
98 views

Why does this algorithm work?

Given a matrix, $P$, why does finding its eigenvalues, say they are $\{\lambda_1, \lambda_2\}$ then the general form of $p_{ij}^{(n)}=A_{ij}\lambda_1^n+B_{ij}\lambda_2^n$? Thanks. Added: Context: $P$ ...
2
votes
2answers
166 views

equation in matrix form

i have an equation $$\sum_{i=0}^m [w^{(i)}[(c-a^{(i)})-\frac{d^{(i)}(c-a^{(i)})\cdot d^{(i)}}{dis^2}]]=0$$ Where: $a,b$ are two end points in a 3D Line. $d$ is a vector and vector $d=b-a$. the ...
1
vote
1answer
251 views

Wikipedia Article — Legendre Transform

I was reading the wiki article on Legendre Transform. I would be grateful if someone could explain the section at http://en.wikipedia.org/wiki/Legendre_transformation#Examples ie how they arrived at ...