Tagged Questions

14 views

Usefulness of Laplacians for directed graphs

Are laplacians for directed graphs used in any algorithms ? For example laplacians for the undirected graphs are used in algorithms such as spectral clustering.
101 views

Clarification about solution of linear SVM problem

I'm reading this tutorial about SVMs. I'd like to have two clarifications: at page 4 (bottom), why is that, after using (1.10) the summation is extended to only $m \in S$? In (1.10) the summation ...
33 views

different Interpolation techniques

what are the differences between spline and Lagrange interpolation, and are there any other kinds that might be similar that perform well ?
18 views

Curve fitting to connect certain points

Well the image says everything, anyone has any idea how to, or where should i look to be able to draw the BLACK curve ? in fact i need a function that would connect the summits of these red dotted ...
52 views

Inner Product Space vs. Vector Space

I had no trouble understanding what a vector space is: a constraint on the type of vectors you can create, such that certain operations could be performed with them. For example, a vector space of ...
33 views

support vector machine - vector algebra

I am trying to understand the basics of SVM algebra, but fail to understand per below: Let us formalize an SVM with algebra. A decision hyperplane can be defined by an intercept term $b$ and a ...
40 views

How to represent a separating hyperplane in two dimensions

I am reading a book about support vector machine, and I don't understand some of the math in it. Consider the training sample ${(x_{i}, d_{i})}^{N}_{i=1}$ where $x_{i}$ is the input pattern for ...
378 views

Gradient Descent on Non-Convex Function Works But How?

For Netflix Prize competition on recommendations one method used a stochastic gradient descent, popularized by Simon Funk who used it to solve an SVD approximately. The math is better explained here ...
95 views

Linear discriminant analysis - Properties of weight vectors and decision surfaces

I'm learning about "linear discriminant analysis" on "Statistical Pattern Recognition" of A.R. Webb and K.D. Copsey (chapter 5 of 3rd edition). The general idea is introduced where we suppose to have ...
135 views

Why the largest eigenvalue is the bound?

I am new to Linear Algebra, say introduction level (know the definition and some basic properties, theories about it but not really familiar with doing math on it). Please explain to me: If A is an ...
156 views

rank one update for Cholesky factor

I have covariance matrix known to be $$K = \sum_{i=1}^Nx_ix_i^T$$ where the dimension of $x$ is big (like 50000) so I don't want to really compute any outer-product to expand it as a full matrix. ...
44 views

Path connectedness of the set of separating hyperplanes?

Let $X \subset \mathbb{R}^m$ be a finite set of points, classified by a function $f:X \to \{-1, 1\}$. We say that a hyperplane $H = \{x \in \mathbb{R}^m : \langle x, v \rangle + c = 0\}$ is a ...
108 views

expectation of norm of orthogonal projector

The question has to do with calculating the expected squared norm of a random projection. We have a 2D subspace $T := span\{U1, U2\}$ where $U1$ is a random vector uniformly distributed over unit ...
110 views

how is the gradient derived here?

I'm taking an online machine learning class and in lecture 9 which covers gradient descent, I can't quite follow how he derives the direction vector of the descent (around the 57:15 mark). He's ...
191 views

Comparing k nearest neighbors (knn) and least squares bias and variance

I'm reading the textbook The Elements of Statistical Learning. In section 2.3.3, it says "The linear decision boundary from least squares is very smooth, and apparently stable to fit. [...] In ...
39 views

Finding an optimal set of weights for combining correlated classifiers

In order to combine classifiers that are correlated with one another, I would need to solve the following optimization problem: Find a vector $\mathbf{w}$ that minimizes $\mathbf{w}^T M \mathbf{w}$ ...
70 views

Problem with equation derivation

I'm studying support vector machines and the book I'm using states in one particular part the following: We know that: $$\vec{y} = \vec{x} + v\vec{w}$$ and that $$|\;\vec{y}-\vec{x}\;| = 2M$$ and so ...
138 views

derivation of Support Vector Machine

I was watching Andrew Ng's machine learning lecture on SVM. There is one line that puzzles me. $$x^{(i)} - \gamma^{(i)} \frac{w}{||w||}$$ I dont understand how can the line above give the ...
319 views

Reproducing Kernel Hilbert Spaces for Dummies

I am in the middle of some machine learning paper that states that for function $f$, imposing the norm constraint, $\|f \|=1$, corresponds to an orthogonal projection onto the direction selected in ...
303 views

How to find Basis vectors of a matrix $X$, given basis vectors of its kernel matrix $XX^T$?

If we know basis vectors for $K=XX^T$ (e.g. will be eigenvectors here since $K$ is symmetric), how can we find base vectors for $X$?
115 views

Linear algebra: finding a Tikhonov regularizer matrix

A more general soft constraint is the Tikhonov regularization constraint $$\mathbf{w}^\text{T}\Gamma^\text{T}\Gamma\mathbf{w} \leq C$$ which can capture relationships among the $w_i$ (the matrix ...
997 views

Uniqueness of symmetric positive definite matrix decomposition

We know that any symmetric positive semi-definite matrix $K$ can be written as $K= AA^T$, where $A$ has real components. One way to get to $A$ is to compute eigen value decomposition of $K= P^T DP$ ...
223 views

Minimize $||Ax-b||$ but for $A$, not $x$

I have a machine learning regression problem. I need to minimize $$\sum_i||Ax_i-b_i||_2^2$$ However I am trying to find matrix $A$, not the usual $x$, and I have lots of example data for $x_i$ and ...
273 views

Concise foundational math books

I am tackling some topics in Probability Theory and Machine Learning and while I have plenty of resources dedicated to those disciplines I am lacking in a good basic math foundation. Does anyone know ...
161 views

linear kernel pca get corresponding dimension

I am implementing my own version of linear kernel principal component analysis for better understanding the algorithm. I faced a problem which seems to be specific ...
442 views

Math problem in Pattern Recognition and Machine Learning

I had been carefully following Bishop's Pattern Recognition book and came across some of difficulties due to my merely basic math ground.First begin by the Gaussian section on page 80. The ...
356 views

Simple Least Squares Regression?

I have a vector X of 50 real numbers and a vector Y of 50 real numbers. I want to model them as y = ax + b How do I determine a and b such that it minimizes the ...
153 views

Solving $X^{T}Xw = X^{T}y$ for $w$ in Octave / MATLAB

I have a matrix $X$ and a vector $y$, how do I solve the following equation for $w$: $$X^{T}Xw = X^{T}y$$ in Octave and/or MATLAB?
84 views

Regression on Linear Model?

I have 50 or so training examples involving a set of 200 or so real numbers (x1,x2,...,x200) (normalized to a 0 mean and std dev 1), and a single output real (y) in the range 0.0..1.0. I want to fit ...
928 views

What is the significance of theoretical linear algebra in machine learning/computer vision research?

I am a computer science research student working in application of Machine Learning to solve Computer Vision problems. Since, lot of linear algebra(eigenvalues, SVD etc.) comes up when reading ...
I am having trouble try to show that this linear regression summation: $J(w) = \sum^m_{i=1} u_i (w^T x_i - y_i)^2$ can be rewritten in the following matrix form: $J(w) = (Xw - y)^T U(Xw - y)$ ...