How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

0
votes
0answers
7 views

what kind of similarity measurement can I apply for real-vaule features [on hold]

As I know for some normalized feature(like histogram). I can apply Histogram intersection, Log-likelihood statistic, ...
1
vote
0answers
26 views

Integrating an expression over a vector $\mathbf{w}$

doing my homework for a Machine Learning course, I have to calculate the following expression: $\newcommand{\IDENTITY}{\mathbf{I}} \newcommand{\W}{\mathbf{w}} \newcommand{\WT}{\mathbf{w}^T} ...
0
votes
0answers
8 views

Does any one know the relationship of the number of support vector and the data dimension in SVM?

Does any one know the relationship of the number of support vector and the number of data dimension in SVM? Is it possible that #support vector < #data dimension? If yes, for #support vector < ...
0
votes
0answers
10 views

compress data by linearization

i have a continuous data and i want to compress it by linearization(see picture). What algorithm can be used to minimize both number of lines and total compression error?
0
votes
1answer
13 views

Normalization of data in decision tree

After reading through a few references, I have come to know that for machine learning in general, it is necessary to normalize features so that no features are arbitrarily large ($centering$) and all ...
1
vote
0answers
60 views

Reducing a linear algebra expression to quadratic form

I am trying to solve the following exercise for my Machine Learning course. Expand this expression so that there are only quadratic terms: $(\mathbf{x} - \mathbf{\mu})^T \mathbf{\Sigma}^{-1} ...
0
votes
2answers
29 views

Identity regarding convexity of the logistic loss function

I found the following identity regarding the logistic loss function in these lecture notes (slide 16) from Berkeley university: $$\log(1 + e^{-z}) = \max_{0 \leq v \leq 1} -zv + v\log(v) + ...
0
votes
0answers
17 views

How to find a separating hyperplane?

I know about support vector machine, and it's quadratic programming approach which delivers the best separating hyperplane. My question is: is there a relatively simple algorithm to find a ...
0
votes
1answer
21 views

Given a sample of input/output data, predict new outputs

My problem is the following : I have a number of inputs with the corresponding deterministic outputs. There is no error on either input or output. The link between the two is completely unknown to me. ...
1
vote
0answers
30 views

Computations for LDA: Eigendecomposition

While reading the book Elements of Statistical Learning p. 113, the author used eigendecomposition of the covariance matrix $\hat{\Sigma}_k =\mathbf{U}_k\mathbf{D}_k\mathbf{U}_k^T$ where ...
0
votes
2answers
36 views

Exposition of solving the quadratic programming problem for SVMs

I'm looking to find a mathematically rigorous exposition on how to solve the quadratic programming problem $$\min ||x||^2 \textrm{ subject to } Ax\leq b$$ where $x\in\mathbb{R}^n$, ...
0
votes
1answer
44 views

Deriving equation in vector notation

I had some trouble deriving an equation from the book 'Elements of statistical Learning' p. 108 equation 4.9. This heavily relies on linear algebra, so I was wondering how the author came to his final ...
0
votes
0answers
23 views

Help needed for K-nearest neighbor distance metric and distance weight formulas.

I am currently working on K-nearest neighbor algorithm with Distance Weighting. I have read the Distance Metric and the Distance Weighting parts. I want to fully understand how the Distance metric and ...
0
votes
0answers
4 views

Are there other names for multilayer perceptrons or multidimensional interpolants based on Kolmogorov's approximation work?

Are there other names for multilayer perceptrons that are used outside of the neural net community? At its core, multilayer perceptrons form a multidimensional interpolant of the form $$ ...
0
votes
1answer
17 views

Mathematics disciplines underpinning Machine Learning

I have an undergrad degree in computational mathematics (though that was about 10 years ago), and spent my professional career in software development. If I wanted to understand what's happening ...
0
votes
0answers
17 views

Expectation Maximization Algorithm for Gaussian Mixture Model

Can we use the Expectation Maximization algorithm for estimation of Gaussian Mixture Model with full covariance matrices? If yes then can you please give me a reference paper? So far all the machine ...
0
votes
1answer
39 views

How would I use derivatives for suggesting an option to my user?

I was learning derivatives. I understood the theoretical concept behind it. When I was searching for the real-life example in machine learning I came across one of the answers in this question How do ...
0
votes
1answer
23 views

Question on understanding Dirichlet process

I have questions on understanding this article about Dirichlet process. If you look at the beginning of section 2.1, it shows three equations 2.1, 2.2, 2.3. The question is I don't understand what ...
5
votes
0answers
43 views

Careers in applied math with an MS other than in finance and data/machine learning?

Since I like math, I would like a career that uses alot of applied math. I'm about to complete my Master's and could do my thesis in numerical solutions of PDEs I'm already aware of careers such as ...
2
votes
0answers
41 views

notation for minimum and maximum?

I'm trying to figure out the correct notation for this situation for use in Machine Learning. I have various ratings (for texts): ...
-1
votes
0answers
24 views

Distributive Law on Sum Product

I am reading a tutorial on Conditional Randome Fileds, Here is the link: http://people.cs.umass.edu/~mccallum/papers/crf-tutorial.pdf in the equation 1.24 it defines: $p(x,y) = \prod_{t=1}^{T} ...
1
vote
2answers
42 views

Why we consider log likelihood instead of Likelihood in Gaussian Distribution

I am reading Gaussian Distribution from a machine learning book. It states that - We shall determine values for the unknown parameters mu and sigma^2 in the gaussian by maximizing the ...
0
votes
1answer
62 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
0
votes
1answer
111 views

Gradient descent (with line search) for convex functions viewed as alternation

I have fundamental confusion about gradient descent (with line search) and the reason it works. I try to explain my view here, and please tell me where it goes wrong. Let $f: \mathbb{R}^n \to ...
2
votes
0answers
64 views

Numerically approximate the maximum of an element of a vector after a series of matrix multiplications.

Where S is a sigmoidal function, A_i is a matrix, and x is an input vector, and ...
1
vote
0answers
17 views

Baseline predictors model

I implemented baseline predictors model (like it is told in Recommender systems handbook pp 148-149). b_ui = mu + b_i + b_u where mu is overall average rating ...
0
votes
2answers
38 views

Creating a polynomial function with no x-intercept

I have an understanding of polynomials and how to create a function based on the leading coefficient, degrees, x-intercepts, etc. My question is how do i go about creating a polynomial function that ...
1
vote
0answers
23 views

Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...
0
votes
2answers
34 views

Why divide by $2m$

I'm taking a machine learning course. The professor has a model for linear regression. Where $h_\theta$ is the hypothesis (proposed model. linear regression, in this case), $J(\theta_1)$ is the cost ...
-1
votes
1answer
15 views

Usefulness of Laplacians for directed graphs

Are laplacians for directed graphs used in any algorithms ? For example laplacians for the undirected graphs are used in algorithms such as spectral clustering.
2
votes
1answer
33 views

How can we show a data set satisfies the manifold assumption?

In machine learning, we often assume that a data set lies on a low-dimensional manifold (the manifold assumption), but is there any formal proof saying that assuming the data set satisfies certain ...
0
votes
0answers
15 views

Sparse data and covariance matrix computation

Background I am trying to apply Gaussian Discriminant Analysis (GDA) on the MNIST dataset of hand-written digits, with 10 classes for 10 digits. In this dataset, each point is a vector of 784 ...
3
votes
1answer
40 views

Math formulas on Clustering

I am currently studying Clustering in Machine Learning. I have found a document regarding guessing the right number of clusters. I am reading the first part of it, having difficulties in understanding ...
1
vote
0answers
24 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
1
vote
1answer
32 views

Transform a k-CNF formulae to conjunctions of boolean literals

The question comes from Mehryar Mohri's Foundations of Machine Learning. In Example 2.5 the book transform a $k$-CNF formula to conjunctions of boolean literals, but I can't understand the trick in ...
0
votes
1answer
15 views

InformationGain on Two Continuos classes instead on inary

I've a problem regarding an excersise with information gain. I can't seem to get the right answer, because the excersises differs from what we learned. Usually, a target class is a binary variable ...
0
votes
1answer
55 views

Sigmoid function in neural network

I am studying a doctoral thesis on control-theory and have trouble understanding the notions and the notation introduced there. I am doing this out of interest on the subject, so I haven't had a ...
4
votes
1answer
104 views

Clarification about solution of linear SVM problem

I'm reading this tutorial about SVMs. I'd like to have two clarifications: at page 4 (bottom), why is that, after using (1.10) the summation is extended to only $m \in S$? In (1.10) the summation ...
0
votes
0answers
11 views

Back propagation on a function

I'm trying to run a back propagation to learn a simple function. I'm not sure what criteria decides the number of hidden layers and so forth. E.g, for a function like f(x)= x^4 - 15x^2; ...
4
votes
3answers
157 views

Hyperplanes and Support Vector Machines

I have the following question regarding support vector machines: So we are given a set of training points $\{x_i\}$ and a set of binary labels $\{y_i\}$. Now usually the hyperplane classifying the ...
0
votes
0answers
11 views

One-class Support Vector Machine Sensitivity Drops when the number of training sample increase

I am using One-Class SVM for outlier detections. It appears that as the number of training samples increases, the sensitivity TP/(TP+FN) of One-Class SVM detection result drops, and classification ...
1
vote
0answers
21 views

feature selection for continuous variables

I wonder how exactly "feature selection" should be performed in case of continuous feature values. When feature values are discrete it is very straitforward to apply feature selection, but what to do ...
0
votes
2answers
44 views

optimization problem: finding an hyperplane separating one point from a set of pointy maximizing the distance

I have this problem: I have a set of n-dimensional points $P$. I have one more n-dimensional point $q$. The points in $P$ are linearly separable from $q$ (i.e. it always exists an hyperplane $n^t x ...
2
votes
1answer
58 views

What is the difference between reinforcement learning, trial and error, and fictitious play?

I have three question about three algorithms. I have a game with $n$ players. The action space of player $i$ is given by $\mathcal{A}_i=\{a_1, a_2, \cdots, a_m\}=\mathcal{A}$. The joint action space ...
2
votes
1answer
38 views

Building intuition for tensors in machine learning

I'm trying to understand tensors in the context of machine learning, but all the resources that mention tensors that I've found so far were building the intuitions through physics applications. As ...
0
votes
0answers
29 views

Proof that feature normalization cause faster convergence of gradient descent

How to prove that if I do feature normalization (scaling of the $x_1,\ldots,x_n$ to be all in range $[0,1]$) to a convex function $f(x_1,\ldots,x_n)$ that returns real scalar, then gradient descent ...
2
votes
1answer
43 views

Maximizing expected profit

Suppose that a person is going to sell Fizzy Cola at a football game and must decide in advance how much to order. Suppose that he makes a gain of $m$ cents on each quart that he sells at the game but ...
0
votes
0answers
28 views

Adaptive whitening / decorrelation

I have multidimensional data as a set of vectors. I am currently whitening this data and removing the mean vector. I end up with decorrelated data with zero mean and variance equal to 1. I'm using ...
0
votes
1answer
30 views

Equality of Information Gain and Mutual Information

I am curious about definition of information gain and mutual information in the context of feature selection. If looks like two these measures define exactly the same thing, however I didn't find ...
0
votes
1answer
22 views

Understand the English paragraph on association rule.

I am currently studying Association Rule Pattern Mining. I am reading the explanation on wikipedia about it. Somehow, I feel like I have a problem in understanding the paragraph below. Can somebody ...