How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

0
votes
0answers
12 views

Expectation Maximization Algorithm for Gaussian Mixture Model

Can we use the Expectation Maximization algorithm for estimation of Gaussian Mixture Model with full covariance matrices? If yes then can you please give me a reference paper? So far all the machine ...
0
votes
1answer
34 views

How would I use derivatives for suggesting an option to my user?

I was learning derivatives. I understood the theoretical concept behind it. When I was searching for the real-life example in machine learning I came across one of the answers in this question How do ...
0
votes
0answers
13 views

I want to know how can data mining techniques be used to predict Out-of-stock in FMCG industry? [on hold]

Since we cannot forecast Out of stock, how can we use machine learning to predict out of stock?
0
votes
1answer
21 views

Question on understanding Dirichlet process

I have questions on understanding this article about Dirichlet process. If you look at the beginning of section 2.1, it shows three equations 2.1, 2.2, 2.3. The question is I don't understand what ...
5
votes
0answers
35 views

Careers in applied math with an MS other than in finance and data/machine learning?

Since I like math, I would like a career that uses alot of applied math. I'm about to complete my Master's and could do my thesis in numerical solutions of PDEs I'm already aware of careers such as ...
2
votes
0answers
37 views

notation for minimum and maximum?

I'm trying to figure out the correct notation for this situation for use in Machine Learning. I have various ratings (for texts): ...
0
votes
0answers
9 views

Normalized graph Laplacian with negative degrees

In spectral embedding algorithms, we can apply a kernel function to each pair of extracted feature vectors when constructing the adjacency matrix $W$. Suppose that the kernel function fulfills the ...
0
votes
0answers
13 views

Distributive Law on Sum Product

I am reading a tutorial on Conditional Randome Fileds, Here is the link: http://people.cs.umass.edu/~mccallum/papers/crf-tutorial.pdf in the equation 1.24 it defines: $p(x,y) = \prod_{t=1}^{T} ...
0
votes
0answers
20 views

Proper way to combine wavelet coefficients from multiple rounds of analysis

I am doing signal analysis for a time series and the assumption of signal is $$S = F + e$$ Where $S$ is the original signal, $F$ is the frequency component and $e$ is white noise (auto-regressive ...
1
vote
2answers
33 views

Why we consider log likelihood instead of Likelihood in Gaussian Distribution

I am reading Gaussian Distribution from a machine learning book. It states that - We shall determine values for the unknown parameters mu and sigma^2 in the gaussian by maximizing the ...
0
votes
1answer
52 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
0
votes
0answers
8 views

How to build a prediction model for exam score based on previous scores [migrated]

I am trying to construct a formula, which will take student's previous exam results (for ex: SAT) taken at particular dates and predict his future test result, given that he continues to study with ...
0
votes
1answer
86 views

Gradient descent (with line search) for convex functions viewed as alternation

I have fundamental confusion about gradient descent (with line search) and the reason it works. I try to explain my view here, and please tell me where it goes wrong. Let $f: \mathbb{R}^n \to ...
2
votes
0answers
64 views

Numerically approximate the maximum of an element of a vector after a series of matrix multiplications.

Where S is a sigmoidal function, A_i is a matrix, and x is an input vector, and ...
1
vote
0answers
16 views

Baseline predictors model

I implemented baseline predictors model (like it is told in Recommender systems handbook pp 148-149). b_ui = mu + b_i + b_u where mu is overall average rating ...
0
votes
2answers
38 views

Creating a polynomial function with no x-intercept

I have an understanding of polynomials and how to create a function based on the leading coefficient, degrees, x-intercepts, etc. My question is how do i go about creating a polynomial function that ...
1
vote
0answers
19 views

Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...
0
votes
2answers
32 views

Why divide by $2m$

I'm taking a machine learning course. The professor has a model for linear regression. Where $h_\theta$ is the hypothesis (proposed model. linear regression, in this case), $J(\theta_1)$ is the cost ...
-1
votes
1answer
14 views

Usefulness of Laplacians for directed graphs

Are laplacians for directed graphs used in any algorithms ? For example laplacians for the undirected graphs are used in algorithms such as spectral clustering.
2
votes
1answer
31 views

How can we show a data set satisfies the manifold assumption?

In machine learning, we often assume that a data set lies on a low-dimensional manifold (the manifold assumption), but is there any formal proof saying that assuming the data set satisfies certain ...
0
votes
0answers
13 views

Sparse data and covariance matrix computation

Background I am trying to apply Gaussian Discriminant Analysis (GDA) on the MNIST dataset of hand-written digits, with 10 classes for 10 digits. In this dataset, each point is a vector of 784 ...
3
votes
1answer
37 views

Math formulas on Clustering

I am currently studying Clustering in Machine Learning. I have found a document regarding guessing the right number of clusters. I am reading the first part of it, having difficulties in understanding ...
1
vote
0answers
20 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
1
vote
1answer
28 views

Transform a k-CNF formulae to conjunctions of boolean literals

The question comes from Mehryar Mohri's Foundations of Machine Learning. In Example 2.5 the book transform a $k$-CNF formula to conjunctions of boolean literals, but I can't understand the trick in ...
0
votes
1answer
15 views

InformationGain on Two Continuos classes instead on inary

I've a problem regarding an excersise with information gain. I can't seem to get the right answer, because the excersises differs from what we learned. Usually, a target class is a binary variable ...
0
votes
1answer
47 views

Sigmoid function in neural network

I am studying a doctoral thesis on control-theory and have trouble understanding the notions and the notation introduced there. I am doing this out of interest on the subject, so I haven't had a ...
4
votes
1answer
101 views

Clarification about solution of linear SVM problem

I'm reading this tutorial about SVMs. I'd like to have two clarifications: at page 4 (bottom), why is that, after using (1.10) the summation is extended to only $m \in S$? In (1.10) the summation ...
0
votes
0answers
11 views

Back propagation on a function

I'm trying to run a back propagation to learn a simple function. I'm not sure what criteria decides the number of hidden layers and so forth. E.g, for a function like f(x)= x^4 - 15x^2; ...
3
votes
3answers
153 views

Hyperplanes and Support Vector Machines

I have the following question regarding support vector machines: So we are given a set of training points $\{x_i\}$ and a set of binary labels $\{y_i\}$. Now usually the hyperplane classifying the ...
0
votes
0answers
10 views

One-class Support Vector Machine Sensitivity Drops when the number of training sample increase

I am using One-Class SVM for outlier detections. It appears that as the number of training samples increases, the sensitivity TP/(TP+FN) of One-Class SVM detection result drops, and classification ...
1
vote
0answers
21 views

feature selection for continuous variables

I wonder how exactly "feature selection" should be performed in case of continuous feature values. When feature values are discrete it is very straitforward to apply feature selection, but what to do ...
0
votes
2answers
31 views

optimization problem: finding an hyperplane separating one point from a set of pointy maximizing the distance

I have this problem: I have a set of n-dimensional points $P$. I have one more n-dimensional point $q$. The points in $P$ are linearly separable from $q$ (i.e. it always exists an hyperplane $n^t x ...
1
vote
1answer
49 views

What is the difference between reinforcement learning, trial and error, and fictitious play?

I have three question about three algorithms. I have a game with $n$ players. The action space of player $i$ is given by $\mathcal{A}_i=\{a_1, a_2, \cdots, a_m\}=\mathcal{A}$. The joint action space ...
2
votes
1answer
30 views

Building intuition for tensors in machine learning

I'm trying to understand tensors in the context of machine learning, but all the resources that mention tensors that I've found so far were building the intuitions through physics applications. As ...
0
votes
0answers
27 views

Proof that feature normalization cause faster convergence of gradient descent

How to prove that if I do feature normalization (scaling of the $x_1,\ldots,x_n$ to be all in range $[0,1]$) to a convex function $f(x_1,\ldots,x_n)$ that returns real scalar, then gradient descent ...
2
votes
1answer
40 views

Maximizing expected profit

Suppose that a person is going to sell Fizzy Cola at a football game and must decide in advance how much to order. Suppose that he makes a gain of $m$ cents on each quart that he sells at the game but ...
0
votes
0answers
26 views

Adaptive whitening / decorrelation

I have multidimensional data as a set of vectors. I am currently whitening this data and removing the mean vector. I end up with decorrelated data with zero mean and variance equal to 1. I'm using ...
0
votes
1answer
23 views

Equality of Information Gain and Mutual Information

I am curious about definition of information gain and mutual information in the context of feature selection. If looks like two these measures define exactly the same thing, however I didn't find ...
0
votes
1answer
20 views

Understand the English paragraph on association rule.

I am currently studying Association Rule Pattern Mining. I am reading the explanation on wikipedia about it. Somehow, I feel like I have a problem in understanding the paragraph below. Can somebody ...
0
votes
0answers
21 views

Transition matrix in left-right hidden semi-Markov model

I'm developing a hidden semi-Markov model left-right . In a left-right model a sequence of $M$ states starts in state $1$ and ends in state $M$, with no repetition of states. Since the model is ...
0
votes
1answer
33 views

different Interpolation techniques

what are the differences between spline and Lagrange interpolation, and are there any other kinds that might be similar that perform well ?
0
votes
0answers
42 views

Pdf of this estimator

We have a set of unidimensional data, $X_1, \ldots , X_n$ drawn from the positive reals. We define a model for its distribution: The data are drawn from a uniform distribution on the interval $[a, ...
0
votes
1answer
18 views

Curve fitting to connect certain points

Well the image says everything, anyone has any idea how to, or where should i look to be able to draw the BLACK curve ? in fact i need a function that would connect the summits of these red dotted ...
0
votes
1answer
53 views

Probability Estimator

Hi I was going through the MIT 2005 Machine Learning homework assignments and I was having trouble understanding a few concepts in probability theory. I would be obliged if anyone could validate my ...
0
votes
0answers
10 views

Q th order polynomial transform to represent all the curves in $\mathbb{R^d} $

In space $ \mathcal{X} = \mathbb{R^2} $, to get all possible quadratic curves in $ \mathcal{X} $, we need feature transform $\mathbf{z} = \Phi_2(\mathbf{x})$, where $\mathbf{x} \in \mathbb{R^2}$, and ...
0
votes
1answer
22 views

Gradient descent with adaptive learning ratio.

I have a neural network, trained with SGD (stochastic gradient descent) with learning ratio $\alpha$. Each iteration I try to recalculate the weights with a rule: $$\Delta \vec{w} = -\alpha ...
0
votes
0answers
18 views

How to evaluate the difference between two classes of data which are highly overlapped

I’m trying to implement a classifier based on a dataset comprising two classes of high dimensional time-series observations (the values of the two classes of observations are highly similar). I ...
0
votes
1answer
32 views

Similarity metric between two sets of points with varying densities

How can I create a similarity metric that describes the top left set of points as more similar to the bottom left set of points than the top right set of points? Clearly least-squares distance doesn't ...
0
votes
0answers
24 views

Expected in-sample error of linear regression with respect to a dataset D

In my textbook, there is a statement mentioned on the topic of linear regression/machine learning, and a question, which is simply quoted as, Consider a noisy target, $ y = (w^{*})^T \textbf{x} + ...
2
votes
1answer
118 views

Function to predict processing service overload

We have a black box that for each input request a, it outputs a computed response b. The computation time for a given request varies in a stable way over time. Stable means here that it is still ...