How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

0
votes
1answer
8 views

Why is $\frac{1}{n}\sum_{i=1}^{n}(\langle\vec{x}_i,\vec{w}\rangle-y_i)^2 = \frac{1}{n}(X\vec{w}-\vec{y}_i)^T (X\vec{w}-\vec{y}_i)$?

I am reading about Ridge Regression in Machine Learning (in particular, the calculation of the empirical risk w.r.t. the square loss function) and do not understand the following step: ...
0
votes
0answers
15 views

What is the Bernoulli class conditional distribution?

What is the Bernoulli class conditional distribution? I am trying to implement a procedure for computing a naive Bayes classifier for binary features with a Bernoulli class conditional distribution. ...
2
votes
2answers
28 views

chain rule conditional entropy

I have to prove the chain-rule for conditional entropy. I kept getting stuck on one step, so I looked up a proof and found this: \begin{align}H(Y\mid X)&= \sum_{x\in\mathcal X, y\in\mathcal ...
0
votes
0answers
8 views

trie based computation of the string spectrum kernel [on hold]

I have found a trie based computation algorithm for computing the string spectrum kernel in the book "kernel methods for pattern Analysis" i still dont understand the algorithm, can anyone help me. ...
1
vote
0answers
51 views
+100

Interpolation and mapping between scattered vectors in two unequally dimensioned spaces

Imagine two spaces: An ‘input’ space with dimension $m$. An ‘output’ space with dimension $n$. $m \geq n$ There are points in each of these spaces defined such that some characteristic is ...
0
votes
1answer
26 views

An explanation of how this solution is derived

I am having difficulty understanding the solution to this problem. Since the solution is in the form of Bayes theorem I expected something along the lines that looked similar to Bayes theorem. ...
0
votes
0answers
15 views

Use of activation derivative in back propagation algorithm

I'm a little confused how the activation derivative in back propagation work. Firstly, when I remove the activation derivative from the back propagation algorithm and replace it with a constant the ...
0
votes
0answers
18 views

Nando de Freitas' Machine Learning Homework 2 Questions 1 & 2 Solutions

I've been following Nando de Freitas' Machine Learning course from UBC. While I have been enjoying the course I thought it would be good to see if I could do the homework along with it. So I'm on ...
0
votes
0answers
4 views

Difference between Nearest Neighbour and Nearest Centroid

I'm trying to understand the difference between Nearest neighbour classifiers and Nearest centroid classifier. Using the nearest neighbour, one selects a data point ...
0
votes
0answers
12 views

Neural network for linear regression [closed]

I found this great source that matched the exact model I needed: http://ufldl.stanford.edu/tutorial/supervised/LinearRegression/ The important bits go like this. You have a plot x->y. Each x-value ...
0
votes
0answers
6 views

How to use Support Vector Machines with Mixed data?

I have a dataset regarding student records with a mix of continuous, discrete & categorical data - the categorical data takes both nominal and ordinal forms. Ex: Continuous - GPA Ex: Discrete - ...
-1
votes
0answers
14 views

What is the difference between the MAP probability and the expected probability? [closed]

Like the title says, I am a bit confused as to which is which.
1
vote
0answers
14 views

Outer product approximation of Hessian for least squares

On p251 of Bishop's machine learning book, the Hessian for least squares is derived (as a preliminary step to the outer product approximation): $ E = \frac{1}{2} \sum_{n=1}^{N} (y_n - t_n)^2$ $H = ...
0
votes
0answers
15 views

Lower bound of Nearest Neighbour Rule

It is stated often as a matter of fact that the lower bound for Nearest Neighbour rule is the Baye's rate. However when I tried to mathematically prove it,I hit a dead end. For reference : Error for ...
1
vote
0answers
12 views

Supported Vector Machine constraint condition.

I think I can follow the following statement, but I can't reach the conclusion "subject to $\sum_{i=1}^n c_i y_0 = 0$". Could someone give me a pointer from which to reach this constraint? Thank you! ...
1
vote
1answer
17 views

Bernoulli Naive Bayes Classification

I am having trouble understanding the following text regarding Bernoulli Naive Bayes. Specifically, the author mentions that $i$ is a feature. However, what is the difference between $x_i$ and $i$? ...
-1
votes
0answers
6 views

How we get L as minimize(w.r.t L) |M-L| subject to rank(L) less than r in Classical PCA? [closed]

D = L+E,L(data matrix), D(corrupted data matrix). E(Error Matrix) I know how to solve Principal Component Analysis using eigen vectors of covariance matrix.
0
votes
1answer
21 views

Gradient of cost function

I have tried to calculate the gradient of the LMS cost function as follows but have a problem. $$J(\theta) = \frac12(y - X'\theta)^2$$ where $y$ is a scalar, theta and $X$ is a $n$ dimensional vector ...
1
vote
0answers
41 views

Lower bound on dimension for nearest neighbor classifier to fail at k=1 and pass at k=3

What is the minimum dimensionality of a dataset of a finite number of points where 1-NN has an accuracy of 0% but 3-NN has an accuracy of 100%. This is certainly possible in 3 dimensions and my ...
0
votes
0answers
32 views

Two different ways to compute PCA?

I am working on the PCA. On the internet I found two different ways to compute it - but they produce different results. First there is this solution: http://stanford.io/2060AxA . They are taking the ...
2
votes
0answers
17 views

Comparing two textbooks for machine learning

I am a Ph.D student in Electrical Engineering. I am going to study the field of machine learning and I found some textbooks to study this field. 1) Probabilistic Graphical Models: Principles and ...
1
vote
0answers
114 views

What does this probability notation denote: $P( Y=y | X =x)$?

What does this probability notation denote: $P( Y=y | X =x)$? I came across this while looking at my notes for optimal prediction rule. Optimal prediction rule: for each $x ∈ X$ , chose $y ...
0
votes
0answers
8 views

When does the UCB algorithm find best arm in Multi-armed Bandits problem?

Some article or paper said "UCB(or a derivative algorithm) can find the best arm among multiple arms in Multi-armed Bandits problem". But I don't remember which one said so, and even worse, I'm not ...
0
votes
0answers
13 views

Equivalence between Label propagation and Likelihood estimation over a Markov Random Field

On page 13/14 of label propagation the near equivalence is set up between what they call 33/34 (reproduced below). This near equivalence is not obvious to me. $$P_{F'}(Y) = \frac{1}{Z} \exp \left( ...
1
vote
2answers
79 views

Neural network cost function - why squared error?

Question: Why is the squared error most often used for training neural networks? Context: Neural networks are trained by adjusting the link weights. The key factor that informs these adjustments is ...
0
votes
0answers
8 views

How to evaluate a point in a weighted Mixture of Gaussians model?

Example: I have a MoG comprised of 2 1-d gaussians. The first gaussian has a weight of 0.8 the other 0.2. I have a sample point which I can easily evaluate on each individual gaussian. The ...
1
vote
0answers
22 views

Expectation Maximization - question about the 2 different formulations of E-Step

I saw the following 2 formulations for the E-Step: $Q(θ,θ_t )=E_{θ_t} ( log(p_θ (X,Z))│X=x)$ $Q(θ,θ_t )= E_{Z|X,θ_t } [log(Pr_θ(X=x,Z))]$ I can't understand why they are equal.
0
votes
1answer
25 views

Solving SVM classifier with two weight vectors

I am trying to implement a paper that basically proposes the following way to train two classifiers on some data with two types of labels. I do not know how to tweak existing solvers for SVM to do the ...
0
votes
1answer
33 views

Meaning of “linear in the xi variables”

I'm afraid my maths is so feeble I was not sure how to entitle the question. Essentially I was hoping to ask what the below passage means in the context of the passage that follows? (If that is ...
0
votes
0answers
14 views

Variational inference on a Normal distribution: is my choice of priors passable?

I am trying to understand the basics of Variational Inference. In order to do so I designed a very simple problem: using the free-form mean field method to approximate the posteriori distribution of ...
0
votes
0answers
10 views

Efficient exploration of the “least important” nodes in a large graph

We call Markov Chain Crawler (MCC) a graph learner that is given query access to a Markov Chain Teacher (MCT) which itself is given a specific Markov Chain. At the beginning, the MCC is given some ...
0
votes
1answer
15 views

Does the Markov Blanket of a node include the node itself?

The definition states the Markov Blanket includes the parents of the children of the node, so does this include the node itself too?
0
votes
0answers
17 views

Neural network for regression

The way I understand regression for neural networks is weights being added to each x-input from the dataset. I want something slightly different. I want weights ...
4
votes
1answer
219 views

What is an example of a SVM kernel, where one implicitly uses an infinity-dimensional space?

Reading the Wikipedia article about SVMs, I noticed More formally, a support vector machine constructs a hyperplane or set of hyperplanes in a high- or infinite-dimensional space, which can be ...
0
votes
0answers
45 views

Gradient Boosting Loss Function Derivation

http://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf XGBoost, a popular machine learning library, refers to the above slides in its documentation. I am working through the derivation of the ...
0
votes
0answers
48 views

What are some methods for prototyping polynomials

I work for an eCommerce company that sells office supplies, and each day, the number of orders we take follows a specific shape/curve (we collect order counts at one minute intervals, so we end up ...
0
votes
0answers
15 views

Where do I need to use regularization parameter lambda for better results?

In polynomial curve fitting problem as below, if $y(x,w)$ is the output when $x$ is the input vector, $w$ are the coefficients and and $M$ is the order of polynomial.. And if $t_n$ is the target ...
0
votes
0answers
45 views

Kernel-induced metric

Given a kernel k on input space X defining RKHS (Reproducing kernel Hilbert space) H. Let Φ : X → H denote the corresponding feature map (think of Φ(x) = k(x, .)). Let x, z ∈ X . How can I show that ...
0
votes
0answers
8 views

Bayesian Networks: simple example when to use discrete network and when to use linear Gaussian network

So I am not sure when to use which. Is there a simple example that a non maths pro would understand when to use which? I use libpgm and the pgmlearner provides different functions to train on data. I ...
1
vote
0answers
17 views

How does Restricted Boltzmann Machine (RBM) try to model the distribution of data?

An RBM is a non-directed graphical model that defines the distribution over some input vector X. I know it's going to model the distribution of that those vectors in my training data X using a layer ...
2
votes
0answers
37 views

Does linear regression form a subspace?

The author writes Given a vector of inputs $X^T = (X_1, \dots ,X_p)$, we can predict an output $Y$ via $$ \hat{Y} = \beta_0 + \sum_{j = 1}^p X_j \beta_j$$ He goes on to note that if we include a 1 in ...
1
vote
1answer
42 views

Logistic regression: Prove that the cost function is convex

I'm reading this You can do a find on "convex" to see the part that relates to my question. Background: $h_\theta(X) = sigmoid(\theta^T X)$ --- hypothesis/prediction function $y \in \{0,1\}$ ...
0
votes
0answers
16 views

How to decide a binary hidden variable's state in RBM using CD-Algorithm or Gibbs Sampling?

I'm a 3rd-grade college student and recently I'm reading tutorial materials about Restricted Boltzmann Machines for better understanding the paper Hilton published in ICML 2007. I'm little confused ...
0
votes
0answers
38 views

Is it legal to use gradient descend method in neural networks?

"The usual" neural network for me is system which was cascading via usage of functions from this class: Linear functions Heaviside step and it's approximations I have some troubles with ...
2
votes
0answers
32 views

What temperature of Softmax layer should I use during neural network training?

I've written GRU (gated recurrent unit) implementation in C#, it works fine. But my Softmax layer has no temperature parameter (T=1). I want to implement "softmax with temperature": $$ P_{i} = ...
0
votes
0answers
7 views

The Elements of Statistical Learning: How does this nature cubic spline have K basis functions with K knots with the given solution? See Details.

I'm referring to this found in chap 5: Picture from ELS If K=2 (as in Sec 5.2), then we have N1, N2, N3 and N4 basis functions. So 4 basis functions and 2 knots. I know that a natural cubic spline ...
0
votes
0answers
7 views

Can I view the problem of principal component analysis as an problem of rank minimization?

Can I view the PCA problem as a following optimization problem? $$\min \text{rank}(X)$$ $$s.t. \quad ||M-L||_2 \le \epsilon$$ where M is the observation matrix and L is the low-rank principle ...
0
votes
1answer
62 views

Principal Component Analysis Summarization by the mean

Let’s consider $$ S=\{x_1 , . . . , x_N \} $$ with $$ x_1 , . . . , x_N ∈ R^d . $$ How can i prove that the solution of $$ argmin_{b∈ R^d} \frac{1}{N}\sum_{i=1}^{N}||x_i-b||^2 $$ is given by $$ ...
1
vote
0answers
13 views

General correlation between function with itself and other input data

I've collected data for a function F = f(g(x)), for different function shapes g(x). The goal is to predict values of ...
0
votes
1answer
39 views

Best possible distribution for solving maximum-likelihood for a staircase data?

I have $n$ iid sample data $x_1,x_2,x_3..., x_n$ from a probability distribution function . The sample density is defined over $[0,1]$ and is of the form: $$f(x) = \left\{\matrix{a, & ...