How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

0
votes
0answers
8 views

Softmax Regression Gradient Derivation

I'm implementing softmax regression and am deriving the max-log-likelihood update for gradient descent by hand first. Coming from the Stanford UFLDL site, they show the gradient of the cost function ...
0
votes
4answers
56 views

What maths courses are needed for Machine Learning

I may sound dump. But I really like to know what maths courses are needed for Machine Learning. I am not computer science graduate but seriously interested in AI, ML, Neural network etc, and I know ...
0
votes
0answers
10 views

ADAM: Stochastic optimization [on hold]

What is the main difference between ADAM (adaptive moment estimation) and the stochastic gradient decent? Also what is the significance of the 1st and 2nd moment vectors and the decay rates in the ...
-1
votes
0answers
11 views

Touch times to authenticate user [migrated]

for a project I gathered touch data of different users when they tap a rhythm repeatedly on the touch screen in a game. ...
0
votes
1answer
17 views

Need some help understanding the notation for Online Machine Learning algorithms

I'm reading the Wikipedia article on Online Machine Learning and some of the algorithms mentioned there seem to be missing some context: ...
0
votes
0answers
34 views

What does $\frac{d^k h}{dx^k}$ mean in the context of vectors and regularization in machine learning?

I was watching a machine learning videos from the caltech course CS 156 and they have a slide where they talk about how radial basis functions (RBFs) can be derived from the following variational ...
0
votes
0answers
14 views

Pattern recognition and machine learning - Bishop: Difficulty deriving (2.111) and (2.112)

I'm having a hard time deriving (2.111) and (2.112) in this book (it's on page 110/703). Here is what it says: Finally, we seek an expression for the conditionalp(x|y). Recall that the results for ...
1
vote
1answer
52 views

Studying machine learning

I'm currently studying machine learning using Bishop's book "Pattern Recognition and Machine Learning". The main disadvantage of this book (for me) is a lack of practical applications. Also it seems ...
0
votes
0answers
31 views

What is topological learning?

I am getting this term topological learning in few places for example a reference is below at section 1.1.2: http://virenjain.org/thesis/VirenJainThesis_official.pdf Can anyone point out what ...
1
vote
1answer
32 views

In an ODE dynamic system, is there a convient way or algorithms for estimating the parameters which make the ODE solution satisfing some constraint?

I have construct a ODE dynamic system like this $$molA(t)==sa$$ $$molB'(t)=sb-db\;molB(t)+\frac{kab\;molA(t)\;molB(t)}{molB(t)+Jab}-\frac{kgb\;molG(t)\;molB(t)}{molB(t)+Jgb} $$ $ molC'(t)=sc-dc\ ...
0
votes
0answers
10 views

Problem with understanding the induction when proving Sauer Lemma.

I will replicate the proof here which is from the book "Learning from Data" B(N, k) is the maximum number of dichotomies on N points such that no subset of size k of the N points can be shattered by ...
0
votes
0answers
10 views

PCA when SVD is a skinny SVD

A = m * n matrix. When $m \ge n$, it is easy to see that the V matrix in the full SVD ($A = U*S*V^T$, where U and V are both orthonormal square matrix) and V in a skinny SVD are the same. When $m \lt ...
0
votes
0answers
82 views

Homogeneous polynomials on sphere - need an example that is used in machine learning.

My question is about an example of use of homogeneous polynomials on sphere as a hypothesis space in learning problem. In order to ask a question I need to make a quick introduction: I'm reading an ...
0
votes
0answers
20 views

periodic radial basis function

A have a point cloud ,described in spherical coordinates, which I need to fit with a smooth surface. I'm trying to do this with a bivariate radial basis function network, which operates on a spherical ...
0
votes
0answers
16 views

Chernoff Bound: Prove that $P[u \geq \alpha] \leq (e^{-s\alpha} U(s))^N$.

Let $u_1, ..., u_N$ be random variables, and let $u = \frac{1}{N} \sum_{n=1}^N u_n$. If $U(s) = E_{u_n}(e^{su_n})$ (for any $n$), prove that $P[u \geq \alpha] \leq (e^{-s\alpha} U(s))^N$. $s > 0$ ...
0
votes
0answers
16 views

Having trouble creating my Neural Network inputs

I'm currently working on a neural network that should have $N$ parameters in input. Each parameters can have $M$ different values (discrete values), let's say $\{A,B,C,\dotsc,M\}$. It also has a ...
2
votes
0answers
26 views

How to use Hoeffding Inequality?

I am new to Hoeffding Inequality and can someone kindly explain to me how to use it? I need to solve the following problem. If $\mu = 0.9$, use Hoeffding Inequality to bound the probability that a ...
1
vote
0answers
13 views

How to use 2D Translation and Rotational error to get offset value for new point?

Here I am trying to detect FIDUCIAL points on PCB in real time using camera. After googling for Two days and reading many post and blog. I found that I have to do something called translational error ...
0
votes
0answers
8 views

Is there any way to compute the probability density value for a given power series function, as implied by a Gaussian Process distribution?

So a Gaussian Process defined over the real line is specified by a mean function $\mu(x)$ and a "kernel" function $k(x_1,x_2)$ both defined for all real inputs. The mean function can be anything ...
0
votes
1answer
16 views

Fisher's linear discriminant

Followed by this book I faced with lack of understanding of4.1.4 section. An author obtained $$\mathbf w\propto \mathbf S_W^{-1}(\mathbf m_2 - \mathbf m_1)$$ and suggested to find a threshold $y_0$ ...
1
vote
1answer
22 views

Logistic regression coefficients problem

I'm using logistic regression model to do a multi-class classification (4 classes). I want to look at the logistic regression coefficients to see the importance of different features. I got model ...
2
votes
0answers
23 views

VC-Dimension of Balls intersected with half-spaces

In $d$ dimensional Euclidean space, the VC-dimension of both the set of balls and the set of half-spaces is $d+1$. It follows that the VC-dimension of balls intersected with half-spaces is $O(d \log ...
0
votes
0answers
23 views

Holder's inequality/Cauchy-Schwartz for Bregman Divergence?

Consider the Bregman divergence. $$ D_F(p, q) = F(p)-F(q)-\langle \nabla F(q), p-q\rangle. $$ And its dual norm: $D_{F*}(p, q) $ where $ F^*(y) = \arg\min_x \left\{ \langle x, y\rangle - F(x) ...
2
votes
0answers
37 views

Hoeffding’s inequality extension

In Hoeffding’s inequality we assume that the random variables $X_i$ ,$i=1,..,n$ are i.i.d. and bounded . Is there any extension to Hoeffding’s inequality for the case that $X_i$ are identically ...
0
votes
0answers
13 views

How to solve this optimization problem (Interpolation on trigram, bigram, and unigram for language model)?

I am a newbie in optimization and learn about the language model in NLP. I am studying the basic interpolation method to estimate the probability of the current word given the last 2 words, $P(w_i | ...
0
votes
0answers
22 views

Size of remaining search space for Vehicle Routing Problem given a partial solution

The vehicle routing problem is a NP-hard problem that, in its most basic form, involves scheduling routes for v vehicles that have to make n deliveries in total. So a solution (schedule) has the form ...
0
votes
0answers
20 views

Maximizing Autoencoder Hidden Unit Function

Given \begin{align} a = f\left(\sum_{j=1}^{100} W_j x_j \right). \end{align} where $f$ is the sigmoid function, $W$ and $x$ are $100 \times 1$ matrices with the constrain \begin{align} ||x||^2 = ...
6
votes
1answer
674 views

How to quantify the differencen between 2/4 and 20/40?

Assume I have two methods to do prediction. The first method makes 4 predictions and 2 out of 4 are correct. The second method makes 40 predictions and 20 out of 40 are correct. The prediction ...
1
vote
1answer
21 views

Kernel Principal Component Analysis (PCA)

I learn kernel PCA from wikipedia. In this article, the eigen equation is \begin{equation} N \lambda \vec{\alpha} = \boldsymbol{K} \vec{\alpha} \end{equation} where $\lambda$ is the eigen value, ...
1
vote
1answer
24 views

I would like to know how to do log transformation of hyperparameters in Gaussian Process Classification.

I am using Gaussian Process classification and I want to do log transform of the hyperparameters so that they are all positive. From this www.lce.hut.fi/research/mm/gpstuff/GPstuffDoc.pdf document, I ...
2
votes
0answers
33 views

Why can matrices be reversed when implementing the hypothesis function?

I'm learning about the hypothesis function used in linear regression. $$h(\theta) = \theta_0X_0 + \theta_1X_1$$ Where $\theta$ is a $1\times 2$ matrix and $X$ is a $n\times 2$ matrix (with the first ...
1
vote
1answer
14 views

Class or type variables as features in polynomial regression algrorithm

I am new in machine learning area, and trying to use polynomial regression for my problem. I have data - advertisements of the cars from kolesa.kz website. Data contains mark, model, mileage, engine ...
0
votes
0answers
14 views

Why does minimization of sum-of-squares yield a predictor that satisfies the same constraint as the training targets?

Bishop's book [1] describes a least-squares approach for classification with a linear model: $$y_k(x)=w_k^Tx + w_{k0}$$ and sum-of-square-errors cost function. Then it mentions an interesting fact: ...
0
votes
0answers
16 views

Dealing with gradient of generative adversarial network

I am currently working on a recurrent implementation of something called a "Generative Adversarial Network". (link: http://arxiv.org/abs/1406.2661 ) Simply explained these are two neural networks, ...
0
votes
0answers
11 views

What is the difference between the error wrt to the true distribution $\mathcal{D}$ and the empirical distribution $S$?

I was reading a paper and on page 4 they talk about the error of predictor with respect to the true distribution $\mathcal{D}$ and the empirical distribution $S$. In other words: $$Err_{ \mathcal{D}} ...
1
vote
1answer
19 views

What does the notation $\{ \pm 1 \}^X$ in relation to functions and hypothesis classes means in the context of PAC learning over half spaces?

I was reading the following paper (on PAC learning over half-spaces) and encountered the following notation for a hypothesis class (on page 4): $$\mathcal{H} \subset \{ \pm 1 \}^X$$ However, it was ...
3
votes
2answers
110 views

How high a priority does discrete math have for people who want to become machine learning practitioners?

Machine learning seems to depend on such math fields as probability, statistics, calculus, and linear algebra. @pranav suggested discrete math would be an important prerequisite. However, someone ...
1
vote
1answer
11 views

Logistic Regression - malty classification

I want to understand why the probability of P(D|p) is presented as a product of mentioned probabilities. I read a lot of texts, but everywhere the explanations are full of terminologies to confuse ...
2
votes
0answers
45 views

Machine Learning and Probability/Stochastics

Main question: What connections are there between machine learning and stochastics (Probability theory, analysis, processes, SDEs)? Background: I've just been accepted into a master's programme for ...
0
votes
1answer
20 views

Are there any general strategies to prove $K(x,y)$ is a machine learning kernel? (I.e. always defines a covariance matrix)?

So there are certain functions of two variables such as the standard Gaussian/radial function $K(x_i,x_j) = e^{-(x_i-x_j)^2}$ which are "kernels" as machine learning calls them, meaning that for any ...
1
vote
2answers
30 views

What machine learning approach requires minimum input data to be significant enough to consider?

I have to a choose machine learning method (binary logistic, SVM, random forest, discriminant analysis, neural networks) for finding significant predictors of a disease relapse. I have sets of 70 and ...
1
vote
0answers
12 views

How to show these covariance functions form a kernel? (I.e. a covariance matrix for any finite set of points)

In machine learning (specifically, Gaussian processes), a "kernel" is a two argument function such that for any set of $N$ "input points," (any $N$, any points), the $N \times N$ matrix of pairwise ...
2
votes
1answer
41 views

High Dimensional Rotation Matrices As Product of In-Plane Rotations

Lately I've been thinking a lot about how to find high-dimensional rotation matrices. In particular, can any rotation in $n$-dimensional space be represented as the product of $2$D plane rotations? ...
0
votes
0answers
6 views

affect of pseudo inputs on covariance matrix

In "A Unifying View of Sparse Approximate Gaussian Process Regression" , it is mentioned that the choice of pseudo inputs affect the final outcome. That affect should be from how this choice affects ...
2
votes
1answer
60 views

How to show that $p(t|x,\mathbf x,\mathbf t)= \int p(t|x,\mathbf w)p(\mathbf w|\mathbf x, \mathbf t)d\mathbf w $

The following paragraph is approximately cited from Bishop's book, Pattern Recognition and Machine Learning. In curve fitting problem, we have training data $\mathbf x$ and $\mathbf t$, along ...
0
votes
1answer
42 views

Math notation clarification

I'm working on learning more about logistic regression and I came across an equation with some confusing notation that I've never seen before: $$ \frac{\delta}{\delta \theta_{y'}^{(j)}} l(\theta) = ...
0
votes
0answers
18 views

Sample complexity of coin bias problem

I am reading a paper involving learning in Multi-armed bandit case (its okay if you don't know what that is. Just trying to give context here.) To give sample complexity lower bound, they reduce their ...
0
votes
0answers
28 views

Global stochastic maximization of a multi-parameter function

I have a function $F:\mathbb{R}^n\to[0,1]$ such that $$ F(\lambda) = \mathbb{E}_x[f(\lambda;x)] = \int f(\lambda;x)\mu(x)dx,$$ and I want to find $\tilde\lambda$ that maximizes F, i.e. ...
1
vote
0answers
39 views

Create periodic function from combining non-periodic functions

I'm studying recurrent neural networks which often use tanh as an activator function which is not periodic. However in research and papers it's shown that these recurrent neural nets can exhibit ...
0
votes
0answers
21 views

Expectation Maximization question

I came across this question while practicing EM question but I don't understand how to apply EM in this scenario. What's the latent variable here? Is it the grade of each student? What will be the ...