How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

5
votes
1answer
638 views

How to quantify the differencen between 2/4 and 20/40?

Assume I have two methods to do prediction. The first method makes 4 predictions and 2 out of 4 are correct. The second method makes 40 predictions and 20 out of 40 are correct. The prediction ...
1
vote
1answer
13 views

Kernel Principal Component Analysis (PCA)

I learn kernel PCA from wikipedia. In this article, the eigen equation is \begin{equation} N \lambda \vec{\alpha} = \boldsymbol{K} \vec{\alpha} \end{equation} where $\lambda$ is the eigen value, ...
0
votes
0answers
15 views

I would like to know how to do log transformation of hyperparameters in Gaussian Process Classification.

I am using Gaussian Process classification and I want to do log transform of the hyperparameters so that they are all positive. From this www.lce.hut.fi/research/mm/gpstuff/GPstuffDoc.pdf document, I ...
2
votes
1answer
14 views

The reason for using normalized data as inputs of Neural Networks [on hold]

Hello I would like to forecast time series and in many algorithms are used as inputs normalized data -1, 1. What is the reason for this normalization? My original data have scale from 1 to 2. Thank ...
2
votes
0answers
30 views

Why can matrices be reversed when implementing the hypothesis function?

I'm learning about the hypothesis function used in linear regression. $$h(\theta) = \theta_0X_0 + \theta_1X_1$$ Where $\theta$ is a $1\times 2$ matrix and $X$ is a $n\times 2$ matrix (with the first ...
0
votes
2answers
54 views

books about relation between Mathematics and reality and life? [on hold]

Which books I should read to understand better Mathematics? Intuition books to understand better Maths. The books show clearly the relation between Mathematics and reality and life.
1
vote
1answer
13 views

Class or type variables as features in polynomial regression algrorithm

I am new in machine learning area, and trying to use polynomial regression for my problem. I have data - advertisements of the cars from kolesa.kz website. Data contains mark, model, mileage, engine ...
0
votes
0answers
13 views

Why does minimization of sum-of-squares yield a predictor that satisfies the same constraint as the training targets?

Bishop's book [1] describes a least-squares approach for classification with a linear model: $$y_k(x)=w_k^Tx + w_{k0}$$ and sum-of-square-errors cost function. Then it mentions an interesting fact: ...
0
votes
0answers
9 views

Dealing with gradient of generative adversarial network

I am currently working on a recurrent implementation of something called a "Generative Adversarial Network". (link: http://arxiv.org/abs/1406.2661 ) Simply explained these are two neural networks, ...
0
votes
0answers
10 views

What is the difference between the error wrt to the true distribution $\mathcal{D}$ and the empirical distribution $S$?

I was reading a paper and on page 4 they talk about the error of predictor with respect to the true distribution $\mathcal{D}$ and the empirical distribution $S$. In other words: $$Err_{ \mathcal{D}} ...
1
vote
1answer
17 views

What does the notation $\{ \pm 1 \}^X$ in relation to functions and hypothesis classes means in the context of PAC learning over half spaces?

I was reading the following paper (on PAC learning over half-spaces) and encountered the following notation for a hypothesis class (on page 4): $$\mathcal{H} \subset \{ \pm 1 \}^X$$ However, it was ...
3
votes
2answers
97 views

How high a priority does discrete math have for people who want to become machine learning practitioners?

Machine learning seems to depend on such math fields as probability, statistics, calculus, and linear algebra. @pranav suggested discrete math would be an important prerequisite. However, someone ...
1
vote
1answer
10 views

Logistic Regression - malty classification

I want to understand why the probability of P(D|p) is presented as a product of mentioned probabilities. I read a lot of texts, but everywhere the explanations are full of terminologies to confuse ...
2
votes
0answers
38 views

Machine Learning and Probability/Stochastics

Main question: What connections are there between machine learning and stochastics (Probability theory, analysis, processes, SDEs)? Background: I've just been accepted into a master's programme for ...
0
votes
1answer
14 views

Are there any general strategies to prove $K(x,y)$ is a machine learning kernel? (I.e. always defines a covariance matrix)?

So there are certain functions of two variables such as the standard Gaussian/radial function $K(x_i,x_j) = e^{-(x_i-x_j)^2}$ which are "kernels" as machine learning calls them, meaning that for any ...
1
vote
0answers
20 views

Can boosting be thought of as a genetic algorithm? [migrated]

Can boosting be classified as a genetic algorithm or as an instance of simulated annealing? Or, is it a completely different paradigm? Essentially, I'm trying to rectify discrete optimization ...
1
vote
2answers
23 views

What machine learning approach requires minimum input data to be significant enough to consider?

I have to a choose machine learning method (binary logistic, SVM, random forest, discriminant analysis, neural networks) for finding significant predictors of a disease relapse. I have sets of 70 and ...
1
vote
0answers
9 views

How to show these covariance functions form a kernel? (I.e. a covariance matrix for any finite set of points)

In machine learning (specifically, Gaussian processes), a "kernel" is a two argument function such that for any set of $N$ "input points," (any $N$, any points), the $N \times N$ matrix of pairwise ...
2
votes
1answer
36 views

High Dimensional Rotation Matrices As Product of In-Plane Rotations

Lately I've been thinking a lot about how to find high-dimensional rotation matrices. In particular, can any rotation in $n$-dimensional space be represented as the product of $2$D plane rotations? ...
0
votes
0answers
6 views

affect of pseudo inputs on covariance matrix

In "A Unifying View of Sparse Approximate Gaussian Process Regression" , it is mentioned that the choice of pseudo inputs affect the final outcome. That affect should be from how this choice affects ...
0
votes
0answers
34 views

ID3 algorithm and binary trees

You have a sample set of 8. You need to make a table such that if you run ID3 on it you get a binary tree with 5 leaf nodes. I'm stuck. I did lots of trial and error.
2
votes
1answer
58 views

How to show that $p(t|x,\mathbf x,\mathbf t)= \int p(t|x,\mathbf w)p(\mathbf w|\mathbf x, \mathbf t)d\mathbf w $

The following paragraph is approximately cited from Bishop's book, Pattern Recognition and Machine Learning. In curve fitting problem, we have training data $\mathbf x$ and $\mathbf t$, along ...
0
votes
1answer
39 views

Math notation clarification

I'm working on learning more about logistic regression and I came across an equation with some confusing notation that I've never seen before: $$ \frac{\delta}{\delta \theta_{y'}^{(j)}} l(\theta) = ...
0
votes
0answers
16 views

Sample complexity of coin bias problem

I am reading a paper involving learning in Multi-armed bandit case (its okay if you don't know what that is. Just trying to give context here.) To give sample complexity lower bound, they reduce their ...
0
votes
0answers
26 views

Global stochastic maximization of a multi-parameter function

I have a function $F:\mathbb{R}^n\to[0,1]$ such that $$ F(\lambda) = \mathbb{E}_x[f(\lambda;x)] = \int f(\lambda;x)\mu(x)dx,$$ and I want to find $\tilde\lambda$ that maximizes F, i.e. ...
1
vote
0answers
32 views

Create periodic function from combining non-periodic functions

I'm studying recurrent neural networks which often use tanh as an activator function which is not periodic. However in research and papers it's shown that these recurrent neural nets can exhibit ...
0
votes
0answers
20 views

Expectation Maximization question

I came across this question while practicing EM question but I don't understand how to apply EM in this scenario. What's the latent variable here? Is it the grade of each student? What will be the ...
0
votes
0answers
13 views

How does Link Analysis work?

Currently, I am into Link Analysis in data mining. I am kind of having a hard time to understand the Link Analysis. I have studied Association Rules and this is my next goal to understand. I have been ...
0
votes
0answers
53 views

Which math fields do I need to learn for machine learning?

I decided to become a serious machine learning practitioner and make new ML algorithms for myself. It seems I need to learn math to understand machine learning algorithms and make new ones. Because ...
0
votes
1answer
24 views

Perceptrons that recognize AND, OR, NOT

I'm trying to figure out how to create a set of perceptron weights: one for AND, one for OR, one for NOT. I'm not sure where to begin, but any hints are greatly appreciated!
0
votes
1answer
37 views

Beginner's questions to Hidden Markov Models

I have started reading about Hidden Markov Models, and have some (more or less) minor questions about things I am not sure I understood correctly. I hope asking here is fine: (1) Assumption about the ...
4
votes
2answers
119 views

Does set theory help understand machine learning or make new machine learning algorithms?

When I was in a university, I didn't major in math but took some math classes. However, I dropped out of math classes pretty quick. Some person recommended that I learn some ...
1
vote
1answer
29 views

For a PAC learnable hypothesis Show that its sample complexity $m_{\mathcal{H}}$ is monotonically non-increasing in each of its parameters

Not sure if this is the right place to post this, if this isn't i'll be grateful if someone will direct me where best to post it. I'm independently taking the course Introduction to Machine language ...
0
votes
0answers
19 views

adaboost weighting scheme’s equality

As you all know, ada-boost weighting is as follows, $$ \begin{cases} e^{-\alpha} & \quad \text{for right classified}\\ e^\alpha & \quad \text{for miss-classified} \end{cases} $$ ...
0
votes
0answers
11 views

Scaling and cross-validation in statistical models

Let's say i have a two dimensional dataset (X and Y variables). My goal is to fit a model that best describes the X-Y relationship Using a training subset of the dataset and then evaluate the ...
0
votes
0answers
13 views

Does every kernel function need to be a dot product in practice?

everyone. I just recently began studying about machine learning, and I have a question about the application of kernel functions. Intuitively, a kernel function is a similarity measure, right? Let's ...
1
vote
1answer
19 views

ANSML - Proving of the matrix identity $\nabla_AtrABA^TC = CAB+C^TAB^T$

(ANSML is a tag I would like to use for Andrew Ng's Stanford Machine Learning - 2008) In this course, there were four matrix identities that I would like to prove. \begin{align} \nabla_a \text{tr}AB ...
1
vote
1answer
26 views

Trouble understanding how Naive Bayes Classifier is derived

I've come across the Naive Bayes Classifier while studying machine learning, but the trouble I'm having is with some of the probability theory used to derive the formula for finding the optimal ...
0
votes
1answer
56 views

Learning finite automata from symbol set and given sample

Good day. We have a finite automaton F1, for example, . We need to get automaton F2 that accepts strings like accepted by ...
0
votes
0answers
16 views

Predictive Density Independent in Gaussian Process Regression?

I am a little confused in Gaussian process regression. In a GP regression, let $Y=[Y_a, Y_b]\sim \mathcal{N}(0, K+\sigma^2I)$, where $Y_b$ is the target of training samples. The task is to predict ...
0
votes
0answers
17 views

Application of line integral or surface integral to machine learning?

I am exploring the kernel methods in machine learning, and found an interesting post on this. In my point of view, kernel method is a way of reducing dimensions. I have an intuitive understanding that ...
0
votes
0answers
16 views

PDE VS machine learning when solving complex systems?

I am wondering how PDE can be used in machine learning theory. I have got idea from this post also this question Based on what I learn from machine learning discriminative and generative models, I ...
1
vote
2answers
43 views

What is the rigorous justification for using inner products as a function of similarity between two vectors?

In machine learning, it is a common thing to define similarity measures, specially using the so call Kernel function. Kernel functions are defined though through inner products of feature vectors: ...
0
votes
1answer
46 views

Cluster probabilites: Bayesian network (sprinkler example, Russel/ Norvig) as a clustered network

like others here I am also learning with Russel's and Norvig's book about artificial intelligence. My question is about the conditional probability tables of a clustered multiply connected network ...
0
votes
1answer
46 views

Where can I find the solutions to exercises of Probabilistic Graphical Models?

I am self-learning Probabilistic Graphical Models written by Daphne Koller. And for testing how well I learned, I did the exercises in the textbook. But I have no solutions to these exercises. Can ...
0
votes
0answers
28 views

Motivation for gradient descent method over OLS/MLE for simple linear regression?

I am beginner in machine learning and I am currently trying to find the motivation for gradient descent method. I am confused why we want to employ gradient descent method for linear regression? I see ...
1
vote
1answer
21 views

Understanding, Non-Negative Sparse Coding algorithm

I have a question regarding sparse coding, Non-negative sparse coding. Iterate until convergence: $ \mathbf{A_i} \leftarrow \arg \! \min_{A \geq 0} || \mathbf{X}_i - \mathbf{B}_i\mathbf{A}||_F^2 + ...
6
votes
1answer
181 views

Mathematical introduction to machine learning

At first glance, this is once again a reference request for "How to start machine learning". However, my mathematical background is relatively strong and I am looking for an introduction to machine ...
1
vote
0answers
27 views

GMM EM-algorithm VS the Multinomial Logit/Probit Model of Discrete Choice Modeling

I am taking two courses where I learn GMM and MNL separately. However, I do see some similarities between they two: like we need indicator variable for discrete choice modeling when using the MLE, ...
1
vote
0answers
17 views

Fisher Expected Information for a Gaussian Process model

Suppose I have a two dimensional Gaussian process model (GP), defined by a squared exponential correlation function s.t: $$R(x_{i},x_{j}) = \exp\left(-\frac{|x_{i} - x_{j}|^2}{2}\right).$$ I am ...