How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

0
votes
0answers
26 views

Which math fields do I need to learn for machine learning?

I decided to become a serious machine learning practitioner and make new ML algorithms for myself. It seems I need to learn math to understand machine learning algorithms and make new ones. Because ...
0
votes
0answers
14 views

Perceptrons that recognize AND, OR, NOT

I'm trying to figure out how to create a set of perceptron weights: one for AND, one for OR, one for NOT. I'm not sure where to begin, but any hints are greatly appreciated!
0
votes
1answer
28 views

Beginner's questions to Hidden Markov Models

I have started reading about Hidden Markov Models, and have some (more or less) minor questions about things I am not sure I understood correctly. I hope asking here is fine: (1) Assumption about the ...
2
votes
1answer
36 views

Does set theory help understand machine learning or make new machine learning algorithms?

When I was in a university, I didn't major in math but took some math classes. However, I dropped out of math classes pretty quick. Some person recommended that I learn some set theory because it'll ...
0
votes
0answers
19 views

For a PAC learnable hypothesis Show that its sample complexity $m_{\mathcal{H}}$ is monotonically non-increasing in each of its parameters

Not sure if this is the right place to post this, if this isn't i'll be grateful if someone will direct me where best to post it. I'm independently taking the course Introduction to Machine language ...
0
votes
0answers
13 views

adaboost weighting scheme’s equality

As you all know, ada-boost weighting is as follows, $$ \begin{cases} e^{-\alpha} & \quad \text{for right classified}\\ e^\alpha & \quad \text{for miss-classified} \end{cases} $$ ...
0
votes
0answers
10 views

Scaling and cross-validation in statistical models

Let's say i have a two dimensional dataset (X and Y variables). My goal is to fit a model that best describes the X-Y relationship Using a training subset of the dataset and then evaluate the ...
0
votes
0answers
13 views

Does every kernel function need to be a dot product in practice?

everyone. I just recently began studying about machine learning, and I have a question about the application of kernel functions. Intuitively, a kernel function is a similarity measure, right? Let's ...
0
votes
0answers
10 views

ANSML - Proving of the matrix identity $\nabla_AtrABA^TC = CAB+C^TAB^T$

(ANSML is a tag I would like to use for Andrew Ng's Stanford Machine Learning - 2008) In this course, there were four matrix identities that I would like to prove. \begin{align} \nabla_a \text{tr}AB ...
0
votes
0answers
8 views

Scatter plot predict/forecast values based on historic values

I do not know if this is the correct forum, as you guys are good at math I'll give it a shot here. Now I've been thinking about this for awhile and have not found out any statistical/mathematical ...
1
vote
1answer
20 views

Trouble understanding how Naive Bayes Classifier is derived

I've come across the Naive Bayes Classifier while studying machine learning, but the trouble I'm having is with some of the probability theory used to derive the formula for finding the optimal ...
0
votes
1answer
43 views

Learning finite automata from symbol set and given sample

Good day. We have a finite automaton F1, for example, . We need to get automaton F2 that accepts strings like accepted by ...
0
votes
0answers
15 views

Predictive Density Independent in Gaussian Process Regression?

I am a little confused in Gaussian process regression. In a GP regression, let $Y=[Y_a, Y_b]\sim \mathcal{N}(0, K+\sigma^2I)$, where $Y_b$ is the target of training samples. The task is to predict ...
0
votes
0answers
5 views

Application of line integral or surface integral to machine learning?

I am exploring the kernel methods in machine learning, and found an interesting post on this. In my point of view, kernel method is a way of reducing dimensions. I have an intuitive understanding that ...
0
votes
0answers
8 views

PDE VS machine learning when solving complex systems?

I am wondering how PDE can be used in machine learning theory. I have got idea from this post also this question Based on what I learn from machine learning discriminative and generative models, I ...
0
votes
1answer
29 views

Find a line such that sum of perpendicular distances of points to the line is minimized

Given a set of points (column vectors) $S = \{p_1, p_2, \cdots, p_n\} \subset \Re^d$, let $A \in \Re^{n \times d}$ be a matrix of which each row is just $p_i^T$. It is easy to find a unit vector $s_1$ ...
1
vote
2answers
37 views

What is the rigorous justification for using inner products as a function of similarity between two vectors?

In machine learning, it is a common thing to define similarity measures, specially using the so call Kernel function. Kernel functions are defined though through inner products of feature vectors: ...
0
votes
1answer
33 views

Cluster probabilites: Bayesian network (sprinkler example, Russel/ Norvig) as a clustered network

like others here I am also learning with Russel's and Norvig's book about artificial intelligence. My question is about the conditional probability tables of a clustered multiply connected network ...
2
votes
0answers
24 views

Difficulty in understanding pattern recognition and machine learning (Bishop) [closed]

I started with Pattern recognition and machine learning by Bishop, but after completing the first chapter(which took lot of time) I feel I am facing lot of problem in understanding the mathematics ...
0
votes
1answer
18 views

Where can I find the solutions to exercises of Probabilistic Graphical Models?

I am self-learning Probabilistic Graphical Models written by Daphne Koller. And for testing how well I learned, I did the exercises in the textbook. But I have no solutions to these exercises. Can ...
0
votes
0answers
23 views

Motivation for gradient descent method over OLS/MLE for simple linear regression?

I am beginner in machine learning and I am currently trying to find the motivation for gradient descent method. I am confused why we want to employ gradient descent method for linear regression? I see ...
1
vote
0answers
16 views

Understanding, Non-Negative Sparse Coding algorithm

I have a question regarding sparse coding, Non-negative sparse coding. Iterate until convergence: $ \mathbf{A_i} \leftarrow \arg \! \min_{A \geq 0} || \mathbf{X}_i - \mathbf{B}_i\mathbf{A}||_F^2 + ...
6
votes
1answer
152 views

Mathematical introduction to machine learning

At first glance, this is once again a reference request for "How to start machine learning". However, my mathematical background is relatively strong and I am looking for an introduction to machine ...
1
vote
0answers
24 views

GMM EM-algorithm VS the Multinomial Logit/Probit Model of Discrete Choice Modeling

I am taking two courses where I learn GMM and MNL separately. However, I do see some similarities between they two: like we need indicator variable for discrete choice modeling when using the MLE, ...
1
vote
0answers
16 views

Fisher Expected Information for a Gaussian Process model

Suppose I have a two dimensional Gaussian process model (GP), defined by a squared exponential correlation function s.t: $$R(x_{i},x_{j}) = \exp\left(-\frac{|x_{i} - x_{j}|^2}{2}\right).$$ I am ...
0
votes
1answer
37 views

Manifold learning: How should this method be interpreted?

I am trying to learn about manifold learning techniques; a family of dimensionality reduction methods in machine learning. According to this idea, there is a low ($d$) dimensional, hidden space where ...
1
vote
2answers
44 views

Thus the eigenvectors of a scatter matrix form a base?

Suppose I have a data set of $m$ vectors in $\mathbb{R}^d$, $D = \{x_1,\ldots,x_m\}$. Let $S = \sum_{i=1}^{m}x_ix_i^T$ be the scatter matrix. My question is: thus the eigenvectors of $S$ form a base ...
4
votes
1answer
43 views

Intuitive explanation of the term manifold

I am reading Christopher Bishop's "Pattern Recognition and Machine Learning" and in the first chapter, where he talks about the curse of dimensionality, he gives the following example: Consider, ...
0
votes
0answers
11 views

Convert KL-divergence to probability

First of all excuse my English is pretty horrible. I'm using the KL-divergence for a metric between compare histograms. I wanted to see if it is possible to convert this value of the KL-divergence at ...
0
votes
0answers
21 views

Minimizer of $\frac{\lambda}{2} \| \theta - \theta^{(k)}\| + \text{Loss}_\text{hinge}(y \theta \cdot x)$

How do you find the minimizer of: $$\min_{\theta \in \mathbb{R}^d}\left\{ \frac{\lambda}{2} \| \theta - \theta^{(k)}\|^2 + \text{Loss}_\text{hinge}(y \theta \cdot x)\right\}$$ if $\theta^{(k)} \in ...
0
votes
0answers
13 views

SVM Soft Margin Lagrange form

I study the Lagrange multipliers form of SVM. I am particulary interested in values that $\alpha_i$ can get. The following is the Langange multipliers form of hard margin SVM. $min_{w,b} ...
1
vote
0answers
24 views

Practical exercise in SVM

Suppose we have four positive points $\{0,1,2,3\}$ and three negative points $\{-3,-2,-1\}$. We want to learn soft-margin linear SVM $\min_{w}0.5 \left \| w \right \| +C \sum \epsilon_i$ the ...
1
vote
1answer
39 views

Covering numbers of metric space

I'm currently reading a paper "On the foundations of machine learning" by F.Cucker and S.Smale and I got stuck on an apparently simple problem. In order to prove an inequality that gives bound on ...
1
vote
1answer
43 views

Book recommendation for wavelet analysis

I am master student doing research in data mining, i read a paper about wavlet analysis for data mining, so i think it may help me in the future. But in my undergraduate degree the last course in ...
0
votes
0answers
5 views

Resulting function after regularization for the courve overfitting problem

A solution for the over fitting problem is the regularization as follows: The function that can overfit if the points number is too low and the order of the equation is greater than the points number ...
0
votes
1answer
21 views

Application of wavelet analysis in computer science

I am doing research in computer science (data mining), do you think wavelet analysis is useful for me?
1
vote
0answers
26 views

Why use two slack variables in the support vector regression formulation?

I am learning support vector regression but cannot fully understand the rational of the slack variable tricks in its formulation. The original optimization problem for SVR is as follows: ...
0
votes
0answers
17 views

For normally distributed points in R^n, what is the probability that p randomly chosen points are linearly separable from all other points?

I am trying to use this to solve a more general problem: the expected number of linearly separable subsets of size s of a set of normally distributed points. Help is much appreciated.
0
votes
1answer
15 views

Scaling Cumulative Probability Distribution function values

We have a cumulative probability distribution function (cdf), we want to scale it down for using it in anomaly detection. The mapping should look like this. CDF value: 0.1 ... 0.5 ... 0.9 ... ...
1
vote
0answers
11 views

SVM maximum-margin distance

I study SVM It's known that the distance between two hyperplanes is $\frac{2}{\left \| w \right \|}$. The problem is I cannot prove this. Let's start. We have two hyperplanes $w \cdot x +b = 1$ and ...
0
votes
0answers
32 views

possible number of hypothesis givin a training set

training set contains p number of papers. each paper is annotated has research or non-research. To develop the research paper filter, we consider the W most frequent phrases in a paper. the research ...
2
votes
2answers
49 views

Prove that a multivariable function has a global minimum

I'm doing an Introduction to Machine Learning course by myself using some open university coursebook and it has the following question which I've tried to solve, but to no avail: Let there be a ...
0
votes
0answers
10 views

Is there a distance metric between a Gaussian Process and a Gaussian Mixture Model density?

Basically, if I do a nonparametric fit to my data using a Gaussian Process, I can smooth it based on an assumption of correlated noise. But the model I'm actually interested in making to fit the data ...
2
votes
2answers
40 views

Lacking in the Intuition behind the Logistic Regression Cost and Update Functions

I am lacking in intuition about the logistic regression cost and update functions. For example, in the cost function of where why is log used. Is it just to make computations easier? Could log ...
0
votes
0answers
9 views

Does the totally normalized version a sub modular f satisfy $\bar{f}(a|A) =f(a|A) - (a|V \setminus {a})$?

To provide context, I was reading the following slides and was confused about slide 55 on the presentation. Specifically what confused me is that, if we have $\bar{f}$ (i.e. a totally normalized ...
0
votes
1answer
26 views

Vector representation in PCA

Recently we studied PCA. As input, we have vectors: $v_1,...,v_n \in R^d$ and $k$, where $k$ is the desirable dimension. Then, we introduce matrix $A:$ $$A = \frac{1}{n} \sum_{i=1}^{n} ...
1
vote
1answer
22 views

Given an incomplete object-object matrix containing relative size differences of the objects, how do I find the missing entries?

For example, let's say I have 3 objects $o_1, o_2, o_3$ and I am given that $o_2$ is 1 more than $o_1$, and $o_3$ is 2 less than $o_1$. I am given this information in the form of an incomplete matrix ...
0
votes
2answers
50 views

What mathematics should I study to understand Neural Nets / Machine Learning?

I am strongly fascinated by neural nets, and perhaps other forms of machine learning. There are so many (potential) applications: teaching a robot with shaft encoders to drive along different ...
0
votes
0answers
22 views

Normalizing of a non-negative matrix factorization (NMF) $M = AW$

Le M be a entry-wise non-negative ($\forall M_{i,j} \geq 0$) matrix, where its columns sum to 1 ($\sum^{m}_{i=1}{M}_{i,j} = 1$) and let M be of the following form: $$ M = \tilde{A} \tilde{W}$$ where ...
0
votes
0answers
62 views

Is information entropy $H(X)$ a sub modular function?

I was trying to learn more about sub modular functions and wanted to see an example of proving that some function is sub modular. Wikipedia said that Entropy was an example so I decided to try it out ...