How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

2
votes
0answers
30 views

How to show that $p(t|x,\mathbf x,\mathbf t)= \int p(t|x,\mathbf w)p(\mathbf w|\mathbf x, \mathbf t)d\mathbf w $

The following paragraph is approximately cited from Bishop's book, Pattern Recognition and Machine Learning. In curve fitting problem, we have training data $\mathbf x$ and $\mathbf t$, along ...
1
vote
0answers
13 views

Kernel function related proof [on hold]

This is not my area of expertise -- nonetheless, I need some sort of semi-convincing proof of the following equation, which has been cited in several machine learning articles I've read: $$ d_j = ...
0
votes
1answer
38 views

Math notation clarification

I'm working on learning more about logistic regression and I came across an equation with some confusing notation that I've never seen before: $$ \frac{\delta}{\delta \theta_{y'}^{(j)}} l(\theta) = ...
0
votes
0answers
15 views

Sample complexity of coin bias problem

I am reading a paper involving learning in Multi-armed bandit case (its okay if you don't know what that is. Just trying to give context here.) To give sample complexity lower bound, they reduce their ...
0
votes
0answers
23 views

Global stochastic maximization of a multi-parameter function

I have a function $F:\mathbb{R}^n\to[0,1]$ such that $$ F(\lambda) = \mathbb{E}_x[f(\lambda;x)] = \int f(\lambda;x)\mu(x)dx,$$ and I want to find $\tilde\lambda$ that maximizes F, i.e. ...
1
vote
0answers
21 views

Create periodic function from combining non-periodic functions

I'm studying recurrent neural networks which often use tanh as an activator function which is not periodic. However in research and papers it's shown that these recurrent neural nets can exhibit ...
0
votes
0answers
10 views

Heuristics for streaming data matching [migrated]

I have an index composed by thousands of documents. Slightly modified copies of those documents are sent to my application in small chunks, and I need to check, from those chunks, which document has ...
0
votes
0answers
10 views

Expectation Maximization question

I came across this question while practicing EM question but I don't understand how to apply EM in this scenario. What's the latent variable here? Is it the grade of each student? What will be the ...
0
votes
0answers
12 views

How does Link Analysis work?

Currently, I am into Link Analysis in data mining. I am kind of having a hard time to understand the Link Analysis. I have studied Association Rules and this is my next goal to understand. I have been ...
0
votes
0answers
43 views

Which math fields do I need to learn for machine learning?

I decided to become a serious machine learning practitioner and make new ML algorithms for myself. It seems I need to learn math to understand machine learning algorithms and make new ones. Because ...
0
votes
1answer
23 views

Perceptrons that recognize AND, OR, NOT

I'm trying to figure out how to create a set of perceptron weights: one for AND, one for OR, one for NOT. I'm not sure where to begin, but any hints are greatly appreciated!
0
votes
1answer
36 views

Beginner's questions to Hidden Markov Models

I have started reading about Hidden Markov Models, and have some (more or less) minor questions about things I am not sure I understood correctly. I hope asking here is fine: (1) Assumption about the ...
3
votes
2answers
96 views

Does set theory help understand machine learning or make new machine learning algorithms?

When I was in a university, I didn't major in math but took some math classes. However, I dropped out of math classes pretty quick. Some person recommended that I learn some ...
0
votes
0answers
26 views

For a PAC learnable hypothesis Show that its sample complexity $m_{\mathcal{H}}$ is monotonically non-increasing in each of its parameters

Not sure if this is the right place to post this, if this isn't i'll be grateful if someone will direct me where best to post it. I'm independently taking the course Introduction to Machine language ...
0
votes
0answers
18 views

adaboost weighting scheme’s equality

As you all know, ada-boost weighting is as follows, $$ \begin{cases} e^{-\alpha} & \quad \text{for right classified}\\ e^\alpha & \quad \text{for miss-classified} \end{cases} $$ ...
0
votes
0answers
10 views

Scaling and cross-validation in statistical models

Let's say i have a two dimensional dataset (X and Y variables). My goal is to fit a model that best describes the X-Y relationship Using a training subset of the dataset and then evaluate the ...
0
votes
0answers
13 views

Does every kernel function need to be a dot product in practice?

everyone. I just recently began studying about machine learning, and I have a question about the application of kernel functions. Intuitively, a kernel function is a similarity measure, right? Let's ...
0
votes
0answers
10 views

ANSML - Proving of the matrix identity $\nabla_AtrABA^TC = CAB+C^TAB^T$

(ANSML is a tag I would like to use for Andrew Ng's Stanford Machine Learning - 2008) In this course, there were four matrix identities that I would like to prove. \begin{align} \nabla_a \text{tr}AB ...
0
votes
0answers
11 views

Scatter plot predict/forecast values based on historic values

I do not know if this is the correct forum, as you guys are good at math I'll give it a shot here. Now I've been thinking about this for awhile and have not found out any statistical/mathematical ...
1
vote
1answer
21 views

Trouble understanding how Naive Bayes Classifier is derived

I've come across the Naive Bayes Classifier while studying machine learning, but the trouble I'm having is with some of the probability theory used to derive the formula for finding the optimal ...
0
votes
1answer
52 views

Learning finite automata from symbol set and given sample

Good day. We have a finite automaton F1, for example, . We need to get automaton F2 that accepts strings like accepted by ...
0
votes
0answers
16 views

Predictive Density Independent in Gaussian Process Regression?

I am a little confused in Gaussian process regression. In a GP regression, let $Y=[Y_a, Y_b]\sim \mathcal{N}(0, K+\sigma^2I)$, where $Y_b$ is the target of training samples. The task is to predict ...
0
votes
0answers
6 views

Application of line integral or surface integral to machine learning?

I am exploring the kernel methods in machine learning, and found an interesting post on this. In my point of view, kernel method is a way of reducing dimensions. I have an intuitive understanding that ...
0
votes
0answers
8 views

PDE VS machine learning when solving complex systems?

I am wondering how PDE can be used in machine learning theory. I have got idea from this post also this question Based on what I learn from machine learning discriminative and generative models, I ...
0
votes
1answer
34 views

Find a line such that sum of perpendicular distances of points to the line is minimized

Given a set of points (column vectors) $S = \{p_1, p_2, \cdots, p_n\} \subset \Re^d$, let $A \in \Re^{n \times d}$ be a matrix of which each row is just $p_i^T$. It is easy to find a unit vector $s_1$ ...
1
vote
2answers
37 views

What is the rigorous justification for using inner products as a function of similarity between two vectors?

In machine learning, it is a common thing to define similarity measures, specially using the so call Kernel function. Kernel functions are defined though through inner products of feature vectors: ...
0
votes
1answer
37 views

Cluster probabilites: Bayesian network (sprinkler example, Russel/ Norvig) as a clustered network

like others here I am also learning with Russel's and Norvig's book about artificial intelligence. My question is about the conditional probability tables of a clustered multiply connected network ...
2
votes
0answers
31 views

Difficulty in understanding pattern recognition and machine learning (Bishop) [closed]

I started with Pattern recognition and machine learning by Bishop, but after completing the first chapter(which took lot of time) I feel I am facing lot of problem in understanding the mathematics ...
0
votes
1answer
25 views

Where can I find the solutions to exercises of Probabilistic Graphical Models?

I am self-learning Probabilistic Graphical Models written by Daphne Koller. And for testing how well I learned, I did the exercises in the textbook. But I have no solutions to these exercises. Can ...
0
votes
0answers
23 views

Motivation for gradient descent method over OLS/MLE for simple linear regression?

I am beginner in machine learning and I am currently trying to find the motivation for gradient descent method. I am confused why we want to employ gradient descent method for linear regression? I see ...
1
vote
0answers
17 views

Understanding, Non-Negative Sparse Coding algorithm

I have a question regarding sparse coding, Non-negative sparse coding. Iterate until convergence: $ \mathbf{A_i} \leftarrow \arg \! \min_{A \geq 0} || \mathbf{X}_i - \mathbf{B}_i\mathbf{A}||_F^2 + ...
6
votes
1answer
156 views

Mathematical introduction to machine learning

At first glance, this is once again a reference request for "How to start machine learning". However, my mathematical background is relatively strong and I am looking for an introduction to machine ...
1
vote
0answers
24 views

GMM EM-algorithm VS the Multinomial Logit/Probit Model of Discrete Choice Modeling

I am taking two courses where I learn GMM and MNL separately. However, I do see some similarities between they two: like we need indicator variable for discrete choice modeling when using the MLE, ...
1
vote
0answers
16 views

Fisher Expected Information for a Gaussian Process model

Suppose I have a two dimensional Gaussian process model (GP), defined by a squared exponential correlation function s.t: $$R(x_{i},x_{j}) = \exp\left(-\frac{|x_{i} - x_{j}|^2}{2}\right).$$ I am ...
0
votes
1answer
37 views

Manifold learning: How should this method be interpreted?

I am trying to learn about manifold learning techniques; a family of dimensionality reduction methods in machine learning. According to this idea, there is a low ($d$) dimensional, hidden space where ...
1
vote
2answers
44 views

Thus the eigenvectors of a scatter matrix form a base?

Suppose I have a data set of $m$ vectors in $\mathbb{R}^d$, $D = \{x_1,\ldots,x_m\}$. Let $S = \sum_{i=1}^{m}x_ix_i^T$ be the scatter matrix. My question is: thus the eigenvectors of $S$ form a base ...
4
votes
1answer
43 views

Intuitive explanation of the term manifold

I am reading Christopher Bishop's "Pattern Recognition and Machine Learning" and in the first chapter, where he talks about the curse of dimensionality, he gives the following example: Consider, ...
0
votes
0answers
11 views

Convert KL-divergence to probability

First of all excuse my English is pretty horrible. I'm using the KL-divergence for a metric between compare histograms. I wanted to see if it is possible to convert this value of the KL-divergence at ...
0
votes
0answers
21 views

Minimizer of $\frac{\lambda}{2} \| \theta - \theta^{(k)}\| + \text{Loss}_\text{hinge}(y \theta \cdot x)$

How do you find the minimizer of: $$\min_{\theta \in \mathbb{R}^d}\left\{ \frac{\lambda}{2} \| \theta - \theta^{(k)}\|^2 + \text{Loss}_\text{hinge}(y \theta \cdot x)\right\}$$ if $\theta^{(k)} \in ...
0
votes
0answers
14 views

SVM Soft Margin Lagrange form

I study the Lagrange multipliers form of SVM. I am particulary interested in values that $\alpha_i$ can get. The following is the Langange multipliers form of hard margin SVM. $min_{w,b} ...
1
vote
0answers
25 views

Practical exercise in SVM

Suppose we have four positive points $\{0,1,2,3\}$ and three negative points $\{-3,-2,-1\}$. We want to learn soft-margin linear SVM $\min_{w}0.5 \left \| w \right \| +C \sum \epsilon_i$ the ...
1
vote
1answer
41 views

Covering numbers of metric space

I'm currently reading a paper "On the foundations of machine learning" by F.Cucker and S.Smale and I got stuck on an apparently simple problem. In order to prove an inequality that gives bound on ...
1
vote
1answer
45 views

Book recommendation for wavelet analysis

I am master student doing research in data mining, i read a paper about wavlet analysis for data mining, so i think it may help me in the future. But in my undergraduate degree the last course in ...
0
votes
0answers
5 views

Resulting function after regularization for the courve overfitting problem

A solution for the over fitting problem is the regularization as follows: The function that can overfit if the points number is too low and the order of the equation is greater than the points number ...
0
votes
1answer
21 views

Application of wavelet analysis in computer science

I am doing research in computer science (data mining), do you think wavelet analysis is useful for me?
1
vote
0answers
26 views

Why use two slack variables in the support vector regression formulation?

I am learning support vector regression but cannot fully understand the rational of the slack variable tricks in its formulation. The original optimization problem for SVR is as follows: ...
0
votes
0answers
18 views

For normally distributed points in R^n, what is the probability that p randomly chosen points are linearly separable from all other points?

I am trying to use this to solve a more general problem: the expected number of linearly separable subsets of size s of a set of normally distributed points. Help is much appreciated.
0
votes
1answer
15 views

Scaling Cumulative Probability Distribution function values

We have a cumulative probability distribution function (cdf), we want to scale it down for using it in anomaly detection. The mapping should look like this. CDF value: 0.1 ... 0.5 ... 0.9 ... ...
1
vote
0answers
11 views

SVM maximum-margin distance

I study SVM It's known that the distance between two hyperplanes is $\frac{2}{\left \| w \right \|}$. The problem is I cannot prove this. Let's start. We have two hyperplanes $w \cdot x +b = 1$ and ...
0
votes
0answers
32 views

possible number of hypothesis givin a training set

training set contains p number of papers. each paper is annotated has research or non-research. To develop the research paper filter, we consider the W most frequent phrases in a paper. the research ...