How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

0
votes
0answers
12 views

Why is the notion of an XRP, as opposed to an IID variable, useful in programming?

The most general notion which shares the main properties of i.i.d. variables are exchangeable random variables, introduced by Bruno de Finetti. Exchangeability means that while variables may not be ...
0
votes
0answers
25 views

Is this function commonly known or has some name?

I have used this function for fitting in my research, and I wonder if there is a name for it, or is it commonly known in some reduced form? $f(x)=\alpha\frac{e^{-\gamma x}}{x^\beta}+\delta$ Actual ...
2
votes
0answers
35 views

How to solve this confusing iterative limit equation

I have this equation which is intended to be used on a data set of integers. I am attempting to create a software algorithm using this equation, but I am very confused about how to implement it. ...
1
vote
1answer
48 views

Equations for Classification & Probability Problem

There are 4 containers (classes) to keep balls of different colors (red, green, blue, orange). We know that Container A is for red balls because it contains 80% red balls. B for green balls (90%), and ...
1
vote
1answer
36 views

How can I calculate area under the curve?

I have problems understanding how can I calculate manually area under the curve for my predictions, knowing the real values. I understand the idea behind confusion matrix, can calculate true ...
0
votes
0answers
15 views

Adding one more feature to my feature set which has no effect in calculation and act as a distiguishable feature [closed]

I have a sort of problem as follow: I have $10000$ thousand of tweets and and I have some features which are labeled $1$ or $2$ . I wanna add another feature but the problem is exactly here: I want ...
0
votes
0answers
10 views

Statistical/ML models when observations have different amounts of input

Let's say we're predicting an employee's performance review score for the following year based on his/her performance review scores from each previous year of their employment. We might have these ...
1
vote
1answer
41 views

Cost Function of Neural Network (Forward Propagation)

This question is related to Andrew Ng's machine learning course on Coursera. Basically, when I calculate the cost function of a neural network, I use the following formula that was described by Ng: $$ ...
2
votes
2answers
29 views

How can dimension reduction lead to better results?

Can someone please explain why a model fitted using a linear combination of the parameters can have better results (lower error) than a plain vanilla one with all the parameters? Can I think about ...
0
votes
0answers
8 views

Is EM-algorithm only for missing data?

Currently studying EM algorithm and have been through a few articles, they all say it is for missing data. I believe there is some implication in the term "missing data". I wonder if EM is designed ...
1
vote
2answers
25 views

Plain English interpretation needed for the sentence to understand EM-algorithm?

I am trying to read an EM-algorithm article on the web, however, as soon as I started I have face a sentence interpretation problem with this like "... in the presence of missing or hidden data" in ...
1
vote
0answers
29 views

Gaussian Bayes Classification with dependent variables..

Gaussian Bayes Classification: two classes: $y \in \{-1,+1\}$ Dependencies for a vector of features ($x_1,x_2,x_3)$: $x_1=z,x_2=2z,x_3=t+3$, where $$P(z\mid y=+1) = \aleph(z;\mu_+,1),\qquad P(z\mid ...
0
votes
0answers
25 views

Derivatives of KL Divergence

We know that KL divergence $D(P||Q) = \sum_{i} p_{i}\log(\frac{p_{i}}{q_{i}})$, where P and Q are vectors. So, I think the derivatives of $D$ with respect to $Q$ is $\frac{\partial D}{\partial ...
1
vote
1answer
22 views

Literature on discriminant analysis

Can anyone suggest a good book on discriminant analysis - comprehensible and detailed? (Kendall and Stuart write about the subject too concisely.) Thanks in advance.
1
vote
0answers
17 views

System of equations for non linear functions

Im readign a book about Deep learning and in this book the author states something like the following when explaining why we need an error function. A neural net can bee seen as a system of ...
4
votes
1answer
66 views

PMF estimation: concentration inequalities for the $l_1$ and $l_\infty$ errors

Assume that you are given $n$ i.i.d samples $X_1, ..., X_n$ drawn from a discrete distribution $p = (p_1, ..., p_k)$. We would like to estimate $p$ using the empirical estimator \begin{equation} ...
0
votes
0answers
23 views

Optimize performance in Naive Bayes classifier with non gaussian conditional probability

Looking at the Python scikit-learn implementation of the Naive Bayes classifier one can use the Gaussian distribution for the conditional probability $P(x_i|y)$. There is an assumption here that the ...
0
votes
0answers
12 views

is there a min max normalization function with weight to middle towards 1?

So the standard min max is $\ \frac{x - min(x)}{ max(x) - min(x)}$ Given a range like 0 to 500 I would like 250 to normalize to 1 and 0 and 500 to 0. What is the proper way to do this? Apologies if ...
2
votes
1answer
38 views

Math notation: What does $I$ mean in this context?

Sorry if this is a noobish question, but I don't know what $I$ means in this context: $$\hat{f}(X) = \sum_{m=1}^5 c_m I\{(X_1, X_2) \in R_m\}$$ I am reading about Decision Trees in The Elements of ...
1
vote
1answer
33 views

Is there any difference between statistical learning and machine learning?

Straight to the point, I'm a math student and I have a course this year called Statistical Learning. From the description, the course contains: Large datasets analysis, regression, principal ...
0
votes
0answers
11 views

Gaussian Process with explicit basis functions

I am considering the Gaussian process with explicit basis functions as discussed in the book (section 2.7): http://www.gaussianprocess.org/gpml/chapters/RW2.pdf Has anyone tried to derive formulas ...
1
vote
1answer
27 views

How to calculate a posterior probability with a given Gaussian Mixture Model?

I'm building a GMM-based classifier in speech processing and I'm using GMM as a probabilistic scoring mechanism (therefore I don't intrinsically care about the underlying mixture components). For ...
0
votes
0answers
19 views

Struggling to understand multi-class logistic regression

It is well defined that given a data set of $N$ $i.i.d$ observations $\mathbf{X} = \{\vec{\mathbf{x}}_1, \dots, \vec{\mathbf{x}}_n\}$, along with corresponding target values $\vec{\mathbf{t}} = {t_1, ...
2
votes
0answers
23 views

Why does minimizing $H[f] =\sum^{N}_{i=1}(y_i-f(x_i))^2+\lambda \| Pf \|^2 $ leads to solution of the form $ f(x) =\sum^N_{i=1}c_iG(x; x_i)+p(x)$?

I was reading the following paper of dimensionality reduction (1) and also one on theory of networks for approximations and learning (2) and was trying to understand how the regularization problem ...
0
votes
0answers
14 views

ARD Kernel - explanation

The following text discusses that the ARD kernel is a regular gaussian kernel but one where $\Sigma$ is diagnonal and one where the $\sigma$'s go to infinity. It seems that the $\kappa$(x,x') would ...
1
vote
0answers
11 views

How close is $\operatorname{argmax}_p E[\log(f(p,\alpha)]$ to $\operatorname{argmax}_p \log(E[f(p,\alpha)])$?

Here $\alpha$ is a random variable and the expectation is taken with respect to that variable. I am wondering if it's the same in any case or there's a theorem quantifying how close both things are. ...
0
votes
0answers
24 views

What type of self-adjoint operator does $\hat{P}$ has to be for Green's function to result in a radial exponetial $e^{-\| x-t \|^2}$

I was reading the following paper on hyper basis function (HBF) (similar to radial basis function RBF network) and was trying to understand when is it the case that the network has radial basis ...
0
votes
0answers
26 views

Optimize to Find the Mahalanobis Distance to Minimize the Term

I have an optimization problem defined as following: Assuming we have a data set $ { \left\{ \left( {x}_{i}, {y}_{i} \right) \right\}}_{i = 1}^{N} $ where $ {x}_{i} \in {\mathbb{R}}^{d} $ and $ ...
0
votes
0answers
17 views

How do you compute the weighted sum of data points for learning the centers of a hyper basis function network (HBF)?

I was reading the following paper on hyper basis function (HBF) (similar to radial basis function RBF network) and was trying to figure out how one learns the movable centers of the hyper basis ...
1
vote
0answers
31 views

how to test if Linear Discriminant Analysis (LDA) I implemented works?

I have implemented Linear Discriminant Analysis (LDA) in C by referring various sources. Now, I would like to test the system with a simple configuration. How can I do that? I work on a speech ...
1
vote
1answer
35 views

How do the rows of a change of basis matrix form a basis for expressing columns?

I am reading this article on Principal Component Analysis (PCA) and in section III-B (page 3) it has strange definition I don't understand. In the toy example $\mathbf{X}$ is an $m \times n$ ...
2
votes
0answers
19 views

EM algorithm with constrained equation

I am reading a paper where author uses EM for the following equation to find the parameters $\theta$(and $\beta$) : $$ J=\sum_m \alpha_{m}\sum_i\sum_j w_{mij}\log\sum_k \theta_{ik}\beta_{mjk} $$ ...
0
votes
0answers
15 views

Is total least square solution only valid for isotropic error

Let $\mathbf{y} = \mathbf{Ax}$ represent a system of equation where, $\mathbf{y}\in\mathbb{R}^n, \mathbf{A}\in\mathbb{R}^{n\times m}$. However due to error in sensor, what we observe is the following ...
0
votes
1answer
36 views

Mathematical Intuition behind the tf-idf formula in Statistics

I was reading: https://en.wikipedia.org/wiki/Tf%E2%80%93idf#Definition But I cannot seem to understand exactly why the formula was constructed the way it is. What I do Understand: iDF should at ...
1
vote
0answers
57 views

Derivation of back-propagation equation $\frac{\partial E(\theta)}{\partial W^k}=x*\delta h^k+\tilde{h}^k*\delta y$ for convolutional autoencoders

I was reading the following paper on convolution stacked auto-encoders and they had the following convolution neural network (for auto-encoders, notice I didn't write the offset term [to avoid ...
0
votes
0answers
18 views

Why use an un-squared L2-norm regularizer?

Presumably it would be useful if you want to penalize small values more and large values less, but L1-norm shouldn't work too bad in those cases either and you get the benefit of sparsity. The only ...
3
votes
3answers
78 views

How does kernel work work

all, I have been learning kernel method for a long time. But I am still not very sure how it works. In my opinion, it works as follows: say $f(x) = \sum_i\alpha_ik(x_i, x)$. First we need to decide ...
3
votes
1answer
44 views

What's Being Returned Here?

I'm working my way through this paper, and I'm having a bit of trouble understanding what it's telling me to do. Here's the specific excerpt that's tripping me up: A (finite) one-shot game is a tuple ...
3
votes
0answers
25 views

What does “the activation of a basis” mean?

In the paper Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer, Andrew Y. Ng, Self-taught learning: transfer learning from unlabeled data, ICML '07 Proceedings of the 24th international ...
2
votes
1answer
52 views

What does it mean to convolve matrices of finite dimension?

If one is given two matrices $I$ and $K$ what does the notation: $$ I * K $$ mean rigorously/precisely? I do know the definition of convolution: $$ s[i, j] = (I * K)[i, j] = \sum_m \sum_n I[m,n] ...
0
votes
0answers
15 views

Softmax Regression Gradient Derivation

I'm implementing softmax regression and am deriving the max-log-likelihood update for gradient descent by hand first. Coming from the Stanford UFLDL site, they show the gradient of the cost function ...
0
votes
4answers
65 views

What maths courses are needed for Machine Learning

I may sound dump. But I really like to know what maths courses are needed for Machine Learning. I am not computer science graduate but seriously interested in AI, ML, Neural network etc, and I know ...
0
votes
1answer
20 views

Need some help understanding the notation for Online Machine Learning algorithms

I'm reading the Wikipedia article on Online Machine Learning and some of the algorithms mentioned there seem to be missing some context: ...
0
votes
0answers
35 views

What does $\frac{d^k h}{dx^k}$ mean in the context of vectors and regularization in machine learning?

I was watching a machine learning videos from the caltech course CS 156 and they have a slide where they talk about how radial basis functions (RBFs) can be derived from the following variational ...
0
votes
0answers
21 views

Pattern recognition and machine learning - Bishop: Difficulty deriving (2.111) and (2.112)

I'm having a hard time deriving (2.111) and (2.112) in this book (it's on page 110/703). Here is what it says: Finally, we seek an expression for the conditionalp(x|y). Recall that the results for ...
1
vote
1answer
62 views

Studying machine learning

I'm currently studying machine learning using Bishop's book "Pattern Recognition and Machine Learning". The main disadvantage of this book (for me) is a lack of practical applications. Also it seems ...
0
votes
0answers
33 views

What is topological learning?

I am getting this term topological learning in few places for example a reference is below at section 1.1.2: http://virenjain.org/thesis/VirenJainThesis_official.pdf Can anyone point out what ...
1
vote
1answer
33 views

In an ODE dynamic system, is there a convient way or algorithms for estimating the parameters which make the ODE solution satisfing some constraint?

I have construct a ODE dynamic system like this $$molA(t)==sa$$ $$molB'(t)=sb-db\;molB(t)+\frac{kab\;molA(t)\;molB(t)}{molB(t)+Jab}-\frac{kgb\;molG(t)\;molB(t)}{molB(t)+Jgb} $$ $ molC'(t)=sc-dc\ ...
1
vote
1answer
46 views

Problem with understanding the induction when proving Sauer Lemma.

I will replicate the proof here which is from the book "Learning from Data" B(N, k) is the maximum number of dichotomies on N points such that no subset of size k of the N points can be shattered by ...
0
votes
0answers
18 views

PCA when SVD is a skinny SVD

A = m * n matrix. When $m \ge n$, it is easy to see that the V matrix in the full SVD ($A = U*S*V^T$, where U and V are both orthonormal square matrix) and V in a skinny SVD are the same. When $m \lt ...