How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

0
votes
0answers
15 views

How is logistic loss and cross-entropy related?

I found that Kullback-Leibler loss, log-loss or cross-entropy is the same loss function. Is the logistic-loss function used in logistic regression equivalent to the cross-entropy function? If yes, can ...
2
votes
1answer
21 views

Analytic solution for matrix factorization using alternating least squares

The standard form for ridge regression aims to minimize the following cost function. $$ \min\ \ \sum_i(y_i-x_i^T\beta)^2 + \lambda\sum_j\beta^2_j $$ As described here, it's possible to differentiate ...
1
vote
1answer
29 views

Gaussian Process Regression

Observations: $$ X= \begin{pmatrix} x_1 \\ x_2 \\ \end{pmatrix} = \begin{pmatrix} 0 & 1 \\ 0.5 & 2 \\ \end{pmatrix} $$ $$ y= ...
0
votes
1answer
21 views

proof that a density proportional to Gaussian is Gaussian

I try to develop bayesian estimation for one dimensional Gaussian with unknown $\mu$ and known $\sigma$. I got $$p(x\mid D) = \int p(x\mid\mu)p(\mu\mid D) \, d\mu =\int \frac{1}{\sigma ...
3
votes
2answers
51 views

Intuition about gradient

https://en.wikipedia.org/wiki/Gradient Gradient is a vector which we can obtain from any differentable function taking its partial derivatives. From Wiki: "...the gradient points in the direction of ...
-2
votes
0answers
18 views

Is there a guitar music learning machine? [closed]

In the non-zero sum guitar music publishing game the guitar music composer makes music public by intonation but cannot read or write music. The publisher can write music acquired by auditory ...
0
votes
0answers
20 views

Gram matrix of Gaussian kernel is not positive definite

I am developing a machine learning software, where I am trying to apply kernel methods. I have N uniformly sampled scalar values, $\{x_1,\dots,x_N\}$ from a given interval $[a,b]$. My aim is to ...
0
votes
0answers
9 views

direct connection between gradient descent and follow the (perturbed) leader algorithm or weighted majority? [migrated]

Is there a direct conversion between gradient descent ([1], Alg 1 ) and any of the following algorithms? 1) Weighted Majority: http://onlineprediction.net/?n=Main.WeightedMajorityAlgorithm 2) ...
0
votes
1answer
23 views

How we come up with: $\mathcal{w}^T\mathcal{w} = \sum_{i,j} y^{(i)}y^{(j)} \alpha_{j}\alpha_{j} (\mathbf{x}^{(i)})^T {\mathbf{x}^{(i)}}$?

The primal optimization problem for finding the optimal margin classifier is: \begin{align} \arg\min_{\mathbf{w}}\frac{1}{2}\|\mathbf{w}\|^2_2 \\ \text{subject to } \quad ...
-1
votes
1answer
24 views

simTuring Machines

Is a player piano a Turing Machine reading data on the piano roll when perforations in paper control mechanical stops that intonate sounds according to the player algorithm? And by analogy then is a ...
0
votes
1answer
21 views

How to get a valid distance metric?

I have got a problem to devise a distance metric to get the similarity measurement of vectors. Someone suggested me to use dot product, which seems to me the same as the Cosine similarity metric; ...
1
vote
0answers
27 views

Extracting $G$ from $K^{-1}=G^TG$, knowing $K$

In a machine learning project I am working on, I have came across the following problem. I have a symmetric, $d\times d$ matrix $K_{oij}=k(x_i,x_j)$ where $k$ is a Gaussian kernel function. $x_1,\dots ...
0
votes
0answers
14 views

Determining a good genome for a genetic algorithm to alter neural network characteristics?

I am developing an application to run a genetic algorithm over the input characteristics of a neural network. I am currently looking for help finding a good "genome" to use along with good example ...
0
votes
0answers
11 views

Application of Gaussian Width in “ a subspace missing a set ”

As in Gaussian Mean Width, it says the concept of Gaussian width of a set $K$, which is a subset of a unit sphere $S^{n-1} \in R^n$. As a popular application, we use it to estimate if there ...
0
votes
0answers
29 views

Why does $\int f(x)(y-r(x))\;dP(y,x) = 0$?

My question is, why does: $$\int f(x)(y-r(x))\;dP(y,x) = 0,$$ where $r(x) = \int y \;dP(y|x)$ and $P$ is a probability distribution function. It was also given (in my book) that: $$\int ...
2
votes
1answer
41 views

Reproducing kernel Hilbert space, why?

Let $K: X \times X \rightarrow \mathbb{C}$ be a positive definite kernel on a set $X$, i.e. for any $x_1, \cdots, x_n \in X$, the matrix $$ [K(x_i, x_j)]_{ij} \in \mathbb{C}^{n \times n} $$ is ...
0
votes
0answers
21 views

What is the Gini impurity index of an empty set?

Now, this may be a silly question because in practice you would never calculate the gini impurity on an empty set of observations. However, I did notice that while the shannon entropy is 1.0 for an ...
2
votes
0answers
15 views

Non parametric estimators for noisy funcions

Suppose there is a function $f(a,b,c,\ldots)$ of $M$ variables (fixed numbers, not random variables). Add some Gaussian noise to this function: $$ g(a,b,c,\ldots) = f(a,b,c,\ldots) + ...
0
votes
0answers
18 views

An example shows the difference between inference in Bayesian network and Junction Tree

Why inference in Junction tree is more efficient? There are directed graph BN and the corresponded undirected graph transformed by Junction tree algorithm. The literature describes that inference in ...
1
vote
0answers
10 views

Relation between RKHS and space of continuous functions

Consider a Mercer Kernel $K\colon \mathcal{X}\times \mathcal{X}\to \mathbb{R}$, $\mathcal{X}$ being a compact subset of $\mathbb{R}^m$, and its (unique) associated Reproducing Kernel HIlbert Space ...
1
vote
1answer
93 views

Unsupervised learning algorithms to detect anomaly in waves.

I have a sample of graphs (more than 10000...). that look like in the image below: I am searching for an unsupervised learning algorithms that can help me to detect anomalous observations. Here ...
0
votes
0answers
29 views

Please enlighten me with Platt's SMO algorithm (for SVM)

From A_Roadmap_to_SVM_SMO.pdf, pg 12. Assume I am using linear kernel, how will I be able to get both the first and second inner product? My guess, inner product of datapoint with datapoint j ...
0
votes
1answer
15 views

Gradients of marginal likelihood of Gaussian Process with squared exponential covariance, for learning hyper-parameters

The derivation of gradient of the marginal likelihood is given in http://www.gaussianprocess.org/gpml/chapters/RW5.pdf But the gradient for the most commonly used covariance function, squared ...
0
votes
1answer
23 views

Markov Chain (Learning)

If I have a Matrix like the one below, what is the probability $p_t$ that at a certain time $t$, we are still not able to arrive at state $z$ $$ \begin{array}{c|lcr} \text{States} & x & y ...
0
votes
1answer
9 views

Converting normalised values into original

I have a normalisation formula as follows, which takes a list of numbers, such as $1,2,3,4,5,6,7,8,9,10$, and returns the normalized values which guarantees that $\tilde{x_i} \in [0,1]$. ...
0
votes
1answer
66 views

Proof: $\forall \pi : V^\pi(s) = max_{a \in A_s} Q^\pi(s,a) \forall s \in S \Rightarrow \pi \text{ is optimal}$

This is a homework example for reinforcement (machine) learning that I have to solve. I just would like to know if my thoughts on this are somehow correct. Let $\pi$ be a policy and $V^\pi(s)$ the ...
1
vote
0answers
60 views

Normalizing Vectors to get short numbers

$\vec{A}$ is vector agent, $\vec{O}$ is vector Object, $m$ is a constant integer. The following expression is repeated with a different O for every loop cycle: ...
0
votes
1answer
30 views

How to Derive Point on Plane from Normal Vector : geometric Margins

Consider the snippet below from Andrew Ng's lecture notes on Support Vector Machines. He goes on to state that $B = x^{(i)} - \gamma^{(i)} \frac{w}{\|w\|}$. I am having a hard time seeing why this ...
2
votes
1answer
26 views

Rademacher complexity of regularized linear function class: does it depend on dimension or not?

I am going through some lecture notes on Learning Theory here: http://ttic.uchicago.edu/~tewari/LT_SP2008.html trying to learn about Rademacher complexities. I'm getting confused about the Rademacher ...
0
votes
1answer
25 views

Machine learning Linear regression cost function

I am doing a project in deep learning and I have been taking Andrew's machine learning course from youtube. I am having difficulty in understanding the working of cost function. given the equation ...
0
votes
1answer
22 views

Proof of why the partition function Z in probabilistic graphical models (PGM) is NP-complete

I was wondering if someone knew why computing the partition function for probabilistic graphical models is NP-Hard? I would like to see a full blown rigorous proof, however, I am as happy to get a ...
0
votes
1answer
14 views

Improving data gaussianity using neural networks

I wanted to know if there is a way to use neural networks (deep neural networks or autoencoders) for a data gaussianization. I wonder how could the output data distribution be monitored and ...
3
votes
1answer
78 views

Variances for K-Means clustering

Can somebody help me understand formulas with an example in the below image? The below image is about K-means clustering. The formulas are about calculations for the variance for within-clusters and ...
0
votes
0answers
18 views

How to prove or disject that the following kernels are a mercer kernel

These kernels are for my master degree in computer science, I have fellings that the first one is not a mercer kernel, but the second one is I have this thought because I simulate different matrices ...
0
votes
1answer
17 views

Question about consistency in the junction tree algorithm (graphical models

I have a question about consistency in graphical models. It is often stated that when running the junction tree algorithm on a clique (cluster) tree, the marginals of all nodes are locally and ...
0
votes
0answers
22 views

How PCA solution for whitenning can be obtained?

Definition: The matrix W, is a whitening matrix in the following linear transform: $u=Wx$ when the covariance matrix of the output vector, u, satisfies: $<uu^T>=I$ where I represents Identity ...
0
votes
1answer
18 views

Normalization in Linear Regression

In linear regression problems it is important not to have a curve that overfits the input data or training examples. In other words, the curve should generalise your training data so you can predict ...
0
votes
1answer
25 views

reinforcement learning as an observer

My task is to build a system that can make predictions about a player's future in-game actions by observing his/her history of interaction with the environment. Reinforcement learning is about ...
-1
votes
1answer
38 views

Drawing Probability Density Function

Can someone help me to draw this pdf? I really don't have an idea how to convert a function to pdf. Thank you p(x | c) = 1/3 for 1 <= x <= 4 and P(c) = 0.5
1
vote
0answers
14 views

Coefficients in Representer theorem

I have a Mercer Kernel, $K\colon X \times X \rightarrow \mathbb{R}$, i.e. it is continuous, symmetric and postive definite on a compact domain $X \subset \mathbb{R}^n$. Also, I have a set of $m$ ...
2
votes
1answer
78 views

Step by step LMS for learning a linear function

Disclaimer Since this is an exercise assignment I'm not looking for a complete solution but for help that enables me to solve it on my own The task Given the error function ...
0
votes
0answers
14 views

Preparing data for WEKA decision tree J48

I'm trying to deal with WEKA and J48 algorithm. Looks like I have to present all my numerical values like age, income, height, weight as classes: age_from_18_to_25, age_from_26_to_40, e.t.c. Here is ...
1
vote
0answers
35 views

What is the left derivative of the hinge loss function in the context of subgradients?

Let: $$|a|_+ = max\{0,a\}$$ Then the Hinge loss function (in the context of classification in Machine Learning) is: $$V(-yf(x)) = |1 - yf(x)|_+$$ Note that $y \in \{-1,1\}$ Let $f(x) = \langle w, ...
1
vote
1answer
71 views

books on the application of linear algebra on statistics/finance/machine learning

I am reading "linear algebra done right" by Axler and like it a lot. One thing though, in the end I would like to put these theory to use and as a math textbook it doesn't cover much application. ...
0
votes
1answer
27 views

Optimization options to select multiple items with different features and values

I'm trying to identify which approach would work best to select a set of elements that have different features that minimise a certain value. To be more specific, I might have a group of elements with ...
0
votes
0answers
17 views

How are parameters constrained in a kernel function?

(Kernel as in the kernel trick used in machine learning) Suppose you have the following kernel function: $$ k(x_m,x_m) = \theta_0*\exp\left\{ - \frac{\theta_1}{2}\|x_n - x_m \|^2\right\} + \theta_2 ...
0
votes
2answers
30 views

How to expand inner product square?

How does this $||x-x'||$ expand to the equation below? $\|x-x'\|^2 = (x^T)x + (x')^T x' - 2x^T x'$
0
votes
1answer
36 views

Understanding how to solve a Cost Function?

I'm having trouble seeing the relationship in the following equation. Let's assume $J(0,1)$ and $m=4$. First I figure out my hypothesis function ...
0
votes
0answers
12 views

Is it true that the ideal predictor that minimizes the logistic function and the exponential function are the same?

Is it true that the ideal predictor that minimizes the logistic function and the exponential function are the same? i.e. it true that (obviously assuming we know the probability distribution): ...
3
votes
4answers
229 views

Formal proof that mean minimize squared error function

On an important book of Machine Learning, I've found this proof. We want to minimize the cost function $J_0(X_0)$ defined by the formula $$J_0(x_0) = \sum_{k=1}^n \|x_0 - x_k \|^2.$$ The solution to ...