How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

0
votes
0answers
3 views

Application of Gaussian Width in “ a subspace missing a set ”

As in Gaussian Mean Width, it says the concept of Gaussian width of a set $K$, which is a subset of a unit sphere $S^{n-1} \in R^n$. As a popular application, we use it to estimate if there ...
0
votes
0answers
20 views

Why does $\int f(x)(y-r(x))\;dP(y,x) = 0$?

My question is, why does: $$\int f(x)(y-r(x))\;dP(y,x) = 0,$$ where $r(x) = \int y \;dP(y|x)$ and $P$ is a probability distribution function. It was also given (in my book) that: $$\int ...
2
votes
1answer
36 views

Reproducing kernel Hilbert space, why?

Let $K: X \times X \rightarrow \mathbb{C}$ be a positive definite kernel on a set $X$, i.e. for any $x_1, \cdots, x_n \in X$, the matrix $$ [K(x_i, x_j)]_{ij} \in \mathbb{C}^{n \times n} $$ is ...
0
votes
0answers
19 views

What is the Gini impurity index of an empty set?

Now, this may be a silly question because in practice you would never calculate the gini impurity on an empty set of observations. However, I did notice that while the shannon entropy is 1.0 for an ...
2
votes
0answers
10 views

Non parametric estimators for noisy funcions

Suppose there is a function $f(a,b,c,\ldots)$ of $M$ variables (fixed numbers, not random variables). Add some Gaussian noise to this function: $$ g(a,b,c,\ldots) = f(a,b,c,\ldots) + ...
0
votes
0answers
11 views

An example shows the difference between inference in Bayesian network and Junction Tree

Why inference in Junction tree is more efficient? There are directed graph BN and the corresponded undirected graph transformed by Junction tree algorithm. The literature describes that inference in ...
1
vote
0answers
9 views

Relation between RKHS and space of continuous functions

Consider a Mercer Kernel $K\colon \mathcal{X}\times \mathcal{X}\to \mathbb{R}$, $\mathcal{X}$ being a compact subset of $\mathbb{R}^m$, and its (unique) associated Reproducing Kernel HIlbert Space ...
-1
votes
0answers
25 views
+50

Unsupervised learning algorithms to detect anomaly in waves.

I have a sample of graphs (more than 10000...). that look like in the image below: I am searching for an unsupervised learning algorithms that can help me to detect anomalous observations. Here ...
0
votes
0answers
29 views

Please enlighten me with Platt's SMO algorithm (for SVM)

From A_Roadmap_to_SVM_SMO.pdf, pg 12. Assume I am using linear kernel, how will I be able to get both the first and second inner product? My guess, inner product of datapoint with datapoint j ...
0
votes
0answers
13 views

slaters condition - Duality - KKT condition [closed]

Can someone give a more intuitive idea to Slater's conditions and how it is related to KKT condition and duality ?
0
votes
0answers
8 views

Gradients of marginal likelihood of Gaussian Process with squared exponential covariance, for learning hyper-parameters

The derivation of gradient of the marginal likelihood is given in http://www.gaussianprocess.org/gpml/chapters/RW5.pdf But the gradient for the most commonly used covariance function, squared ...
0
votes
1answer
22 views

Markov Chain (Learning)

If I have a Matrix like the one below, what is the probability $p_t$ that at a certain time $t$, we are still not able to arrive at state $z$ $$ \begin{array}{c|lcr} \text{States} & x & y ...
0
votes
1answer
9 views

Converting normalised values into original

I have a normalisation formula as follows, which takes a list of numbers, such as $1,2,3,4,5,6,7,8,9,10$, and returns the normalized values which guarantees that $\tilde{x_i} \in [0,1]$. ...
0
votes
1answer
63 views

Proof: $\forall \pi : V^\pi(s) = max_{a \in A_s} Q^\pi(s,a) \forall s \in S \Rightarrow \pi \text{ is optimal}$

This is a homework example for reinforcement (machine) learning that I have to solve. I just would like to know if my thoughts on this are somehow correct. Let $\pi$ be a policy and $V^\pi(s)$ the ...
1
vote
0answers
58 views

Normalizing Vectors to get short numbers

$\vec{A}$ is vector agent, $\vec{O}$ is vector Object, $m$ is a constant integer. The following expression is repeated with a different O for every loop cycle: ...
0
votes
1answer
29 views

How to Derive Point on Plane from Normal Vector : geometric Margins

Consider the snippet below from Andrew Ng's lecture notes on Support Vector Machines. He goes on to state that $B = x^{(i)} - \gamma^{(i)} \frac{w}{\|w\|}$. I am having a hard time seeing why this ...
2
votes
1answer
23 views

Rademacher complexity of regularized linear function class: does it depend on dimension or not?

I am going through some lecture notes on Learning Theory here: http://ttic.uchicago.edu/~tewari/LT_SP2008.html trying to learn about Rademacher complexities. I'm getting confused about the Rademacher ...
0
votes
1answer
24 views

Machine learning Linear regression cost function

I am doing a project in deep learning and I have been taking Andrew's machine learning course from youtube. I am having difficulty in understanding the working of cost function. given the equation ...
0
votes
1answer
20 views

Proof of why the partition function Z in probabilistic graphical models (PGM) is NP-complete

I was wondering if someone knew why computing the partition function for probabilistic graphical models is NP-Hard? I would like to see a full blown rigorous proof, however, I am as happy to get a ...
0
votes
1answer
14 views

Improving data gaussianity using neural networks

I wanted to know if there is a way to use neural networks (deep neural networks or autoencoders) for a data gaussianization. I wonder how could the output data distribution be monitored and ...
2
votes
2answers
59 views

Variances for K-Means clustering

Can somebody help me understand formulas with an example in the below image? The below image is about K-means clustering. The formulas are about calculations for the variance for within-clusters and ...
0
votes
0answers
18 views

How to prove or disject that the following kernels are a mercer kernel

These kernels are for my master degree in computer science, I have fellings that the first one is not a mercer kernel, but the second one is I have this thought because I simulate different matrices ...
0
votes
1answer
14 views

Question about consistency in the junction tree algorithm (graphical models

I have a question about consistency in graphical models. It is often stated that when running the junction tree algorithm on a clique (cluster) tree, the marginals of all nodes are locally and ...
0
votes
0answers
22 views

How PCA solution for whitenning can be obtained?

Definition: The matrix W, is a whitening matrix in the following linear transform: $u=Wx$ when the covariance matrix of the output vector, u, satisfies: $<uu^T>=I$ where I represents Identity ...
0
votes
1answer
18 views

Normalization in Linear Regression

In linear regression problems it is important not to have a curve that overfits the input data or training examples. In other words, the curve should generalise your training data so you can predict ...
0
votes
1answer
25 views

reinforcement learning as an observer

My task is to build a system that can make predictions about a player's future in-game actions by observing his/her history of interaction with the environment. Reinforcement learning is about ...
-1
votes
1answer
38 views

Drawing Probability Density Function

Can someone help me to draw this pdf? I really don't have an idea how to convert a function to pdf. Thank you p(x | c) = 1/3 for 1 <= x <= 4 and P(c) = 0.5
1
vote
0answers
13 views

Coefficients in Representer theorem

I have a Mercer Kernel, $K\colon X \times X \rightarrow \mathbb{R}$, i.e. it is continuous, symmetric and postive definite on a compact domain $X \subset \mathbb{R}^n$. Also, I have a set of $m$ ...
2
votes
1answer
78 views

Step by step LMS for learning a linear function

Disclaimer Since this is an exercise assignment I'm not looking for a complete solution but for help that enables me to solve it on my own The task Given the error function ...
0
votes
0answers
10 views

Preparing data for WEKA decision tree J48

I'm trying to deal with WEKA and J48 algorithm. Looks like I have to present all my numerical values like age, income, height, weight as classes: age_from_18_to_25, age_from_26_to_40, e.t.c. Here is ...
1
vote
0answers
32 views

What is the left derivative of the hinge loss function in the context of subgradients?

Let: $$|a|_+ = max\{0,a\}$$ Then the Hinge loss function (in the context of classification in Machine Learning) is: $$V(-yf(x)) = |1 - yf(x)|_+$$ Note that $y \in \{-1,1\}$ Let $f(x) = \langle w, ...
1
vote
1answer
68 views

books on the application of linear algebra on statistics/finance/machine learning

I am reading "linear algebra done right" by Axler and like it a lot. One thing though, in the end I would like to put these theory to use and as a math textbook it doesn't cover much application. ...
0
votes
1answer
27 views

Optimization options to select multiple items with different features and values

I'm trying to identify which approach would work best to select a set of elements that have different features that minimise a certain value. To be more specific, I might have a group of elements with ...
0
votes
0answers
16 views

How are parameters constrained in a kernel function?

(Kernel as in the kernel trick used in machine learning) Suppose you have the following kernel function: $$ k(x_m,x_m) = \theta_0*\exp\left\{ - \frac{\theta_1}{2}\|x_n - x_m \|^2\right\} + \theta_2 ...
0
votes
2answers
26 views

How to expand inner product square?

How does this $||x-x'||$ expand to the equation below? $\|x-x'\|^2 = (x^T)x + (x')^T x' - 2x^T x'$
0
votes
1answer
32 views

Understanding how to solve a Cost Function?

I'm having trouble seeing the relationship in the following equation. Let's assume $J(0,1)$ and $m=4$. First I figure out my hypothesis function ...
0
votes
0answers
12 views

Is it true that the ideal predictor that minimizes the logistic function and the exponential function are the same?

Is it true that the ideal predictor that minimizes the logistic function and the exponential function are the same? i.e. it true that (obviously assuming we know the probability distribution): ...
3
votes
4answers
196 views

Formal proof that mean minimize squared error function

On an important book of Machine Learning, I've found this proof. We want to minimize the cost function $J_0(X_0)$ defined by the formula $$J_0(x_0) = \sum_{k=1}^n \|x_0 - x_k \|^2.$$ The solution to ...
0
votes
1answer
9 views

Support of vector $w$ in graph sparsity

I'm reading about graph sparsity and I have one problem in a paper I'm reading I don't understand, maybe someone can clarify: Graph Sparsity: In graph sparsity, we have a directed acyclic graph ...
0
votes
1answer
15 views

Differentiable L-1 Regularization

In machine learning we are often faced with optimization problems where we want to minimize some energy function using L1 regularization over some of the parameters, e.g.: $$ E(a,w) = [\text{sum of ...
0
votes
1answer
48 views

how to calculate the marginal distribution of probabilistic principal component analysis

In the book Pattern recognition and machine learning from Bishop equation 12.33 states: $\mathbf{x} = \mathbf{W} \mathbf{z} + \boldsymbol\mu + \boldsymbol\epsilon$ Here $\mathbf{z}$ has a normal ...
0
votes
0answers
24 views

Simple confusion matrix question

Thanks in advance for the help. I have a set of data with n samples that I plan to use with knn to make some classifications. I want to test out the performance before applying it to my validation ...
1
vote
0answers
19 views

How do you show the connection of reproducing kernels to feature maps?

This question is in the context of Hilbert Reproducing Hilbert Spaces and reproducing Kernels and their relation to feature maps (and machine learning). We have a Hilbert space $\mathcal{F}$ and ...
0
votes
0answers
24 views

Conservative perceptron update rule - convex optimization

Suppose I have a condition on a perceptron update rule should be a little conservative. For example, it minimizes the distance between the new update and previous classifier $w_i$, i.e. $||w_{i+1} - ...
0
votes
1answer
37 views

Linear regression using gradient descent in Octave seems to fail

I was trying to implement linear regression with gradient descent using the equation presented on the machine learning course on Coursera: ...
1
vote
1answer
53 views

Help understanding machine learning cost function

I am taking an online class on Machine Learning and I'm trying to fully understand how the cost function work. Can someone explain to me exactly what is going on in the function below: Cost function ...
0
votes
0answers
15 views

Correlation vs Similarity

I want to know what is the difference between the correlation matrix and a similarity matrix, for e.g. if I have a matrix A consisting of items and I want to compute which items are similar to one ...
1
vote
1answer
36 views

Linear Regression with independent but non-identical noise

If I have this linear regression equation: $$y=X\beta+\epsilon $$ ($x$ and $\beta$ are vectors) The likelihood function can be written as $$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi ...
2
votes
1answer
103 views

Derivative of Softmax loss function

I am trying to wrap my head around backpropagation in a neural network with a softmax classifier, which uses the softmax function: \begin{equation} p_j = \frac{e^o_j}{\sum_k e^{o_k}} \end{equation} ...
1
vote
0answers
29 views

Integrating an expression over a vector $\mathbf{w}$

doing my homework for a Machine Learning course, I have to calculate the following expression: $\newcommand{\IDENTITY}{\mathbf{I}} \newcommand{\W}{\mathbf{w}} \newcommand{\WT}{\mathbf{w}^T} ...