How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

1
vote
0answers
15 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
1
vote
1answer
27 views

Transform a k-CNF formulae to conjunctions of boolean literals

The question comes from Mehryar Mohri's Foundations of Machine Learning. In Example 2.5 the book transform a $k$-CNF formula to conjunctions of boolean literals, but I can't understand the trick in ...
0
votes
1answer
15 views

InformationGain on Two Continuos classes instead on inary

I've a problem regarding an excersise with information gain. I can't seem to get the right answer, because the excersises differs from what we learned. Usually, a target class is a binary variable ...
0
votes
1answer
34 views

Sigmoid function in neural network

I am studying a doctoral thesis on control-theory and have trouble understanding the notions and the notation introduced there. I am doing this out of interest on the subject, so I haven't had a ...
4
votes
1answer
99 views

Clarification about solution of linear SVM problem

I'm reading this tutorial about SVMs. I'd like to have two clarifications: at page 4 (bottom), why is that, after using (1.10) the summation is extended to only $m \in S$? In (1.10) the summation ...
0
votes
0answers
11 views

Back propagation on a function

I'm trying to run a back propagation to learn a simple function. I'm not sure what criteria decides the number of hidden layers and so forth. E.g, for a function like f(x)= x^4 - 15x^2; ...
4
votes
3answers
111 views

Hyperplanes and Support Vector Machines

I have the following question regarding support vector machines: So we are given a set of training points $\{x_i\}$ and a set of binary labels $\{y_i\}$. Now usually the hyperplane classifying the ...
0
votes
0answers
7 views

One-class Support Vector Machine Sensitivity Drops when the number of training sample increase

I am using One-Class SVM for outlier detections. It appears that as the number of training samples increases, the sensitivity TP/(TP+FN) of One-Class SVM detection result drops, and classification ...
1
vote
0answers
18 views

feature selection for continuous variables

I wonder how exactly "feature selection" should be performed in case of continuous feature values. When feature values are discrete it is very straitforward to apply feature selection, but what to do ...
1
vote
2answers
25 views

optimization problem: finding an hyperplane separating one point from a set of pointy maximizing the distance

I have this problem: I have a set of n-dimensional points $P$. I have one more n-dimensional point $q$. The points in $P$ are linearly separable from $q$ (i.e. it always exists an hyperplane $n^t x ...
1
vote
1answer
42 views

What is the difference between reinforcement learning, trial and error, and fictitious play?

I have three question about three algorithms. I have a game with $n$ players. The action space of player $i$ is given by $\mathcal{A}_i=\{a_1, a_2, \cdots, a_m\}=\mathcal{A}$. The joint action space ...
-1
votes
0answers
17 views

What machine learning tool is best suited for taking time series and descriptive and making a binomial classification. [migrated]

I have an interesting task of utilizing log data from computer servers in a server farm and predicting if a particular server is likely to fail in the next 24 hours. My data set will be comprised of ...
2
votes
1answer
24 views

Building intuition for tensors in machine learning

I'm trying to understand tensors in the context of machine learning, but all the resources that mention tensors that I've found so far were building the intuitions through physics applications. As ...
0
votes
0answers
13 views

Nonnegative Matrix Factorization in Machine Learning

I have a matrix $X^{m\times n}$, and I need to factorize it into $W^{m \times p}H^{p \times n}$, $p$ is the number of factors s.t. $p << min(m,n)$, and $m$ is the number of variables while $n$ ...
0
votes
0answers
27 views

Proof that feature normalization cause faster convergence of gradient descent

How to prove that if I do feature normalization (scaling of the $x_1,\ldots,x_n$ to be all in range $[0,1]$) to a convex function $f(x_1,\ldots,x_n)$ that returns real scalar, then gradient descent ...
0
votes
0answers
31 views

normalize function -inf to inf to integrate to one.

I have a function of the probability distribution (method of k nearest neighbor). How I normalize this function , that the area under the graph is equal to unity. I mean it will make the prob from ...
2
votes
1answer
40 views

Maximizing expected profit

Suppose that a person is going to sell Fizzy Cola at a football game and must decide in advance how much to order. Suppose that he makes a gain of $m$ cents on each quart that he sells at the game but ...
0
votes
0answers
25 views

i cant solve EM algorithm

suppose data set D={(1,3),(4,5),(2,*)} come from separable density p(x1,x2) = p(x1)*p(x2) . witch * is missing value and we have: with start Ɵ0 = (3,6) calculate one step of E and M steps. ...
0
votes
0answers
13 views

Adaptive whitening / decorrelation

I have multidimensional data as a set of vectors. I am currently whitening this data and removing the mean vector. I end up with decorrelated data with zero mean and variance equal to 1. I'm using ...
0
votes
1answer
21 views

Equality of Information Gain and Mutual Information

I am curious about definition of information gain and mutual information in the context of feature selection. If looks like two these measures define exactly the same thing, however I didn't find ...
0
votes
1answer
18 views

Understand the English paragraph on association rule.

I am currently studying Association Rule Pattern Mining. I am reading the explanation on wikipedia about it. Somehow, I feel like I have a problem in understanding the paragraph below. Can somebody ...
0
votes
0answers
17 views

Transition matrix in left-right hidden semi-Markov model

I'm developing a hidden semi-Markov model left-right . In a left-right model a sequence of $M$ states starts in state $1$ and ends in state $M$, with no repetition of states. Since the model is ...
0
votes
1answer
30 views

different Interpolation techniques

what are the differences between spline and Lagrange interpolation, and are there any other kinds that might be similar that perform well ?
0
votes
0answers
40 views

Pdf of this estimator

We have a set of unidimensional data, $X_1, \ldots , X_n$ drawn from the positive reals. We define a model for its distribution: The data are drawn from a uniform distribution on the interval $[a, ...
0
votes
1answer
17 views

Curve fitting to connect certain points

Well the image says everything, anyone has any idea how to, or where should i look to be able to draw the BLACK curve ? in fact i need a function that would connect the summits of these red dotted ...
0
votes
1answer
53 views

Probability Estimator

Hi I was going through the MIT 2005 Machine Learning homework assignments and I was having trouble understanding a few concepts in probability theory. I would be obliged if anyone could validate my ...
0
votes
0answers
9 views

Q th order polynomial transform to represent all the curves in $\mathbb{R^d} $

In space $ \mathcal{X} = \mathbb{R^2} $, to get all possible quadratic curves in $ \mathcal{X} $, we need feature transform $\mathbf{z} = \Phi_2(\mathbf{x})$, where $\mathbf{x} \in \mathbb{R^2}$, and ...
0
votes
1answer
20 views

Gradient descent with adaptive learning ratio.

I have a neural network, trained with SGD (stochastic gradient descent) with learning ratio $\alpha$. Each iteration I try to recalculate the weights with a rule: $$\Delta \vec{w} = -\alpha ...
0
votes
0answers
17 views

How to evaluate the difference between two classes of data which are highly overlapped

I’m trying to implement a classifier based on a dataset comprising two classes of high dimensional time-series observations (the values of the two classes of observations are highly similar). I ...
0
votes
1answer
31 views

Similarity metric between two sets of points with varying densities

How can I create a similarity metric that describes the top left set of points as more similar to the bottom left set of points than the top right set of points? Clearly least-squares distance doesn't ...
0
votes
0answers
20 views

Expected in-sample error of linear regression with respect to a dataset D

In my textbook, there is a statement mentioned on the topic of linear regression/machine learning, and a question, which is simply quoted as, Consider a noisy target, $ y = (w^{*})^T \textbf{x} + ...
2
votes
1answer
117 views

Function to predict processing service overload

We have a black box that for each input request a, it outputs a computed response b. The computation time for a given request varies in a stable way over time. Stable means here that it is still ...
0
votes
0answers
10 views

Learning a multivariate polynomial with dependent coefficients

I have a polynomial of the form of $ K^2((a-i)^2 + (b-j)^2 + c^2) = (ct)^2$ where $a,b,c,t$ are unknowns. I have multiple observation points for the values of $i,j,K$. Can I use some technique to ...
0
votes
0answers
23 views

What method should I use for this optimization / feature selection project

I'm going to describe a problem and I'm not sure how to best solve it. I will describe the situation. When answering please recommend a method and maybe a software library. I'm using Python for my ...
0
votes
1answer
21 views

Expected error of best possible linear fit?

I asked the following question on stat SE, but I could not get a mathematically rigorous answer, and I have decided to ask here again. In my textbook, there is a statement mentioned on the topic of ...
0
votes
1answer
21 views

Is it a wrong expression for the local log-likelihood of logistic regression?

In page 206 of the book 'Elements of statistical learning', the author wrote: The local log-likelihood for this $J$ class model can be written $\sum_{i=1}^NK_\lambda (x_0, ...
1
vote
0answers
22 views

How does Linear Regression classification work?

I am currently trying to understand the following: Logistic regression is a probabilistic, linear classifier. It is parametrized by a weight matrix $W$ and a bias vector $b$. Classification is ...
1
vote
1answer
45 views

Essential mathematics for Image Processing

What are the most essential mathematical concepts one has to be familiar with for succeeding in the field of Image/Signal Processing and Machine Learning. I am somewhat familiar with Tensors, ...
0
votes
1answer
28 views

Maximum-Likelihood Estimator: What problems occur if data is not i.i.d.?

This is a question from an exam: You want to estimate the parameters for a gaussian distribution using the Maximum-Likelihood Method for an i.i.d. set of data. What role does the property ...
1
vote
0answers
18 views

Machine learning algorithm for relative similarity

I'm trying to find a good starting place (or existing algorithm) to determine the similarity of various items to one another based on subjective assessments of two items' relative similarity to a ...
1
vote
1answer
34 views

What connections between machine learning and dynamical systems?

I have a background of ("pure") dynamical systems and ergodic theory, but I am switching to machine learning. Can some machine learning questions be treated from a dynamical systems/ergodic theory ...
0
votes
0answers
9 views

Invalid Kernels with $a<0 \;\rm{and}\; b<0$

How can we prove for $a<0$ and $b<0$, $k(x,y)=(x^Ty+a)^b$ is not a valid kernel? For $b<0$, can we write $k(x,y)$ cannot be represented as an inner product?
0
votes
1answer
17 views

4 points not separable by SVM

We know in a support vector machine: Considering we have a linear feature mapping $\phi(x_n)=x_n$ and the XOR problem. We have 2 classes in $R^2$, class 1 $ t_+=+1$ and class 2 $t_-=-1$ and 4 ...
0
votes
1answer
31 views

What is the meaning of the semicolon in $h(x;\theta)$?

The context is machine learning, and the full expression is $h(x;\theta) = \operatorname{sign}(\theta_1 x_1 + \cdots + \theta_d x_d)$. $x$ is a feature vector and $\theta$ parameterizes a set of ...
0
votes
1answer
16 views

Why is the logistic function a special case of the sigmoid function?

I am reading the Wikipedia article about the logistic function used in logistic regression, but I don't understand the following A logistic function or logistic curve is a common special case of ...
1
vote
2answers
38 views

Curse of Dimensionality … as illustrated by Christopher Bishop

I'm reading Christopher Bishop's book "Neural Networks for Pattern Recognition". I'm on pg 7 about curse of dimensionality. Here is the relevant part: For simplicity assume the dimensionality we ...
0
votes
1answer
21 views

What is this distribution formulated with w, m and sum sign?

I have a binary classification problem, part of which is defined as follows : p(x|y=1) $\sim w (m_1 , \sum_1$) and p(x|y=0) $\sim w (m_0 , \sum_0$) Where $\sum_1$ is a covariance matrix : $$ ...
1
vote
1answer
32 views

Locally minimizing a concave function

What will happen if we minimize a concave function via gradient descent? Where does it get stuck? Intuitively a concave function has more structure than an arbitrary function, and seem to be easier ...
0
votes
0answers
13 views

Why do the two conditions listed in this paper imply the third?

I am working my way through this paper. On page 4, the author says: Why does A(λ', λ) have to be greater than 0? If the likelyhood with λ' is higher than the likelyhood with λ and A(λ', λ)<0 ...
0
votes
1answer
85 views

How do you minimize “hinge-loss”?

A lot of material on the web regarding Loss functions talk about "minimizing the Hinge Loss". However, nobody actually explains it, or at least gives some example. The best material I found is here ...