Tagged Questions

How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

learn more… | top users | synonyms

0
votes
1answer
31 views

Drawing Probability Density Function

Can someone help me to draw this pdf? I really don't have an idea how to convert a function to pdf. Thank you p(x | c) = 1/3 for 1 <= x <= 4 and P(c) = 0.5
0
votes
0answers
9 views

Coefficients in Representer theorem

I have a Mercer Kernel, $K\colon X \times X \rightarrow \mathbb{R}$, i.e. it is continuous, symmetric and postive definite on a compact domain $X \subset \mathbb{R}^n$. Also, I have a set of $m$ ...
2
votes
1answer
65 views

Step by step LMS for learning a linear function

Disclaimer Since this is an exercise assignment I'm not looking for a complete solution but for help that enables me to solve it on my own The task Given the error function ...
0
votes
0answers
4 views

Preparing data for WEKA decision tree J48

I'm trying to deal with WEKA and J48 algorithm. Looks like I have to present all my numerical values like age, income, height, weight as classes: age_from_18_to_25, age_from_26_to_40, e.t.c. Here is ...
1
vote
0answers
19 views

What is the left derivative of the hinge loss function in the context of subgradients?

Let: $$|a|_+ = max\{0,a\}$$ Then the Hinge loss function (in the context of classification in Machine Learning) is: $$V(-yf(x)) = |1 - yf(x)|_+$$ Note that $y \in \{-1,1\}$ Let $f(x) = \langle w, ...
1
vote
1answer
50 views

books on the application of linear algebra on statistics/finance/machine learning

I am reading "linear algebra done right" by Axler and like it a lot. One thing though, in the end I would like to put these theory to use and as a math textbook it doesn't cover much application. ...
0
votes
1answer
26 views

Optimization options to select multiple items with different features and values

I'm trying to identify which approach would work best to select a set of elements that have different features that minimise a certain value. To be more specific, I might have a group of elements with ...
0
votes
0answers
15 views

How are parameters constrained in a kernel function?

(Kernel as in the kernel trick used in machine learning) Suppose you have the following kernel function: $$ k(x_m,x_m) = \theta_0*\exp\left\{ - \frac{\theta_1}{2}\|x_n - x_m \|^2\right\} + \theta_2 ...
0
votes
2answers
16 views

How to expand inner product square?

How does this $||x-x'||$ expand to the equation below? $\|x-x'\|^2 = (x^T)x + (x')^T x' - 2x^T x'$
0
votes
1answer
28 views

Understanding how to solve a Cost Function?

I'm having trouble seeing the relationship in the following equation. Let's assume $J(0,1)$ and $m=4$. First I figure out my hypothesis function ...
0
votes
0answers
10 views

Is it true that the ideal predictor that minimizes the logistic function and the exponential function are the same?

Is it true that the ideal predictor that minimizes the logistic function and the exponential function are the same? i.e. it true that (obviously assuming we know the probability distribution): ...
3
votes
4answers
146 views

Formal proof that mean minimize squared error function

On an important book of Machine Learning, I've found this proof. We want to minimize the cost function $J_0(X_0)$ defined by the formula $$J_0(x_0) = \sum_{k=1}^n \|x_0 - x_k \|^2.$$ The solution to ...
0
votes
1answer
9 views

Support of vector $w$ in graph sparsity

I'm reading about graph sparsity and I have one problem in a paper I'm reading I don't understand, maybe someone can clarify: Graph Sparsity: In graph sparsity, we have a directed acyclic graph ...
0
votes
1answer
13 views

Differentiable L-1 Regularization

In machine learning we are often faced with optimization problems where we want to minimize some energy function using L1 regularization over some of the parameters, e.g.: $$ E(a,w) = [\text{sum of ...
0
votes
1answer
46 views

how to calculate the marginal distribution of probabilistic principal component analysis

In the book Pattern recognition and machine learning from Bishop equation 12.33 states: $\mathbf{x} = \mathbf{W} \mathbf{z} + \boldsymbol\mu + \boldsymbol\epsilon$ Here $\mathbf{z}$ has a normal ...
0
votes
0answers
19 views

Simple confusion matrix question

Thanks in advance for the help. I have a set of data with n samples that I plan to use with knn to make some classifications. I want to test out the performance before applying it to my validation ...
1
vote
0answers
18 views

How do you show the connection of reproducing kernels to feature maps?

This question is in the context of Hilbert Reproducing Hilbert Spaces and reproducing Kernels and their relation to feature maps (and machine learning). We have a Hilbert space $\mathcal{F}$ and ...
0
votes
0answers
23 views

Conservative perceptron update rule - convex optimization

Suppose I have a condition on a perceptron update rule should be a little conservative. For example, it minimizes the distance between the new update and previous classifier $w_i$, i.e. $||w_{i+1} - ...
0
votes
1answer
28 views

Linear regression using gradient descent in Octave seems to fail

I was trying to implement linear regression with gradient descent using the equation presented on the machine learning course on Coursera: ...
1
vote
1answer
40 views

Help understanding machine learning cost function

I am taking an online class on Machine Learning and I'm trying to fully understand how the cost function work. Can someone explain to me exactly what is going on in the function below: Cost function ...
0
votes
0answers
14 views

Correlation vs Similarity

I want to know what is the difference between the correlation matrix and a similarity matrix, for e.g. if I have a matrix A consisting of items and I want to compute which items are similar to one ...
0
votes
0answers
9 views

Internet Traffic Predictor using Elman Neural Network

I want to build a Network Traffic Load Predictor, which predicts based on the device make/type(e.g. Android, iPhone, Windows Laptop, Macbook, Xbox, Playstation etc.), network bandwidth available and ...
1
vote
1answer
35 views

Linear Regression with independent but non-identical noise

If I have this linear regression equation: $$y=X\beta+\epsilon $$ ($x$ and $\beta$ are vectors) The likelihood function can be written as $$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi ...
1
vote
1answer
43 views

Derivative of Softmax loss function

I am trying to wrap my head around backpropagation in a neural network with a softmax classifier, which uses the softmax function: \begin{equation} p_j = \frac{e^o_j}{\sum_k e^{o_k}} \end{equation} ...
1
vote
0answers
29 views

Integrating an expression over a vector $\mathbf{w}$

doing my homework for a Machine Learning course, I have to calculate the following expression: $\newcommand{\IDENTITY}{\mathbf{I}} \newcommand{\W}{\mathbf{w}} \newcommand{\WT}{\mathbf{w}^T} ...
0
votes
0answers
8 views

Does any one know the relationship of the number of support vector and the data dimension in SVM?

Does any one know the relationship of the number of support vector and the number of data dimension in SVM? Is it possible that #support vector < #data dimension? If yes, for #support vector < ...
0
votes
0answers
11 views

compress data by linearization

i have a continuous data and i want to compress it by linearization(see picture). What algorithm can be used to minimize both number of lines and total compression error?
0
votes
1answer
30 views

Normalization of data in decision tree

After reading through a few references, I have come to know that for machine learning in general, it is necessary to normalize features so that no features are arbitrarily large ($centering$) and all ...
1
vote
0answers
76 views

Reducing a linear algebra expression to quadratic form

I am trying to solve the following exercise for my Machine Learning course. Expand this expression so that there are only quadratic terms: $(\mathbf{x} - \mathbf{\mu})^T \mathbf{\Sigma}^{-1} ...
0
votes
2answers
34 views

Identity regarding convexity of the logistic loss function

I found the following identity regarding the logistic loss function in these lecture notes (slide 16) from Berkeley university: $$\log(1 + e^{-z}) = \max_{0 \leq v \leq 1} -zv + v\log(v) + ...
0
votes
0answers
26 views

How to find a separating hyperplane?

I know about support vector machine, and it's quadratic programming approach which delivers the best separating hyperplane. My question is: is there a relatively simple algorithm to find a ...
0
votes
1answer
28 views

Given a sample of input/output data, predict new outputs

My problem is the following : I have a number of inputs with the corresponding deterministic outputs. There is no error on either input or output. The link between the two is completely unknown to me. ...
1
vote
0answers
36 views

Computations for LDA: Eigendecomposition

While reading the book Elements of Statistical Learning p. 113, the author used eigendecomposition of the covariance matrix $\hat{\Sigma}_k =\mathbf{U}_k\mathbf{D}_k\mathbf{U}_k^T$ where ...
0
votes
2answers
44 views

Exposition of solving the quadratic programming problem for SVMs

I'm looking to find a mathematically rigorous exposition on how to solve the quadratic programming problem $$\min ||x||^2 \textrm{ subject to } Ax\leq b$$ where $x\in\mathbb{R}^n$, ...
0
votes
1answer
46 views

Deriving equation in vector notation

I had some trouble deriving an equation from the book 'Elements of statistical Learning' p. 108 equation 4.9. This heavily relies on linear algebra, so I was wondering how the author came to his final ...
0
votes
0answers
37 views

Help needed for K-nearest neighbor distance metric and distance weight formulas.

I am currently working on K-nearest neighbor algorithm with Distance Weighting. I have read the Distance Metric and the Distance Weighting parts. I want to fully understand how the Distance metric and ...
0
votes
0answers
5 views

Are there other names for multilayer perceptrons or multidimensional interpolants based on Kolmogorov's approximation work?

Are there other names for multilayer perceptrons that are used outside of the neural net community? At its core, multilayer perceptrons form a multidimensional interpolant of the form $$ ...
0
votes
1answer
26 views

Mathematics disciplines underpinning Machine Learning

I have an undergrad degree in computational mathematics (though that was about 10 years ago), and spent my professional career in software development. If I wanted to understand what's happening ...
0
votes
0answers
29 views

Expectation Maximization Algorithm for Gaussian Mixture Model

Can we use the Expectation Maximization algorithm for estimation of Gaussian Mixture Model with full covariance matrices? If yes then can you please give me a reference paper? So far all the machine ...
0
votes
1answer
43 views

How would I use derivatives for suggesting an option to my user?

I was learning derivatives. I understood the theoretical concept behind it. When I was searching for the real-life example in machine learning I came across one of the answers in this question How do ...
0
votes
1answer
23 views

Question on understanding Dirichlet process

I have questions on understanding this article about Dirichlet process. If you look at the beginning of section 2.1, it shows three equations 2.1, 2.2, 2.3. The question is I don't understand what ...
5
votes
0answers
57 views

Careers in applied math with an MS other than in finance and data/machine learning?

Since I like math, I would like a career that uses alot of applied math. I'm about to complete my Master's and could do my thesis in numerical solutions of PDEs I'm already aware of careers such as ...
2
votes
0answers
48 views

notation for minimum and maximum?

I'm trying to figure out the correct notation for this situation for use in Machine Learning. I have various ratings (for texts): ...
1
vote
2answers
47 views

Why we consider log likelihood instead of Likelihood in Gaussian Distribution

I am reading Gaussian Distribution from a machine learning book. It states that - We shall determine values for the unknown parameters mu and sigma^2 in the gaussian by maximizing the ...
0
votes
1answer
72 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
0
votes
1answer
121 views

Gradient descent (with line search) for convex functions viewed as alternation

I have fundamental confusion about gradient descent (with line search) and the reason it works. I try to explain my view here, and please tell me where it goes wrong. Let $f: \mathbb{R}^n \to ...
2
votes
0answers
65 views

Numerically approximate the maximum of an element of a vector after a series of matrix multiplications.

Where S is a sigmoidal function, A_i is a matrix, and x is an input vector, and ...
1
vote
0answers
26 views

Baseline predictors model

I implemented baseline predictors model (like it is told in Recommender systems handbook pp 148-149). b_ui = mu + b_i + b_u where mu is overall average rating ...
0
votes
2answers
39 views

Creating a polynomial function with no x-intercept

I have an understanding of polynomials and how to create a function based on the leading coefficient, degrees, x-intercepts, etc. My question is how do i go about creating a polynomial function that ...
1
vote
0answers
32 views

Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...