0
votes
1answer
21 views

Given a sample of input/output data, predict new outputs

My problem is the following : I have a number of inputs with the corresponding deterministic outputs. There is no error on either input or output. The link between the two is completely unknown to me. ...
0
votes
1answer
62 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
1
vote
0answers
22 views

Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...
0
votes
2answers
34 views

Why divide by $2m$

I'm taking a machine learning course. The professor has a model for linear regression. Where $h_\theta$ is the hypothesis (proposed model. linear regression, in this case), $J(\theta_1)$ is the cost ...
1
vote
0answers
24 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
0
votes
0answers
28 views

Expected in-sample error of linear regression with respect to a dataset D

In my textbook, there is a statement mentioned on the topic of linear regression/machine learning, and a question, which is simply quoted as, Consider a noisy target, $ y = (w^{*})^T \textbf{x} + ...
0
votes
1answer
22 views

Expected error of best possible linear fit?

I asked the following question on stat SE, but I could not get a mathematically rigorous answer, and I have decided to ask here again. In my textbook, there is a statement mentioned on the topic of ...
0
votes
1answer
22 views

Is it a wrong expression for the local log-likelihood of logistic regression?

In page 206 of the book 'Elements of statistical learning', the author wrote: The local log-likelihood for this $J$ class model can be written $\sum_{i=1}^NK_\lambda (x_0, ...
1
vote
0answers
34 views

How does Linear Regression classification work?

I am currently trying to understand the following: Logistic regression is a probabilistic, linear classifier. It is parametrized by a weight matrix $W$ and a bias vector $b$. Classification is ...
2
votes
1answer
40 views

How can I plot this?

Given a bunch of data $x_i$ , $y_i$, how do I plot $$f(\theta_2,\theta_2)= \frac{1}{2M} \sum_{i=1}^{M} (\theta_1\cdot x_i -\theta_2 y_i)^2$$ in matlab? I know it should be parabolic, but my code ...
2
votes
1answer
120 views

Understanding Regularization parameters in Machine Learning/Statistics

Suppose I have the following $k$ degree polynomial regression model with a data set of size $n$ which includes a $k$-dimensional feature vector $x$ and an outcome denoted $t_i$ for each vector in the ...
2
votes
1answer
238 views

What is the difference between Curve Fitting and Regression(Machine Learning)?

I know that Machine Learning regression algorithms try to find the function of the data. That is, if we have 1000 data points (x,y), to find a general continuous function that follows the trends of ...
1
vote
1answer
35 views

Combine linear models of different sets of data.

I'm working on a large data set D that can be partitioned into some disjoint subsets D1, D2, ..., Dn. For each subset Di, I have a linear model Mi that minimizes the residual error for data in Di. ...
2
votes
2answers
37 views

Regression model when under-estimations costs us more than over-estimations

We have a factory and we are planning how many items produce in 2014. During the learning process we minimize the mean squared error. But, under-estimations costs us more than over-estimations. Let's ...
1
vote
2answers
45 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
0
votes
1answer
70 views

Is a traditional Multi Layer Perceptron Network capable of non-linear regression? Which activation function should be used for that purpose?

I need to use a Multi Layer Perceptron Network in order to perform some non-linear regression. Any ideas if it's possible to perform a task like that and how? Which activation function should be used ...
0
votes
0answers
135 views

Which machine learning algorithm to use?!

I have a training set which is set of essays written by students for a question. These essays are all scored by human evaluators with labels such as 1, 2 , 3 which is actually marks allotted for those ...
0
votes
1answer
125 views

Show posterior probability takes the form of the logistic function

Suppose you have a D-dimensional data vector $x$ = ($x_1$, ..., $x_n$) and associated class variable $y \in \{0, 1\}$, which is Bernoulli with parameter $\alpha$. Assume the dimensions of $x$ are ...
7
votes
2answers
2k views

derivative of cost function for Logistic Regression

I am going over the lectures on Machine Learning at Coursera. I am struggling with the following. How can the partial derivative of ...
1
vote
1answer
144 views

Machine Learning, why not use matrix multiplication instead of gradient descent?

If we want to minimize our Cost function for a given set of data, why do we use gradient descent and continually guess values until we find a min value for theta when when can just use matrix ...
0
votes
0answers
76 views

Why Local Minimum is calculated for a derivative function instead of actual function?

In Machine learning regression problem, why the local minimum is computed for a derivative function instead of the actual function? Example: http://en.wikipedia.org/wiki/Gradient_descent The ...
1
vote
0answers
46 views

Gaussian prior from feature to input space

if I have Gaussian prior ($\exp\left(\dfrac{-\sum_i w_i}{2\gamma^2}\right)$) on my weights in a linear classifier, how can I transform this so I can apply it for my kernel parameters $\alpha$? I have ...
8
votes
3answers
224 views

Minimize $||Ax-b||$ but for $A$, not $x$

I have a machine learning regression problem. I need to minimize $$ \sum_i||Ax_i-b_i||_2^2 $$ However I am trying to find matrix $A$, not the usual $x$, and I have lots of example data for $x_i$ and ...
0
votes
2answers
198 views

Why Logistic Regression for Classification Problems?

In a class on machine learning, we covered classification problems. In such a problem, you are studying a property of some object, say malignity of tumors in a patient. You are first given a training ...
2
votes
1answer
231 views

An intuitive explanation for neural networks as function approximators ?

We use normal linear regression for modelling functions on datasets . But Can someone explain how neural networks help in approximating more complex ,especially non-linear functions ? intuitively , ...
10
votes
6answers
12k views

regression vs classification

THis is more machine learning questions, but perhaps someone will be able to help. I would like to know what is the diference between regression and classification when we try to generate output for a ...
1
vote
2answers
366 views

Simple Least Squares Regression?

I have a vector X of 50 real numbers and a vector Y of 50 real numbers. I want to model them as y = ax + b How do I determine a and b such that it minimizes the ...
0
votes
1answer
84 views

Regression on Linear Model?

I have 50 or so training examples involving a set of 200 or so real numbers (x1,x2,...,x200) (normalized to a 0 mean and std dev 1), and a single output real (y) in the range 0.0..1.0. I want to fit ...
1
vote
1answer
130 views

can an artificial neural network with only one hidden layer fit all purposes/applications/functions?

I have heard that only a single layer is needed for an ANN to fit any possible function (input to output). Is this true and where is this investigated/state/found? Then what is the advantage of having ...
0
votes
1answer
197 views

dependency between regression coefficients and probability distribution

let's consider regression problem. Given set of training data $\{(x_i,y_i)\}_{i=1}^N$, $x_i \in \mathbb{R}^n$ and $y_i \in \mathbb{R}$, find prediction function $y = f(x)$, e.g. in RBF regression case ...
5
votes
2answers
284 views

Theoretical basis for overfitting

There are many examples in which making more "precise" predictions gives worse performance (e.g. Runge's phenomenon). My professor implied that there was a sound basis for choosing "simple" functions ...