1
vote
1answer
29 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
0
votes
0answers
35 views

How to take partial derivative of summation?

This is about L$^1$ iteratively reweighted least squares. Where did the (-a$_i$$_k$) come from?
0
votes
0answers
60 views

Can't find gradient for MLE for mult-class logistic regression

$$P(k | x_i;w)= \frac{exp(w_k^tx_i)}{\sum_{j=1}^K exp(w_k^tx_i)}$$ $y_i^k$ is a vector that uses 1-of-k encoding. Thus, if $y_i=k$, then the vector $y_i$ has a 1 in the kth spot and a 0 everywhere ...
0
votes
0answers
20 views

Directional Derivative of a function containing an Indicator function.

I'm trying to understand a passage in Koenker's Quantile regression book (p.33). It says: (note that y,x, are vectors and w is the direction vector) With the first part of the outcome no problem: ...
0
votes
1answer
79 views

Jacobian in Levenberg-Marquardt for 4-Parameter equation

I am trying to fully understand how I can use Levenberg-Marquardt to minimise a 4 parameter equation. There are lots of fancy programs to do this but the documentation about the mathematics is ...
0
votes
1answer
209 views

Question about derivatives, Jacobian Matrix and non-linear least square curve fitting

I am working with a software package that performs non-linear least squares curve fitting. The goal of curve fitting is to calculate the coefficients of an equation given x,y points. The software ...
0
votes
0answers
76 views

Why Local Minimum is calculated for a derivative function instead of actual function?

In Machine learning regression problem, why the local minimum is computed for a derivative function instead of the actual function? Example: http://en.wikipedia.org/wiki/Gradient_descent The ...
0
votes
1answer
80 views

Logistic regression with weighted instances

I'm working on implementing a logistic regression algorithm in code. It's based this link. Unfortunately, the paper doesn't talk about weighting the individual examples $x_{i}$. I think the relevant ...
1
vote
1answer
56 views

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method?

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method? Link: http://www.youtube.com/watch?v=YwZYSTQs-Hk Thanks in advance!
1
vote
2answers
687 views

Log-likelihood gradient and Hessian

Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions: $f(x) = x^T \beta$ $p(x) = ...