# Questions tagged [machine-learning]

How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

170 questions
Filter by
Sorted by
Tagged with
192k views

### Derivative of Softmax loss function

I am trying to wrap my head around back-propagation in a neural network with a Softmax classifier, which uses the Softmax function: \begin{equation} p_j = \frac{e^{o_j}}{\sum_k e^{o_k}} \end{equation}...
105k views

106 views

### Is this implementation use HBOS mathematic?

I'm experimenting with an unsupervised statistical-based outlier detection so-called XBOS on top of the KMeans clustering algorithm. It is claimed that XBOS generates outlier scores as HBOS does. I'm ...
276 views

### Extension of binary classification to multi-class classification

Multi-class classification is a generalization of logistic regression wherein we are dealing with binary classification. The latter problem is a setting where a number should be mapped to either $0$ ... 1k views

111 views

### Averaged log-likelihood with a latent variable for mixture models

In class we've defined the following: $$Q(\theta; \theta^t) = \sum_z P(Z=z\mid X=x; \theta^t) \log P(X=x; Z=z;\theta)$$ It's part of the EM algorithm. Here, $\theta^t$ are the assumed parameters at ...
1k views

### Derivative of Softmax loss function (with temperature T)

I am try to calculate the derivative of cross-entropy, when the softmax layer has the temperature T. That is: \begin{equation} p_j = \frac{e^{o_j/T}}{\sum_k e^{o_k/T}} \end{equation} This question ...
1 vote
395 views

### What mathematics should I study to understand Neural Nets / Machine Learning?

I am strongly fascinated by neural nets, and perhaps other forms of machine learning. There are so many (potential) applications: teaching a robot with shaft encoders to drive along different ...
1 vote
484 views

### Mathematics disciplines underpinning Machine Learning

I have an undergrad degree in computational mathematics (though that was about 10 years ago), and spent my professional career in software development. If I wanted to understand what's happening ...
1 vote
1k views

### Expected squared prediction error conditioned on training set

I'm reading Elements of Statistical Learning by Hastie and Tibshirani, and I am thoroughly confused by the way they conditioned expected squared prediction error in section 2.5 (p.26): \begin{align*} ...
128 views

### How can I find the rank and spark relation here?

Hi, I can not get rank/spark relation correctly, I know that rank is a number of linearly independent columns of a matrix and spark is linearly dependent ones. In this question I understand option a), ...
22k views

### Why divide by $2m$

I'm taking a machine learning course. The professor has a model for linear regression. Where $h_\theta$ is the hypothesis (proposed model. linear regression, in this case), $J(\theta_1)$ is the cost ...
21k views

11k views

### The median distance from the origin to the closest data point and the curse of dimensionality

I'm reading The Elements of Statistical Learning. I have a question about the curse of dimensionality. In section 2.5, p.22: Consider $N$ data points uniformly distributed in a $p$-dimensional unit ...