# Questions tagged [machine-learning]

How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

170 questions
Filter by
Sorted by
Tagged with
192k views

### Derivative of Softmax loss function

I am trying to wrap my head around back-propagation in a neural network with a Softmax classifier, which uses the Softmax function: $$p_j = \frac{e^{o_j}}{\sum_k e^{o_k}}$$...
• 2,157
105k views

• 1,747
106 views

### Is this implementation use HBOS mathematic?

I'm experimenting with an unsupervised statistical-based outlier detection so-called XBOS on top of the KMeans clustering algorithm. It is claimed that XBOS generates outlier scores as HBOS does. I'm ...
• 73
276 views

### Extension of binary classification to multi-class classification

Multi-class classification is a generalization of logistic regression wherein we are dealing with binary classification. The latter problem is a setting where a number should be mapped to either $0$ ...
1k views

• 121
111 views

### Averaged log-likelihood with a latent variable for mixture models

In class we've defined the following: $$Q(\theta; \theta^t) = \sum_z P(Z=z\mid X=x; \theta^t) \log P(X=x; Z=z;\theta)$$ It's part of the EM algorithm. Here, $\theta^t$ are the assumed parameters at ...
• 159
1k views

### Derivative of Softmax loss function (with temperature T)

I am try to calculate the derivative of cross-entropy, when the softmax layer has the temperature T. That is: $$p_j = \frac{e^{o_j/T}}{\sum_k e^{o_k/T}}$$ This question ...
• 23
1 vote
395 views

### What mathematics should I study to understand Neural Nets / Machine Learning?

I am strongly fascinated by neural nets, and perhaps other forms of machine learning. There are so many (potential) applications: teaching a robot with shaft encoders to drive along different ...
• 883
1 vote
484 views

### Mathematics disciplines underpinning Machine Learning

I have an undergrad degree in computational mathematics (though that was about 10 years ago), and spent my professional career in software development. If I wanted to understand what's happening ...
• 133
1 vote
1k views

### Expected squared prediction error conditioned on training set

I'm reading Elements of Statistical Learning by Hastie and Tibshirani, and I am thoroughly confused by the way they conditioned expected squared prediction error in section 2.5 (p.26): \begin{align*} ...
128 views

### How can I find the rank and spark relation here?

Hi, I can not get rank/spark relation correctly, I know that rank is a number of linearly independent columns of a matrix and spark is linearly dependent ones. In this question I understand option a), ...
22k views

### Why divide by $2m$

I'm taking a machine learning course. The professor has a model for linear regression. Where $h_\theta$ is the hypothesis (proposed model. linear regression, in this case), $J(\theta_1)$ is the cost ...
• 1,305
21k views

• 477
11k views

### The median distance from the origin to the closest data point and the curse of dimensionality

I'm reading The Elements of Statistical Learning. I have a question about the curse of dimensionality. In section 2.5, p.22: Consider $N$ data points uniformly distributed in a $p$-dimensional unit ...
• 493
13k views

### How to calculate Vapnik-Chervonenkis dimension

it's my first post here, so I apologize if I broke a rule! I'm reading Introduction to Machine Learning and got stuck on VC dimension. Here's a quote from the book: "...we see that an axis-aligned ...
• 325