# Tagged Questions

How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

3 views

### effect of number of samples on performance of unsupervised clustering algorithms

I want to apply a k means classification algorithm on a set of data. How does the performance of the clustering algorithm depend on the number of samples available to it? Specifically, instead of ...
22 views

### A heuristic explanation of the Curse of Dimensionality

From Principles and Theory for Data Mining and Machine Learning, Clarke et al. (2009): This phrase [the "Curse of Dimensionality"] was first used by Bellman (1961)... The result is that ...
55 views

### Implementing gradient descent based on formula

The gradient descent algorithm is given as : repeat { $$\displaystyle \theta_j := \theta_j - \frac{1}{m} \alpha \sum_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)}) x^{(i)}_j$$ } Given these values : <...
7 views

### Combining datasets based on their prior distributions in classification

Given $k$ datasets $D_1,...,D_k$, each dataset consists of a a collection of features $X_i$ and the corresponding labels $y_i$. The seperation of datasets is based on a preprocessing of the data. ...
13 views

### Problem OOB in random forest

I have a 6 levels of one group and i have to do a random forest classification. My problem is that OOB in test set is too low and cv give me a almost 0 error so i don't understand if this can be a big ...
76 views

### What is a good book for math students to learn machine learning in depth?

I am a math master student and have done fundamental math courses like probability theory, measure theory, linear algebra and know a little bit about functional analysis. What is good way for me to ...
26 views

### Number of neighbors as a function of dimension

I apologize in advance for perhaps an imprecise formulation of the question. If I have a point in 1D, it has precisely 2 nearest neighbors independent of choices. In 2D, if I allow arbitrary ...
49 views

### Mathematical notation for neural network

there are so many "styles" to express neural net in mathematical notation, for example Michael Nielson defines $w_{jk}$ as weight from $k$-th neuron to $j$-th, Andrew Ng defines it otherwise, some ...
26 views

### Does this calculus argument involving rank-4 tensors make sense?

Edit: Completely rewritten to be shorter and easier to digest. Background and the Actual Question: I'm trying to derive a gradient formula (back propagation) for a machine learning application. The ...
29 views

### Convergence sequence of mean implies convergence in mean / weakly consistence of subsequence of regression function estimates

Let $(X_n)$ be a sequence of positive random variables. Suppose that the limit of expectation of this sequence $\lim_{n\rightarrow\infty}\mathbb{E}[X_n]=0$. This imply that $(X_n)$ converges to zero ...
18 views

### Alternatives to Kruskal-Wallis or one way Anova test for small size samples

I have a group of measurements 'grouped by year' and on each year I have only one recorded measure, such as the example below: ...
14 views

### KKT condition of linearly inseparable Support Vector Machine (SVM)

In the paper Sequential Minimal Optimization:A Fast Algorithm for Training Support Vector Machines, the optimization problem for linearly inseparable SVM is \begin{align} \min\limits_{\boldsymbol{w},...
25 views

### Softmax Derivation Help

I've been reading a paper that derives logistic regression from a few assumptions . Here is the link. If you go to page 5 and look at equation 18 the author claims that this essentially says the ...
14 views

### What is KL-Divergence? Why Do I need it? How do I use it?

I am currently studying KL Divergence. But It seems very confusing that I don't maybe understand why do I ever need it and what is that for? As I have been reading stuff about Mutual Information, it ...
40 views

### The derivative of the absolute value |x| [duplicate]

I read about the derivative of the absolute value |x|, but why the absolute value is not differentiable at point zero, and when it becomes 1 or -1 {geometrically}? Thanks
17 views

### Solving $I^* = \arg\min_{I'} \left( \|\phi_\ell(I) - \phi_\ell(I')\|_2^2 + R(I') \right)$ with gradient descent

I am trying to create the results from this a paper that is trying to understand the types of features a convolutional neural network is learning to recognize. I don't think understanding ...
78 views

### Does any technical definition of embedding accept a “non-injective” function as opposed to only “injective”?

Embedding is defined to be a one-to-one structure preserving mapping. My question is if the one-to-one condition is really critical. Like if linear mappings from high-dimensional space to low-...
24 views

### Roadmap to Differential Geometry for Machine Learning

Recently within machine learning, there are a lot of works on non-convex optimization and natural gradients methods etc which are based on differential geometry, it gives rise to increased need to ...
12 views

### Precision-Recall Graph: F1 Score v.s. Break-Even Point

To evaluate two classifiers from the aspects of Precision-Recall, two measures are often used: F1 score and Break Even Point (BEP for short. I failed to find any document about it from wiki, and it is ...
21 views

### formulate the nearest-neighbour classifier for a general nonlinear kernel

I have an input vector x and the nearest input vector $x_n$ from the training set. The distance is defined as $||x-x_n||^2$. How can I express it in terms of scalar products and then make use of ...
16 views

### Is the EM-algorithm the same thing that variational inference in LDA?

I am new in the probabilistic topic modeling, and I need to understand deeply the LDA process, I understand what want to do the inference process in LDA, and I understand too that there is 2 "types" ...
16 views

### Gradient descent rule for a particular matrix in a parse tree RNN

Consider the following structure of a recursive neural network. My input is a parse tree, which is a sentence parsed into a binary tree such that an entry is a leaf if and only if it is a word, else ...
24 views

### Support Vector Machines: Hype or Hallelujah? - what is alfa? [closed]

I at the moment trying to understand how SVM works with the help of this paper The paper itself explains things pretty well, but there is an alfa term, which doesn't seem to be documented anywhere? ...
37 views

### Undestanding SVM

I am the moment trying to understand how SVM works.. I understand the concept of finding a seperating hyperplane with the highest margin, but i do not understand how it works in mathmatically. Mor ...
47 views

### Gaussian process for machine learnig

Here is my question in the equation 2.11 A is N by N matrix, so there is not feasible if N is large the textbook say in the euqation 2.12, we only need to invert size n by n. But I think $K$ is 1 ...
33 views