# Tagged Questions

26 views

### Normalization of data in decision tree

After reading through a few references, I have come to know that for machine learning in general, it is necessary to normalize features so that no features are arbitrarily large ($centering$) and all ...
44 views

### Deriving equation in vector notation

I had some trouble deriving an equation from the book 'Elements of statistical Learning' p. 108 equation 4.9. This heavily relies on linear algebra, so I was wondering how the author came to his final ...
20 views

### Expectation Maximization Algorithm for Gaussian Mixture Model

Can we use the Expectation Maximization algorithm for estimation of Gaussian Mixture Model with full covariance matrices? If yes then can you please give me a reference paper? So far all the machine ...
43 views

### Why we consider log likelihood instead of Likelihood in Gaussian Distribution

I am reading Gaussian Distribution from a machine learning book. It states that - We shall determine values for the unknown parameters mu and sigma^2 in the gaussian by maximizing the ...
66 views

### Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
38 views

### Creating a polynomial function with no x-intercept

I have an understanding of polynomials and how to create a function based on the leading coefficient, degrees, x-intercepts, etc. My question is how do i go about creating a polynomial function that ...
23 views

### Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...
15 views

### Sparse data and covariance matrix computation

Background I am trying to apply Gaussian Discriminant Analysis (GDA) on the MNIST dataset of hand-written digits, with 10 classes for 10 digits. In this dataset, each point is a vector of 784 ...
40 views

### Math formulas on Clustering

I am currently studying Clustering in Machine Learning. I have found a document regarding guessing the right number of clusters. I am reading the first part of it, having difficulties in understanding ...
28 views

### How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
21 views

### feature selection for continuous variables

I wonder how exactly "feature selection" should be performed in case of continuous feature values. When feature values are discrete it is very straitforward to apply feature selection, but what to do ...
23 views

### Understand the English paragraph on association rule.

I am currently studying Association Rule Pattern Mining. I am reading the explanation on wikipedia about it. Somehow, I feel like I have a problem in understanding the paragraph below. Can somebody ...
18 views

### How to evaluate the difference between two classes of data which are highly overlapped

I’m trying to implement a classifier based on a dataset comprising two classes of high dimensional time-series observations (the values of the two classes of observations are highly similar). I ...
35 views

9 views

### Invalid Kernels with $a<0 \;\rm{and}\; b<0$

How can we prove for $a<0$ and $b<0$, $k(x,y)=(x^Ty+a)^b$ is not a valid kernel? For $b<0$, can we write $k(x,y)$ cannot be represented as an inner product?
39 views

### Curse of Dimensionality … as illustrated by Christopher Bishop

I'm reading Christopher Bishop's book "Neural Networks for Pattern Recognition". I'm on pg 7 about curse of dimensionality. Here is the relevant part: For simplicity assume the dimensionality we ...
12 views

### Is Expectation Propagation (EP) affected by the prior?

I understand EP by reading Minka's thesis: http://research.microsoft.com/en-us/um/people/minka/papers/ep/minka-ep-uai.pdf I'm trying to apply it to solve a Bayesian inference problem. However, I'm ...
23 views

### Deriving the optimal value for the intercept term in SVM

I was reading andrew ng's machine learning lecture notes on SVM. I came across the following equation (finding the optimal value for the intercept term $b$ in the SVM problem): However, I have no ...
31 views

### Normalizing multiple different features from unknown distributions

I'm doing some "exploratory" data analysis over a large set of classes/proteins, with a few hundred different features (I.E. Continuous variables) extracted from the data. The features are calculated ...
19 views

### What is the rationale behind ROC curves?

I am not sure how ROC curves work. I see that the X-Axis is the false positive rate while the Y axis is the true positive rate. 1) I don't understand how for a given statistical learning model, you ...
36 views

### Given $N$ coins, find a coin with minimal bias based on $N$ samples

General description: Given $N$ coins $Z_1,...,Z_N$ (Bernoulli RVs), where the $i$-th coin has probability $p_i$ for "Head", I'm trying to find $\min\limits _{i\in[N]}p_{i}$. I'm interested in a ...
27 views

116 views

### expectation of norm of orthogonal projector

The question has to do with calculating the expected squared norm of a random projection. We have a 2D subspace $T := span\{U1, U2\}$ where $U1$ is a random vector uniformly distributed over unit ...
33 views

### Finding Kernel function for the data set

For a set of data points how to find an appropriate kernel function to map it to higher dimension so that it will be linearly seperable
42 views

### Testing using training data

I've been trying to prove that estimates of a classifier's performance using training data is a bad thing. Does "bad" mean it is biased? This is part of a larger proof. If somebody knows of previous ...
105 views

### Building Bayesian Networks, Causality and Cyclic Reasoning

I am studying Bayesian Statistics and I am trying to get a good understanding on Bayesian Networks, which seems to be vital in order to make something useful in Machine Learning. Most of the texts I ...
109 views

### L2-Regularized\Penalized Logistic Regression

Suppose you have an $n$ dimensional data vector $x = (x_1, \ldots, x_n)$ and two classes $y = 0$ or $y = 1$. Assuming the dimensions of $x$ are conditionally independent given $y$, and that the ...
57 views

### How to compare training and test errors in statistics?

I have a data set and I need to compare the performance of various statistical models: Least Squares, LASSO, Ridge Regression, to name a few of the key ones. What are standard techniques for ...
211 views

### Comparing k nearest neighbors (knn) and least squares bias and variance

I'm reading the textbook The Elements of Statistical Learning. In section 2.3.3, it says "The linear decision boundary from least squares is very smooth, and apparently stable to fit. [...] In ...
63 views

### K Nearest Neighbors classification Special Case with Identical Points

The question is about KNN algorithm for classification - the class labels of training samples are discrete. Suppose that the training set has n points that are ...
2k views

### derivative of cost function for Logistic Regression

I am going over the lectures on Machine Learning at Coursera. I am struggling with the following. How can the partial derivative of ...
69 views

### online learning to maximize profit

I have a software which takes input as investment and gives the output as return on a particular stock. Now profit metric $x_i$ is defined as the ratio of return $g_i$ to maximum possible return ...
$C_m$: m = most likely class (wanted to write C subscript MAP for "maximum a posteriori" but couldn't do MAP with MathJax) ...
Consider two multidimensional random vectors $x$ and $z$ having Gaussian distributions $P(x)=N(x\mid\mu_x,\Sigma_x)$ and $P(z)=N(z\mid\mu_z,\Sigma_z)$, respectively, together with their sum $y=x+z$. ...