Questions tagged [data-mining]

This tag is for questions about data mining, which is an interdisciplinary subfield of computer science. It is the computational process of discovering patterns in large data sets.

Filter by
Sorted by
Tagged with
7
votes
2answers
102 views

How do I find the formula (or rules) that created a list of numbers with seemingly no pattern?

Newbie here, and I apologize if this is the wrong forum for this type of question... I have a group of 200 or so alphanumeric codes from an unknown source. Here's an example piece of the data set: <...
5
votes
1answer
6k views

Normalization of data in decision tree

After reading through a few references, I have come to know that for machine learning in general, it is necessary to normalize features so that no features are arbitrarily large ($centering$) and all ...
4
votes
1answer
5k views

Mutual Information for clustering

I'm working on a document clustering application and decided to use Normalized Mutual Information as one of the measures of effectivenes. But I don't really understand how to implement this in that ...
4
votes
1answer
66 views

How to find almost periodic lattices in a set of high-dimensional points?

sorry for lame question, but I just have no maturity in this direction. Let's say I have very large set ( millions ) of high-dimensional vectors ( typical dimensionality is 64). These vectors ...
3
votes
1answer
516 views

What is KL-Divergence? Why Do I need it? How do I use it?

I am currently studying KL Divergence. But It seems very confusing that I don't maybe understand why do I ever need it and what is that for? As I have been reading stuff about Mutual Information, it ...
3
votes
2answers
1k views

Distance between unequal-dimension vectors (and data)?

It is easy to find simple distance measures for equal-dimension vectors, such as Euclidean Distance or Correlation. What about unequal-dimension vectors, such as, for instance, $(a,b,c)$ and $(d,e)$? ...
3
votes
1answer
2k views

How can one prove that mahalanobis distance is a metric?

How can one prove that mahalanobis distance is a metric? How can one show that these four properties of a metric are valid for mahalanobis distance? 1) d(x, y) ≥ 0 (non-negativity, or separation ...
3
votes
0answers
46 views

Number of rules (Data Mining)

This is a paraphrasing of a statement from Data Mining by Witten et al., 4th edition, on section 1.6. Consider the set $S = \{1, \dots, 288\}$. A rule is defined as a one-element singleton set ...
3
votes
0answers
236 views

Does there exist closed-form solutions for calculations in the EM-algorithm for Gaussian Mixtures?

Our data can be represented as $X_{n\times d}$ matrix, where we have $n$ data points lying in $\mathbb{R}^d$. We assume that there are $k$ underlying Gaussian models, $\mathcal{N}_d(\mu_j, \Sigma_j)$, ...
3
votes
1answer
37 views

Clustering elements according to covariance matrix

I'm doing a little bit of topic modelling (which is not really my area) with twitter tweets. The situation is the following: I have a (sort of) covariance matrix where the entrie $C_{ij}$ corresponds ...
3
votes
0answers
376 views

Gini coefficient vs Gini impurity - decision trees

The problem refers to decision trees building. According to Wikipedia 'Gini coefficient' should not be confused with 'Gini impurity'. However both measures can be used when building a decision tree - ...
3
votes
2answers
180 views

Are A and B conditionally independent

Are A and B conditionally independent given the class label? I calculated that $$P(A=1) = \frac{1}{2}$$ $$P(B=1) = \frac{2}{5}$$ $$P(A=1,B=1)=\frac{1}{5}$$ My answer is yes. I do it by anding $(A\...
2
votes
1answer
2k views

why use a small learning rate in gradient descent

I am new to neural networks and recently found out about gradient descent. Something does not sit right with me. x←x−λ∇fk(x) Why does this formula work? Wouldn'...
2
votes
1answer
21 views

Can i use the coefficients of a trained model of Logistic Regression as a result itself without using the model on unseen data?

I'm trying to figure out if i can use a logistic regression as a predictive model, to estimate the probability of response of a user in CRM by having the predictors and i also have the class (...
2
votes
1answer
186 views

Probability and Statistics Books for Distributions and Introduction to Data Mining/Machine Learning

In college, I took a probability class using Sheldon Ross' A First Course in Probability. It was not my best semester to say the least. However, I am returning back to probability and statistics as it ...
2
votes
1answer
891 views

Naive Bayes Classification Example

Given the following data: $$\begin{array}{c|c|c|c|c|} \text{Instance} & \text{A} & \text{B} &\text{C} &\text{Class} \\ \hline \text{1} & 1 & 2 & 1 & 1 \\ \hline \text{...
2
votes
2answers
87 views

Are mathematicians discovering algorithms by mining them with computers? [closed]

Throughout history many famous equations have been intuitively derived by famous mathematicians - Einstein's discovery of mass–energy equivalence, Schrödinger's equation for the orbitals of electrons, ...
2
votes
2answers
97 views

Plain English interpretation needed for the sentence to understand EM-algorithm?

I am trying to read an EM-algorithm article on the web, however, as soon as I started I have face a sentence interpretation problem with this like "... in the presence of missing or hidden data" in ...
2
votes
3answers
148 views

Why can we use entropy to measure the quality of a language model?

I am reading the < Foundations of Statistical Natural Language Processing >. It has the following statement about the relationship between information entropy and language model: ...The ...
2
votes
2answers
79 views

How could we know the Dirichlet allocation is describing the topic rather than something else?

Dirichlet distribution is widely used in document modelling and document clustering. I tried to understand its rational. I read from this article that: Different Dirichlet distributions can be ...
2
votes
1answer
290 views

Creating a lift chart for a classification tree

This is likely a simple question but I'm new to data mining techniques and am trying to compare two different predictive models. I've created a logistic regression and a classification tree and would ...
2
votes
1answer
28 views

exercise 1.3 from Mining of Massive Data Sets book

Hello there is a question given in Mining of Massive Data Sets book http://infolab.stanford.edu/~ullman/mmds/ch1.pdf it is on page 15 exercise 1.3.2 My solution is following: as there are $10$ ...
2
votes
0answers
36 views

Calculate PageRank for small web

Calculate PageRank for: A links to B, B links to C and C links to B and C where the damping factor $\beta=0.8$ I have: $M=\begin{bmatrix} 0&0&\frac{1}{2} \\ 1&0&\frac{1}{2} \\ 0&...
2
votes
0answers
462 views

Spectral relaxation of k-means clustering

I am working on a presentation on Spectral relaxation of k-means clustering (http://papers.nips.cc/paper/1992-spectral-relaxation-for-k-means-clustering.pdf) and I am a bit stuck. I understand ...
2
votes
0answers
309 views

Optimization of English Braille: Using the fewest dots

Background: The English Braille system is laid out in such a way so that the letters can be referenced by their position in the alphabet. Of the six dots available for each character, the top four ...
2
votes
0answers
58 views

How can I infer order from partially ordered discrete sequences?

A really interesting problem that I can't stop thinking about! Have run in to this a couple of times but yet to find a smart approach to either solve or frame this problem. This is my try at ...
2
votes
0answers
775 views

How do I measure the goodness of cosine similarity scores across different vector spaces?

I am a computer scientist working on a problem that requires some statistical measures, though (not being very well versed in statistics) I am not quite sure what statistics to use. Overview: I ...
1
vote
2answers
74 views

How can I interpret the image for prior probability?

The image included below is about Bayesian statistics. While looking at the lecture, the lecturer expressed the probability distribution of prior probability as a uniform distribution. Somehow, I feel ...
1
vote
1answer
48 views

Why/when is there $\frac{1}{2}$ in front of Least Squares Estimator?

So in PCA I encountered a formulation for LSE, which is: $$\frac{1}{2} \sum_{i=1}^N ||x_i - \tilde{x_i}||^2$$ Where $\tilde{x}$ is a "restriction" of $x$ such that only parts of the observations are ...
1
vote
2answers
161 views

F-measure of a binary classifier

Prove that the F-measure of any binary classifier is $\leq\dfrac{precision+recall}{2}$ Let $P=precision$ and $R=recall$ I have that the F measure $=\dfrac{2PR}{P+R}$ Note that $precision=\dfrac{tp}...
1
vote
2answers
65 views

Are there $a,b,c,d \in \mathbb N$ such that $\frac{a + b}{a + b + c + d} < \frac{a}{a + c} < \frac{b}{b + d}$?

Consider the following $2 \times 2$ contingency table: \begin{array}{c|cc|c} & C & \overline C & \Sigma \\ \hline V & 4000 & 3500 & 7500 \\ \overline V & 2000 & 500 &...
1
vote
1answer
37 views

Efficient methods to compute PCA for an incrementally growing matrix

I am wondering if there exists an efficient method to compute PCA. I am drafting the question into a presudo code: ...
1
vote
1answer
34 views

In PCA, why for every $x \in \mathbb{R}^n$, $x=\sum_{k=1}^n u^T_k x \space u_k$?

In PCA, why for every $x \in \mathbb{R}^n$, $x=\sum_{k=1}^n (u^T_k x) \space u_k$? Where $\{u_1,...,u_n\}$ is orthonormal basis and $||u||^2=u^T_i u_i=1 \forall i$. Is this some standard vector ...
1
vote
1answer
2k views

How Jaccard similarity can be approximated with minhash similarity?

In Page 81 of this book, Mining of Massive Data Sets. It says the following: Now, consider the probability that $h(S_1) = h(S_2)$. If we imagine the rows permuted randomly, and we proceed from the ...
1
vote
1answer
252 views

Smoothing Spline Example

I am learning the smoothing spline method. I saw that smoothing spline is a penalty term to reduce overfitting in linear regression. Given dataset {$(x_1,y_1),(x_2,y_2)..(x_n,y_n)$}So the formular ...
1
vote
1answer
118 views

Physical Meaning Behind Matrix Factorization

As we all know, Matrix Factorization is an effective method to do rating prediction jobs in recommender systems. Thanks to the work of Yahuda Koren. My question is why MF can do this job ? What's the ...
1
vote
1answer
65 views

Given two arrays, how can one determine the the intersection?

I have two datasets (A & B). They each have 1000 numbers. 99% of the time: A < x <= B However, 1% of the time B < x < A. How can I solve for x, ...
1
vote
1answer
20 views

Weighting function for a scatter plot of ratio and difference across several orders of magnitude

I am comparing straight line distances to the shortest discovered path. I have millions of points with a pair (straight line distance,...
1
vote
1answer
232 views

Intuitive difference between cosine similarity and bilinear similarity

Given a pair of strings in vector form $(s_i,s_j)$, I can find cosine similarity of pairs as follows: $cosine(s_i,s_j)=s_i.*s_j / (\|s_i\|\|s_j\|)$ Similarly, bilinear similarity is defined as: ...
1
vote
1answer
18 views

In raw stress, what does putting $\sum_{i <j} d_{ij}$ in denominator do?

In raw stress, what does putting $\sum_{i <j} d_{ij}$ in denominator do? That is $$\frac{\sum_{i<j}(\hat{d_{ij}}-d_{ij})^2}{\sum_{i<j} d_{ij}^2}$$ Where the nominator is the raw stress. I ...
1
vote
1answer
54 views

Unification of data set using machine learning

I have just started learning data science so pardon me the statements that does not make any sense. Consider this situation - I have a data set which is made of examples containing personal info ...
1
vote
1answer
386 views

Hierarchical Clustering using Centroids

Perform a hierarchical clustering (with five clusters) of the one-dimensional set of points $2, 3, 5, 7, 11, 13, 17, 19, 23$ assuming clusters are represented by their centroid (average) and at each ...
1
vote
1answer
73 views

Is EM-algorithm only for missing data?

Currently studying EM algorithm and have been through a few articles, they all say it is for missing data. I believe there is some implication in the term "missing data". I wonder if EM is designed ...
1
vote
1answer
757 views

What is the relationship or difference between MLE and EM algorithm?

I am trying to study EM algorithm and Maximum Likelihood Estimation. Somehow, they both sound the same to me but can't really say the difference. Maybe I don't really understand any of them. I have ...
1
vote
1answer
92 views

How to find the minimum of $f(a_1, b_1, \ldots , a_m, b_m) = \sum_{j=1}^{n} (y_j - \sum_{k=1}^{m} a_kx_j^{b_k})^2$?

$$f(a_1, b_1, \ldots , a_m, b_m) = \sum_{j=1}^{n} (y_j - \sum_{k=1}^m a_kx_j^{b_k})^2$$ $$2m < n$$ $x$ and $y$ are constants, and $a$ and $b$ are variables to find. I took deviation out of it and ...
1
vote
1answer
166 views

Unsupervised learning algorithms to detect anomaly in waves.

I have a sample of graphs (more than 10000...). that look like in the image below: I am searching for an unsupervised learning algorithms that can help me to detect anomalous observations. Here what ...
1
vote
3answers
92 views

Separating populations and estimating line-fit parameters

Given a dataset containing two populations, each of which can be described by a linear relationship between two variables in each sample with high R$^2$, how does one separate the two populations (and ...
1
vote
2answers
54 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
1
vote
1answer
1k views

Naive Bayes to Predict a class label

How do I use Naïve Bayes to predict a class label for a test sample $(A=1, B=1, C=1)$ I know Bayes Theorem is: $$P(C|A) = [P(A|C) P(C)]/P(A)$$ I have no idea how to do this, please help.
1
vote
0answers
59 views

Definition of Entropy (Information Theory)

In Information Theory, entropy is defined as: $$-\sum_{i}P_ilog(P_i)$$ where $-P_ilog(P_i)$ looks like this (using log base 2): From just a generic English definition of entropy, meaning lack of ...