# Questions tagged [data-analysis]

Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

701 questions
5k views

### Where to start learning about topological data analysis?

I was wondering if anyone could help me out with finding a nice introductory introductory text for topological data analysis (I'm speaking as somebody who has two semesters of experience with topology,...
7k views

### Polynomial fitting where polynomial must be monotonically increasing

Given a set of monotonically increasing data points (in 2D), I want to fit a polynomial to the data which is monotonically increasing over the domain of the data. If the highest x value is 100, I don'...
965 views

### Roadmap for learning Topological Data Analysis?

I'm a math major who has recently graduated and I will be starting full time work in 'data analysis'. Having finished with decent marks and still being incredibly interested in mathematics, I was ...
3k views

### Mathematics base for data mining and artificial intelligence algorithms. [closed]

Could you give me some clarification about data mining and artificial intelligence algorithms? What mathematics base they used for? Could you give me starting point, in mathematics, to understand ...
1k views

### Removing noise when the signal is not smooth

Suppose we have (an interval of) a time series of measurements: We assume it can be explained as a "simple" underlying signal overlaid by noise. I'm interested in finding a good algorithm to estimate ...
657 views

### Finding the orientation of a noisy ellipse

This question comes from a neuroscience study which generates $12$ vectors. The vectors are evenly spaced, $30 n$ degrees for $n=0,\dots, 11$, each with their tail centered on the origin. I am ...
285 views

### Monitoring a data stream

You are monitoring a data stream which is delivering very many $32$-bit quantities at a rate of $10$ Megabytes per second. You know that either: $A$: All values occur equally often, $or$ $B$: Half ...
99 views

### How do I find the formula (or rules) that created a list of numbers with seemingly no pattern?

Newbie here, and I apologize if this is the wrong forum for this type of question... I have a group of 200 or so alphanumeric codes from an unknown source. Here's an example piece of the data set: <...
41k views

### Value range of normalization methods? min-max, z-score, decimal scaling

I am working my way through Normalization (data transformation) of data and was curious about four methods: min-max normalization, 2. z-score, 3. z-score mean absolute deviation, and 4. decimal ...
31k views

### Use a set of data points from a graph to find a derivative

I have a data logger that is recording the temperature readings from thermocouples at a specific interval. This gives me data points that I can graph where the x-coordinate is time and the y-...
613 views

### Chaos theory and fractal geometry: Constructing from data

I understand that fractal geometry represents behaviour of 'chaotic' system, if I am not wrong. And also, fractals are generated by a recursive function. But, lets say I have random data lying with me....
136 views

### Explicit definition of a sequence

Suppose there are 6 sequences $a=(a_n)_{n\geq 0}, b=(b_n)_{n\geq 0},c=(c_n)_{n\geq 0},d=(d_n)_{n\geq 0},e=(e_n)_{n\geq 0},f=(f_n)_{n\geq 0}$, the data can be seen here: Data. I found out by trial and ...
697 views

### Relationship between eigenvectors of correlation and covariance matrices

For the purpose of computing principal components of a dataset, represented as matrix $X$ of dimensions $n \times p$ with $n$ samples and $p$ features, we can compute sample covariance matrix $S$, and ...
659 views

### Lottery Ticket Question

A lottery ticket contains six different numbers, chosen from 1 to 39. The winning ticket will match all six numbers in the correct order, plus a bonus number, which may match the other six numbers. ...
29k views

### Find the median given a table of relative frequencies

I came across the following GRE question. I had no problem finding the mean. However, the answer for the median is given to be 1. I don't understand how they arrive at this. Find the mean and ...
935 views

### Difference between Topological Data Analysis and Graph Technology

I'm trying to understand the difference between Oracle's graph technology, which apparently has an inherent understanding of topology, and Ayasdi's Topological Data Analysis technology. Are these two ...
247 views

### Highest points in heart cycle graph

I'm working on an application that reads the heart cycle from a device, and I've aimed to get this image: Now, I need to get the highest points that appear in every cycle in order to calculate the ...
642 views

### Testing for chaos in data

I have data for 3 variables ,each with respect to the discrete time values. How do I check for the existence of chaos for this 3D discrete system?(I don't have the analytic eqs.,just the data). MY ...
1k views

### Abstract Algebra for Business Intelligence / Data Mining

Twitter released its Algebird library, an abstract algebra library for scala. At a high level, what are the maths involved in abstract algebra that have an impact on aggregating large datasets?
79 views

### Is persistent homology mathematically “interesting”? [closed]

Currently, I am occupying myself with (non-applied perspectives on) Persistent Homology, having a background in non-applied maths (representation theory). After having spent some time on the topic ...
3k views

### Way to compare three numbers with an algorithm.

Say I have values 10, 3, 6 What would be a way of finding a subjective difference between them. Or in other words, how close they are to each-other. So if i had 3, 2.5, 2.8; they would be close ...
4k views

766 views

### Normality test vs. Fitting a Gaussian curve

I have a set of one-dimensional data, and I suspect that the data is normally distributed. Before embarking in a normality test, I decided to fit a Gaussian curve to the histogram of relative ...
2k views

### Parameters estimation for gaussian function with offset

I've read the paper Least square fitting of a Gaussian function to a histogram by Leo Zhou on how to perform a Least Square Fitting of a gaussian function to a histogram. The Gaussian function used ...
195 views

### Detecting increasing pulse trains

I have a one dimensional point process representing the times of events which is also mixed in with lots of data that I regard as noise. The interval between the events in the point process are ...
365 views

### Fitting a surface to 2D measurements

I am looking for a way to fit a surface given a set of measured data $(x, y) \mapsto z$. A typical example would consist of anywhere between $10$ and $30$ measurements spread evenly over a disc. ...
51 views

### How do I determine a good function to fit a set of data?

I have a calculated some rates based on data, but I was wondering if there is a way to determine a good math function that might fit this data. For example, consider the following: ...
75 views

### Are the results from persistent homology complete?

Do the homology groups for each Betti number as produced by persistent homology completely capture all the topological characteristics of the space? If not, what topological features are not captured? ...
2k views

### When to use cumulative moving average vs a simple moving average?

After reviewing the Wikipedia page on moving averages, the difference between the simple moving average and cumulative moving average are clear: 1) Simple moving average only considers the last n ...
75 views

### What do the parameters of a multinomial logistic regression correspond to?

I've recently started learning about data science/statistics and learned how to derive such models as linear regressors and logistic regressors. What I don't understand, however, is what the ...
589 views

### Logistic functions - how to find the growth rate

We have the formule for a model with logistic growth: $$N_t = N_{t-1} + g\, N_{t-1}\left( 1 -\dfrac{N_{t-1}}{K}\right)$$ where $g$ defines the growth rate and $K$ is the carrying capacity. Let's ...
444 views

### What is KL-Divergence? Why Do I need it? How do I use it?

I am currently studying KL Divergence. But It seems very confusing that I don't maybe understand why do I ever need it and what is that for? As I have been reading stuff about Mutual Information, it ...
183 views

### How do I use specific data points on a graph to determine an equation?

I need to find a function $f(x)$ such that the following data points would fit on it: $$f(1) = 0 \\ f(2) = 0.5 \\ f(4) = 1.0 \\ f(8) = 1.5 \\ \cdots$$ and so on. So the pattern is every time $x$ ...
20 views

### Number of distinct scatterplots from $p$ variables in a data set

Consider the following quote from the text An Introduction to Statistical Learning: In practice, we often encounter data sets that contain many more than two variables. In this case, we cannot ...
157 views

### Need to find function by the data

I have found interesting sequence, but I can't find its function. Here are the input and output data: ...
252 views

### Prerequisites for Gunnar Carlsson's Topology and Data

I am planning on doing a project on topological data analysis in the near future and intend to use Gunnar Carlsson's paper "Topology and Data" as my introduction to the field. I am familiar with point-...
319 views

### Link between PCA and the Eckart-Young theorem

We can find a really good explanation about the link between: finding the subspace upon which the projected data cloud has the maximal variance (or "inertie") finding the subspace upon which the ...
605 views

### Tropical geometry: practical applications?

In 1960, E. Wigner published a paper entitled "The Unreasonable Effectiveness of Mathematics in the Natural Sciences". Theoretical mathematical structures pave the way to further advances and ...
128 views

### Median of Medians

Given a set A with median Am = 10 and set B with median Bm = 20 is it true that the median of the combined set C is $10 \le$ Cm$\le 20$ ? My first thought was that this wasn't true so I tried to find ...
87 views

### What is the chance of a sample in white noise to be a local extremum?

I draw a long sequence of values from a uniform or normal distribution. Observing the resulting sequence, what is the chance of any point in the sequence (aside from the first and last where it is not ...