Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data-analysis]

Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

15
votes
4answers
6k views

Polynomial fitting where polynomial must be monotonically increasing

Given a set of monotonically increasing data points (in 2D), I want to fit a polynomial to the data which is monotonically increasing over the domain of the data. If the highest x value is 100, I don'...
3
votes
1answer
18k views

Boxplots and bar graphs

I'm studying for the GRE and came across several questions that I was unable to answer in a practice booklet, even after looking at the answer and trying to work backwards, and searching google and ...
1
vote
2answers
165 views

Creating a function incrementally

Let's say that I need a computer program to generate a function that fits a data set which has $(x,y)$ data points. Let's say that I get the data set one point at a time, I don't know how many data ...
1
vote
2answers
81k views

calculation of median of grouped data

While calculating the median of grouped data of total frequency $N$, in order to find the median class which value should be taken into consideration to match against cumulative frequency : $\frac N2$ ...
21
votes
2answers
5k views

Where to start learning about topological data analysis?

I was wondering if anyone could help me out with finding a nice introductory introductory text for topological data analysis (I'm speaking as somebody who has two semesters of experience with topology,...
4
votes
2answers
5k views

What is the most scientific way to assign weights to historical data?

This is a common question I usually face while processing historical data. I have year on year data of an event for the past N years.I would like to assign weights to the data of these N years so that ...
1
vote
2answers
231 views

Mathematical Background Required for Advanced Machine Learning Concepts

What are the must-know concepts and best resources for preparing the mathematical background for advanced machine learning studies? Currently, looking into the book What is Mathematics? by Richard ...
4
votes
2answers
4k views

Power set of a set containing a set

I need some help understanding this concept a little better. I understand the general power sets, but only worked nice and easy examples where only the set consisted of only single elements like $\{a,...
2
votes
1answer
63 views

Optimize database indexes, the sequel

Background For those not familiar with databases. Indices help to speed up database searches, but they come at the cost of memory. Since you like your database to be fast, they are best cached in RAM ...
2
votes
0answers
123 views

Need help understanding the proof of the Johnson-Lindenstrauss lemma by Dasgupta and Gupta

The Johnson-Lindenstauss lemma is the following: Let $\varepsilon\in(0,1)$, $\mathcal{P}\subseteq\mathbb{R}^d$ be a finite subset and $k\in\mathcal{O}(\varepsilon^-2\log{n})$. Then there exists a (...
1
vote
1answer
870 views

What asymmetric sigmoid function defined on [0, ∞[ could I use to model my data?

I need to find a model to fit my data, defined on [0, ∞[ or even on ]0, ∞[. It seems that some kind of (reversed) sigmoid function would work, but my data aren't symmetric, so traditional sigmoid ...
4
votes
1answer
763 views

Normality test vs. Fitting a Gaussian curve

I have a set of one-dimensional data, and I suspect that the data is normally distributed. Before embarking in a normality test, I decided to fit a Gaussian curve to the histogram of relative ...
4
votes
1answer
365 views

Fitting a surface to 2D measurements

I am looking for a way to fit a surface given a set of measured data $(x, y) \mapsto z$. A typical example would consist of anywhere between $10$ and $30$ measurements spread evenly over a disc. ...
3
votes
1answer
502 views

Techniques to find regression parameters for multiple datasets where a subset of parameters should be the same for all datasets

I have five sets of observations of measured y as some function of measured $x_1, x_2, x_3,\ldots$ and I want to fit five functions to these observations. They have the form $$ y = f(x_1, x_2, x_3,\...
2
votes
3answers
239 views

Finding similarity between elements using statistics

I have a dataset of DJs in which I'm trying to find DJs similar to a specific DJ. Each DJ has a set of a genres with a certain percentage. How can I find the similarity between 2 DJs? The following is ...
1
vote
3answers
2k views

Find parameters for exponential function fitting to datapoints

I have a set of datapoints, in this case the temperature of an object adjusting to the environment temperature over time. Because I know these kind of processes take the form of $$f(x)=Ae^{x/B}+C$$ I ...
1
vote
2answers
117 views

How to interpret the result of correlation between two discrete signals

Before starting, let me say I'm not a mathematician, I'm just a curious engineer, with a master's degree in engineering and computer science. I have been using this book The Scientist and Engineer's ...
1
vote
2answers
454 views

How do I smoothly merge two power laws?

I've got data that, when plotted on a log-log scale, looks like two straight lines that gently bend and blend into each other. See the picture below. I don't know how to find a function to fit that. ...
0
votes
0answers
236 views

Is the sum of all cross-correlation samples representative of target existence likelihood?

Answers to this question take the peak in the cross correlation as the measure to the likelihood of the trigger signal exist in the received signal - this is pretty much text book. My question is ...
0
votes
1answer
123 views

Probability of a Collision in Selection of Permutations

Suppose I have a set of characters, say $\{a-z,A-Z,0-9\}$ $(n=62)$. Then I select a random permutation each containing $10$, e.g. $k=10$, characters. What would be the probability of a collision &...
0
votes
1answer
23 views

How to fit Poission distribution with large mean?

I have one set of discrete data plotted below. I suspect the data follows Poisson distribution with mean $\mu \simeq 400$ and standard deviation $\sigma \simeq \sqrt{400} = 20$. $$ n(k) = \frac{\mu ^...
0
votes
2answers
67 views

Optimize database indexes

Q: I have 14 columns, how many indexes do i need to create to cover all possibilities? Examples of possibilities: col1 col12 col5, col3, col4 col7, col2, col12, col1 all 14 columns -- Order can be ...