Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

9
votes
3answers
462 views

Removing noise when the signal is not smooth

Suppose we have (an interval of) a time series of measurements: We assume it can be explained as a "simple" underlying signal overlaid by noise. I'm interested in finding a good algorithm to ...
0
votes
1answer
71 views

What is the name of this data normalization?

I have a two-dimensional array ($569\times30$ double) which should be normalized using this formula: $x'_{ij} = \dfrac{x_{ij} - \overline\mu_j}{\overline\delta_j} $ What is the name of this ...
0
votes
1answer
70 views

Finding outliers in a linear series

I have a series of observations made against time. Given perfect observations, the trend should be reasonably linear. As I expect there to be measurement errors, I wish to exclude them from further ...
1
vote
1answer
498 views

Error in a numerical derivative

I have a graph of data, say temperature ($T$) vs time($t$), I know the error bounds in each $\Delta T$. The range of t is from 0 $\to$ 1600 s, with small steps say 0.001 s. If I numerically take ...
0
votes
1answer
69 views

Determine a curve's dominant direction over time

I have a sequence of numbers in a certain order, for example {60,61,62,1,63,,64,102,65,66,200} If this is drawn as a graph, this will be a curve going upwards, but with some aggressive jumps upwards ...
1
vote
2answers
2k views

What is the distribution of a data set

I understand what the probability distribution is. I also have a personal understanding/interpretation of the concept of distribution of a dataset. Whenever I see this expression I imagine a graph ...
5
votes
1answer
660 views

Abstract Algebra for Business Intelligence / Data Mining

Twitter released its Algebird library, an abstract algebra library for scala. At a high level, what are the maths involved in abstract algebra that have an impact on aggregating large datasets?
3
votes
2answers
123 views

How do I use specific data points on a graph to determine an equation?

I need to find a function $f(x)$ such that the following data points would fit on it: $$f(1) = 0 \\ f(2) = 0.5 \\ f(4) = 1.0 \\ f(8) = 1.5 \\ \cdots $$ and so on. So the pattern is every time $x$ ...
0
votes
1answer
142 views

Microsoft Excel not graphing, $x = y^{1/2}$

Im not sure if this is a math question or a su question. The experiment was relating the period of one "bounce" when you hang a weight on a spring and let it bounce. I have this data here, one ...
1
vote
0answers
47 views

Fitting multiple functions with shared parameters to data

consider a set of real-valued continuous functions $$f_i(t;a)$$ for $1\leq i\leq n$ with some shared non-negative parameters $a=(a_1,\ldots,a_d)\in\mathbb{R}^d$. Furthermore, assume that there exists ...
1
vote
2answers
129 views

Finding an equation to fit data points

I'm a software developer by trade and one of the things I did today was recursively generate some dummy data for a tree. I used two parameters in my recursive algorithm: number of levels and number of ...
1
vote
0answers
92 views

Filter/Removal of periodic 'delta-peaks'

Currently I am measuring data (Counts over Time). Due to measurement problems I have some nasty peaks in this data. These peaks are periodical, very sharp (~3 datapoints over a range of 10000) and ...
2
votes
1answer
95 views

What's the best function to use to fit this data?

{7502, 6610, 4988, 2861, 2242, 1746, 1433, 1312, 949, 793, 767, 664, 660, 653, 588, 578, 575, 572, 553, 500, 445, 418, 374, 165, 108} I was looking at the number of seeds for torrent files for ...
0
votes
1answer
47 views

arctic sea ice data by year

I want to demonstrate the usefulness of regression lines to my class, and I was hoping to do so with some data about the Arctic sea ice. What I'd like is the maximal area covered by the ice per year, ...
1
vote
1answer
66 views

Variable frequency redistribution

I know that there is a way to "redistribute of the frequencies of a variable" as stated here: Dorian Pyle book chapter 7 section 2 paragraph 3 (7.2.3): The easiest way to adjust distribution ...
3
votes
1answer
221 views

Fitting a surface to 2D measurements

I am looking for a way to fit a surface given a set of measured data $(x, y) \mapsto z$. A typical example would consist of anywhere between $10$ and $30$ measurements spread evenly over a disc. ...
0
votes
1answer
339 views

Given a set of data (distance vs time) obtain a closer approximate for velocity

This is actually a physics experiment. I've collected a set of data Distance and time it takes for a cart to move down an incline plane. From here, I've calculated the average time (...
2
votes
1answer
2k views

Standard Deviation vs Standard Error

I am confused about the difference between standard deviation and error. When do I use which? From my lecture: Standard deviation quantifies spread of data about mean Standard error measures spread ...
1
vote
1answer
82 views

What preposition to use when fitting data?

Please could some real experts give an opinion on this question on English.SE: What preposition to use when fitting data? Do we fit data with, by, or as a linear function? Forgive me as I do ...
11
votes
4answers
2k views

Mathematics base for data mining and artificial intelligence algorithms. [closed]

Could you give me some clarification about data mining and artificial intelligence algorithms? What mathematics base they used for? Could you give me starting point, in mathematics, to understand ...
1
vote
1answer
250 views

Html Tag counting - Rate of Change formula

I've been trying to a find a statistics-esque formula for calculating the rate of change for html tags which are either added or removed from various websites. So, for example, with the scraper I'm ...
0
votes
1answer
5k views

How to perform simple linear interpolation on a data set

With the following data set, what is the best way to interpolate the data for each time. ...
1
vote
0answers
52 views

How to quantify the volitility of a time series data?

I am currently working on a time series data and I would like to quantify how volatile it is. Here volatile I mean how "shaky" the series is. If the series is smooth than it is not volatile. I have ...
2
votes
1answer
78 views

How to transform data distributed around zero to make it closer to normal?

I have data that ranges continuously from $-1$ to $+1$, with lots of zeros in the middle. I want to transform the data to a normal distribution. How would I do this? My normal approach with data ...
1
vote
0answers
344 views

How should I calculate “fill rate”? People inquire about an item and then buy it days/months later?

So I have posts(people posting a want for an item) and then I have fills(the person actually getting the item) There are hundreds of posts per day but the fill for that item can be days or weeks away ...
1
vote
1answer
360 views

Equal interval data classification technique formula confusion

The formula for breaking a sequence of values into groups of equal ranges is: max - min / number of categories I would like to know why is it necessary to subtract the maximum value from the minimum ...
1
vote
1answer
191 views

How to create function that is a power law

If I have a starting and ending y value and know the range along the x-axis, how can I create a function that adheres to a power-law? Ex: I want the function to start at (1, 10), end at (1000,0). ...
1
vote
0answers
105 views

Jacobian approximation at given point without explicit derivatives expression

after solving a NLproblem with optimization method, I would like to compute confidence intervals, prediction bounds and standard deviation for these optimal parameters. Explicit formulas I have read ...
0
votes
2answers
75 views

Normalize $X$ to $0$ to $10$ scale with asymptotes at either end

I am trying to find a scaling function that mimics the gas gauge in a car. I would like to map a value to a $0$ to $10$ scale, on which I have two known points. For example: $X_1 = 2$, $Y_1 = 2.5$, ...
1
vote
2answers
2k views

time derivative of discrete data in simulink

I'm implementing a program in Java that was delivered in Simulink. My expertise is limited, and I'm stuck on converting a derivative block. The simulink code applies a du/dt block to the input data. ...
0
votes
1answer
372 views

Calculating number of draws in a series of team matches

May be this could be an easy problem but somehow cannot arrive to any conclusive result to this indecisiveness problem For a certain number of teams ($n$) playing a series of matches ($m$), and given ...
2
votes
2answers
170 views

Working out the “best” score based on quantity and ratio

I'm not an expert with mathematics so I hope I'm posting in the right place! I've got a lot of products which can be rated good, bad and OK by a user. What I'm having trouble with is finding which is ...
1
vote
3answers
183 views

'Squashing' a graph of data?

I have a vector of sample data that describes a divergent oscillation like in this image. I want to transform this data (just the data, not the system or anything), so that the data does converge ...
0
votes
1answer
234 views

How do i estimate an equation of best fit of a sort-of exponential equation given it's points?

There is an unnessarly long table of values of an item in a game that grants experience to a player based on one's level in the chosen skill. I think it would be much more appropriate to cite the ...
1
vote
3answers
724 views

Find parameters for exponential function fitting to datapoints

I have a set of datapoints, in this case the temperature of an object adjusting to the environment temperature over time. Because I know these kind of processes take the form of $$f(x)=Ae^{x/B}+C$$ I ...
0
votes
2answers
91 views

preliminary evaluation for forecasting models

Suppose I would like to use a method for data prediction, and that I have some empirical data (i.e., sequence of samples of the form [time, value]). Would it be possible to know in advance, based on ...
1
vote
1answer
85 views

Waveform filter - what's in the black box?

I'm trying to come up with an algorithm or filter that has the i/o characteristics in the image below (waveform plotted from audio data) .. I input a sine wave or saw and the resulting graphic is an ...
1
vote
0answers
34 views

How to symmetrically adjust data by removing certain value

Say I have a data set in the following format: {a:200, b: 100; c:400} The sum of a, b and c equals 700. Now, I have another value which is supposed to be the actual total say 500. How do I adjust ...
0
votes
1answer
53 views

Graphically representing low number of infractions against high activity volume

I have been presented with a dataset that looks at work volume and safety infractions. A sample table: A sample chart: The issue we are having is trying to graphically represent it so that ...
2
votes
1answer
1k views

Understanding a Density Plot

I am currently plotting density plots in R to view the distribution of my data. An example density plot is here I understand N is the number of values in the data set and bandwidth is the the ...
3
votes
3answers
398 views

What would this curve be called?

In an resistor-capacitor circuit, the voltage stored is measured by taking the two values in Farads and Ohms - and the curve looks like this: What would this curve be called, logarithmic? ...
0
votes
1answer
130 views

Fitting two separate curves to a graph consistently?

I have a set of graphs with a large primary peak and a secondary peak that intersects the first and I want to fit two curves (one per peak) and find the point at which the two curves intersect for ...
2
votes
1answer
416 views

Curve fitting with upper and lower bounds for derivatives

I compute (at a great cost) upper and lower bounds $f_u(x)$ and $f_l(x)$ of an unknown function $f(x)$ at points $x$ in $[0,1]$. Now I am interested in an estimation of the derivative $f'(x)$. I ...
2
votes
2answers
167 views

weighted ratings

I have a form where users can rate presentations and they can say how knowledgeable they are on the subject. The range for both sets is 1-5. (1 being lousy and 5 being great) So for example, a user ...
1
vote
1answer
418 views

Given sample data with three data points, how do I predict or guess the 3rd data point when only given two data points?

I'm tracking data on a backup job that runs nightly on our server and using the historical data to predict data and job time growth. I have the following three data points for most of the records: ...
10
votes
4answers
2k views

Polynomial fitting where polynomial must be monotonically increasing

Given a set of monotonically increasing data points (in 2D), I want to fit a polynomial to the data which is monotonically increasing over the domain of the data. If the highest x value is 100, I ...
2
votes
1answer
8k views

Boxplots and bar graphs

I'm studying for the GRE and came across several questions that I was unable to answer in a practice booklet, even after looking at the answer and trying to work backwards, and searching google and ...
1
vote
2answers
137 views

Trying to find a meaningful statistical indicator to reflect user interaction with a website

A given user can interact in multiple ways with a website. Let's simplify a bit and say say a user can: Post a message Comment a message "like" something on the website via Facebook (after that we ...
1
vote
1answer
134 views

problem constructing an equation that connects variables

need a suggestion/advice/inputs from the mathematicians here. I have collected data of number of instructions/sec executed by a processor every second, number of loads/stores performed every second ...