Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

0
votes
2answers
74 views

Numerical integration of a derivative dataset

I have an experimentally measured derivative data ($\frac {dy}{dx}$) at a range of times i.e $\frac {dy}{dx}$ for $0 \le t \le tf$. Integrating for $t=0$ is fine, since $y(x,0)$ is known. However, ...
1
vote
0answers
51 views

How to understand the task about bootstrap?

I have the following task. Let $X_1,\ldots,X_n$ be a sample (i.i.d.), $T_n = \overline{X}_n^2=\left(\sum\limits_{i=1}^nX_i\right)^2,$ and $\hat\alpha_k ...
0
votes
1answer
274 views

How to determine a function from outputs

Given the following inputs and outputs to a function, how would I determine an algorithm to fit? ...
1
vote
1answer
364 views

Normalize data with large spread in values.

I'm currently trying to rank a set of data. The issue is that my initial rank comes from a search on google and the returning result set. The spread in values is ranging from 33 all the way up ...
1
vote
0answers
43 views

problem in dimensionality reduction

I am using multidimensional scaling to plot my data in R. However there is a hierarchy in my dataset which i want to exploit and I am using the delaunay triangulation to visualize the plot. So now I ...
1
vote
1answer
178 views

Mathematical calculation to regenerate numeric series

I am searching for a mathematical technique by which I can perform the following, Let me explain first what i want to do. consider a numeric series: ...
1
vote
2answers
8k views

Use a set of data points from a graph to find a derivative

I have a data logger that is recording the temperature readings from thermocouples at a specific interval. This gives me data points that I can graph where the x-coordinate is time and the ...
0
votes
1answer
162 views

Dividing data of a given distribution

Having a dataset with distribution $p$ (e.g. uniform or normal), if we divide the dataset into $n$ parts with equal size, is it valid to say that each part still has distribution $p$?
0
votes
1answer
112 views

noisy 1-second GPS data [closed]

So, I come from a programming background, but not a mathematical one. I have a problem that I'm sure has a great mathematical solution! so, I have GPS data points for every second of a drive a car ...
1
vote
1answer
51 views

Box Plot Log Scaled?

When making box plots to represent data, is it logical to set our axis scale to logarithmic? Or does that defeat the whole point of a box plot?
1
vote
1answer
369 views

Group mean vector and common group covariance matrix in discriminant analysis?

I'm looking over the answers for old statistics exams in preparation for my own exam, while reviewing a question concerning discriminant analysis I found something really odd. I have input variables ...
0
votes
0answers
44 views

What is the name of the matrices which reorder edges in an adjancency so that they are closer to the diagonal? (modularity/structure)

I have seen adjacency matrices presented where the edges formed box like formations along the diagonal (not block matrices). This helped view the modularity and community structure of the group of ...
1
vote
1answer
96 views

Comparing two ratios with tolerances

I'm trying to figure out what the most robust way to compare two ratios is. What I have is a set of proportions between two or more quantities. For example, it might be 0.5:0.5, or 0.45:0.2:0.35. Then ...
1
vote
0answers
282 views

Classification: Why k-Nearest Neighbor method is more appropriate for a Mixture of Gaussians?

I'm reading a book named "The Elements of Statistical Learning" in which it states 2 scenarios when we are trying to predict the class label: Scenario 1: The training data in each class were ...
2
votes
1answer
130 views

Find data to perform regression analysis

I'm trying to find some data (two continuous variables that I believe are correlated) online for which I can perform a regression anaylsis, my assignment sheet says: The data may be found anywhere ...
1
vote
0answers
29 views

Extrapolating parameters for a fit different from the one originally used for regression analysis (no longer having access to data)

Say I have some data, I assume a fit of the form: $\alpha_1 e^{\beta_1 k}$, and I extract some values $\alpha_1>0$ and $\beta_1<0$ for $k=[1,\ldots]$. Say this appears to be a "near perfect" ...
1
vote
0answers
139 views

How can I create a good subset of data proportional to the full data set

I am programming a neural network and I need to use some data to train it. I am following the steps of an experiment which tried to predict direction of stock price index movement using neural ...
1
vote
0answers
38 views

How to find BUE of N? where $X_i$ iid $ P(X_i=j)=1/N$, for $j=1, \cdots, N$

Let $X_1, \cdots, X_n$ be iid with $P(X_i=j)=1/N$, for $j=1, \cdots, N$, where $N$ is an unknown positive integer. I would guess that the order statistics $X_{(n)}$ is the complete sufficient ...
0
votes
0answers
18 views

Linear trend has to pass through a point [duplicate]

Possible Duplicate: Linear trend has to pass through a point I need to interpolate a linear trend surface through a number of points but with the condition that the surface has to pass ...
0
votes
0answers
173 views

Test of goodness of fit - normal distribution of file sizes on HDD

I'm trying to prove, that sizes of files stored on HDD comes from normal distribution. I already made a java code that gave me list of all files on disk with filesizes in Bytes: ...
2
votes
1answer
546 views

Data Type Classification in Statistics

It is my understanding that in statistics one has 4 basic data types: nominal, ordinal, ratio, and interval. I see cases where people refer to "count data" (which is a random variable whose range is ...
0
votes
4answers
326 views

Can you help me solve these questions related to a Logical theory?

In a group of 200 people, number of people having at least primary education (assuming - Category I): number of people having at least middle school education (Category II): number of people having ...
0
votes
1answer
82 views

Determining Sample Size - Design Problem

I have a hypothetical problem were say I have a sample size of about 5000 people. I then will collect data on them say, how man calories of food they eat per day for a month. Now I can't obviously run ...
1
vote
1answer
128 views
0
votes
1answer
103 views

algebraic expression for a counting function

consider a 2d "counting" function given by the $(x,y)$ points/pairs: (1,1), (2,2),(3,2), (4,3),(5,3),(6,3), (7,4),(8,4),(9,4),(10,4), (11,5),(12,5),(13,5),(14,5),(15,5), ... ie $x$ increments in ...
9
votes
3answers
503 views

Removing noise when the signal is not smooth

Suppose we have (an interval of) a time series of measurements: We assume it can be explained as a "simple" underlying signal overlaid by noise. I'm interested in finding a good algorithm to ...
0
votes
1answer
73 views

What is the name of this data normalization?

I have a two-dimensional array ($569\times30$ double) which should be normalized using this formula: $x'_{ij} = \dfrac{x_{ij} - \overline\mu_j}{\overline\delta_j} $ What is the name of this ...
0
votes
1answer
71 views

Finding outliers in a linear series

I have a series of observations made against time. Given perfect observations, the trend should be reasonably linear. As I expect there to be measurement errors, I wish to exclude them from further ...
1
vote
1answer
545 views

Error in a numerical derivative

I have a graph of data, say temperature ($T$) vs time($t$), I know the error bounds in each $\Delta T$. The range of t is from 0 $\to$ 1600 s, with small steps say 0.001 s. If I numerically take ...
0
votes
1answer
70 views

Determine a curve's dominant direction over time

I have a sequence of numbers in a certain order, for example {60,61,62,1,63,,64,102,65,66,200} If this is drawn as a graph, this will be a curve going upwards, but with some aggressive jumps upwards ...
1
vote
2answers
2k views

What is the distribution of a data set

I understand what the probability distribution is. I also have a personal understanding/interpretation of the concept of distribution of a dataset. Whenever I see this expression I imagine a graph ...
5
votes
1answer
701 views

Abstract Algebra for Business Intelligence / Data Mining

Twitter released its Algebird library, an abstract algebra library for scala. At a high level, what are the maths involved in abstract algebra that have an impact on aggregating large datasets?
3
votes
2answers
128 views

How do I use specific data points on a graph to determine an equation?

I need to find a function $f(x)$ such that the following data points would fit on it: $$f(1) = 0 \\ f(2) = 0.5 \\ f(4) = 1.0 \\ f(8) = 1.5 \\ \cdots $$ and so on. So the pattern is every time $x$ ...
0
votes
1answer
144 views

Microsoft Excel not graphing, $x = y^{1/2}$

Im not sure if this is a math question or a su question. The experiment was relating the period of one "bounce" when you hang a weight on a spring and let it bounce. I have this data here, one ...
1
vote
0answers
48 views

Fitting multiple functions with shared parameters to data

consider a set of real-valued continuous functions $$f_i(t;a)$$ for $1\leq i\leq n$ with some shared non-negative parameters $a=(a_1,\ldots,a_d)\in\mathbb{R}^d$. Furthermore, assume that there exists ...
1
vote
2answers
133 views

Finding an equation to fit data points

I'm a software developer by trade and one of the things I did today was recursively generate some dummy data for a tree. I used two parameters in my recursive algorithm: number of levels and number of ...
1
vote
0answers
96 views

Filter/Removal of periodic 'delta-peaks'

Currently I am measuring data (Counts over Time). Due to measurement problems I have some nasty peaks in this data. These peaks are periodical, very sharp (~3 datapoints over a range of 10000) and ...
2
votes
1answer
95 views

What's the best function to use to fit this data?

{7502, 6610, 4988, 2861, 2242, 1746, 1433, 1312, 949, 793, 767, 664, 660, 653, 588, 578, 575, 572, 553, 500, 445, 418, 374, 165, 108} I was looking at the number of seeds for torrent files for ...
0
votes
1answer
47 views

arctic sea ice data by year

I want to demonstrate the usefulness of regression lines to my class, and I was hoping to do so with some data about the Arctic sea ice. What I'd like is the maximal area covered by the ice per year, ...
1
vote
1answer
66 views

Variable frequency redistribution

I know that there is a way to "redistribute of the frequencies of a variable" as stated here: Dorian Pyle book chapter 7 section 2 paragraph 3 (7.2.3): The easiest way to adjust distribution ...
3
votes
1answer
237 views

Fitting a surface to 2D measurements

I am looking for a way to fit a surface given a set of measured data $(x, y) \mapsto z$. A typical example would consist of anywhere between $10$ and $30$ measurements spread evenly over a disc. ...
0
votes
1answer
358 views

Given a set of data (distance vs time) obtain a closer approximate for velocity

This is actually a physics experiment. I've collected a set of data Distance and time it takes for a cart to move down an incline plane. From here, I've calculated the average time (...
2
votes
1answer
2k views

Standard Deviation vs Standard Error

I am confused about the difference between standard deviation and error. When do I use which? From my lecture: Standard deviation quantifies spread of data about mean Standard error measures spread ...
1
vote
1answer
85 views

What preposition to use when fitting data?

Please could some real experts give an opinion on this question on English.SE: What preposition to use when fitting data? Do we fit data with, by, or as a linear function? Forgive me as I do ...
11
votes
4answers
2k views

Mathematics base for data mining and artificial intelligence algorithms. [closed]

Could you give me some clarification about data mining and artificial intelligence algorithms? What mathematics base they used for? Could you give me starting point, in mathematics, to understand ...
1
vote
1answer
269 views

Html Tag counting - Rate of Change formula

I've been trying to a find a statistics-esque formula for calculating the rate of change for html tags which are either added or removed from various websites. So, for example, with the scraper I'm ...
0
votes
1answer
5k views

How to perform simple linear interpolation on a data set

With the following data set, what is the best way to interpolate the data for each time. ...
1
vote
0answers
53 views

How to quantify the volitility of a time series data?

I am currently working on a time series data and I would like to quantify how volatile it is. Here volatile I mean how "shaky" the series is. If the series is smooth than it is not volatile. I have ...
2
votes
1answer
79 views

How to transform data distributed around zero to make it closer to normal?

I have data that ranges continuously from $-1$ to $+1$, with lots of zeros in the middle. I want to transform the data to a normal distribution. How would I do this? My normal approach with data ...
1
vote
0answers
389 views

How should I calculate “fill rate”? People inquire about an item and then buy it days/months later?

So I have posts(people posting a want for an item) and then I have fills(the person actually getting the item) There are hundreds of posts per day but the fill for that item can be days or weeks away ...