Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

11
votes
4answers
2k views

Mathematics base for data mining and artificial intelligence algorithms. [closed]

Could you give me some clarification about data mining and artificial intelligence algorithms? What mathematics base they used for? Could you give me starting point, in mathematics, to understand ...
9
votes
4answers
2k views

Polynomial fitting where polynomial must be monotonically increasing

Given a set of monotonically increasing data points (in 2D), I want to fit a polynomial to the data which is monotonically increasing over the domain of the data. If the highest x value is 100, I ...
9
votes
3answers
358 views

Removing noise when the signal is not smooth

Suppose we have (an interval of) a time series of measurements: We assume it can be explained as a "simple" underlying signal overlaid by noise. I'm interested in finding a good algorithm to ...
5
votes
2answers
126 views

Highest points in heart cycle graph

I'm making an application that reads the heart cycle from a device, and I've aimed to get this image: Now, I need to get the highest points that appear in every cycle in order to calculate the ...
5
votes
1answer
522 views

Abstract Algebra for Business Intelligence / Data Mining

Twitter released its Algebird library, an abstract algebra library for scala. At a high level, what are the maths involved in abstract algebra that have an impact on aggregating large datasets?
5
votes
0answers
43 views

Careers in applied math with an MS other than in finance and data/machine learning?

Since I like math, I would like a career that uses alot of applied math. I'm about to complete my Master's and could do my thesis in numerical solutions of PDEs I'm already aware of careers such as ...
4
votes
2answers
6k views

Value range of normalization methods? min-max, z-score, decimal scaling

I am working my way through Normalization (data transformation) of data and was curious about four methods: min-max normalization, 2. z-score, 3. z-score mean absolute deviation, and 4. decimal ...
4
votes
2answers
175 views

Detecting increasing pulse trains

I have a one dimensional point process representing the times of events which is also mixed in with lots of data that I regard as noise. The interval between the events in the point process are ...
3
votes
3answers
257 views

What would this curve be called?

In an resistor-capacitor circuit, the voltage stored is measured by taking the two values in Farads and Ohms - and the curve looks like this: What would this curve be called, logarithmic? ...
3
votes
2answers
104 views

How do I use specific data points on a graph to determine an equation?

I need to find a function $f(x)$ such that the following data points would fit on it: $$f(1) = 0 \\ f(2) = 0.5 \\ f(4) = 1.0 \\ f(8) = 1.5 \\ \cdots $$ and so on. So the pattern is every time $x$ ...
3
votes
2answers
59 views

Need to find function by the data

I have found interesting sequence, but I can't find its function. Here are the input and output data: ...
3
votes
1answer
188 views

Fitting a surface to 2D measurements

I am looking for a way to fit a surface given a set of measured data $(x, y) \mapsto z$. A typical example would consist of anywhere between $10$ and $30$ measurements spread evenly over a disc. ...
3
votes
1answer
49 views

Queue and Stack in Computer Science and relation to MATH

For Creating a Queue Q we use two Stack S1,S2. for insert x in the end of Q we do Push(S1,x). for deleting element from front of Q, (if Q is not empty) we do Pop(S2). if on Q that at first is empty, ...
3
votes
1answer
92 views

Sigmoid function with separate control of derivative at 0 and sharpness of bend.

For a physical relationship, $f(x)$, I'm trying to model, i have a fairly good determination of some of the boundaries, such that $ f(0) = 0$ $f'(0) = B$ $ \lim_{x\to\infty} f = A $ So far, what I ...
3
votes
1answer
52 views

The number of nuts in a package of “Premium Cashews” is normally distributed with a mean of 433 and a standard deviation of 6 nuts.

The number of nuts in a package of "Premium Cashews" is normally distributed with a mean of 433 and a standard deviation of 6 nuts. Packages with fewer than 420 nuts or more than 445 nuts will be ...
3
votes
1answer
223 views

How do I 'reverse engineer' the standard deviation?

My problem is fairly concrete and direct. My company loves to do major business decisions based on many reports available on the media. These reports relates how our products are fairing in ...
3
votes
1answer
20 views

Where can I find a longitudinal study with a binary response?

I've spent the last couple of months trying to find real data (not simulated) of a longitudinal study, but I can't seem to find one. Any topic and time frame is fine, but the dependent variable must ...
3
votes
1answer
120 views

Logistic functions - how to find the growth rate

We have the formule for a model with logistic growth: $$ N_t = N_{t-1} + g\, N_{t-1}\left( 1 -\dfrac{N_{t-1}}{K}\right)$$ where $g$ defines the growth rate and $K$ is the carrying capacity. Let's ...
2
votes
2answers
60 views

Converting data to a specified range

I am trying to convert data to a particular range of 0-10. Actual Data may vary from - 50000 - 26214400. I have broken this down in to 4 parts as follows - ...
2
votes
3answers
90 views

$4\log_2(n)=n$ How to Solve for $n$?

As the title suggests, my log skills are pretty lacking. Need to learn how to get from $4\log_2(n)=n$ to $n=16$ ($\log$ base $2$). I've searched Google and it seems I am missing some core concept ...
2
votes
1answer
97 views

Is there a scientific way to assign weights to historical data?

This is a common question I usually face while processing historical data. I have year on year data of an event for the past N years.I would like to assign weights to the data of these N years so that ...
2
votes
1answer
74 views

Find data to perform regression analysis

I'm trying to find some data (two continuous variables that I believe are correlated) online for which I can perform a regression anaylsis, my assignment sheet says: The data may be found anywhere ...
2
votes
2answers
148 views

weighted ratings

I have a form where users can rate presentations and they can say how knowledgeable they are on the subject. The range for both sets is 1-5. (1 being lousy and 5 being great) So for example, a user ...
2
votes
1answer
6k views

Boxplots and bar graphs

I'm studying for the GRE and came across several questions that I was unable to answer in a practice booklet, even after looking at the answer and trying to work backwards, and searching google and ...
2
votes
1answer
22 views

Data Analysis and Probability

Two dice with a difference ... They appear identical, but one die is fair and the other is loaded so that the probability of throwing a 6 is 0.25. Bev picks a die at random and throws it, and it shows ...
2
votes
1answer
74 views

How to appropriately normalize financial data?

I am evaluating a normalization of financial data. Two colleagues feel certain of a certain approach, and I am befuddled. The approach shapes the results how they want (i.e., brings possibly extreme ...
2
votes
1answer
50 views

Optimize database indexes, the sequel

Background For those not familiar with databases. Indices help to speed up database searches, but they come at the cost of memory. Since you like your database to be fast, they are best cached in RAM ...
2
votes
1answer
140 views

Techniques to find regression parameters for multiple datasets where a subset of parameters should be the same for all datasets

I have five sets of observations of measured y as some function of measured $x_1, x_2, x_3,\ldots$ and I want to fit five functions to these observations. They have the form $$ y = f(x_1, x_2, ...
2
votes
1answer
388 views

Data Type Classification in Statistics

It is my understanding that in statistics one has 4 basic data types: nominal, ordinal, ratio, and interval. I see cases where people refer to "count data" (which is a random variable whose range is ...
2
votes
1answer
94 views

What's the best function to use to fit this data?

{7502, 6610, 4988, 2861, 2242, 1746, 1433, 1312, 949, 793, 767, 664, 660, 653, 588, 578, 575, 572, 553, 500, 445, 418, 374, 165, 108} I was looking at the number of seeds for torrent files for ...
2
votes
2answers
143 views

Working out the “best” score based on quantity and ratio

I'm not an expert with mathematics so I hope I'm posting in the right place! I've got a lot of products which can be rated good, bad and OK by a user. What I'm having trouble with is finding which is ...
2
votes
1answer
23 views

Aproximate data with this equation (or linearize the equation)

I have found an equation that describes the behaviour of a phisical system: $$ y=a_1e^{-a_2x} + a_3 + a_4x + a_5e^{{-a_6} / {(1-x)}}$$ Now I have data of that physical system and I want to ...
2
votes
0answers
157 views

curve fit but unknown function

I face a strange problem. I have data $[X,Y]$ with no noise. If I plot $Y$ as a function of $X$, it looks like a straight line and the regression coefficient is quite good. For the simplest model ...
2
votes
0answers
97 views

The correct probability distribution / way to analyze daily changes

I am working on a report which is being sent through to end users that should flag to them any "large changes" in the day-to-day values for the past 30 days for something we would assume the ...
2
votes
0answers
367 views

Exponent p-value generated in Excel

Excel gave me a p-value of 1.44909E-09 Notice is does not say .09 but 09 This is confusing me, I am trying to analyze my data but am stuck at this point. If it were E-9 it could be ...
2
votes
1answer
195 views

Kalman filter for acceleremoter and gyro

I have read a lot of literature on this topic, but many questions still remain. (Many formulae are here below, so, who prefers to look at them in a more comfortable manner can find a MathCAD sheet ...
2
votes
1answer
179 views

Lloyd's algorithm in normed vector spaces

How do I run Lloyd's algorithm in a normed vector space? The space: L*a*b* color space, finite sRGB segment, $R^3$ The distance metric: CIE94 using L*C*h* information derived from the L*a*b* ...
1
vote
3answers
45 views

fit sine wave to data

I have some astronomical data which I know has a sinusoidal component to it of the form y = Asin(Ωt+Φ). The period of the sinusoid is equal to a sidereal day. So ...
1
vote
1answer
76 views

What preposition to use when fitting data?

Please could some real experts give an opinion on this question on English.SE: What preposition to use when fitting data? Do we fit data with, by, or as a linear function? Forgive me as I do ...
1
vote
2answers
55 views

Curve fitting: $a f_1(x)+b f_2(x)+c f_3(x)+d f_4(x)$

I want to fit my data $y$ with a method such as $$a f_1(x)+b f_2(x)+c f_3(x)+d f_4(x)$$ However I don't know how to obtain coefficients $(a,b,c,d)$ of the known functions ...
1
vote
2answers
65 views

Can we express “exactly X” with set theory?

I have a question that I think that is a better fit here than a programming SE as SQL is founded on set theory and my question is more about set theory than about a programming question. Assuming I ...
1
vote
2answers
77 views

Determining function from simple xy graph

I'm not sure if this question is too vague for here. Will delete if necessary. I have a set of points, (1,782), (2,893), (3,992),... and I'm trying to determine the function that was used to generate ...
1
vote
1answer
66 views

Extremely poor polyfit, what am I doing wrong?

I have a dataset with me: http://pastebin.com/YZArky1j, which I am trying to polyfit. This is what I used to perform the polyfit: ...
1
vote
1answer
42 views

Data compression from lack of patterns

From my understanding, lossless data compression takes advantages of statistical patterns. I want to know if it is possible to compress data based on the lack of patterns, such as a random string of ...
1
vote
2answers
202 views

Least squares fit with a trick

Fitting a circle with least squares is easy once you get the trick for $c = r^2 - a^2 -b^2$ and you got a linear set of equations. my problem is as follows: $$z = \alpha x + \beta y + \alpha \beta$$ ...
1
vote
2answers
116 views

Finding an equation to fit data points

I'm a software developer by trade and one of the things I did today was recursively generate some dummy data for a tree. I used two parameters in my recursive algorithm: number of levels and number of ...
1
vote
3answers
168 views

'Squashing' a graph of data?

I have a vector of sample data that describes a divergent oscillation like in this image. I want to transform this data (just the data, not the system or anything), so that the data does converge ...
1
vote
1answer
24 views

Find median given large amount of data

Listing all the possibilities would take time. Would there be a faster and more efficent way to find the median?
1
vote
1answer
28 views

How to work out the formula that connects several numbers

I have an interesting problem. Say I have lots of datasets like this: a = 21 b = 23 c = 58 d = 498 etc (lots of other values) X = 85 I need to find the formula ...
1
vote
1answer
56 views

Given two arrays, how can one determine the the intersection?

I have two datasets (A & B). They each have 1000 numbers. 99% of the time: A < x <= B However, 1% of the time B < x < A. How can I solve for x, ...