Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

0
votes
1answer
24 views

Is Expectation Propagation (EP) affected by the prior?

I understand EP by reading Minka's thesis: http://research.microsoft.com/en-us/um/people/minka/papers/ep/minka-ep-uai.pdf I'm trying to apply it to solve a Bayesian inference problem. However, I'm ...
0
votes
1answer
564 views

slope of curve represented by discrete points

I have data which are visualized in this chart: I need to compute slope of increasing / decreasing parts of the curve. I can't use any 2 points because of noise in data. Maybe numerical derivative ...
1
vote
1answer
38 views

Problem with significant figures

I know that if I am multiplying/dividing my answer is the least number of significant figure in the data and if I am adding/subtracting my answer is the least number of decimal places i the data but ...
1
vote
1answer
229 views

Maths concept for salient point in graph data

I have collected data concerning the total post counts of users in an online forum (see graphic). What I am hoping to do is compare the language of 'first posts' with the language of 'later posts'. ...
0
votes
0answers
51 views

Simple maths for an algorithm to searching a table to find out if one type of entry has a causal effect upon another type?

I asked this over on StackExchange and was sent over here... I have a database table which is recording two different types of events which seem to be related to one another. It records the date and ...
1
vote
1answer
206 views

Given a set of data points, how to use gradient descent to find the minimum in the function that passes from those data points?

I have a function with n parameters. I don't know the formula of the function but I can generate as many data points as I want using the function that I have. My question is, how can I find the set of ...
1
vote
1answer
28 views

Correct equation

What is the correct equation which describes the following dates: 270, 180, 135, 112.25, 101.25. I've obtained data whit the equations: 180 + 90 = 270 135 + 45 = 180 112.5 + 22.5 = ...
1
vote
0answers
35 views

How to compute a “weather” score for repeated trials over time, with a bias for recency

I'm doing some analysis on repeated software tests that can either pass or fail. For each test, I've collected a set of historical (pass/fail, timestamp) records, and I'd like to compute a score to ...
1
vote
1answer
53 views

Distance measures for binary data

I was wondering what are some good distance measures for binary data that have the following properties. I know that there are measures like the Jaccard index and the Dice Index, but they don't ...
3
votes
1answer
256 views

Sigmoid function with separate control of derivative at 0 and sharpness of bend.

For a physical relationship, $f(x)$, I'm trying to model, i have a fairly good determination of some of the boundaries, such that $ f(0) = 0$ $f'(0) = B$ $ \lim_{x\to\infty} f = A $ So far, what I ...
2
votes
1answer
123 views

How to appropriately normalize financial data?

I am evaluating a normalization of financial data. Two colleagues feel certain of a certain approach, and I am befuddled. The approach shapes the results how they want (i.e., brings possibly extreme ...
2
votes
1answer
404 views

Analyzing data to determine seasonal indices?

The quarterly sales levels for the past four years of a leading tyre manufacturer are displayed on the table table How to analyze the data to determine the specific seasonal indices,and what ...
1
vote
0answers
392 views

Kalman filter with sensors having different sampling rate

I would like to implement Kalman filter scheme with two different kind of sensors - say 9DOF IMU (gyro/accelerometer/magnetometer) and GPS sensor (position/velocity/altitude). Problem is, as i ...
1
vote
1answer
180 views

Looking for data science-oriented “Applied Maths for dummies” summer school

I'm a PhD candidate who critically lake of maths skills for his data science projects. I'm looking for a summer school that would enable me to take off in this aspects. This could be ...
0
votes
2answers
167 views

2D data fitting

I have some numbers as a function of 2 variables: $(x, y) \mapsto z$. I would like to know which function $z=z(x,y)$ best fits my data. Unfortunately, I don't have any hint, I mean, there's no ...
2
votes
1answer
995 views

Is there a scientific way to assign weights to historical data?

This is a common question I usually face while processing historical data. I have year on year data of an event for the past N years.I would like to assign weights to the data of these N years so that ...
0
votes
0answers
63 views

Is it possible to get function from bunch of points?

I have some points which I know the position in 3D. and that points made in sequence. for example, . (x1,y1,z1) ...
0
votes
1answer
69 views

matching two curves

I have two curves extracted from some data and I'm trying to detect when there is a difference. Each of these 3 plots represents a different data set. From these example curves below, I'm looking for ...
1
vote
1answer
57 views

How to convert one number range to another? (Prioritizing a list of skills based on current familiarity and usefulness)

I'm trying to come up with a formula to prioritize a list of skills that a learner should tackle based on: current familiarity with the skill, and the usefulness of the skill. Familiarity is a $1$ ...
0
votes
0answers
150 views

Two Part Question Regarding Function Analysis and Encryption

I've been to my professors office hours, and my teaching assistants are not making this clear for me. Encryption is such a great field of study and I really would like to get off on the right foot. ...
2
votes
3answers
99 views

$4\log_2(n)=n$ How to Solve for $n$?

As the title suggests, my log skills are pretty lacking. Need to learn how to get from $4\log_2(n)=n$ to $n=16$ ($\log$ base $2$). I've searched Google and it seems I am missing some core concept ...
0
votes
1answer
42 views

Decision to market one or the other game, based on the expected demand

For example, there is a new video game to market, lets call it Game 1, and there is a 0.60 probability that its competitor will produce a new game similar to it. (This 0.60 may not need to be known ...
1
vote
0answers
51 views

Approximating a probability distribution given multiple datasets

I am beginner in mathematics/statistics and apologise in advance for my faulty use of language. I am working on a problem in statistical genetics. Given a dataset, $D$, I obtained probabilities for ...
1
vote
0answers
19 views

Correct sampling methods for this data set / requirements

I am just looking for a push in the right direction as to what kind of sampling methods I can use to fulfill this set of sampling criteria. I had thought stratified sampling, but I'm not sure if ...
0
votes
4answers
68 views

Finding function for data

I'm in need of a function for this data which occurs in an application I am making. I've been staring at it for hours and I can't figure it out, hope someone here can help me. Here's the data: f(x) = ...
0
votes
1answer
93 views
2
votes
1answer
31 views

Aproximate data with this equation (or linearize the equation)

I have found an equation that describes the behaviour of a phisical system: $$ y=a_1e^{-a_2x} + a_3 + a_4x + a_5e^{{-a_6} / {(1-x)}}$$ Now I have data of that physical system and I want to ...
0
votes
4answers
149 views

Predicting the movement of people on a grid.

Let there be a $30$ by $30$ grid that is there are $900$ tiles.The rows are numbered from 1-30 while the columns are numbered from (i) to (xxx). There are total of $100$ people and each of them is ...
1
vote
3answers
65 views

Separating populations and estimating line-fit parameters

Given a dataset containing two populations, each of which can be described by a linear relationship between two variables in each sample with high R$^2$, how does one separate the two populations (and ...
1
vote
2answers
79 views

Can we express “exactly X” with set theory?

I have a question that I think that is a better fit here than a programming SE as SQL is founded on set theory and my question is more about set theory than about a programming question. Assuming I ...
1
vote
2answers
49 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
1
vote
2answers
65 views

Best way to generalize a distribution of data into smaller subsets?

Suppose I had a data set as follows: 168.95, 176.83, 178.13, 179.61, 179.44, 172.83, 173.37, 174.06, 174.94, 175.43, 175.73, 175.5, 173.78, 174.06, 172.71, 174.3, 178.38, 178.43, 177.18, 175.34, ...
0
votes
2answers
177 views

Function to fit a curve

I am trying to fit a function to data that looks like the following. I have no idea what the function is, but by visual inspection it seems to be a parabola (or some other convex polynomial) + a ...
1
vote
0answers
65 views

Why do candlestick plots seem to show a cyclical structure?

One oddity I notice is that if a random or quasi-random data series such as price data is plotted, it is similar to a random walk, but if the same data series is plotted using candlesticks ...
-1
votes
1answer
247 views

How do you set the range for ListPlot (mathematica)

I have a large set (about 2050 values) of data that I would like to make a graph out of. The data is in two columns in Excel. I copied the first column and made the list x={0, 3.6, 7.1, 10.7...} and ...
3
votes
1answer
62 views

The number of nuts in a package of “Premium Cashews” is normally distributed with a mean of 433 and a standard deviation of 6 nuts.

The number of nuts in a package of "Premium Cashews" is normally distributed with a mean of 433 and a standard deviation of 6 nuts. Packages with fewer than 420 nuts or more than 445 nuts will be ...
2
votes
1answer
35 views

Continuity correction: Change P(2 ≤ x < 9) to continuous?

Convert discrete probability into continuous probability using continuous correction: attempt: Discrete: P(2 ≤ x < 9) therefore continuous should be Continuous: P(1.5 < X < 8.5) Is this ...
1
vote
1answer
32 views

Rating based on 2 criterias

I'm trying to write rating system which scores items based on two criteria with different importance. Criteria_1 will have importance 0.2 and Criteria_2 will have 0.8. Maybe a bit of trickiness adds ...
3
votes
1answer
1k views

How do I 'reverse engineer' the standard deviation?

My problem is fairly concrete and direct. My company loves to do major business decisions based on many reports available on the media. These reports relates how our products are fairing in ...
0
votes
1answer
24 views

How do I calculate the function of an (expected) logarithmic distribution, if I know the area under the curve & one data point?

I'm trying to get the function that lets me determine the value of y, where x is an arbitrary number. In this case, I'm looking at sales charts, and I'm making the assumption that the function is ...
1
vote
0answers
128 views

Significance Tests & Confidence Intervals for Non-normal data

I have 2 nonnormally distributed samples of different sizes (N1~=N2). To evaluate whether there is a significant difference between these samples, I used the Mann_whitney U test (ranksum in MATLAB) ...
1
vote
0answers
29 views

“Sane” quantisation of moderate-dimensional analogue data

Warm-up problem (with solution): Suppose I have a measuring device that spits out a stream of real numbers, which I will model as i.i.d. random variables with range $\mathbb{R}$ and continuous ...
0
votes
1answer
56 views

Large Regression dataset

For a project I need a large regression (least squares) dataset: If $n$ is the number of samples and $p$ the number of features, then I need $p < n$ and $p,n$ both very large. For example ...
1
vote
2answers
90 views

Determining function from simple xy graph

I'm not sure if this question is too vague for here. Will delete if necessary. I have a set of points, (1,782), (2,893), (3,992),... and I'm trying to determine the function that was used to generate ...
1
vote
1answer
126 views

Weird formula for linear regression

I'll try to make the matter as clear as possible given the circumstances. My boss asked me to look at an old report a former employee wrote around a couple of months ago. Apparently the report ...
1
vote
1answer
32 views

Non-close-form Regression Research

As I try to process some physic experiment data that I don't have the closed form formula with unknown parameters, I have to use some regression models like polynomials or normal distributions . The ...
0
votes
1answer
145 views

Hypothesis Testing, simple against composite.

i've got some trouble understanding this exercise from Amemiya's Introduction to statistics: Given the density $f(x) = \frac{1}{\theta}$ from $0 < x < \theta$, and 0 elsewhere, we are to test ...
1
vote
1answer
989 views

Comparing annualised volatility from monthly and annual data

I fear there is a very simple answer to this question and its killing me that I can't see it. I am interested in calculating historical volatility: I have monthly index values starting in Jan 2005 ...
1
vote
0answers
19 views

Isolating clusters in data

This is a problem that I keep encountering in one form or another every so often. Given a large N dimension space with a sufficiently large (M) sample of points. [1] If I were to be asked to find a ...
1
vote
0answers
21 views

grouping of skewed dara

I have a set of data composed of the measurements of spaces. There are 2 distinct sizes represented in this data set. (distinct in that the larger size is larger by some unknown multiple) I am ...