Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

0
votes
0answers
25 views

Fitting discrete distribution (negative binomial) for early data values

I'm having some difficulties with fitting a discrete distribution function (I'm specifically using the negative binomial distribution). Here's my setting: I have a source of incoming items, each with ...
0
votes
4answers
128 views

Predicting the movement of people on a grid.

Let there be a $30$ by $30$ grid that is there are $900$ tiles.The rows are numbered from 1-30 while the columns are numbered from (i) to (xxx). There are total of $100$ people and each of them is ...
1
vote
3answers
41 views

Separating populations and estimating line-fit parameters

Given a dataset containing two populations, each of which can be described by a linear relationship between two variables in each sample with high R$^2$, how does one separate the two populations (and ...
1
vote
2answers
63 views

Can we express “exactly X” with set theory?

I have a question that I think that is a better fit here than a programming SE as SQL is founded on set theory and my question is more about set theory than about a programming question. Assuming I ...
1
vote
2answers
45 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
1
vote
2answers
42 views

Best way to generalize a distribution of data into smaller subsets?

Suppose I had a data set as follows: 168.95, 176.83, 178.13, 179.61, 179.44, 172.83, 173.37, 174.06, 174.94, 175.43, 175.73, 175.5, 173.78, 174.06, 172.71, 174.3, 178.38, 178.43, 177.18, 175.34, ...
0
votes
2answers
90 views

Function to fit a curve

I am trying to fit a function to data that looks like the following. I have no idea what the function is, but by visual inspection it seems to be a parabola (or some other convex polynomial) + a ...
1
vote
0answers
45 views

Why do candlestick plots seem to show a cyclical structure?

One oddity I notice is that if a random or quasi-random data series such as price data is plotted, it is similar to a random walk, but if the same data series is plotted using candlesticks ...
-1
votes
1answer
121 views

How do you set the range for ListPlot (mathematica)

I have a large set (about 2050 values) of data that I would like to make a graph out of. The data is in two columns in Excel. I copied the first column and made the list x={0, 3.6, 7.1, 10.7...} and ...
3
votes
1answer
52 views

The number of nuts in a package of “Premium Cashews” is normally distributed with a mean of 433 and a standard deviation of 6 nuts.

The number of nuts in a package of "Premium Cashews" is normally distributed with a mean of 433 and a standard deviation of 6 nuts. Packages with fewer than 420 nuts or more than 445 nuts will be ...
1
vote
1answer
25 views

Continuity correction: Change P(2 ≤ x < 9) to continuous?

Convert discrete probability into continuous probability using continuous correction: attempt: Discrete: P(2 ≤ x < 9) therefore continuous should be Continuous: P(1.5 < X < 8.5) Is this ...
1
vote
1answer
21 views

Rating based on 2 criterias

I'm trying to write rating system which scores items based on two criteria with different importance. Criteria_1 will have importance 0.2 and Criteria_2 will have 0.8. Maybe a bit of trickiness adds ...
0
votes
0answers
13 views

Computing a threshold value for paired data

I have a dataset of vector pairs; for each pair ($x_1$, $x_2$) I have two values $V$($x_1$,$x_2$) and $U$($x_1$,$x_2$) (two metrics). I also know a threshold value $U^*$ such that any pair ($x,y$) ...
3
votes
1answer
201 views

How do I 'reverse engineer' the standard deviation?

My problem is fairly concrete and direct. My company loves to do major business decisions based on many reports available on the media. These reports relates how our products are fairing in ...
0
votes
1answer
15 views

How do I calculate the function of an (expected) logarithmic distribution, if I know the area under the curve & one data point?

I'm trying to get the function that lets me determine the value of y, where x is an arbitrary number. In this case, I'm looking at sales charts, and I'm making the assumption that the function is ...
0
votes
0answers
12 views

Likelihood function of truncated data

I am having a little trouble understanding the concept and derivation of the likelihood of truncated data. For example, if I want to find the likelihood function based on a sample from a ...
1
vote
0answers
64 views

Significance Tests & Confidence Intervals for Non-normal data

I have 2 nonnormally distributed samples of different sizes (N1~=N2). To evaluate whether there is a significant difference between these samples, I used the Mann_whitney U test (ranksum in MATLAB) ...
1
vote
0answers
22 views

“Sane” quantisation of moderate-dimensional analogue data

Warm-up problem (with solution): Suppose I have a measuring device that spits out a stream of real numbers, which I will model as i.i.d. random variables with range $\mathbb{R}$ and continuous ...
0
votes
1answer
48 views

Large Regression dataset

For a project I need a large regression (least squares) dataset: If $n$ is the number of samples and $p$ the number of features, then I need $p < n$ and $p,n$ both very large. For example ...
1
vote
2answers
77 views

Determining function from simple xy graph

I'm not sure if this question is too vague for here. Will delete if necessary. I have a set of points, (1,782), (2,893), (3,992),... and I'm trying to determine the function that was used to generate ...
0
votes
0answers
20 views

Average data output from randomized data input?

Apologies about the title, I am not sure exactly how to word it. Currently I am receiving data from a Bluetooth Low Energy peripheral (RSSI in dB). Even if both the receiving device and the ...
1
vote
1answer
66 views

Weird formula for linear regression

I'll try to make the matter as clear as possible given the circumstances. My boss asked me to look at an old report a former employee wrote around a couple of months ago. Apparently the report ...
1
vote
1answer
27 views

Non-close-form Regression Research

As I try to process some physic experiment data that I don't have the closed form formula with unknown parameters, I have to use some regression models like polynomials or normal distributions . The ...
0
votes
1answer
98 views

Hypothesis Testing, simple against composite.

i've got some trouble understanding this exercise from Amemiya's Introduction to statistics: Given the density $f(x) = \frac{1}{\theta}$ from $0 < x < \theta$, and 0 elsewhere, we are to test ...
0
votes
1answer
306 views

Comparing annualised volatility from monthly and annual data

I fear there is a very simple answer to this question and its killing me that I can't see it. I am interested in calculating historical volatility: I have monthly index values starting in Jan 2005 ...
0
votes
0answers
14 views

In classification algorithms what is the idea behing choosing the first n values?

I came across the cluster pruning algorithm the other day. The algorithm first chooses √N random leaders for N points and after that assigns the new points to the nearest leaders. What is the idea ...
1
vote
0answers
13 views

Isolating clusters in data

This is a problem that I keep encountering in one form or another every so often. Given a large N dimension space with a sufficiently large (M) sample of points. [1] If I were to be asked to find a ...
1
vote
0answers
7 views

grouping of skewed dara

I have a set of data composed of the measurements of spaces. There are 2 distinct sizes represented in this data set. (distinct in that the larger size is larger by some unknown multiple) I am ...
1
vote
1answer
127 views

Mathematical model building with dependent and independent variables

I have been working with data and building models on data. I have developed models in regression using cubic and power series. It works fine for variables with one dependent and one independent ...
0
votes
1answer
30 views

Rescaling Dataset by a constant value

I have two sets of measurements that measure one thing. I would like to graph the relative heights of the measurements, however one set is nearly consistently higher than the other one. I would like ...
0
votes
1answer
48 views

Experimental Data Analysis

Suppose I have performed 2 different experiments to measure the number of cell in a mice. In the first experiment, we measure the cell in odd day (day 5,7,9,11,13) and in the second, we measure the ...
1
vote
0answers
19 views

mutual information for power low distributed data?

I have a dataset with power-low distribution, I would like to measure some kind of correlation/mutual information between the data to its class,(for feature selection task), Can I use ...
2
votes
0answers
153 views

curve fit but unknown function

I face a strange problem. I have data $[X,Y]$ with no noise. If I plot $Y$ as a function of $X$, it looks like a straight line and the regression coefficient is quite good. For the simplest model ...
1
vote
1answer
80 views

Can you please help me model this curve?

I have no idea how to model this curve. Can someone please help?
0
votes
2answers
161 views

A simple GRE data-analysis question - but I dont know what it is asking

It is a GRE data analysis question. I don't if I am dumb in Math, or in English. But I could not understand what is the question a want me to answer. Please give me a hint.
0
votes
1answer
87 views

compering distances between points in different dimension

I have pairs of points, each pair is in the same dimension, and I need to measure the distance between each pair: For example: ...
1
vote
1answer
56 views

Given two arrays, how can one determine the the intersection?

I have two datasets (A & B). They each have 1000 numbers. 99% of the time: A < x <= B However, 1% of the time B < x < A. How can I solve for x, ...
0
votes
0answers
41 views

How to search for an object in more efficient way?

I have a doubt how to check if structure contains an object. Simple sketch of my structure: Root - is a group which can contain groups, keys, values. Group - can contain groups keys, values. Key - ...
4
votes
2answers
175 views

Detecting increasing pulse trains

I have a one dimensional point process representing the times of events which is also mixed in with lots of data that I regard as noise. The interval between the events in the point process are ...
0
votes
1answer
44 views

Type of Distribution for Survey Answers 1 to 10

I'm investigating survey data where participants are asked to answer based on a scale of 1 to 10 typically, the average is around 8.5 with most responses between 7-10, with a lower numbers like 3 ...
1
vote
2answers
200 views

Least squares fit with a trick

Fitting a circle with least squares is easy once you get the trick for $c = r^2 - a^2 -b^2$ and you got a linear set of equations. my problem is as follows: $$z = \alpha x + \beta y + \alpha \beta$$ ...
1
vote
1answer
34 views

How can I mathematically determine the next available index for a child row in this data set?

I have a data set that looks like this: $$ \begin{array}{c|lcr} & 0 & 1 & 2&3&4&5&6&7&8&9 \\ \hline parent & x & x & x & x & ...
1
vote
0answers
263 views

Can you compare data sets with percentages with the Chi Square test?

I’m confused about the Chi Square test.. Apparently you have to use it when comparing frequencies only, but what if both data sets are percentages? e.g. comparing the percentage of smokers in specific ...
0
votes
1answer
26 views

data interpretation

let us consider following picture and question: INCOME: 1. Money from fund raising programmes, 2. Grant from the government, 3. Contributions from individuals, 4. Contributions from corporations, ...
2
votes
1answer
50 views

Optimize database indexes, the sequel

Background For those not familiar with databases. Indices help to speed up database searches, but they come at the cost of memory. Since you like your database to be fast, they are best cached in RAM ...
0
votes
2answers
46 views

Optimize database indexes

Q: I have 14 columns, how many indexes do i need to create to cover all possibilities? Examples of possibilities: col1 col12 col5, col3, col4 col7, col2, col12, col1 all 14 columns -- Order can ...
2
votes
0answers
96 views

The correct probability distribution / way to analyze daily changes

I am working on a report which is being sent through to end users that should flag to them any "large changes" in the day-to-day values for the past 30 days for something we would assume the ...
1
vote
1answer
271 views

Mutual Information for clustering

I'm working on a document clustering application and decided to use Normalized Mutual Information as one of the measures of effectivenes. But I don't really understand how to implement this in that ...
1
vote
3answers
145 views

Testing for chaos in data

I have data for 3 variables ,each with respect to the discrete time values. How do I check for the existence of chaos for this 3D discrete system?(I don't have the analytic eqs.,just the data). MY ...
0
votes
1answer
67 views

How to normalize these numbers for better visualization?

My dataset is like this: a1 4565380 a2 676477 a3 359939 ... b1 222431 b2 12222 ... g1 139 ... h1 134 i1 10 j1 11 and goes on.. The problem is when I ...