Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

2
votes
1answer
38 views

Fitting a curve given points

My set-up is the following, I have two variables $N$ and $TTR$, and I have these points for each variable: $N$ = [35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50] $TTR$ = [0.818, ...
0
votes
0answers
11 views

Distribution for frequency data

I am conducting statistical tests on a large set of frequency data. The frequency data is in the form of times of occurrence of events at a number of locations. Locations can be assumed to be ...
0
votes
1answer
29 views

Finding the number that's less than 90% of result set

Forgive if this sounds vague or poorly worded. I'm struggling to pinpoint the right terminology of what I'm looking to do. Hoping someone here can push me in the right direction. So I have a set of ...
0
votes
1answer
65 views

Determine equation from graph

Background: I'm working on a script to read/parse a file generated by a piece of software I use to create music mixes. One aspect I'm having difficulty with is translating the volume value from it's ...
0
votes
0answers
36 views

Calculating Average “Productivity”

Let me apologise in advance if my terminology is incorrect or if what seems logical to me is completely nonsensical. I am an analyst without a mathematical background and have tried to self teach off ...
1
vote
2answers
33 views

How sensitive is a natural cubic spline?

I am interested in using natural cubic splines to generate possible replacement values in the quality control of data. I would like to do this as close to real-time as I can. That is, I would like ...
1
vote
1answer
27 views

How to describe a statistical dataset more precisely?

I am a newbie in stat and I need to be able to do this for a GIS application: Say I have the following dataset of only two possible values $0$ and $1$: $0,0,0,1,1,1,0,1$ The mean would equal ...
0
votes
0answers
15 views

Is data normally distributed over attributes or need to be normalized ( scale )

I have been spend lot of time related to my data distribution, for the following matrix, there are need to cluster similar factories as follow : - each factory made some products for example : factory ...
1
vote
1answer
86 views

On normalized error measures

I have function values $f_1,\ldots,f_n$ that are approximated by data $y_1,\ldots,y_n$. I am looking for a measure that describes the error in the data $y_1,\ldots,y_n$ and I want the measure to take ...
0
votes
0answers
14 views

replicating a probability model for classification

I'm trying to replicate a probability model for a binary classification based on some data. Until now I been able to replicate the left panel, the histogram and the gaussians curves, but I dont know ...
0
votes
0answers
9 views

Building agglomerative hierarchical clustering algorithm manually

I am currently trying to solve this problem but I'm quite unsure if my answer is correct. I've build the tree different types of measures, MIN, MAX and AVERAGE but I might have some of them wrong. Can ...
0
votes
0answers
11 views

Complementation Rule for MVDs

I have that $X \rightarrow \rightarrow Y$ is a Multivalued Dependency in $R$ and let $Z=R-(X \cup Y)$ With $t_{1}[x]=t_{2}[x]=t_{3}[x]=t_{4}[x]$ $t_{1}[y]=t_{3}[y]$ and $t_{2}[y]=t_{4}[y]$ ...
0
votes
0answers
11 views

Comparing Temperature Curves

I'm collecting a bunch of temperature data from water cool down tests and need some advice on how to best compare data from different tests. I have a test stand and as I change test variables between ...
0
votes
0answers
13 views

Bayesian model to estimate the parameter of a Bernoulli law

Suppose we have iid boolean variables $X_1,...,X_T = X_{1:T}$ and the associated deterministic parameters $k_1,...,k_T=k_{1:T}$ and $c_1,...,c_T=c_{1:T}$, where for each $t \in \mathbb{N}$, $k_{t} \in ...
1
vote
1answer
52 views

Stem and Leaf Plot for Strongly Skewed Data

How to draw a stem and leaf plot for the following large data, the data is for different EU states so there is a large difference between the data. I have to plot stem and leaf for each table i.e. ...
1
vote
1answer
50 views

File Compression Statistics

Statically, how is the compression of a file possible? As in, a file n bytes long can have $2^{n*8}$ (the 'times 8' part is because there are 8 bits in a byte). So, if you decrease the size then ...
0
votes
0answers
19 views

Data Management Frequency Interval

I am given the results of 30 students for a Data Management test: $78, 81, 55, 60, 65, 86, 44, 90, 59, 81, 77, 71, 65, 39, 80, 72, 70, 64, 68, 78, 88, 73, 61, 70, 75, 96, 51, 73, 65, 67.$ The ...
0
votes
0answers
22 views

Ordinal scale: does aggregating data by arithmetic mean is a concrete problem?

Sometimes when people have some ordinal values, they use a misleading number representation. Then, they aggregate data with arithmetic mean. For example, we have to aggregate the grades of an exam ...
0
votes
0answers
12 views

Introducing weights into spectral clustering

Suppose I have a data set with points $x_i$ and a dissimilarity measure $d_{ij}$ between each pair, as well as a weight $w_{ij}$ that qualifies the quality of this dissimilarity. I have two problems: ...
0
votes
0answers
46 views

How to do Modified Mann Kendall test for trend analysis?

In mann Kendal trend analysis if auto-correlation in the time series exist we do pre-whitening to remove the affect of lag-1 correlation. However, it has been reported that pre-whitening reduces the ...
0
votes
0answers
14 views

Find repeated attribute patterns in collection of objects

I have objects $o_1,...,o_k$where every object has a set of properties $p_1,...,p_j$. I'm looking for property patterns that are repeated among these objects. It could be perhaps called correlation of ...
0
votes
0answers
18 views

Ortogonal polynomial regression

I need to fit a weighted set of points with orthogonal polynomials based on least squares polynomials. The grid on X is nonuniform, so Chebyshev polynomials can't be applied. The data points are like ...
1
vote
1answer
64 views

Cauchy Schwarz inequality in Normalized Cross Correlation

I'm currently using a normalized cross correlation(NCC) for measure the degree of similarity between two image. Almost two week studying about how NCC is derived from Cauchy Schwarz inequality but ...
0
votes
0answers
14 views

Cohen's Kappa and MovieLens

I'm tryings to analyse the MoveLens 20m data set and I'm interested at looking at user rating agreement levels. Someone has suggested I look at Cohen's Kappa as this kind of inter-rater agreement is ...
0
votes
1answer
22 views

What would be a good algorithm to determine the busiest servers by cpu usage

I have a huge data set that contains time series data for some servers' cpu usage per application. I need to find the busiest servers to for the application and print it. I need to find a method or ...
1
vote
0answers
17 views

Penalty function of multi-peak fit?

The question I have is about the answer from here by @Silvia: http://mathematica.stackexchange.com/questions/26336/how-to-perform-a-multi-peak-fitting I can only understand some of the code but the ...
1
vote
2answers
63 views

What is the order of this algorithm?

I was messing around with a method of manipulating strings in Java when I came across this algorithm. For a string of length n: $$\{1, 2, ... ,n - 1, n\}$$ rearrange it such that the first character ...
0
votes
1answer
41 views

Create a modal line or benchmark

I have some data the following, X is cycle in percentage, and Y is the quantity: ...
2
votes
1answer
39 views

Mean of values ${1 \over m} {\sum_{i=1}^m {s_i \over t_i}}$ vs. Value of sums ${\sum_{i=1}^m {s_i} \over \sum_{i=1}^m {t_i}}$, What is the relation?

Similar to: Average of products VS. product of averages I have $$ R = \{(s_1, t_1), (s_2, t_2), ..., (s_n, t_n)\} $$ With the property that, for all pairs $(s_i, t_i)$, $0 \le s_i < t_i$. $$ ...
0
votes
1answer
44 views

Runge-Kutta (Fehlberg) with multiple time dependant variables and adaptive time steps

I've searched the internet for weeks for a step in the right direction with this. I'm trying to solve a first order ODE- not too difficult I know, and I am capable thus far of implementing the Runge ...
2
votes
1answer
50 views

Distance between unequal-dimension vectors (and data)?

It is easy to find simple distance measures for equal-dimension vectors, such as Euclidean Distance or Correlation. What about unequal-dimension vectors, such as, for instance, $(a,b,c)$ and $(d,e)$? ...
2
votes
1answer
22 views

Extrapolate data points from a series of averages

Given a series of data points: $$\begin{array}{c|c|c|c|c|c|c|c|} & \text{Monday} & \text{Tuesday} & \text{Wednesday} & \text{Thursday} & \text{Friday} & \text{Saturday} ...
0
votes
0answers
27 views

How to infer information from a cumulative frequency graph, and what it means practically

So I'm given a table of data, regarding "An estimator working for a large construction company, is preparing a tender and trying to establish a rate for reinforcing bars in a concrete floor. Data ...
0
votes
0answers
19 views

Is the sum of all cross-correlation samples representative of target existence likelihood?

Answers to this question take the peak in the cross correlation as the measure to the likelihood of the trigger signal exist in the received signal - this is pretty much text book. My question is ...
1
vote
1answer
31 views

Find errored member in a dataset

Assuming a simple data-set looking like this dataset As you can see there is 10 lines, each one representing an entity. For each entity, there is an amount of downloaded and uploaded data which are ...
0
votes
1answer
27 views

How to profile people using clustering

I have a data of customers and i want to split them to segment (profiling). The columns of the data are Amount-Spending, Amount-Bonus, Age, Churn-or-Not. So i clustered my data with k-means. to 3 ...
0
votes
0answers
18 views

What is the algorithm for training and testing a logistic regression model using Newton's method and l2 regularization?

I have a spam dataset with 57 features and 3065 data points. I also have test data with around 1500 data points. The classes are spam/non-spam. I have to fit a logistic regression model on MatLab ...
0
votes
0answers
36 views

How to obtain the best fit?

I have a complex function say $f(x,a,b,c)$ where $x$ is variable and $a,b,c$ are the parameters. Parameters $a,b$ are linked as $d = (1/a^2) - (i*pi/b)$. The limits of x is very small say -0.02 to ...
0
votes
0answers
13 views

Choosing between several methods to replace bad or missing data.

We are doing some quality control on data coming in from stations all over the world. Part of the quality control involves testing the data on several different things like plausibility, etc. If a ...
0
votes
0answers
16 views

time series regression data set

I am trying to find a data set to which I can apply a regression time series model. So I need a data set where the regressors(predictors) of the model are also time series themselves. where can I ...
0
votes
1answer
17 views

Formula that devides date(given in millisecounds)

Just to clarify at the offset, i am a javascript developer and this is a Math question , so my question is as follows: suppose i create to dates in javascript like so: var d1 = new ...
0
votes
1answer
35 views

What is the probability of having the same (binary) datasets?

Suppose we have $m$ binary data points as an outcome of a specific experiment, so the outcomes of those points are fixed. We save our data in a file of $m$ points having value $0$ or $1$. (for example ...
0
votes
0answers
23 views

Finding the weighted mean with varying errors

Simply enough, I've a set of numbers with varying errors: 2.02 +- 0.10 , 2.26 +-0.30 , 3.24 +- 0.30 , 3.33 +- 0.30 , 3.92 +- 0.30 , 4.02 +- 0.10 I'm certain there's a simple formula to compute the ...
0
votes
1answer
21 views

How to find a data value using only standard deviation?

I have a homework problem with the following data values: x, 17, 24, 13, 21. I need to find x, but I do not have the mean. I only have the standard deviation, which is square root of 14, or 3.745. How ...
7
votes
5answers
357 views

Finding the orientation of a noisy ellipse

This question comes from a neuroscience study which generates $12$ vectors. The vectors are evenly spaced, $30 n$ degrees for $n=0,\dots, 11$, each with their tail centered on the origin. I am ...
0
votes
0answers
17 views

Can't get figure out how to get coordinates back in MDS (multidimensional scaling)

I have a 3,5 matrix $X$. I have 5 items in 3 dimensional space. Let $K= X^TX$. $D$ is the squared distance between any 2 points. (So it is a $5x5$ matrix). $H$ = $I - \frac{1}{5} *ones(5,5)$ $B = ...
3
votes
2answers
64 views

What is the chance of a sample in white noise to be a local extremum?

I draw a long sequence of values from a uniform or normal distribution. Observing the resulting sequence, what is the chance of any point in the sequence (aside from the first and last where it is not ...
1
vote
0answers
37 views

Need help understanding the proof of the Johnson-Lindenstrauss lemma by Dasgupta and Gupta

The Johnson-Lindenstauss lemma is the following: Let $\varepsilon\in(0,1)$, $\mathcal{P}\subseteq\mathbb{R}^d$ be a finite subset and $k\in\mathcal{O}(\varepsilon^-2\log{n})$. Then there exists a ...
0
votes
0answers
12 views

Testing Relative Preference Fit (multi-variable)

I'm playing around with a data set that I have built which is meant to represent the subjective preferences of a person related to a particular product. I want to compare, then, the properties of a ...
0
votes
0answers
34 views

Quantify difference between regularity and irregularity

I am solving an equation numerically on a 1D-domain using the finite-element method. I am solving it using two different domains, one regular and one irregular. Naturally, the solution varies slightly ...