Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

0
votes
0answers
15 views

Cohen's Kappa and MovieLens

I'm tryings to analyse the MoveLens 20m data set and I'm interested at looking at user rating agreement levels. Someone has suggested I look at Cohen's Kappa as this kind of inter-rater agreement is ...
0
votes
1answer
25 views

What would be a good algorithm to determine the busiest servers by cpu usage

I have a huge data set that contains time series data for some servers' cpu usage per application. I need to find the busiest servers to for the application and print it. I need to find a method or ...
1
vote
0answers
21 views

Penalty function of multi-peak fit?

The question I have is about the answer from here by @Silvia: http://mathematica.stackexchange.com/questions/26336/how-to-perform-a-multi-peak-fitting I can only understand some of the code but the ...
1
vote
2answers
64 views

What is the order of this algorithm?

I was messing around with a method of manipulating strings in Java when I came across this algorithm. For a string of length n: $$\{1, 2, ... ,n - 1, n\}$$ rearrange it such that the first character ...
0
votes
1answer
41 views

Create a modal line or benchmark

I have some data the following, X is cycle in percentage, and Y is the quantity: ...
2
votes
1answer
40 views

Mean of values ${1 \over m} {\sum_{i=1}^m {s_i \over t_i}}$ vs. Value of sums ${\sum_{i=1}^m {s_i} \over \sum_{i=1}^m {t_i}}$, What is the relation?

Similar to: Average of products VS. product of averages I have $$ R = \{(s_1, t_1), (s_2, t_2), ..., (s_n, t_n)\} $$ With the property that, for all pairs $(s_i, t_i)$, $0 \le s_i < t_i$. $$ \...
0
votes
1answer
56 views

Runge-Kutta (Fehlberg) with multiple time dependant variables and adaptive time steps

I've searched the internet for weeks for a step in the right direction with this. I'm trying to solve a first order ODE- not too difficult I know, and I am capable thus far of implementing the Runge ...
2
votes
1answer
55 views

Distance between unequal-dimension vectors (and data)?

It is easy to find simple distance measures for equal-dimension vectors, such as Euclidean Distance or Correlation. What about unequal-dimension vectors, such as, for instance, $(a,b,c)$ and $(d,e)$? ...
2
votes
1answer
23 views

Extrapolate data points from a series of averages

Given a series of data points: $$\begin{array}{c|c|c|c|c|c|c|c|} & \text{Monday} & \text{Tuesday} & \text{Wednesday} & \text{Thursday} & \text{Friday} & \text{Saturday} &...
0
votes
0answers
31 views

How to infer information from a cumulative frequency graph, and what it means practically

So I'm given a table of data, regarding "An estimator working for a large construction company, is preparing a tender and trying to establish a rate for reinforcing bars in a concrete floor. Data ...
0
votes
0answers
21 views

Is the sum of all cross-correlation samples representative of target existence likelihood?

Answers to this question take the peak in the cross correlation as the measure to the likelihood of the trigger signal exist in the received signal - this is pretty much text book. My question is ...
1
vote
1answer
31 views

Find errored member in a dataset

Assuming a simple data-set looking like this dataset As you can see there is 10 lines, each one representing an entity. For each entity, there is an amount of downloaded and uploaded data which are ...
0
votes
1answer
29 views

How to profile people using clustering

I have a data of customers and i want to split them to segment (profiling). The columns of the data are Amount-Spending, Amount-Bonus, Age, Churn-or-Not. So i clustered my data with k-means. to 3 ...
0
votes
0answers
23 views

What is the algorithm for training and testing a logistic regression model using Newton's method and l2 regularization?

I have a spam dataset with 57 features and 3065 data points. I also have test data with around 1500 data points. The classes are spam/non-spam. I have to fit a logistic regression model on MatLab ...
0
votes
0answers
36 views

How to obtain the best fit?

I have a complex function say $f(x,a,b,c)$ where $x$ is variable and $a,b,c$ are the parameters. Parameters $a,b$ are linked as $d = (1/a^2) - (i*pi/b)$. The limits of x is very small say -0.02 to 0....
0
votes
0answers
17 views

Choosing between several methods to replace bad or missing data.

We are doing some quality control on data coming in from stations all over the world. Part of the quality control involves testing the data on several different things like plausibility, etc. If a ...
0
votes
0answers
18 views

time series regression data set

I am trying to find a data set to which I can apply a regression time series model. So I need a data set where the regressors(predictors) of the model are also time series themselves. where can I ...
0
votes
1answer
17 views

Formula that devides date(given in millisecounds)

Just to clarify at the offset, i am a javascript developer and this is a Math question , so my question is as follows: suppose i create to dates in javascript like so: var d1 = new Date(2005,12,4);...
0
votes
1answer
35 views

What is the probability of having the same (binary) datasets?

Suppose we have $m$ binary data points as an outcome of a specific experiment, so the outcomes of those points are fixed. We save our data in a file of $m$ points having value $0$ or $1$. (for example ...
0
votes
0answers
24 views

Finding the weighted mean with varying errors

Simply enough, I've a set of numbers with varying errors: 2.02 +- 0.10 , 2.26 +-0.30 , 3.24 +- 0.30 , 3.33 +- 0.30 , 3.92 +- 0.30 , 4.02 +- 0.10 I'm certain there's a simple formula to compute the ...
0
votes
1answer
21 views

How to find a data value using only standard deviation?

I have a homework problem with the following data values: x, 17, 24, 13, 21. I need to find x, but I do not have the mean. I only have the standard deviation, which is square root of 14, or 3.745. How ...
7
votes
5answers
376 views

Finding the orientation of a noisy ellipse

This question comes from a neuroscience study which generates $12$ vectors. The vectors are evenly spaced, $30 n$ degrees for $n=0,\dots, 11$, each with their tail centered on the origin. I am ...
0
votes
0answers
18 views

Can't get figure out how to get coordinates back in MDS (multidimensional scaling)

I have a 3,5 matrix $X$. I have 5 items in 3 dimensional space. Let $K= X^TX$. $D$ is the squared distance between any 2 points. (So it is a $5x5$ matrix). $H$ = $I - \frac{1}{5} *ones(5,5)$ $B = ...
3
votes
2answers
65 views

What is the chance of a sample in white noise to be a local extremum?

I draw a long sequence of values from a uniform or normal distribution. Observing the resulting sequence, what is the chance of any point in the sequence (aside from the first and last where it is not ...
1
vote
0answers
40 views

Need help understanding the proof of the Johnson-Lindenstrauss lemma by Dasgupta and Gupta

The Johnson-Lindenstauss lemma is the following: Let $\varepsilon\in(0,1)$, $\mathcal{P}\subseteq\mathbb{R}^d$ be a finite subset and $k\in\mathcal{O}(\varepsilon^-2\log{n})$. Then there exists a (...
0
votes
0answers
12 views

Testing Relative Preference Fit (multi-variable)

I'm playing around with a data set that I have built which is meant to represent the subjective preferences of a person related to a particular product. I want to compare, then, the properties of a ...
0
votes
0answers
38 views

Quantify difference between regularity and irregularity

I am solving an equation numerically on a 1D-domain using the finite-element method. I am solving it using two different domains, one regular and one irregular. Naturally, the solution varies slightly ...
2
votes
3answers
53 views

pdf of sum of two dependent random variables

What is the pdf of sum of two dependent random variables given we know their joint pdf and individual pdfs. I have seen already some posts but none of them answered when they are dependent. Every one ...
0
votes
1answer
27 views

Polynomial Fitting of Circular Data Object

This is a very odd question. I have a one dimensional data set that is graphed on a histogram. I am trying to curve fit this data set (using the class midpoints as the x values, and the frequencies as ...
0
votes
0answers
22 views

Is EM-algorithm only for missing data?

Currently studying EM algorithm and have been through a few articles, they all say it is for missing data. I believe there is some implication in the term "missing data". I wonder if EM is designed ...
2
votes
2answers
67 views

Plain English interpretation needed for the sentence to understand EM-algorithm?

I am trying to read an EM-algorithm article on the web, however, as soon as I started I have face a sentence interpretation problem with this like "... in the presence of missing or hidden data" in ...
1
vote
0answers
90 views

What is the relationship or difference between MLE and EM algorithm?

I am trying to study EM algorithm and Maximum Likelihood Estimation. Somehow, they both sound the same to me but can't really say the difference. Maybe I don't really understand any of them. I have ...
0
votes
0answers
42 views

Qualitative comparison of parameterized sigmoid function (logit)

Having the logistic function to describe a 2 Alternative Forced Choice experiment: $$f(x) = \frac{1}{1+e^{(a+bx)}}$$ which from the results, I'm getting different sigmoids with different steepness. ...
0
votes
1answer
21 views

Analysing data to find the principal parameter

I have a data set $N\times M$, which contains information about motorcycles: $N$ motorcycles have been sold during some time and for each bike there is $M$ parameters regarding the sale such as price, ...
0
votes
0answers
12 views

Statistical significance of deviation from a complex-valued model

I have complex-valued data. At each one of about 100 linearly spaced $x$ values, I have a corresponding measurement of a complex quantity with well-defined Gaussian uncertainties on both the real and ...
0
votes
0answers
43 views

Distribution of the test statistic?

Let $\mathbf{x}_i \sim \mathcal{N}(\boldsymbol\mu, \boldsymbol\Sigma)$. I am trying to find a distribution of the following test statistic $ T(\mathbf{x}) = \frac{\bar{\mathbf{x}}^T \mathbf{S}_\...
0
votes
0answers
20 views

Analysis of rank/likert scale data

If age groups/gender represents independent variable(s) and responses have been collected for 5-6 items in the form of ranking (most preferred - rank 1 to least preferred rank - 5). Which test is an ...
1
vote
0answers
32 views

Markov-Chain Monte-Carlo: Are transformations on the inputs valid?

The problem: I am trying to solve a high dimensional (up to ~50) class of data fitting & modelling problems. The user specifies the problem, so I would like to make the configuration as easy as ...
2
votes
2answers
107 views

weighing avg by columns vs rows outputs different results

First of all, I just know the basics of math, so please be patient. I have an overall score for a company with the product1 and product2. However when I do the overall for each criteria like ...
0
votes
1answer
37 views

How do I find if a series of integers is (somehow) regular

[DISCLAMIER]: please be patient and low profile: no math background here :) I have a database of transactions, i.e. a list of purchases made by different customers. What I have to figure out is ...
0
votes
1answer
50 views

Curvature of a 3D trajectory for which I know data points

In order to simulate an airplane model, I need to change its orientation knowing the curvature of its trajectory. The simulator gives me the plane position, so in order to perform my orientation ...
1
vote
0answers
72 views

Interpolation at “extreme values.”

I am working with a meteorologist on a project. We are pulling data from METAR Observation stations on several variables (such as temperature, dew point, wind speed, etc.) throughout time. ...
1
vote
1answer
42 views

Non-Whole Median Numbers in Real Data [duplicate]

According to this CDC report, the median number of reported sexual partners for females aged 15-44 is 3.2, and for males 5.1. Tables on pages 19 and 20 report these statistics for a variety of ...
2
votes
2answers
86 views

Outlier detection with robust multiple regression model

I have a set of features (eg, location, income, budget, education) that I use to predict a continuous variable (say, amount spent per day on the internet). I am interested in detecting outliers. I ...
1
vote
0answers
42 views

Matlab, Extracting Particular Values from an Excel File

If I have a table of values in Matlab ( here I have a 40k rows, and about a dozen columns), how can I read only particular values from each row? My goal is to extract some of the values from each row, ...
0
votes
0answers
25 views

intersection of 2 paths

Could somebody please advise me on the best approach to take to find the intersection points of 2 paths. Each path is a data set of distance against time (I want to avoid curve fitting) and each path ...
1
vote
0answers
23 views

Averaging Models in Excel

I am working in Excel and have two sets of data points that come up with a percentage for the likelihood of a recession. I am wondering what the best strategy would be to combine these data points? ...
3
votes
1answer
77 views

Calculate trend and represent in text?

First off, I'm terrible at math. I'm writing a script that monitors transactions from clients daily over a 7 day period. Given a set of numbers like below, I would like to calculate a trend and ...
0
votes
1answer
27 views

How to convert table into a distance function?

Been stumped on this past paper question for a while, it's in the context of clustering and the next part is using single linkage bottom-up hierarchical clustering to form a dendrogram using your ...
0
votes
1answer
43 views

How can I maximize the sum of y values across a given number of graphs?

I have any number of graphs. Each graph is a plot of points, not necessarily an equation. I need to pick a point on each graph in way that the following is true: The sum of the y values of each ...