Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

0
votes
0answers
15 views

What is correlation kernel and compare with gaussian kernel

I read a paper that said about correlation kernel that defined: $$W(x-y)=(α/1+d(|y − x|))$$ where $α =  (\int(1+d(y − x)dy)^{-1}$, $(d(|y − x|))$ is spatial Euclidean distance from the central ...
0
votes
0answers
12 views

Get a set of points from Fourier plot on Matlab?

I have a program that collects raw data and also does a Fourier transform to get complex numbers. When I feed the raw data into Matlab to do its own FFT, I can get a nice plot of power on frequency. ...
0
votes
0answers
8 views

Is there any common formula to estimate the salience of peaks of a histogram?

I need to use some quantitative measure to estimate the salience of peaks and the flatness of valleys of a histogram which represents the distribution of lightness of an image. For example, if an ...
0
votes
0answers
7 views

One-class Support Vector Machine Sensitivity Drops when the number of training sample increase

I am using One-Class SVM for outlier detections. It appears that as the number of training samples increases, the sensitivity TP/(TP+FN) of One-Class SVM detection result drops, and classification ...
1
vote
1answer
15 views

How to determine who improved the most considering diminishing returns

My friends and I are competing in a fitness challenge against each other. We will determine the winner based on who has the greatest percentage improvement after three months. Here is the problem: if ...
0
votes
1answer
25 views

Method for detecting anomalies in timeseries?

I am exploring creating alerts for anomalies in a timeseries. Say that I have a graph like this: There are two anomalies here that I would wanted to be alerted on: First, on April 7th with there ...
0
votes
0answers
40 views

How to interpret the indicator function?

I am reviewing a paper titled " Bayesian Sampling Approach to Decision Fusion" by Biao Chen and Pramod K Varshney. This paper uses an indicator function that I am not being able understand. The ...
0
votes
0answers
27 views

What type of equation matches these data points?

Can anyone tell me what equation will match these data points? 377, 384, 1113, 1120, 19861, 19868 Wolfram Plot
0
votes
0answers
15 views

Betfair Odds Percentage Movements & Hedging

I want to determine if the odds on Betfair have decreased by a certain percentage and then calculate the hedged profit when hedging on that percentage, but it's made tricky by the fact that decimal ...
0
votes
0answers
10 views

Implementation of Total Variation Regularization Algorithm (Lagged Diffusivity Algorithm)

I am trying to compute the derivative of an experimentally-measured quantity as a function of time. The data are fairly noisy, which causes problems. For instance, using finite differences (central ...
0
votes
0answers
11 views

Predictions on the statitistical series [migrated]

I'm in need to summarize possible approaches (as well as their pros and cons) to the problem of statistical series prediction. It's pure theoretical question, an example below was given only for ...
0
votes
2answers
86 views

Difference between Topological Data Analysis and Graph Technology

I'm trying to understand the difference between Oracle's graph technology which apparently has an inherent understanding of topology and Ayasdi's Topological Data Analysis technology. Are these two ...
0
votes
0answers
25 views

On the truncation of spherical harmonics

Suppose there is a function $f(\theta,\phi)$ defined on the surface of a sphere, and $\theta$ and $\phi$ are the polar and azimuthal angles respectively. Similar with the fact that a function defined ...
0
votes
1answer
24 views

Is topological data analysis useful for low dimensional data?

I often read how Topological Data Analysis (TDA) is useful especially for highly dimensional data. But, what about (apparently) low dimensional ones? Example: consider measuring the resistance of a ...
-1
votes
0answers
12 views

Data modelling help

If I have $52$ weeks of data (ie) number of transactions & I have $52$ weeks of data (ie) sales of those trasactions I have data for each half an hour interval of the store being open I know ...
0
votes
1answer
22 views

Calculating Inflection Point in Real World Data

I can think of a few ways to get a good grasp of this problem from an experimental perspective, but I'm looking for a more general explanation that can be applied to different situations. If this ...
2
votes
1answer
22 views

Data Analysis and Probability

Two dice with a difference ... They appear identical, but one die is fair and the other is loaded so that the probability of throwing a 6 is 0.25. Bev picks a die at random and throws it, and it shows ...
1
vote
1answer
28 views

How to work out the formula that connects several numbers

I have an interesting problem. Say I have lots of datasets like this: a = 21 b = 23 c = 58 d = 498 etc (lots of other values) X = 85 I need to find the formula ...
0
votes
0answers
17 views

How to evaluate the difference between two classes of data which are highly overlapped

I’m trying to implement a classifier based on a dataset comprising two classes of high dimensional time-series observations (the values of the two classes of observations are highly similar). I ...
1
vote
1answer
43 views

Where to start learning about topological data analysis

I was wondering if anyone could help me out with finding a nice introductory introductory text for topological data analysis (I'm speaking as somebody who has two semesters of experience with ...
0
votes
0answers
23 views

What method should I use for this optimization / feature selection project

I'm going to describe a problem and I'm not sure how to best solve it. I will describe the situation. When answering please recommend a method and maybe a software library. I'm using Python for my ...
1
vote
1answer
58 views

Extremely poor polyfit, what am I doing wrong?

I have a dataset with me: http://pastebin.com/YZArky1j, which I am trying to polyfit. This is what I used to perform the polyfit: ...
0
votes
1answer
28 views

Smoothing Spline Example

I am learning the smoothing spline method. I saw that smoothing spline is a penalty term to reduce overfitting in linear regression. Given dataset {$(x_1,y_1),(x_2,y_2)..(x_n,y_n)$}So the formular ...
0
votes
1answer
29 views

Probability of name collision

Question on statistics & probability: I am trying to find the best way to build a unique key. Data i have: 1) first name 2) last name 3) DOB 4) gender So, assuming those are correct fields (not ...
0
votes
0answers
17 views

Data normalization methods for RNA-seq

Not sure if this is the best place to ask such a question, please re-direct me if there is a more appropriate site. I am using bioinformatics software to look at gene expression levels in populations ...
1
vote
0answers
23 views

Effect of Event on Another

I am having difficulty wrapping my mind around conceptualizing the impact of one event on another. For the scope of this question, let's define the following: Event A: An assault occurs; Event B: A ...
1
vote
0answers
32 views

Bivariate Correlation Unique solution

I have a set of data, let's say "score vs loss ratio", the score is an arbitrary number I gave to a team by looking at their past history. I want to prove my hypothesis which is higher the score, high ...
0
votes
0answers
13 views

Well-behaved interpolation

I have a set of time-dependent scalar data. I would like to remove certain time intervals and fit the remaining data "well-behaved", which I cannot quite exactly define. The intervals may also be at ...
0
votes
1answer
14 views

Variance of Estimated Coefficients in Logistic Regression

I have a logistic regression model with a binary variable as the response and a categorical variable with 3 categories as a predictor. The fitted model is: logit(P(Y=1)) = intercept -0.19*C2 + ...
1
vote
1answer
20 views

Empirical Copula

I am trying to find the empirical copula linking two random variables $X$ and $Y$. I have some data available but it's limited with respect to the variable $Y$ and I am not convinced it's enough data ...
1
vote
1answer
38 views

Data compression from lack of patterns

From my understanding, lossless data compression takes advantages of statistical patterns. I want to know if it is possible to compress data based on the lack of patterns, such as a random string of ...
1
vote
1answer
23 views

How do I creat a box and whisker plot?

I know you have to create a box around the median of a data set of numbers, but I do not know where the end of the box would be placed. Please help!
0
votes
1answer
17 views

Addition of two box plots

I have two box plots that I'd like to add together to form one box plot representing both plots in one. Here's an example of two box plots: At a school 200 boys and 200 girls participated in a test. ...
0
votes
0answers
11 views

Best multivariate approximation scheme for scattered data approximation

what is the best multivariate approximation scheme for scattered data approximation on a set E of points in $R^d$?
0
votes
0answers
26 views

Find spikes in data

I have some datasets and I need to find spikes in them. Imagine the data looks like trading data. If the spike is big enough, I need to log it, otherwise, proceed in the analysis. I tried with a ...
0
votes
0answers
11 views

Is Expectation Propagation (EP) affected by the prior?

I understand EP by reading Minka's thesis: http://research.microsoft.com/en-us/um/people/minka/papers/ep/minka-ep-uai.pdf I'm trying to apply it to solve a Bayesian inference problem. However, I'm ...
0
votes
1answer
40 views

slope of curve represented by discrete points

I have data which are visualized in this chart: I need to compute slope of increasing / decreasing parts of the curve. I can't use any 2 points because of noise in data. Maybe numerical derivative ...
1
vote
1answer
22 views

Problem with significant figures

I know that if I am multiplying/dividing my answer is the least number of significant figure in the data and if I am adding/subtracting my answer is the least number of decimal places i the data but ...
1
vote
1answer
58 views

Maths concept for salient point in graph data

I have collected data concerning the total post counts of users in an online forum (see graphic). What I am hoping to do is compare the language of 'first posts' with the language of 'later posts'. ...
0
votes
0answers
30 views

Simple maths for an algorithm to searching a table to find out if one type of entry has a causal effect upon another type?

I asked this over on StackExchange and was sent over here... I have a database table which is recording two different types of events which seem to be related to one another. It records the date and ...
1
vote
1answer
45 views

Given a set of data points, how to use gradient descent to find the minimum in the function that passes from those data points?

I have a function with n parameters. I don't know the formula of the function but I can generate as many data points as I want using the function that I have. My question is, how can I find the set of ...
1
vote
1answer
24 views

Correct equation

What is the correct equation which describes the following dates: 270, 180, 135, 112.25, 101.25. I've obtained data whit the equations: 180 + 90 = 270 135 + 45 = 180 112.5 + 22.5 = ...
1
vote
0answers
16 views

How to compute a “weather” score for repeated trials over time, with a bias for recency

I'm doing some analysis on repeated software tests that can either pass or fail. For each test, I've collected a set of historical (pass/fail, timestamp) records, and I'd like to compute a score to ...
0
votes
0answers
24 views

Distance measures for binary data

I was wondering what are some good distance measures for binary data that have the following properties. I know that there are measures like the Jaccard index and the Dice Index, but they don't ...
3
votes
1answer
73 views

Sigmoid function with separate control of derivative at 0 and sharpness of bend.

For a physical relationship, $f(x)$, I'm trying to model, i have a fairly good determination of some of the boundaries, such that $ f(0) = 0$ $f'(0) = B$ $ \lim_{x\to\infty} f = A $ So far, what I ...
2
votes
1answer
67 views

How to appropriately normalize financial data?

I am evaluating a normalization of financial data. Two colleagues feel certain of a certain approach, and I am befuddled. The approach shapes the results how they want (i.e., brings possibly extreme ...
0
votes
0answers
15 views

Fit Conditional Probability Given Continuous Response

Given a sample of data, all the techniques aimed to find conditional probabilities of the response variable require that this is dichotomous or at least categorical (see multinomial logistic ...
1
vote
3answers
81 views

Analyzing data to determine seasonal indices?

The quarterly sales levels for the past four years of a leading tyre manufacturer are displayed on the table table How to analyze the data to determine the specific seasonal indices,and what ...
0
votes
0answers
112 views

Kalman filter with sensors having different sampling rate

I would like to implement Kalman filter scheme with two different kind of sensors - say 9DOF IMU (gyro/accelerometer/magnetometer) and GPS sensor (position/velocity/altitude). Problem is, as i ...
0
votes
0answers
26 views

Calculating a smooth 90% limit for differences in a time series

I have $50$ data sets. Each set has three related time series: fast, medium, slow. My end purpose is simple, I want to generate a number that indicates a relative degree of change of the time series ...