Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

0
votes
0answers
19 views

Normalizing Data for Graph

Firstly, sorry for the long post, but I must be detailed in my explanation here. This is a computer science heavy topic, and I've posted it on the CS section of Stack Overflow already, but the main ...
0
votes
1answer
39 views

How do you call it when you remove the top n% and the bottom n% of a dataset?

I am currently writing about a dataset of collected handwritings. I want to show some characteristics of the dataset. For example I think it is interesting to show how long it took users to create the ...
0
votes
2answers
39 views

Creating a polynomial function with no x-intercept

I have an understanding of polynomials and how to create a function based on the leading coefficient, degrees, x-intercepts, etc. My question is how do i go about creating a polynomial function that ...
1
vote
0answers
46 views

$y=\sqrt x \rightarrow \ln y = \ln x$ problem

So I had some raw data, and it looked like there was a square root relation. I plotted graphs of $y=\sqrt{x}$ They look kind of straight. So doing the algebra and taking the ln of both sides, we ...
1
vote
2answers
41 views

Finding similarity between elements using statistics

I have a dataset of DJs in which I'm trying to find DJs similar to a specific DJ. Each DJ has a set of a genres with a certain percentage. How can I find the similarity between 2 DJs? The following is ...
0
votes
1answer
39 views

Median value resulting in negative number

I know the that the formula for finding the median of grouped data that is: $$\mathrm{Median} = L_m + \left [ \frac { \frac{n}{2} - F_{m-1} }{f_m} \right ] \times c$$ And I also know what the ...
0
votes
1answer
29 views

Data interpretation problem

Approximately what is the minimum amount of caffeine, in milligrams, consumed per day by a person who daily drinks two 10-ounce mugs of percolated coffee and one 12-ounce cup of a caffeinated ...
0
votes
0answers
24 views

Low pass filter to maintain edge information

I am looking for a kernel as low pass filter that satisfy as:I must find a kernel that statisfies as follows: In the my reference paper, the author suggest gaussian kernel that is The gaussian ...
1
vote
1answer
28 views

Multidimensional fitting of two data sets

My problem is the following: A laser gives out a bunch of data points which are reflected off a metal surface and recorded by a camera attached to the side of the laser. The image the camera receives ...
0
votes
1answer
32 views

Finding a population Function

I have been given the population of the USA from 1790 - 1980 (increasing in intervals of 10) and I am asked to solve this differential equation. Using t as time in yearrs P as size of population at ...
0
votes
1answer
38 views

Formula to link two exponential values together - doesn't quite work

Basically, I've done a script, and I'm stuck on a formula for it. After I run the code on a cube, based on two different inputs (detail level and vertex average iterations), the resulting size will be ...
0
votes
1answer
20 views

Work out if the relationship between 2 datasets is constant

I have 2 one-dimensional datasets, let's call them a and b. I want to know the correlation between ...
0
votes
0answers
59 views

How do you use R to find the box counting dimension of a two dimensional set of data, or scatter plot?

I'm using the software R to do some analysis on some data sets for a graduate project. R has a package called "fractaldim", in this package is a function for finding the box counting dimension. The ...
0
votes
1answer
27 views

Is there a way to use the Generalized Mean to find the “best” possible mean to use for a specific data set?

I've recently learned about the Generalized Mean as an abstraction of the many different means, includeing the Geometric, Arithmetic, and Harmonic means, as well as others. It is my understanding ...
1
vote
1answer
27 views

Survival analysis in survey dropouts?

I am working with a survey data set and am curious about who abandoned it before finishing all the questions. I have information on how many questions each respondent answered (no skipping allowed) ...
1
vote
0answers
39 views

Form of a function I can fit like a polynomial that has an asymptote on x-axis and is always positive

So I have a small list of pairs of the form $(\mathbb{R}, \mathbb{R}_{\geq 0})$ and I want to fit a function to this data. Additionally I know that as $x$ grows large in either direction $y$ tends to ...
0
votes
1answer
65 views

What is purpose of correlation kernel? IIs it high pass filter or low pass filter?

I am research about correlation kernel and I have some questions that need your help. Let see the pp. 3302-3303 in the http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6517250&tag=1 The special ...
1
vote
0answers
36 views

Linear model for data that follow gaussian distribution

I have a question about linear regression. We have the linear regression of input data $(X,Y)=((x_1,y_1),(x_2,y_2)...(x_n,y_n))$ is $$F=aX+b$$ a,b are factors of the linear line, $y_i$ is {-1,1}. ...
0
votes
1answer
511 views

Finding the range from standard deviation and Gaussian Curve

The figure above shows a normal distribution with mean m and standard deviation d, including approximate percents of the distribution corresponding to the six regions shown. Suppose the heights of a ...
0
votes
0answers
24 views

What is correlation kernel and compare with gaussian kernel

I read a paper that said about correlation kernel that defined: $$W(x-y)=(α/1+d(|y − x|))$$ where $α =  (\int(1+d(y − x)dy)^{-1}$, $(d(|y − x|))$ is spatial Euclidean distance from the central ...
0
votes
0answers
23 views

Get a set of points from Fourier plot on Matlab?

I have a program that collects raw data and also does a Fourier transform to get complex numbers. When I feed the raw data into Matlab to do its own FFT, I can get a nice plot of power on frequency. ...
0
votes
0answers
9 views

Is there any common formula to estimate the salience of peaks of a histogram?

I need to use some quantitative measure to estimate the salience of peaks and the flatness of valleys of a histogram which represents the distribution of lightness of an image. For example, if an ...
0
votes
0answers
11 views

One-class Support Vector Machine Sensitivity Drops when the number of training sample increase

I am using One-Class SVM for outlier detections. It appears that as the number of training samples increases, the sensitivity TP/(TP+FN) of One-Class SVM detection result drops, and classification ...
1
vote
1answer
22 views

How to determine who improved the most considering diminishing returns

My friends and I are competing in a fitness challenge against each other. We will determine the winner based on who has the greatest percentage improvement after three months. Here is the problem: if ...
0
votes
1answer
32 views

Method for detecting anomalies in timeseries?

I am exploring creating alerts for anomalies in a timeseries. Say that I have a graph like this: There are two anomalies here that I would wanted to be alerted on: First, on April 7th with there ...
0
votes
0answers
47 views

How to interpret the indicator function?

I am reviewing a paper titled " Bayesian Sampling Approach to Decision Fusion" by Biao Chen and Pramod K Varshney. This paper uses an indicator function that I am not being able understand. The ...
0
votes
0answers
45 views

What type of equation matches these data points?

Can anyone tell me what equation will match these data points? 377, 384, 1113, 1120, 19861, 19868 Wolfram Plot
0
votes
0answers
65 views

Betfair Odds Percentage Movements & Hedging

I want to determine if the odds on Betfair have decreased by a certain percentage and then calculate the hedged profit when hedging on that percentage, but it's made tricky by the fact that decimal ...
0
votes
0answers
24 views

Nyquist limit explanation

Kindly explain Nyquist in easy words. The actual question is as follows. We can attempt to display sampled data by simply plotting the points and letting the human visual system merge the points into ...
1
vote
2answers
62 views

How to find a line that minimizes the average squared perpendicular distance from the given points to the line?

I have set of points scattered around the origin. How to find a vector, such that the average squared distance (perpendicular distance) from points to the vector is minimised? Added For example, ...
0
votes
0answers
43 views

Implementation of Total Variation Regularization Algorithm (Lagged Diffusivity Algorithm)

I am trying to compute the derivative of an experimentally-measured quantity as a function of time. The data are fairly noisy, which causes problems. For instance, using finite differences (central ...
1
vote
2answers
171 views

Difference between Topological Data Analysis and Graph Technology

I'm trying to understand the difference between Oracle's graph technology, which apparently has an inherent understanding of topology, and Ayasdi's Topological Data Analysis technology. Are these two ...
0
votes
0answers
46 views

On the truncation of spherical harmonics

Suppose there is a function $f(\theta,\phi)$ defined on the surface of a sphere, and $\theta$ and $\phi$ are the polar and azimuthal angles respectively. Similar with the fact that a function defined ...
1
vote
1answer
41 views

Is topological data analysis useful for low dimensional data?

I often read how Topological Data Analysis (TDA) is useful especially for highly dimensional data. But, what about (apparently) low dimensional ones? Example: consider measuring the resistance of a ...
0
votes
1answer
34 views

Calculating Inflection Point in Real World Data

I can think of a few ways to get a good grasp of this problem from an experimental perspective, but I'm looking for a more general explanation that can be applied to different situations. If this ...
2
votes
1answer
24 views

Data Analysis and Probability

Two dice with a difference ... They appear identical, but one die is fair and the other is loaded so that the probability of throwing a 6 is 0.25. Bev picks a die at random and throws it, and it shows ...
1
vote
1answer
28 views

How to work out the formula that connects several numbers

I have an interesting problem. Say I have lots of datasets like this: a = 21 b = 23 c = 58 d = 498 etc (lots of other values) X = 85 I need to find the formula ...
0
votes
0answers
18 views

How to evaluate the difference between two classes of data which are highly overlapped

I’m trying to implement a classifier based on a dataset comprising two classes of high dimensional time-series observations (the values of the two classes of observations are highly similar). I ...
5
votes
2answers
222 views

Where to start learning about topological data analysis

I was wondering if anyone could help me out with finding a nice introductory introductory text for topological data analysis (I'm speaking as somebody who has two semesters of experience with ...
0
votes
0answers
25 views

What method should I use for this optimization / feature selection project

I'm going to describe a problem and I'm not sure how to best solve it. I will describe the situation. When answering please recommend a method and maybe a software library. I'm using Python for my ...
1
vote
1answer
90 views

Extremely poor polyfit, what am I doing wrong?

I have a dataset with me: http://pastebin.com/YZArky1j, which I am trying to polyfit. This is what I used to perform the polyfit: ...
0
votes
1answer
36 views

Smoothing Spline Example

I am learning the smoothing spline method. I saw that smoothing spline is a penalty term to reduce overfitting in linear regression. Given dataset {$(x_1,y_1),(x_2,y_2)..(x_n,y_n)$}So the formular ...
0
votes
2answers
98 views

Probability of name collision

Question on statistics & probability: I am trying to find the best way to build a unique key. Data i have: 1) first name 2) last name 3) DOB 4) gender So, assuming those are correct fields (not ...
0
votes
0answers
41 views

Data normalization methods for RNA-seq

Not sure if this is the best place to ask such a question, please re-direct me if there is a more appropriate site. I am using bioinformatics software to look at gene expression levels in populations ...
1
vote
0answers
24 views

Effect of Event on Another

I am having difficulty wrapping my mind around conceptualizing the impact of one event on another. For the scope of this question, let's define the following: Event A: An assault occurs; Event B: A ...
1
vote
0answers
33 views

Bivariate Correlation Unique solution

I have a set of data, let's say "score vs loss ratio", the score is an arbitrary number I gave to a team by looking at their past history. I want to prove my hypothesis which is higher the score, high ...
0
votes
0answers
16 views

Well-behaved interpolation

I have a set of time-dependent scalar data. I would like to remove certain time intervals and fit the remaining data "well-behaved", which I cannot quite exactly define. The intervals may also be at ...
0
votes
1answer
24 views

Variance of Estimated Coefficients in Logistic Regression

I have a logistic regression model with a binary variable as the response and a categorical variable with 3 categories as a predictor. The fitted model is: logit(P(Y=1)) = intercept -0.19*C2 + ...
1
vote
1answer
30 views

Empirical Copula

I am trying to find the empirical copula linking two random variables $X$ and $Y$. I have some data available but it's limited with respect to the variable $Y$ and I am not convinced it's enough data ...
1
vote
1answer
44 views

Data compression from lack of patterns

From my understanding, lossless data compression takes advantages of statistical patterns. I want to know if it is possible to compress data based on the lack of patterns, such as a random string of ...