Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

5
votes
0answers
40 views

Careers in applied math with an MS other than in finance and data/machine learning?

Since I like math, I would like a career that uses alot of applied math. I'm about to complete my Master's and could do my thesis in numerical solutions of PDEs I'm already aware of careers such as ...
2
votes
0answers
153 views

curve fit but unknown function

I face a strange problem. I have data $[X,Y]$ with no noise. If I plot $Y$ as a function of $X$, it looks like a straight line and the regression coefficient is quite good. For the simplest model ...
2
votes
0answers
96 views

The correct probability distribution / way to analyze daily changes

I am working on a report which is being sent through to end users that should flag to them any "large changes" in the day-to-day values for the past 30 days for something we would assume the ...
2
votes
0answers
354 views

Exponent p-value generated in Excel

Excel gave me a p-value of 1.44909E-09 Notice is does not say .09 but 09 This is confusing me, I am trying to analyze my data but am stuck at this point. If it were E-9 it could be ...
1
vote
0answers
45 views

$y=\sqrt x \rightarrow \ln y = \ln x$ problem

So I had some raw data, and it looked like there was a square root relation. I plotted graphs of $y=\sqrt{x}$ They look kind of straight. So doing the algebra and taking the ln of both sides, we ...
1
vote
0answers
37 views

Form of a function I can fit like a polynomial that has an asymptote on x-axis and is always positive

So I have a small list of pairs of the form $(\mathbb{R}, \mathbb{R}_{\geq 0})$ and I want to fit a function to this data. Additionally I know that as $x$ grows large in either direction $y$ tends to ...
1
vote
0answers
35 views

Linear model for data that follow gaussian distribution

I have a question about linear regression. We have the linear regression of input data $(X,Y)=((x_1,y_1),(x_2,y_2)...(x_n,y_n))$ is $$F=aX+b$$ a,b are factors of the linear line, $y_i$ is {-1,1}. ...
1
vote
0answers
24 views

Effect of Event on Another

I am having difficulty wrapping my mind around conceptualizing the impact of one event on another. For the scope of this question, let's define the following: Event A: An assault occurs; Event B: A ...
1
vote
0answers
32 views

Bivariate Correlation Unique solution

I have a set of data, let's say "score vs loss ratio", the score is an arbitrary number I gave to a team by looking at their past history. I want to prove my hypothesis which is higher the score, high ...
1
vote
0answers
17 views

How to compute a “weather” score for repeated trials over time, with a bias for recency

I'm doing some analysis on repeated software tests that can either pass or fail. For each test, I've collected a set of historical (pass/fail, timestamp) records, and I'd like to compute a score to ...
1
vote
0answers
133 views

Kalman filter with sensors having different sampling rate

I would like to implement Kalman filter scheme with two different kind of sensors - say 9DOF IMU (gyro/accelerometer/magnetometer) and GPS sensor (position/velocity/altitude). Problem is, as i ...
1
vote
0answers
36 views

Approximating a probability distribution given multiple datasets

I am beginner in mathematics/statistics and apologise in advance for my faulty use of language. I am working on a problem in statistical genetics. Given a dataset, $D$, I obtained probabilities for ...
1
vote
0answers
17 views

Correct sampling methods for this data set / requirements

I am just looking for a push in the right direction as to what kind of sampling methods I can use to fulfill this set of sampling criteria. I had thought stratified sampling, but I'm not sure if ...
1
vote
0answers
35 views

Data Envelopment Analysis

I have been given an example for a Linear Programming Model in Data Envelopment Analysis. The given data is (the last two columns are the ratios but so far I've only used them in a graphical method); ...
1
vote
0answers
44 views

Why do candlestick plots seem to show a cyclical structure?

One oddity I notice is that if a random or quasi-random data series such as price data is plotted, it is similar to a random walk, but if the same data series is plotted using candlesticks ...
1
vote
0answers
64 views

Significance Tests & Confidence Intervals for Non-normal data

I have 2 nonnormally distributed samples of different sizes (N1~=N2). To evaluate whether there is a significant difference between these samples, I used the Mann_whitney U test (ranksum in MATLAB) ...
1
vote
0answers
22 views

“Sane” quantisation of moderate-dimensional analogue data

Warm-up problem (with solution): Suppose I have a measuring device that spits out a stream of real numbers, which I will model as i.i.d. random variables with range $\mathbb{R}$ and continuous ...
1
vote
0answers
13 views

Isolating clusters in data

This is a problem that I keep encountering in one form or another every so often. Given a large N dimension space with a sufficiently large (M) sample of points. [1] If I were to be asked to find a ...
1
vote
0answers
7 views

grouping of skewed dara

I have a set of data composed of the measurements of spaces. There are 2 distinct sizes represented in this data set. (distinct in that the larger size is larger by some unknown multiple) I am ...
1
vote
0answers
19 views

mutual information for power low distributed data?

I have a dataset with power-low distribution, I would like to measure some kind of correlation/mutual information between the data to its class,(for feature selection task), Can I use ...
1
vote
0answers
263 views

Can you compare data sets with percentages with the Chi Square test?

I’m confused about the Chi Square test.. Apparently you have to use it when comparing frequencies only, but what if both data sets are percentages? e.g. comparing the percentage of smokers in specific ...
1
vote
0answers
37 views

Unit Root Test interpretation

I am analyzing data collected hourly over $9$ years (around $80000$ hours) and I have following result for Unit Root Test : How can I interpret the data? Can I use the Autoregressive model?
1
vote
0answers
93 views

Least squares fit or decomposition of average points (from data)

I have a set of data consisting of the averages of a function $g(r)$ at regular intervals; i.e. my first datapoint is the average of the function from $0$ to $\Delta r$, the second point is the ...
1
vote
0answers
46 views

How to understand the task about bootstrap?

I have the following task. Let $X_1,\ldots,X_n$ be a sample (i.i.d.), $T_n = \overline{X}_n^2=\left(\sum\limits_{i=1}^nX_i\right)^2,$ and $\hat\alpha_k ...
1
vote
0answers
39 views

problem in dimensionality reduction

I am using multidimensional scaling to plot my data in R. However there is a hierarchy in my dataset which i want to exploit and I am using the delaunay triangulation to visualize the plot. So now I ...
1
vote
0answers
165 views

Classification: Why k-Nearest Neighbor method is more appropriate for a Mixture of Gaussians?

I'm reading a book named "The Elements of Statistical Learning" in which it states 2 scenarios when we are trying to predict the class label: Scenario 1: The training data in each class were ...
1
vote
0answers
27 views

Extrapolating parameters for a fit different from the one originally used for regression analysis (no longer having access to data)

Say I have some data, I assume a fit of the form: $\alpha_1 e^{\beta_1 k}$, and I extract some values $\alpha_1>0$ and $\beta_1<0$ for $k=[1,\ldots]$. Say this appears to be a "near perfect" ...
1
vote
0answers
112 views

How can I create a good subset of data proportional to the full data set

I am programming a neural network and I need to use some data to train it. I am following the steps of an experiment which tried to predict direction of stock price index movement using neural ...
1
vote
0answers
35 views

How to find BUE of N? where $X_i$ iid $ P(X_i=j)=1/N$, for $j=1, \cdots, N$

Let $X_1, \cdots, X_n$ be iid with $P(X_i=j)=1/N$, for $j=1, \cdots, N$, where $N$ is an unknown positive integer. I would guess that the order statistics $X_{(n)}$ is the complete sufficient ...
1
vote
0answers
44 views

Fitting multiple functions with shared parameters to data

consider a set of real-valued continuous functions $$f_i(t;a)$$ for $1\leq i\leq n$ with some shared non-negative parameters $a=(a_1,\ldots,a_d)\in\mathbb{R}^d$. Furthermore, assume that there exists ...
1
vote
0answers
86 views

Filter/Removal of periodic 'delta-peaks'

Currently I am measuring data (Counts over Time). Due to measurement problems I have some nasty peaks in this data. These peaks are periodical, very sharp (~3 datapoints over a range of 10000) and ...
1
vote
0answers
51 views

How to quantify the volitility of a time series data?

I am currently working on a time series data and I would like to quantify how volatile it is. Here volatile I mean how "shaky" the series is. If the series is smooth than it is not volatile. I have ...
1
vote
0answers
296 views

How should I calculate “fill rate”? People inquire about an item and then buy it days/months later?

So I have posts(people posting a want for an item) and then I have fills(the person actually getting the item) There are hundreds of posts per day but the fill for that item can be days or weeks away ...
1
vote
0answers
99 views

Jacobian approximation at given point without explicit derivatives expression

after solving a NLproblem with optimization method, I would like to compute confidence intervals, prediction bounds and standard deviation for these optimal parameters. Explicit formulas I have read ...
1
vote
0answers
33 views

How to symmetrically adjust data by removing certain value

Say I have a data set in the following format: {a:200, b: 100; c:400} The sum of a, b and c equals 700. Now, I have another value which is supposed to be the actual total say 500. How do I adjust ...
0
votes
0answers
29 views

Using significant figures, what would the answer be?

I've been learning about significant figures and I have a few questions. When you multiply/divide, the number of significant figures in the answer should be the same as the term with the least number ...
0
votes
0answers
11 views

Polynomial Data Fitting with Two Variables

I have the following data and I want to find the equation interpolating the given data. For example equation for given sample data is simple $a^2*b+10$ ...
0
votes
0answers
31 views

How to find a graph's equation from its points

I have a set of data that constitutes the graph on the picture. What I want to know is how would I find the equation equivalent to that kind of graph? The X are on the interval $[1,10]$.
0
votes
0answers
10 views

Reading Graphs to solve probability

Each point in the scatterplot above represents a person's systolic and diastolic blood presures. What fraction of the 24 people had either a systolic pressure over 140millimeters of mercury (mm Hg) ...
0
votes
0answers
15 views

Normalizing Data for Graph

Firstly, sorry for the long post, but I must be detailed in my explanation here. This is a computer science heavy topic, and I've posted it on the CS section of Stack Overflow already, but the main ...
0
votes
0answers
18 views

Low pass filter to maintain edge information

I am looking for a kernel as low pass filter that satisfy as:I must find a kernel that statisfies as follows: In the my reference paper, the author suggest gaussian kernel that is The gaussian ...
0
votes
0answers
21 views

Estimation of real estate

I'm working on a project on estimating the price value of real estate. First I have collected a lot of data (500 000 instances) with details such as postal code, number of bedrooms, build year, ...
0
votes
0answers
33 views

How do you use R to find the box counting dimension of a two dimensional set of data, or scatter plot?

I'm using the software R to do some analysis on some data sets for a graduate project. R has a package called "fractaldim", in this package is a function for finding the box counting dimension. The ...
0
votes
0answers
19 views

What is correlation kernel and compare with gaussian kernel

I read a paper that said about correlation kernel that defined: $$W(x-y)=(α/1+d(|y − x|))$$ where $α =  (\int(1+d(y − x)dy)^{-1}$, $(d(|y − x|))$ is spatial Euclidean distance from the central ...
0
votes
0answers
17 views

Get a set of points from Fourier plot on Matlab?

I have a program that collects raw data and also does a Fourier transform to get complex numbers. When I feed the raw data into Matlab to do its own FFT, I can get a nice plot of power on frequency. ...
0
votes
0answers
8 views

Is there any common formula to estimate the salience of peaks of a histogram?

I need to use some quantitative measure to estimate the salience of peaks and the flatness of valleys of a histogram which represents the distribution of lightness of an image. For example, if an ...
0
votes
0answers
11 views

One-class Support Vector Machine Sensitivity Drops when the number of training sample increase

I am using One-Class SVM for outlier detections. It appears that as the number of training samples increases, the sensitivity TP/(TP+FN) of One-Class SVM detection result drops, and classification ...
0
votes
0answers
44 views

How to interpret the indicator function?

I am reviewing a paper titled " Bayesian Sampling Approach to Decision Fusion" by Biao Chen and Pramod K Varshney. This paper uses an indicator function that I am not being able understand. The ...
0
votes
0answers
32 views

What type of equation matches these data points?

Can anyone tell me what equation will match these data points? 377, 384, 1113, 1120, 19861, 19868 Wolfram Plot
0
votes
0answers
27 views

Betfair Odds Percentage Movements & Hedging

I want to determine if the odds on Betfair have decreased by a certain percentage and then calculate the hedged profit when hedging on that percentage, but it's made tricky by the fact that decimal ...