Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

learn more… | top users | synonyms

5
votes
0answers
62 views

Careers in applied math with an MS other than in finance and data/machine learning?

Since I like math, I would like a career that uses alot of applied math. I'm about to complete my Master's and could do my thesis in numerical solutions of PDEs I'm already aware of careers such as ...
3
votes
0answers
90 views
+100

How does 2D kriging interpolation work?

I have a grid of points Example ...
2
votes
0answers
22 views

How can I minimize the error from fitting two functions to different parts of data?

I have some data, and I expect that part of the data will fit to a power law, and part of the data to a line. Here's a made up example of some similar data: Where I've made the red part a quadratic ...
2
votes
0answers
28 views

Predicting profit with price variation

I am currently working on a high school project that aims to predict profit from X amount of items to Y amount of profit based off a deviated sale price. For instance: I sale 10 cookies for 10 ...
2
votes
0answers
167 views

curve fit but unknown function

I face a strange problem. I have data $[X,Y]$ with no noise. If I plot $Y$ as a function of $X$, it looks like a straight line and the regression coefficient is quite good. For the simplest model ...
2
votes
0answers
103 views

The correct probability distribution / way to analyze daily changes

I am working on a report which is being sent through to end users that should flag to them any "large changes" in the day-to-day values for the past 30 days for something we would assume the ...
2
votes
0answers
509 views

Exponent p-value generated in Excel

Excel gave me a p-value of 1.44909E-09 Notice is does not say .09 but 09 This is confusing me, I am trying to analyze my data but am stuck at this point. If it were E-9 it could be ...
1
vote
0answers
12 views

Statistics linear modelling in R

Suppose I have a date set of the form: Test Subject/Sex/No. of mistakes made in the morning/ No. of mistakes made in the afternoon A / M / 2 / 5 B / F / 1 / 4 C / M / 3 / 5 D / F / 1 / 5 Suppose ...
1
vote
0answers
12 views

Curve 'anti-correlation' detection

If I have a data set, what algorithm can I use to detect when one curve moves in clear opposition to the majority of curves? For example: At point 15, the purple line moves up in opposition to the ...
1
vote
0answers
16 views

Find two clusters in an ordinated array

As the titles goes, I have an array of sorted real positive numbers, and some information on the structure of the data. I know that there are two constants $a>>b>0$ and two other numbers ...
1
vote
0answers
46 views

$y=\sqrt x \rightarrow \ln y = \ln x$ problem

So I had some raw data, and it looked like there was a square root relation. I plotted graphs of $y=\sqrt{x}$ They look kind of straight. So doing the algebra and taking the ln of both sides, we ...
1
vote
0answers
39 views

Form of a function I can fit like a polynomial that has an asymptote on x-axis and is always positive

So I have a small list of pairs of the form $(\mathbb{R}, \mathbb{R}_{\geq 0})$ and I want to fit a function to this data. Additionally I know that as $x$ grows large in either direction $y$ tends to ...
1
vote
0answers
36 views

Linear model for data that follow gaussian distribution

I have a question about linear regression. We have the linear regression of input data $(X,Y)=((x_1,y_1),(x_2,y_2)...(x_n,y_n))$ is $$F=aX+b$$ a,b are factors of the linear line, $y_i$ is {-1,1}. ...
1
vote
0answers
24 views

Effect of Event on Another

I am having difficulty wrapping my mind around conceptualizing the impact of one event on another. For the scope of this question, let's define the following: Event A: An assault occurs; Event B: A ...
1
vote
0answers
33 views

Bivariate Correlation Unique solution

I have a set of data, let's say "score vs loss ratio", the score is an arbitrary number I gave to a team by looking at their past history. I want to prove my hypothesis which is higher the score, high ...
1
vote
0answers
35 views

Find spikes in data

I have some datasets and I need to find spikes in them. Imagine the data looks like trading data. If the spike is big enough, I need to log it, otherwise, proceed in the analysis. I tried with a ...
1
vote
0answers
18 views

How to compute a “weather” score for repeated trials over time, with a bias for recency

I'm doing some analysis on repeated software tests that can either pass or fail. For each test, I've collected a set of historical (pass/fail, timestamp) records, and I'd like to compute a score to ...
1
vote
0answers
159 views

Kalman filter with sensors having different sampling rate

I would like to implement Kalman filter scheme with two different kind of sensors - say 9DOF IMU (gyro/accelerometer/magnetometer) and GPS sensor (position/velocity/altitude). Problem is, as i ...
1
vote
0answers
39 views

Approximating a probability distribution given multiple datasets

I am beginner in mathematics/statistics and apologise in advance for my faulty use of language. I am working on a problem in statistical genetics. Given a dataset, $D$, I obtained probabilities for ...
1
vote
0answers
18 views

Correct sampling methods for this data set / requirements

I am just looking for a push in the right direction as to what kind of sampling methods I can use to fulfill this set of sampling criteria. I had thought stratified sampling, but I'm not sure if ...
1
vote
0answers
38 views

Data Envelopment Analysis

I have been given an example for a Linear Programming Model in Data Envelopment Analysis. The given data is (the last two columns are the ratios but so far I've only used them in a graphical method); ...
1
vote
0answers
51 views

Why do candlestick plots seem to show a cyclical structure?

One oddity I notice is that if a random or quasi-random data series such as price data is plotted, it is similar to a random walk, but if the same data series is plotted using candlesticks ...
1
vote
0answers
75 views

Significance Tests & Confidence Intervals for Non-normal data

I have 2 nonnormally distributed samples of different sizes (N1~=N2). To evaluate whether there is a significant difference between these samples, I used the Mann_whitney U test (ranksum in MATLAB) ...
1
vote
0answers
23 views

“Sane” quantisation of moderate-dimensional analogue data

Warm-up problem (with solution): Suppose I have a measuring device that spits out a stream of real numbers, which I will model as i.i.d. random variables with range $\mathbb{R}$ and continuous ...
1
vote
0answers
14 views

Isolating clusters in data

This is a problem that I keep encountering in one form or another every so often. Given a large N dimension space with a sufficiently large (M) sample of points. [1] If I were to be asked to find a ...
1
vote
0answers
7 views

grouping of skewed dara

I have a set of data composed of the measurements of spaces. There are 2 distinct sizes represented in this data set. (distinct in that the larger size is larger by some unknown multiple) I am ...
1
vote
0answers
23 views

mutual information for power low distributed data?

I have a dataset with power-low distribution, I would like to measure some kind of correlation/mutual information between the data to its class,(for feature selection task), Can I use ...
1
vote
0answers
281 views

Can you compare data sets with percentages with the Chi Square test?

I’m confused about the Chi Square test.. Apparently you have to use it when comparing frequencies only, but what if both data sets are percentages? e.g. comparing the percentage of smokers in specific ...
1
vote
0answers
39 views

Unit Root Test interpretation

I am analyzing data collected hourly over $9$ years (around $80000$ hours) and I have following result for Unit Root Test : How can I interpret the data? Can I use the Autoregressive model?
1
vote
0answers
98 views

Least squares fit or decomposition of average points (from data)

I have a set of data consisting of the averages of a function $g(r)$ at regular intervals; i.e. my first datapoint is the average of the function from $0$ to $\Delta r$, the second point is the ...
1
vote
0answers
48 views

How to understand the task about bootstrap?

I have the following task. Let $X_1,\ldots,X_n$ be a sample (i.i.d.), $T_n = \overline{X}_n^2=\left(\sum\limits_{i=1}^nX_i\right)^2,$ and $\hat\alpha_k ...
1
vote
0answers
40 views

problem in dimensionality reduction

I am using multidimensional scaling to plot my data in R. However there is a hierarchy in my dataset which i want to exploit and I am using the delaunay triangulation to visualize the plot. So now I ...
1
vote
0answers
200 views

Classification: Why k-Nearest Neighbor method is more appropriate for a Mixture of Gaussians?

I'm reading a book named "The Elements of Statistical Learning" in which it states 2 scenarios when we are trying to predict the class label: Scenario 1: The training data in each class were ...
1
vote
0answers
27 views

Extrapolating parameters for a fit different from the one originally used for regression analysis (no longer having access to data)

Say I have some data, I assume a fit of the form: $\alpha_1 e^{\beta_1 k}$, and I extract some values $\alpha_1>0$ and $\beta_1<0$ for $k=[1,\ldots]$. Say this appears to be a "near perfect" ...
1
vote
0answers
124 views

How can I create a good subset of data proportional to the full data set

I am programming a neural network and I need to use some data to train it. I am following the steps of an experiment which tried to predict direction of stock price index movement using neural ...
1
vote
0answers
36 views

How to find BUE of N? where $X_i$ iid $ P(X_i=j)=1/N$, for $j=1, \cdots, N$

Let $X_1, \cdots, X_n$ be iid with $P(X_i=j)=1/N$, for $j=1, \cdots, N$, where $N$ is an unknown positive integer. I would guess that the order statistics $X_{(n)}$ is the complete sufficient ...
1
vote
0answers
45 views

Fitting multiple functions with shared parameters to data

consider a set of real-valued continuous functions $$f_i(t;a)$$ for $1\leq i\leq n$ with some shared non-negative parameters $a=(a_1,\ldots,a_d)\in\mathbb{R}^d$. Furthermore, assume that there exists ...
1
vote
0answers
88 views

Filter/Removal of periodic 'delta-peaks'

Currently I am measuring data (Counts over Time). Due to measurement problems I have some nasty peaks in this data. These peaks are periodical, very sharp (~3 datapoints over a range of 10000) and ...
1
vote
0answers
52 views

How to quantify the volitility of a time series data?

I am currently working on a time series data and I would like to quantify how volatile it is. Here volatile I mean how "shaky" the series is. If the series is smooth than it is not volatile. I have ...
1
vote
0answers
318 views

How should I calculate “fill rate”? People inquire about an item and then buy it days/months later?

So I have posts(people posting a want for an item) and then I have fills(the person actually getting the item) There are hundreds of posts per day but the fill for that item can be days or weeks away ...
1
vote
0answers
101 views

Jacobian approximation at given point without explicit derivatives expression

after solving a NLproblem with optimization method, I would like to compute confidence intervals, prediction bounds and standard deviation for these optimal parameters. Explicit formulas I have read ...
1
vote
0answers
33 views

How to symmetrically adjust data by removing certain value

Say I have a data set in the following format: {a:200, b: 100; c:400} The sum of a, b and c equals 700. Now, I have another value which is supposed to be the actual total say 500. How do I adjust ...
0
votes
0answers
12 views

identify nature of missingness for categorical variables

could you please give me some hints for identifying the nature of missingness for categorical variables' missing value? I mean, I gave a fast search on google scholar but I didn't find anything ...
0
votes
0answers
17 views

Unsupervised learning algorithems to detect anomaly in waves.

I have a sample of graphs (more then 10000...). that look like in the image below: I am searching an Unsupervised learning algorithems thet can help me to detect Anomaly observations. Here what i ...
0
votes
0answers
9 views

PCA: what if continous variables are frequencies

I've got a data set that exists out of 5000 variables and 6000 instances. Each element of the data sets describes a frequency. It's a documentTermMatrix. Now I would like to do some data reduction, ...
0
votes
0answers
5 views

Hurst exponent for decay evaluation

i would like to evaluate rate of decay of tail of my histogram. Expressing histogram on loglog plot gives straight line in the range where i have gathered a lot of data. Unfortunately the line is ...
0
votes
0answers
15 views

Analyzing/Modeling DynamoDB Usage

I'm seeking the correct mathematical tools to analyze/model a real world process for the purpose of optimizing resource allocation. I want to analyze consumed throughput of a AWS DynamoDB table and ...
0
votes
0answers
15 views

How can I find the value of the X-axis in the middle of the plateau shown on the chart?

On the chart below, I have 4 data sets. Each has plateau'ed at 50 on the Y-axis. I want to be able to mathematically calculate the value of the X-axis from the middle of the plateau (marked with a red ...
0
votes
0answers
22 views

Maximum Error Tolerances When Forecasting A Random Time Series

I have been working on an approach for forecasting random time series with a given maximum error tolerance for each forecasted point less than 10%-15% of the range of possible values. I have not been ...
0
votes
0answers
16 views

How can I apply inverse Abel Transform to a Data?

I have a data with X values and corresponding Y values. When I plot intensity map (2D histogram) of the data, I get a good image. I want to apply inverse Abel Transform to this image. How can I do it? ...