Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.
0
votes
3answers
73 views
preliminary evaluation for forecasting models
Suppose I would like to use a method for data prediction, and that I have some empirical data (i.e., sequence of samples of the form [time, value]). Would it be possible to know in advance, based on ...
1
vote
2answers
100 views
Use a set of data points from a graph to find a derivative
I have a data logger that is recording the temperature readings from thermocouples at a specific interval. This gives me data points that I can graph where the x-coordinate is time and the ...
1
vote
1answer
160 views
What is the distribution of a data set
I understand what the probability distribution is.
I also have a personal understanding/interpretation of the concept of distribution of a dataset. Whenever I see this expression I imagine a graph ...
1
vote
1answer
489 views
time derivative of discrete data in simulink
I'm implementing a program in Java that was delivered in Simulink. My expertise is limited, and I'm stuck on converting a derivative block.
The simulink code applies a du/dt block to the input data. ...
0
votes
1answer
19 views
What is feature selection?
I have been participating in the kaggle digit recognizer competition and after some initial success someone suggested that I look into feature selection. I've done a bit of research but I am having a ...
0
votes
1answer
28 views
Need help with Lagrange Multipliers in Support Vector Machine
In linear support vector machine with binary classification if we know all the support vectors, geometrically we can create hyperplanes on both sides of the main hyperplane using these support vectors ...
0
votes
1answer
36 views
Normalize data with large spread in values.
I'm currently trying to rank a set of data. The issue is that my initial rank comes from a search on google and the returning result set.
The spread in values is ranging from 33 all the way up ...
0
votes
1answer
36 views
Dividing data of a given distribution
Having a dataset with distribution $p$ (e.g. uniform or normal), if we divide the dataset into $n$ parts with equal size, is it valid to say that each part still has distribution $p$?
0
votes
1answer
30 views
How are user opinions compared?
Consider a situation where people are asked to rate a product on a scale of 1 to 5.
Product A has a rating of 3.95 based on 15,000 ratings. Product B has a rating of 4.2 based on 1000 ratings.
...
2
votes
0answers
55 views
Exponent p-value generated in Excel
Excel gave me a p-value of 1.44909E-09
Notice is does not say .09 but 09
This is confusing me, I am trying to analyze my data but am stuck at this point.
If it were E-9 it could be ...
2
votes
0answers
74 views
Kalman filter for acceleremoter and gyro
I have read a lot of literature on this topic, but many questions still remain.
(Many formulae are here below, so, who prefers to look at them in a more comfortable manner can find a MathCAD sheet ...
1
vote
0answers
37 views
How to understand the task about bootstrap?
I have the following task. Let $X_1,\ldots,X_n$ be a sample (i.i.d.), $T_n = \overline{X}_n^2=\left(\sum\limits_{i=1}^nX_i\right)^2,$ and $\hat\alpha_k ...
1
vote
0answers
35 views
problem in dimensionality reduction
I am using multidimensional scaling to plot my data in R. However there is a hierarchy in my dataset which i want to exploit and I am using the delaunay triangulation to visualize the plot. So now I ...
1
vote
0answers
41 views
Classification: Why k-Nearest Neighbor method is more appropriate for a Mixture of Gaussians?
I'm reading a book named "The Elements of Statistical Learning" in which it states 2 scenarios when we are trying to predict the class label:
Scenario 1: The training data in each class were ...
1
vote
0answers
24 views
Extrapolating parameters for a fit different from the one originally used for regression analysis (no longer having access to data)
Say I have some data, I assume a fit of the form: $\alpha_1 e^{\beta_1 k}$, and I extract some values $\alpha_1>0$ and $\beta_1<0$ for $k=[1,\ldots]$. Say this appears to be a "near perfect" ...
1
vote
0answers
50 views
How can I create a good subset of data proportional to the full data set
I am programming a neural network and I need to use some data to train it.
I am following the steps of an experiment which tried to predict direction of stock price index movement using neural ...
1
vote
0answers
28 views
How to find BUE of N? where $X_i$ iid $ P(X_i=j)=1/N$, for $j=1, \cdots, N$
Let $X_1, \cdots, X_n$ be iid with $P(X_i=j)=1/N$, for $j=1, \cdots, N$, where $N$ is an unknown positive integer.
I would guess that the order statistics $X_{(n)}$ is the complete sufficient ...
1
vote
0answers
86 views
Jacobian approximation at given point without explicit derivatives expression
after solving a NLproblem with optimization method, I would like to compute confidence intervals, prediction bounds and standard deviation for these optimal parameters. Explicit formulas I have read ...
1
vote
0answers
32 views
How to symmetrically adjust data by removing certain value
Say I have a data set in the following format: {a:200, b: 100; c:400}
The sum of a, b and c equals 700. Now, I have another value which is supposed to be the actual total say 500.
How do I adjust ...
1
vote
0answers
22 views
Data decomposition method invariant to small shift and scale?
Is there a data decomposition method similar to eigenvalue that estimates projection matrix to reduce dimensionality but does not project similar vectors too far away in euclidian distance terms from ...
0
votes
0answers
10 views
How to do test and training on graph data without a transductive manner?
If I have a complete network data, such as people's social network e.g. facebook/twitter. I want to build a predictive model and evaluate the performance. How can I have discrete training and test ...
0
votes
0answers
14 views
Calculating mean of data given correlated data pairs (how to apply Bayes theorem)?
Suppose that I have a data set of points $\{A_i\}$ and $\{B_i\}$. These data points are not independent, and the pairing is A1 <--> B1, A2 <--> B2 etc (i.e. the points come in pairs).
I want to ...
0
votes
0answers
21 views
Fitting 2D random walk to data points given on cartesian plane
I have a collection of data points given with coordinates $(x,y)$ in $\Re^2$.
I suspect that these data points can be modeled using a random walk. How can I fit a random walk model to these 2D ...
0
votes
0answers
34 views
Calculating the peak source data rate
For an exam I am soon to sit there is a question I have no idea how to answer, it is in past papers but there is no explanation to work it out in our lecture slides. The question is:
A company ...
0
votes
0answers
36 views
Normalizing data between 0 and 1
I've got some data that I need to be normalized. I've tried several different ways I've found online, none of them work the way I'd like. I've tried ...
0
votes
0answers
60 views
Least squares fit or decomposition of average points (from data)
I have a set of data consisting of the averages of a function $g(r)$ at regular intervals; i.e. my first datapoint is the average of the function from $0$ to $\Delta r$, the second point is the ...
0
votes
0answers
24 views
Variogram practical interprtation
I have 23 stations and I have calculated the variogram as following:
Variogram
The maximum distance between the stations is 2169 meter and the range is 1650 meter (exponential effective range).
I ...
0
votes
0answers
14 views
Finding the number of measurements given a boxplot.
I am trying to do the following GRE question, and have difficulty with part (b). The answer is 40 measurements. I have no idea of the procedure to do this part of the problem.
0
votes
0answers
26 views
What's the problem will I have if the variables in an object function are not independent?
I have a question related to objective function minimization. Suppose the object function can be written as sum( (erf((xi-p)/delta) -yi)*(erf((xi-p)/delta) -yi) ), ...
0
votes
0answers
64 views
Choose a right optimization method
I have a question related to optimization method selection. Suppose now I have some observation data (1D) as well as the underlying function model. For example, I use the following codes to generate ...
0
votes
0answers
33 views
Splitting a dataset with a given distribution?
Having a dataset with a given distribution $p$ (e.g. uniform or normal), if the dataset is randomly split into $n$ sub-datasets with equal size, is it valid to say that each sub-dataset still has ...
0
votes
0answers
13 views
For unequal size clusters, how do you estimate probability for initial centroid
I've read that for equal sized clusters, the probability that centroid could come from any cluster is K-factorial/K exp K.
But what if clusters are unequal? And doesn't intercluster distance matter?
...
0
votes
0answers
37 views
What is the name of the matrices which reorder edges in an adjancency so that they are closer to the diagonal? (modularity/structure)
I have seen adjacency matrices presented where the edges formed box like formations along the diagonal (not block matrices). This helped view the modularity and community structure of the group of ...
0
votes
0answers
78 views
Test of goodness of fit - normal distribution of file sizes on HDD
I'm trying to prove, that sizes of files stored on HDD comes from normal distribution.
I already made a java code that gave me list of all files on disk with filesizes in Bytes:
...
0
votes
0answers
59 views
maximum likelihood discriminant rule
I'm trying to use maximum discriminant rule to classify objects in my database. Could, please, somebody clarify the following for me:
For example we have 3 groups and we already know what groups our ...
0
votes
0answers
22 views
Multi-stage Filter Implementation Help
I'm reading the paper:
"New Directions in Trac Measurement and Accounting: Focusing on the Elephants, Ignoring the Mice"
I want to experiment with multistage filters. There are some details in the ...
0
votes
0answers
162 views
Fama Macbeth regression vs. time fixed effects model
Mathematically, is there a relationship between the estimated coefficients derived from fama macbeth and the coefficients derived from a time fixed effects model?
fama macbeth regression is for each ...
0
votes
0answers
35 views
Forecasting accuracy
Suppose I have some data like (date, no of reservations). For example a database which describes the number of reservations of a restaurant for each date over some period. I would like to build a ...
0
votes
0answers
33 views
Fitting multiple functions with shared parameters to data
consider a set of real-valued continuous functions
$$f_i(t;a)$$
for $1\leq i\leq n$ with some shared non-negative parameters $a=(a_1,\ldots,a_d)\in\mathbb{R}^d$.
Furthermore, assume that there exists ...
0
votes
0answers
46 views
Filter/Removal of periodic 'delta-peaks'
Currently I am measuring data (Counts over Time).
Due to measurement problems I have some nasty peaks in this data. These peaks are periodical, very sharp (~3 datapoints over a range of 10000) and ...
0
votes
0answers
51 views
Matrix methods in data analysis: appropriate without (unique) vector space structure?
Matrix-based methods such as Singular Value Decomposition or Nonnegative Matrix Factorizations are ubiquitous in data analysis (see, eg [Understanding Complex Datasets: Data Mining with Matrix ...
0
votes
0answers
40 views
How to quantify the volitility of a time series data?
I am currently working on a time series data and I would like to quantify how volatile it is.
Here volatile I mean how "shaky" the series is.
If the series is smooth than it is not volatile.
I have ...
0
votes
0answers
238 views
finding sample points of a curve in a MATLAB figure
I am trying to find the points of intersection between two curves in a .fig file of MATLAB. the first curve is constant but I need to change the second curve and find the points for different ones. It ...
0
votes
0answers
189 views
how to calculate a curvature/angle of best fit line segments
I have data points from a power spectrum (fft) output that look like the image below.
A characteristic of the data seems to be a heavy slope and then a flattening out. I would like to be able to ...
0
votes
0answers
38 views
equation to calculate intervals based on dynamic data
I am looking for an equation to calculate appropriate intervals to designate values as very low, low, medium, high, very high for dynamic data.
for example if my dataset ranges from 0 to 100 very ...
-1
votes
0answers
28 views
SER greater or lower than BER
suppose that my symbols are [-3 -1 +1 +3] and the mapping to bits is
-3|00
-1|01
+1|11
+3|10
now lets data=[-3 -3 -3] bit_data=[0 0 0 0 0 0] and quantization data quant_data=[-1 -1 -3] ...
-1
votes
0answers
36 views
Looking for online app or easy way to test if function follows a power law
Is there an on-line app or a VERY easy way to test if a function follows a power law or not? I have a function and I want to see if it adheres to a power law. I'd like to plot a few points and see ...
-3
votes
0answers
48 views
How to calculate the equation of a curve?
How to calculate the equation of a curve ? Following are the Data for the curve.
...
