Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data-analysis]

Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

-1
votes
0answers
11 views

Different pca plots

I was trying to to learn pca(using the iris dataset) with python and i got some results,so i wanted to test the results ir R to make sure it was good.When i checked the results,it gave me a mirror ...
0
votes
0answers
16 views

Survey weighting

I took a survey of 142 people, ...
1
vote
1answer
25 views

Excel: inverse interpolation

Through Excel I obtained a calibration curve as shown in this figure The equation of this curve is $$y = 0,6209x^5 - 1,7958x^4 + 2,0116x^3 - 1,14x^2 + 0,3584x + 1,33$$ Now, through some ...
-1
votes
0answers
15 views

Using statistics in petroleum engineering [closed]

How can engineers use mathematical statistics to determine relationship between number of wells and oil, gas and water production?
-1
votes
0answers
29 views

Party Invitations Problem [closed]

There is a party on some Sunday next year, it is open to all. a)Everyday at some point, a person comes and drops a slip with contact details in a dropbox. b)A person on a given day could comeback ...
0
votes
1answer
25 views

Growth in forward difference

I am looking at a pattern in some data, and I am trying to figure out if it tells me anything about the underlying structure of the dataset, or if this is just an obvious algebraic relationship I am ...
0
votes
0answers
22 views

Compare 2 lists and find out their similarity

2 list of data (inventory levels versus order). I want to compare how similar they are. Correlation coefficient can be calculated in Excel as below. The 2 lists doesn’t look similar but Correlation ...
0
votes
0answers
12 views

modifying gumbel function to suit data

I have a set of data (x and y), which is right-skewed. I am now looking for a function that fits the data best (least squares). So far I was thinking about a curve that has a form of a gumbel or log ...
0
votes
2answers
41 views

Mean, Variance, Standard Variation

I submitted this question the other week to my tutor, but it was wrong. the only thing correct was the mean. He did not tell me why it was wrong or the correct answers. Can anybody tell me why this ...
2
votes
0answers
24 views

Principal Component Analysis - Why Use Eigenvectors of the covariance matrix?

I've pretty much been trying to learn PCA all day, and I feel like I'm so close. Here are my two main questions right now. Hopefully, someone on this thread can help. First of all, I THINK I ...
0
votes
0answers
20 views

Probability of failure needed

There are 38 numbers on a wheel. I pick 5 numbers and maintain those 5 numbers for 35 spins of the wheel. What is the probability that 1 of my 5 numbers will NOT be selected within the 35 spins/rounds?...
0
votes
0answers
8 views

Smooth a variable's trend based on another variable

I have a data with demand (quantity) and price. Their trends follow each other's most of the time. When i plot it, i can see there are times when there is a fallout in the trend. Is it possible to ...
0
votes
0answers
16 views

Analyzing SATQ scores. Please any tips help.

You are given a data file called Calculus_data.tab(a huge file) We take a random sample of 30 from the data file. The Question: Estimate the difference between SATQ scores for male and female ...
1
vote
0answers
29 views

Methods to classify observed data into states

Given a time series (that sometimes unpredictably alternates between random and non-random and is not stationary). What mathematical methods are there to classify the observed set of data into ...
0
votes
0answers
6 views

Data Transfer - Resolution & Bits

Let's say the 16-bit Unicode representation of characters is transmitted along with its coordinates on a page. If the characters can be freely positioned on the page by having the anchor point of a ...
-1
votes
1answer
27 views

Data Transfer Speed

If the data transfer to the printer is done pixel by pixel with 3 * 8-bit colors, how long does it take to transfer a DIN A4 color page (21 cm x 29.7 cm) to the printer using the following transfer ...
3
votes
0answers
36 views

Decoding confidential numbers expressed as percentages using number theory

This is a actual problem I encountered at work. Little long as it requires some explanation Let $n_1, n_2, \ldots n_k$ be natural numbers and let $s$ be their sum. The expressed as a percentage, the ...
0
votes
0answers
11 views

Random Walk Modeling - Log Normalization of Data (MATLAB)

Having trouble clarifying which plot correctly log-normalizes the data. 1:steps(i); is a vector of 1 -30 and is equal to the number of steps taken ...
1
vote
0answers
11 views

Guessing a functional form to use for fitting an exponentially decaying spectrum with peaks

I'm trying to fit an experimental spectrum (Intensity vs frequency) in which the intensity exponentially decays as frequency increases. However, there are peaks amidst the exponentially decaying ...
0
votes
0answers
10 views

Decomposition of posterior porbability

So, I'm currently working through "Data Analysis: A Bayesian Tutorial 2nd Ed." by D.S. Sivia and I am stuck an equation, which I am not able to reproduce (p.40 - 42 for those who might have a copy). ...
0
votes
1answer
29 views

I ran an ANN model and got an extremely low R2 but a pretty good MSE, what does this mean? [closed]

I ran an artificial neuron network on data with about 2,000 rows and 3 features. I got a R2 of .06 which is really low, but a good MSE of .41. Why are these performance evaluators of this model ...
0
votes
0answers
14 views

Linear advection equation with coefficient given at data points

I am working on a problem to solve some particle population balances. In the analysis of some experiments I got an equation of this type: $$\frac{\partial u}{\partial y} - \alpha(x, y) \frac{\partial ...
0
votes
0answers
6 views

Unit vector Identicon

Is there a way to visualize multiple n-dimensional ($n\approx300$) unit vectors with the property that large changes to the vector result in large visual changes, but small changes result in little to ...
0
votes
0answers
9 views

What is a lag-based correlation?

could anyone please advise as to what a lag-based correlation is? Is this the same as a cross-lagged analysis? This type of analysis was recommended to me by a reviewer to elucidate the ...
0
votes
1answer
21 views

Should i remove “know” faulty datapoints from dataset before doing statistical analysis or after importing to statistical analysis program? [closed]

I have a dataset of 21017 rows in excel, this has ~12 columns. At row 10013 i have i have one, (1), measurement that i know must be faulty (As it physically cant happen) I want to import this ...
2
votes
0answers
27 views

Is it appropriate to use clustering to partition the dependent variable into separate datasets for a home price prediction model?

I'm struggling to decide how to deal with a heteroskedasticity problem in a home price prediction model I'm developing. The training set residuals are normally distributed around zero, but they have ...
0
votes
0answers
31 views

How to fit models to data?

I am working on a project in which I am comparing how well and exponential, logistic and Gompertz model fit to tumour growth data. I have the equations for all three models but I don't know how to go ...
1
vote
2answers
80 views

Recursion $T(n)=5T(n/4)+n^3(\log\log(n))$ [closed]

I need help solve this: $\log$ is in base $4$ n=4^4^k $T(n)=5T(n/4)+n^3(\log\log(n))$ I tried to do something and I got stuck here: $T(4^4^k)=5T(4^(4^k-1))+((4^4^k)^3)k$ how can I continue from here I ...
2
votes
0answers
98 views

Compare euclidean distances from different dimensions

EDIT: I'd like to reformulate my primary question. I have a set of points in $\mathbb{R}^D$ and I reduce the dimensions of the points to some $\mathbb{R}^L$. I do this with multiple configurations ...
1
vote
1answer
32 views

How to find the causality of a periodic behaviour

If you use Google Trends (a tool to illustrate the frequency of search terms) and search for the word "war" with USA as geographic region, you will be presented with the graph below showing monthly ...
0
votes
1answer
29 views

Which similarity formula should I use?

I was studying Cosine Similarity and I have just seen this article. https://medium.com/@rahulkuntala9/cosine-similarity-and-handling-categorical-variables-29f907951b5 The author uses Cosine ...
0
votes
0answers
15 views

Can Kabsch's algorithm also provide the covariance of its solution?

Say you have a collection of data points $p_i \in \Bbb R^3$, and a collection of corresponding reference points $r_i \in \Bbb R^3$. Kabsch's algorithm, which relies on SVD decomposition, provides an ...
0
votes
1answer
33 views

Finding out what data is useful to use…

I'm trying to determine the best method for narrowing down data to use in a program I'm writing. I could use some help. Here is a use case... Sometimes we have bad data in our system and it can lead ...
0
votes
1answer
30 views

SVD and PCA - help

I have some data on cars: i have been given the Noise,size,speed,if its electric or not, if its a lorry or not. it looks something like this: $$ \begin{matrix} Noise & Size & Speed &...
0
votes
0answers
21 views

Interpolation on curvilinear grid

How could someone interpolate spatial data on a curvilinear grid, like the one in the picture shown?
3
votes
0answers
53 views

Solving recurrences using substitution method

I have given $$ T(n)= \begin{cases} T(n/3)+T(2n/3)+n,\quad &n>1 \\ 1, \quad &n=1 \end{cases} $$ I tried it again and again but couldn't think beyond, $$ T(n)=T(n/27)+3T(...
0
votes
1answer
16 views

Similarity between observations and an ideal trend

I have a series of well known data showing, for example, the average weight by age in Europe. Let's say yesterday I called a lot of people and I asked for their age and weight. The data coming from ...
1
vote
0answers
20 views

How can I make a best fit line out of data where each point is weighted more heavily than the previous?

I'm thinking that there should be a linear regression technique that allows me to value recent data more that older data. (Maybe this difference in weight could be configurable through a constant). ...
0
votes
0answers
14 views

longitudinal study with repeated measures

I have a data-set which is collected using two devices (A and B) to measure the same variable, for 10 participants during 5 different types of activities . Each activity goes for 15 minutes and there ...
1
vote
2answers
30 views

In Support-Vector-Machine, why is the hyperplane given by $(p-1)$ dimensions?

In SVM, each feature vector is viewed as a data point in a $p$-dimensional plane and different labels are separated by a $(p-1)$ dimensional hyperplane. Please see the wikipedia entry: https://en....
0
votes
0answers
76 views

Sample mean and sample covariance canonical forms

Let $(x_1, x_2,\dotsc, x_n)$ be a sequence of vectors: ($\forall i=1,\dots,n$) \begin{pmatrix} x_i^1 \\ x_i^2\\ \vdots\\ x_i^m \end{pmatrix} In statistics, one often has to compute the ...
0
votes
1answer
30 views

Calculating error on slope of graph

I'm trying to find the rate of change and the error on that rate based on 7 measurements points and the assumption that the trend is linear. My calculations are below: $$ \begin{array}{|c|c|c|c|} \...
1
vote
0answers
32 views

Modeling distributed densities or is it?

Lets say that I am counting pixels over time. Each pixel is either the color red or blue. Let's say that I make up some threshold for the number of red pixels that I count and that when I reach this ...
0
votes
1answer
25 views

Understanding the lower and upper limit of a graph

I was recently presented the following graph: I was given very little context and I have to recreate it. I don't want it to be done for me, but some context would help. Any idea what these upper and ...
0
votes
0answers
10 views

Problem about multivaluable distributions. (Hotelling distribution)

From $n_1$ p-dimensional random vectors: $X_1,...,X_{n_1}$ independents with $X_i\sim N_p(\mu_1,\Sigma)$ and $n_2$ random vectors with the same dimension: $Y_1,...,Y_{n_2}$ independents with $Y_i\sim ...
0
votes
1answer
34 views

Quantile-Quantile (QQ) Plots

I understand how to assess whether an exponential/normal distribution is suitable to model a piece of data when the parameters are given, i.e. finding the theoretical quantiles and plotting against ...
0
votes
0answers
17 views

Regression analysis for Measurement values

I'm in a situation where I'm provided with a signal and a couple of measurement values for that signal. Usually the signal will drift a little when the measurement start (i.e. because of heat). I want ...
0
votes
0answers
17 views

Minimum and Maximum Data Points in a Spectrum

If I am collecting data that will be used to assemble a spectrum, how often should I take a data point to properly sample, but not oversample given the following information? I can control my data ...
1
vote
1answer
18 views

Weighting function for a scatter plot of ratio and difference across several orders of magnitude

I am comparing straight line distances to the shortest discovered path. I have millions of points with a pair (straight line distance,...
0
votes
0answers
18 views

Calculating an average of multiple data sets based on priority.

First and foremost I'm no mathematician, it may be a case of over complication. What I'm trying to do is that I have 2 sets of data, I want to calculate one average for both sets but I want the first ...