Questions tagged [data-analysis]

Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Time series analysis on ACF and PACF plots

So I have a non stationary time series that is hourly, daily and monthly recorded for a year and I have the ACF and PACF plots for the serie. ACF and PACF I applied the the Ljung-Box test and for ...
user avatar
1 vote
0 answers
12 views

Graphical model - log transformed covariance matrix

It is known that the non-zero entries of the inverse covariance matrix constitute an estimate on the edges in a graphical model. I wonder if there is an analog or similar result for the log-...
user avatar
-2 votes
0 answers
40 views

OEIS Previous version [closed]

I am interested in downloading the OEIS of some previous versions. In order to do some data analysis and compare it with today's OEIS. Does anybody know how to download or create a subset of the ...
user avatar
  • 1
-1 votes
0 answers
21 views

Calculate the expectation and variance for normal distribution [closed]

I am solving a question which is about distribution but it is confusing as I am new in the course of applied data analysis. how we calculate the expectation and variance when we have this given ...
user avatar
0 votes
0 answers
11 views

Mini batches and loss in recurrent neural networks (RNNs)

Suppose that we have a sequence $\left\{x^{(k)}\right\}_{k = 1}^{N}$ and that we wish to use a RNN to predict the next element of the sequence given the previous elements of the sequence (e.g., a ...
user avatar
  • 303
0 votes
0 answers
22 views

Statistics confidence interval when I know the data is always positive [closed]

I have a data set where most of the data is close to zero, but there are some considerable outliers. The data is all ratios of positive numbers, so I know none of them will be negative values, but ...
user avatar
  • 139
0 votes
1 answer
27 views

How can I express a lower order partial differential equation (with regards to the variable I want to focus on) as a higher order order PDE?

I'm currently researching a data driven algorithm and I want to express the PDE results in a different manner. This is the kind of form I would like to get, mostly I want to express the t partial ...
user avatar
  • 1
3 votes
2 answers
84 views

Statistical method of giving an upper estimate for $\|f\|_2^2$ from a data

Suppose, that we are given an i.i.d. sample of $(X_i,Y_i), i=1,...,n$ input-output pairs, where $Y_i=f(X_i)+\varepsilon_i$, where $f: [0,1] \to [0,1]$ is the data-generating function, and $\...
user avatar
  • 3,662
1 vote
0 answers
45 views

What skills are necessary for a career in applied topology? [closed]

What skills are necessary for a career in applied topology? I'm currently finishing my Bachelors in pure Mathematics and I love math, but have always wanted to work on anti-aging research in some ...
user avatar
0 votes
0 answers
10 views

Proof of Nesterov Momentum change of variables

Please help me to know how the above phrase has been changed to a lower expression by changing a variable !? show picture
user avatar
0 votes
0 answers
15 views

Best statistical method for percentage of (Good points)/(All points)

I have data where players score points. There are good points and bad points. I want to get the precision of players. I first thought about doing: $$\text{Precision} = \frac{\text{Good}}{\text{Bad + ...
user avatar
  • 2,323
0 votes
0 answers
15 views

Resources with examples for Bayesian Data Analysis in R

I'm taking a course on Bayesian Data Analysis, and currently I need to do a project that requires the use of $R$ as a programming language. The goal of the project is to use basic Bayesian Inference ...
user avatar
  • 334
0 votes
0 answers
13 views

Standard error of slope when data is unknown

Let’s say I did a linear regression on a dataset with 10 observations, and I got the estimated slope as $b=0.5$ and residual variance as $\sigma^2=5.2$ and further, $$(A^TA)^{-1}=\begin{bmatrix} 2.54 &...
user avatar
0 votes
0 answers
16 views

How to group using percentiles?

I'm trying to understand this research below that I added: The outcome variable was categorized using percentiles to Mild, Moderate, and Severe in the Symptom Severity Scale (SSS). The three ...
user avatar
0 votes
0 answers
16 views

Formula for Linear Discriminant Analysis (LDA)

I'm having a bit of trouble understanding one of the sets of formulae for linear discriminant analysis (LDA). At first, I thought LDA used \begin{align} f_{i} = \mu_{i}\mathbf{C}^{-1}\mathbf{x}^{T}_{k}...
user avatar
  • 143
1 vote
0 answers
37 views

Literature on Principal differential analysis

I'm currently dealing with topics in Functional Data Analysis (FDA), specifically Principal Differential Analysis (PDA). By the corresponding R package description, this is related to estimating a ...
user avatar
  • 61
1 vote
1 answer
54 views

Determining whether two samples are from the same distribution

The context for my question is that we have two sets of data which are of the same family of distributions, both normal, both exponential, etc. If they were both normally distributed, a difference of ...
user avatar
0 votes
0 answers
11 views

how to create a data frame in format start-stop for a dependent time covariable who initial state can be 0 or 1. Different results R and SPSS

I´d like to get a data.frame in format start-stop for a time dependent covariate (trabajo) who initial state (trab0) can be 0 or 1, and change at a time (timetrab1) to trab1. In this way, a patient ...
user avatar
0 votes
1 answer
24 views

Finding a distribution to fit this curve

If it is possible, could someone help me fit some kind of function to this curve? I thought a Lorentzian fit would work because of the peak and the tails, but it is causing problems. I will attach an ...
user avatar
0 votes
0 answers
95 views

The probability classifier so obtained is identical to Centroid classifier

I'm studying data mining and I try to prove that the bayes classifier is statistically indentical to nearest centroid classifier. I try to assume for bayes classifier, $P(X=x|Y=c_{k})$ has normal ...
user avatar
0 votes
0 answers
21 views

Distance matrix spaces for different vector dimensions

Suppose $M(k,d)$ denotes the subset of $R^{k x k}$ that contains all possible distance matrices that can be achieved with any $k$ vectors $x_{1},...,x_{k}$, where $x_{i} \in R^{d}$. Also consider that ...
user avatar
  • 25
-1 votes
1 answer
35 views

distance between histogram and density in R [closed]

I have some financial data, here I have the histogram and density with estimated parameters regarding normal distribution. What I want now is to determine the "distance" between the data of ...
user avatar
2 votes
1 answer
64 views

Traveling Salesman Problem where not all cities need to be visited

If there are a total of X possible cities to travel to, but our salesman only has to visit a fixed number of them Y<X, how do we choose the Y cities to travel to and the order? Has this problem ...
user avatar
0 votes
0 answers
28 views

Trying to find a 3D equation which can fit some data and follow specific behaviour!

Hi there I am trying to find a 3D equation that follows some properties, I am having a bit of difficulty. I am unsure how to describe what the equation looks like but here goes: The equation should ...
user avatar
0 votes
0 answers
25 views

SVD of residual matrix non-orthogonal to orthogonal projection?

Suppose we have two data matrices, $X$ which is a $m$ x $n$ matrix, where $m$ >> $n$ $Y$ which is a $n$ x $p$ matrix, consisting of $p$ orthonormal columns, where $n$ > $p$ Next, we find ...
user avatar
0 votes
0 answers
11 views

Distributed Hypothesis Testing--reference question

If you don't know the correct keyword, you can still miss a key literature search: I have a problem in distributed Bayesian detection with a serial (or tandem) network topology. The probability ...
user avatar
0 votes
0 answers
10 views

Can I use an I-MR chart for a process in which the sampling time is random, or not regular?

I am currently studying a process which produces one sample for testing at one time. Not only that, the time at which the sample is taken is also not fixed but it varies. The process parameters also ...
user avatar
0 votes
0 answers
22 views

downsampling of measurement data (time series)

I am designing some software to measure data (voltage over time) and send it to a computer. Now, I am having constraints on the transmission, so I would like to compress data before sending it. ...
user avatar
  • 252
0 votes
1 answer
31 views

What is a clever or efficient way to compute this variant of the Wasserstein distance between persistence diagrams?

A two-dimensional persistence diagram in $[0,1]$ say is just a multiset of points of $\mathbb R^2$. Given two diagrams $P=\{p_1=(a_1,b_1),\ldots, p_n= (a_n,b_n)\}$ and $Q=\{q_1=(c_1,d_1),\ldots, q_n= (...
user avatar
  • 8,561
0 votes
0 answers
20 views

Best way to standardize data set that have a variety of ranges that update over time

So I have a fairly unique scenario, the following is currently the format of the data and how it works: Each column of data can contain a variety of ranges, i.e. one columns may contain values from -....
user avatar
0 votes
0 answers
48 views

How to evaluate the difference/distance between 2 values positive and negative on a scale

Problem 1 : The input is 2 values, that can be in a scale between [-3.89, 10.66] And i need to compare the difference between an oldValue (A) and a newValue (B). So i want to create a variable that ...
user avatar
  • 1
3 votes
2 answers
54 views

Creating a probability density function for a particular dataset [closed]

I want to create a probability density function for a particular dataset. First of all, I calculate the mean and the variance of my dataset. So, I use the mean and the variance to create a probability ...
user avatar
0 votes
0 answers
18 views

Calculating stock returns?

I am currently working on a project to analyze the stock data and predicting future stock prices and if possible building strategies to trade in the future. I just want to know the difference between ...
user avatar
  • 23
0 votes
0 answers
6 views

Posteriori of latent factor model

I'm reading this article https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3419391/. I want to compute the posterior distribution of the factor latent model. The generic form of a latent factor model is $$ ...
user avatar
  • 1
0 votes
0 answers
19 views

Weighted averages for microdata merging

I'm working with microdata files and would like to hear if any one may have a suggestion when merging the data. I'm concerned with making inferences on wages given specific character traits. I think I ...
user avatar
-1 votes
1 answer
65 views

What is a dimension of a convex hull?

Let $S$ be a set of data points in $E^d$, a real Euclidean space of dimension $d$. What would be the dimension of the convex hull of that data? In general, what is the definition of the dimension of ...
user avatar
  • 3
0 votes
0 answers
13 views

simulate the normal gamma inverse density using metropolis hastings

I want to simulate the normal gamma inverse density using Metropolis hasting. The density is defined as : $f\left(x, \sigma^{2} \mid \mu, \gamma, \alpha, \beta\right)=\frac{1}{\sigma \sqrt{2 \pi \...
user avatar
  • 1
0 votes
0 answers
25 views

Creating an ER model - Fashion catalog

A fashion catalog wants information about the photo shoots for the products shown in a data file save bank. Each shooting can be clearly identified by the date of the shooting and the location. ...
user avatar
  • 12.2k
0 votes
1 answer
34 views

Covariant and contravariant matrix in its indices

Consider $W$ a vector space of dimension $n$ with basis $\{e_i\}_{i=1}^n$, and a linear transformation from $W$ to $W$ with the matrix representation $M^i_j = M(e_j)^i$, i.e the $i^th$th coefficient ...
user avatar
  • 1,377
0 votes
0 answers
28 views

what is the the centre of a class interval

So im given a frequency table. Now I want to transfer this info to a histogram. The class intervals are 0-2, 3-5, 6-8 and so on. Something like that. Now, I would expect the class centre to be 1, 4, 7 ...
user avatar
  • 1
1 vote
1 answer
35 views

Calculate number of faces and dices from toss samples

I have a sample of data. That data comes from tossing n dices with k faces and adding the values of each dice. How can I calculate k and n? My approach: I know that the expected value of 1 dice of k ...
user avatar
0 votes
0 answers
49 views

Is standard deviation an associative operation? How does this affect the use of a combiner?

The mean µ of a set of numbers X = {x1, x2 ... , xn} is defined as: enter image description here The standard deviation σ of a set on numbers X is defined as: enter image description here
user avatar
0 votes
0 answers
16 views

Dynamic Data Normalisation

I am facing an issue to compare dynamic data. The situation is something like this as described below. Suppose I have $n$ students in a class and I take an exam that was very tough and since the ...
user avatar
  • 55
1 vote
1 answer
28 views

How to do multivariate analysis on data not normally distributed?

I have a series of continuous measurements that are not normally distributed (multivariate Shapiro-Wilk test p-value = 0.003818, Anderson-Darling test p-value = 0.0178 or lower for each variable) that ...
user avatar
  • 175
0 votes
0 answers
35 views

Log periodic power Law data-fiting

Good day, I have to start working on data fitting problem, but I do not know how to start. I have found this equation can work as the curve around which data will fit. $y_t = A + B(t_c - t)^\beta[1+C \...
user avatar
  • 139
1 vote
0 answers
25 views

"F1 score better" equivalent to "accuracy better"? Or not?

I have been asked to proof: Suppose we have two models, $M1$ and $M2$, that solve the same binary classification problem for which we compute the $F_1$ values and accuracies. I have been asked to ...
user avatar
  • 25
0 votes
0 answers
21 views

Internal covariance matrix of linear fit

In a physics course, I learned about the covariance matrix. It is defined as follows: Consider a dataset $\{\{x_1,y_1,\sigma_1\},\dots,\{x_n,y_n,\sigma_n\}\}$. Here the $x_i$ have negligible ...
user avatar
  • 486
1 vote
1 answer
93 views

Histogram and Normal distribution

I was studying histograms and normal distribution. As far as I know, they are two different tools used for calculating probability and statistics. More specifically they help to visualize and it is an ...
user avatar
  • 191
1 vote
0 answers
60 views

Meaning behind Metropolis-Hastings Acceptance Ratio

I have a question regarding the the term in the acceptance ratio of the Metropolis-Hastings algorithm (Refer to the link to a screenshot). Is the $\mid$ in $g(x_t \mid X^*)$ supposed to mean it's a ...
user avatar
  • 11
1 vote
0 answers
44 views

What technique, method or procedure exists for someone to determine a recurring or explicit function that fits a set of data points?

I have an infinite set of data points (the first seven values are posted bellow) and I'm trying to figure out how someone would go about determining the recurring or explicit function from them? ...
user avatar
  • 179

1
2 3 4 5
20