Questions tagged [data-analysis]

Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Help with Understanding Periodicity Detection Algorithm

I am sorry if I do any mistakes as this is my first post and any misgivings while posting this question as I am not from a mathematical background. I have been trying to implement this algorithm ...
1
vote
1answer
22 views

Multivariate Analysis: formula with unknown origin

I formula to study the stability of a multivariant variable was given to me. The formula is introduced below: $$ \sum_{i=1}^k (p_{i,2}-p_{i,1})\log \bigl(\frac{p_{i,2}}{p_{i,1}}\bigl) $$ Where: $...
0
votes
0answers
8 views

Assigning Probabilities to Alarming Events for predicting System Outages

I have a problem. I have two sets of data, let's assume they are clean and well-organised. Data-set 1: Alarm Events. Data-set 2: System Outages. The two data-sets have only one feature in common - the ...
0
votes
0answers
19 views

Descriptive statistics/summaries for scored questionnaire

I'm collecting data in a questionnaire for facilities. The questionnaire is split into sections, and scores are aggregated (summed) per section. Relevant questions vary based on the kind of facility. ...
0
votes
0answers
10 views

Fitting and Solving Integral Equations

Say I have a collection of data, $d_i$, that come from pixels $i$ arranged in a single row on a detector. Each pixel has a different field of view, which I'll say goes from $a_i$ to $b_i$ (angle from ...
0
votes
1answer
23 views

Showing equivalence of the sum of squared differences

Consider a mean-centered data matrix $\textbf{X}$ and let $\mathbf{\tilde{X}} = z_1w_1^T$ be the best rank-1 approximation to $\textbf{X}$, where $z_1$ is an $n$-column vector and $w_1$ is a $p$-...
1
vote
0answers
27 views

Sample covariance of scores matrix [closed]

The question is: Let X be a mean centered n × p data matrix and let Z be the corresponding n × p scores matrix. Show that the sample covariance of the scores matrix is diagonal. What is the ...
0
votes
0answers
28 views

Is it possible to combine two statistics which may be dependent on each other without access to the underlying data?

I'm trying to find the average salaries for college graduates of specific universities with specific majors. The issue I'm running into is that there isn't good data available for this issue. However, ...
0
votes
0answers
7 views

Number of trajectories in elementary effects method

I'm performing a sensitivity analysis for a project I'm doing and I came across some questions I can't find concrete answers for. First and foremost, how do you determine the "correct" number of ...
0
votes
0answers
13 views

Why is a normalizing constant needed in the definition of individualistic assignment

From Rubin's causal inference textbook: Definition 3.4 (Individualistic Assignment) An assignment mechanism $\operatorname{Pr}(\mathbf{W} | \mathbf{X}, \mathbf{Y}(0), \mathbf{Y}(1))$ is ...
0
votes
0answers
10 views

A data analysis question on finding a model has a strong linear relationship but also a model that will be useful for prediction and inference.

Suppose I have a response (dependent) variable $Y$ and 15 predictor (indepentent) variables $X_1, X_2, ..., X_15$. I have the 30 data measurements for both the response vaiable and each predictor ...
0
votes
0answers
18 views

complexity of on-line correlation

I am computing pairwise correlation between the N rows of a matrix $X_{(N,G)}$ using two methods. 1- Direct computation Zx is the z-scored X along the columns $C_n=\frac{Zx'*Zx}{(N-1)}$ 2- On-line ...
1
vote
0answers
30 views

Percentage deviation

EDIT 1 To clarify, why there is such a huge difference, the value of a subject can change and there were also new subjects in 2018 included which made some subjects of 2017 irrelevant. So I think ...
1
vote
0answers
31 views

What statistics can compare dataset with shared values from same data source?

I have a dataset structured as follows: I have split the data depending on the Structure column containing either "A", "B" or both as below: In this dataset, if ...
0
votes
0answers
15 views

one-to-one mapping of z-transform data

I am trying to make a simple example of a data set of (say) 10 numbers and then its one-to-one corresponding z-transformed data set of 10 numbers. No idea how can I do it? Can I take any 10 random ...
0
votes
0answers
22 views

For a set $\mathcal{S}\subset\mathbb{R}^n$, find a function $g(.)$, so that $s=g(x_1,…,x_r)$, $x_i\in\mathbb{R},r\leq n$, for each $s\in\mathcal{S}$

Thank you in advance for your help. I am trying to fight with a technical issue that I do not really know how to prove. I have a set of $n$ sensors that provide me with $n$ measurements –not ...
2
votes
2answers
31 views

Ways to draw 3 balls

A box contains 2 white balls, 3 black balls and 4 red balls. In how many ways can 3 balls be drawn from the box, if at least one black ball is to be included in the draw? My logic: total ways to draw ...
0
votes
1answer
21 views

Uniform distribution in statistics

I have got data from a sample that is thought to be uniformly distributed. The data is shown below: My question is for what range of values for y, is the data not consistent with the distribution at ...
0
votes
2answers
26 views

Solution for this equation

I have been working on a random function I found with some data I simulated and was wondering if anyone has an idea how to, or perhaps can even find a solution for it. The relationship is: $$ f(x) = ...
0
votes
0answers
4 views

What is the gradient of Q-value W.R.T policy parameters?

I have been recently studying Actor-Critic algorithms, and I ran into this question: Let $Q_{\omega}$ be the critic network, and $\pi_{\theta}$ be the actor. It is known that in order to maximize the ...
0
votes
0answers
7 views

Identify Dataset which one part is normal distributed @ x and the other part @ y

I need to identify a dataset which looks like the following: 100 Points[x,y] are as following: $x \in [0,100], y \in [-1,1]$ and another 100 Points which looks like the following: $x \in [-1,1], y \in ...
0
votes
0answers
14 views

Determining if model parameter is “useful”

I've a system of ordinary differential equations that I am using to model some phenomenon. More specifically, the system I am modeling is a disease system, and the set of equations describe ...
0
votes
1answer
7 views

Finding discrete matrix $A$ from time continuous data - System identification

I'm seeking a way to find discrete $A$ from $P_k = [CA^kB, CA^kM]$ inside a hankel matrix. I can show you an example to find $A$ if we only have $CA^kB$ for $i = 1, 2, 3, 4, 5, \dots, n$ Assume that ...
1
vote
0answers
11 views

How (or which) filter methods we can use to check if thete is no direct relationship between features and target

one approach of feature selection is filter method. In this method we check 2 things: relation between feature i and target. we want to preserve only features that influence the target relation ...
1
vote
0answers
24 views

Properties of set of points

Here is tinder fungus: and it's pores: I was wondering what can be said about this pattern. Using Wolfram Mathematica we can get the following results. Delaunay triangulation: Voronoi diagram: ...
0
votes
0answers
8 views

Item Response Theory for ordinal variables

Suppose I have the following (representative) survey data ...
0
votes
0answers
10 views

Weighted Mean Before or After Linear Fitting?

When given multiple sets (corresponding to multiple trials in an experiment) of coordinates with their associated ordinate uncertainties, with the goal of linearly fitting the data being to find the ...
1
vote
0answers
22 views

Are there examples persistent homology being used to study non-linear data?

You can compute the persistent homology of any point cloud embedded in a metric space. In the real-world applications of persistent homology I've come across so far, the data points all have (...
0
votes
0answers
13 views

Would a GAMM be appropriate to predict outcomes using both non-parametric and parametric dta

Good Evening, I have set of clinical data that is mostly nominal (0,1 type questions) as well as some continuous data variables (lab values mostly, some other demographic calculations). All of this is ...
0
votes
2answers
26 views

Representation of a data set through a curve

I have some doubts regarding the nomenclature given to the process of representing a data set through a curve (or function). I have an exercise that asks me to make a parameterization of a set of ...
0
votes
0answers
8 views

Why does data generated by series of reciprocal powers best fit by series of reciprocal roots?

I have a function of the form: $$y=a/x + b/x^2 + c/x^n$$$$4<n<5$$ I want to derive a function for $x(y)$. I generated many $(x,y)$ data points, and then I attempted to find a function that ...
0
votes
0answers
11 views

How can I use statistical models to predict results without intervention for epidemics

I have a statistical data set for number of cases of a disease during an epidemic (basically the covid 19 pandemic data for a single country). The data follows an exponential curve. I am able to ...
0
votes
0answers
10 views

Intuitive explanation of GP-LVM

I am looking for an intuitive explanation of Gaussian Process Latent Variable Models (GP-LVM). Here is what I understand: GP-LVM is a nonlinear dimensionality-reduction method. I think "nonlinear" ...
0
votes
1answer
14 views

Is it possible to generate two dimensional Pareto distributed data with python?

I would like to generate 2D random data, which is Pareto distributed. a, m = 30., 2. # Shape and mode s = (np.random.pareto(a,1000) + 1) * m I am using the ...
0
votes
1answer
44 views

How do I find an equation for this data sample?

I have gathered some data and I'm trying to find an equation for it. There are two variables, as you can see in the table below. I've noticed the result is equal to the x value as long as x <= y. ...
0
votes
0answers
19 views

Select correct sampling rate for a process, data science

For example, lets say I collected a days worth of a several variables every 0.1 seconds, and I want to collect them every day from now on. If I were to keep the 0.1 second sampling rate my database ...
1
vote
0answers
21 views

Chi-square test for uniform sample gives the wrong result

I'm studing $\chi^2$ test. I generated sample size 50 elements of uniform distribution ($U(x, \frac{-3}{2}, \frac{3}{2})$). After that, I assume that it has the form $N(x, 0, 1)$ and take it as the ...
0
votes
0answers
20 views

Finding the best fit for the data

Hi guys i am trying to use the normal equations to fit some data to find values a and b for the following equation. $$y=a*b^x$$ The data was given to me below and this is what i am currently using ...
0
votes
0answers
17 views

How To Interpret A Periodogram

I have been running analysis on some data and I wanted to use a periodogram to identify any trends. However, I can seem to grasp what information it infact gives. Here is an attatched image for the ...
0
votes
1answer
25 views

How to extrapolate the data?

The question demands me to find "rupture lifetime" for 300MPa load at a temperature of 649$^{\circ}$ C.How do I solve it?All that I know is to apply interpolation and extrapolation which I am ...
0
votes
0answers
9 views

How to calculate weights for simple data

I have a very simple problem and no experience with weighting data. It seems like my problem is so simple that I'm having trouble finding resources to help me add weights to my data. (So far I can ...
0
votes
1answer
27 views

Two sums out of a product

Problem Assuming $\theta \in [0, 1], y_i \in \{0, 1\}$, I'm having trouble deriving the expression on the right from the one on the left: $$ \begin{align} \prod_i \theta^{y_i}(1-\theta)^{(1-y_i)} ~~~...
0
votes
0answers
20 views

Examples of parametric equations for compact $n$-manifolds embedded in $\mathbb{R}^m$, $m>n\ge3$?

I've been messing with some computational topology texts and want to do some manifold/surface reconstruction on various data sets. I'd like to test on some purely geometric examples. Are there any ...
0
votes
2answers
52 views

Mean and Variance of subset of a data set

I have a data set of position measurements of an object. However, the data set is split into subsets. The subsets have equal size. I want to find the mean and variance of the whole data set only with ...
2
votes
0answers
17 views

Averaging Datasets with Inconsistent Time Points

I am trying to find the equivalent of a "mean" and "standard deviation" of some time-dependent datasets, with the added complication that not all datasets were taken at consistent time points. Say, ...
0
votes
1answer
21 views

Calculating Percentiles on a large and constantly changing set

Background: Let S be a set of scores. These scores can be any positive real number. The size of S is large. A specific score s is subject to change frequently. I would like to tell each user (a user ...
0
votes
2answers
15 views

Plotting 1 dimensional data

I have measured the diameter and circumference of various (ca. 50) circular objects and would like to plot the ratio, i.e., circumference:diameter in order to show that those values are close to pi. ...
0
votes
0answers
21 views

Is the Hamming distance sensitive to sequence of bits?

Two binary strings ($110$) and ($100$) have a hamming distance of $1$, similarly ($110$) and ($111$) also have hamming distance $1$. Hamming distance is not sensitive to the sequence of bits. Is there ...
0
votes
0answers
21 views

Counting the width and depth of a ReLU FNN implementation

I am trying to understand the way that the writer is counting the width and depth of a ReLU feedforward neural network (FNN) that we implement in the proof. In this paper, we define the width of a ...
0
votes
0answers
27 views

How do I fit a function to data that is monotonic?

I have a list of numbers that always increase. I would like to fit a function (any function, not necessarily a polynomial). I do have a condition to this function though which is as follows- Let us ...

1
2 3 4 5
17