Skip to main content

Questions tagged [data-analysis]

Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.

Filter by
Sorted by
Tagged with
0 votes
0 answers
18 views

Unequal observation Two-Way ANOVA

Righto, I have some data on fish. I want to compare the weight of male and female (F = 1) fish across different ponds (Ponds 1 to 4) so using a two-way ANOVA. I want to do this in Excel, using a Two-...
Amy R's user avatar
  • 1
0 votes
2 answers
54 views

Combinations of Groups of People and the Probability of People being in the Same Group?

Question: Ms. Vena has 25 students in her class. For the next lesson, she would like to put the students into groups of 5. What is the probability that Samantha and Daniel are in the same group? ...
Yusuf Patel's user avatar
1 vote
0 answers
14 views

Data smoothing with controlled smoothing error

Say I have a data sets $X$ and $T$ of $n$ measurements at $n$ distinct times. The data is slow varying and relatively non-noisy. I want to smooth the data with some curve or a set of piecewise ...
Donatas Šimeliūnas's user avatar
1 vote
1 answer
48 views

sorting functions by amount of conditions for a random dataset to be described using it?

Given a finite dataset like 1, 2, 3, 4 You could find infinite functions, for simplicity I found 2: Add 1 for the next data point, so the sequence continues as 5, 6, etc. 2.Cycle through 1, 2, 3, 4, ...
Anonymous's user avatar
-1 votes
1 answer
36 views

Interpreting this Table [closed]

I was just wondering if there was a given name to these kinds of tables? I have trouble interpreting them and would like to do further research but don't know the name of said table.
Mason Shah's user avatar
0 votes
0 answers
32 views

How to compare real-time physiological data between pre-defined time blocks

I have a set of physiological data for approx. 20 patients measured per second over approximately 8 to 10 minutes. The data is grouped into the following blocks of time: calibration, test, first ...
Richard Tinkler's user avatar
1 vote
0 answers
73 views

Differences Between Modified Kolmogorov-Smirnov and Van der Waerden Tests

I am trying to understand the differences between the Modified Kolmogorov-Smirnov test and the Van der Waerden test. I am new to this and struggling to grasp their distinct features. Specifically, ...
PoreyS's user avatar
  • 77
0 votes
0 answers
5 views

Splitting upon insertion in hierarchical clustering

It's my understanding that, upon the insertion of a new element, complete-link hierarchical clustering can lead the splitting of a cluster so as to maintain its "spherical compactness". Do ...
Tfovid's user avatar
  • 153
0 votes
0 answers
37 views

Appropriateness of one observation per unique combination of dummy variables

I am wondering what conclusions you can draw regarding the coefficients of an OLS model when you only have one observation per combination of unique dummy variables. I have seen someone else do this ...
user25435163's user avatar
0 votes
0 answers
14 views

Finding weights from a linear system

I have a dataset with n individuals and each individual is randomly assigned codes from m possible integer codes. The structure is such that if a person contains multiple codes then each code is ...
Calum's user avatar
  • 399
0 votes
1 answer
16 views

Understanding the Reasoning Behind the Growth Function $m_{\mathcal{H}}(N)=2^N$ for Convex Sets

I am currently reading Learning from Data by Abu-Mostafa et al. and I am struggling to understand the reasoning behind the growth function $m_{\mathcal{H}}(N)=2^N$ for convex sets. Here is the ...
bruno's user avatar
  • 425
1 vote
0 answers
13 views

Challenges with a Multi-label Regression Problem in a Real Estate Dataset

I am currently conducting a research where I aim to predict the selling price and deal execution time of properties in NY. To do this, I have a dataset of various properties sold in NY, containing ...
Marco Di Giacomo's user avatar
0 votes
1 answer
65 views

Percentage of values between 3 standard deviations and 4 standard deviations.

After learning about the empirical rule / 68-95-99.7 rule, I am just wondering about the percentage of values 4 standard deviations from the mean. This question is pretty much the same as the ...
GSmith's user avatar
  • 154
0 votes
1 answer
29 views

Tricky Bar Graph or Histogram [closed]

I am trying to determine if this is a bar graph or a histogram. It's the distribution of American families by income. This has been such a struggle for me, but I'm sure there's a simple explanation. ...
spraka's user avatar
  • 15
0 votes
0 answers
21 views

How do we combine sensitivity in differential privacy with Lipschitz continuous?

The definition of sensitivity in differential privacy is: $s_{GS}(f) = max_{D1,D2}\|f(D_1)-f(D2)\|$ for two neighboring datasets $D_1, D_2$ (take global $l_1$ sensitivity as an example here), many ...
Shan Sha's user avatar
0 votes
0 answers
10 views

Compare and Represent Data With Different Number of Elements

I have N entities that I want to compare them to; for example, A has 5 options, B has 3 options, and C has 6 options. 1 means an option exists, 0 otherwise as in the example next: A = [1 1 0 1 1] (...
mmain's user avatar
  • 101
0 votes
0 answers
8 views

How to compute a weighted mean as a measure of central tendency?

Imagine I have data points in triplicates, as follows $$ L=\begin{pmatrix} p_{11} & p_{12} & p_{13}\\ p_{21} & p_{22} & p_{23}\\ & \vdots &\\ p_{n1} & p_{n2} & p_{n3} \...
sam wolfe's user avatar
  • 3,405
1 vote
1 answer
25 views

Derivation of a method an a code to compute a set the interpolation polynomials known as periodic splines

Consider an approximation of a periodic function sampled at N discrete points $x_0, x_1, ... , x_{N−1}$. We now suppose that the sequence is repeated on both sides of the interval as shown in the ...
Mon's user avatar
  • 37
0 votes
0 answers
13 views

Why I get the eigen value as $0$ for the minimum variance?

enter image description here like equation (8.8) for $M_{ii}^B = <B_i B_i> - <B_i><B_i>$ is coming $0$ for my time series data set. But this eigen value should not be $0$ as ...
Mon's user avatar
  • 37
1 vote
0 answers
72 views

Prove convexity formula applied to positive semi definite matrices

I have to prove that : Let $p \in [0,1], A,B > 0$ and $\alpha \in [0,1]$. Then : $$(\alpha A+(1−\alpha)B)^p \ge \alpha A^p +(1−\alpha )B^p$$ I have the hint that : $X \ge Y \iff X−Y \ge 0$. I also ...
Ethan07's user avatar
  • 11
0 votes
0 answers
27 views

How Do Global Translation and Rotation Relate to the Principal Components in PCA of Molecular Data?

I'm working with Principal Component Analysis (PCA) on molecular data, specifically focusing on the Cartesian coordinates of atoms within molecules. My goal is to preprocess this data for a machine ...
andy90's user avatar
  • 309
0 votes
0 answers
13 views

How to know the underlying structure of Multivariate Data?

"I'm currently working with multivariate data containing 18 features, and I'm looking to understand the underlying distribution of these features. My goal is to leverage this information to ...
Karandeep Saluja's user avatar
0 votes
0 answers
26 views

Proof for property of derivation operator for formal context: $A_1\subseteq A_2 \implies A^{'}_2\subseteq A^{'}_1$.

I wanted to verify the validity of my proof for the property of derivation operator for formal context. I will not go to detail that much and simply define few things before indroducing the property ...
Ilhom Sadriddinov's user avatar
0 votes
0 answers
12 views

How to determine the significance of a difference in a pearson correlation?

I'm developing some methods for processing images. I have applied seven methods (M1 to M2) in 10000 images and measured the effects of each method in each image according to two performance measures. ...
Zaratruta's user avatar
  • 131
-1 votes
1 answer
36 views

Finding a general formula for a data series

I have some numerical data tables, with a column of calculated results based on three inputs in each row. How can I find the formula to apply to the inputs and recalculate the same result ? the data ...
Bikay's user avatar
  • 31
1 vote
0 answers
15 views

How to mathematically prove the "transitive property of nested predictors"?

Note: I posted this question on the Stats.Stackexchange site a week ago, but I think it might be better here. It's about proving a theorem I came up with about experiment data set structures, ...
Chris Science's user avatar
0 votes
0 answers
22 views

notation for clusters of 2D data points

Is there any convention about the notation to use for clusters of $2-$D data points? I have a set of clusters of $2-$D data point. I can denote each cluster with $c_i$, where $i = 1, 2, ..., n$, and $...
Ommo's user avatar
  • 349
0 votes
0 answers
29 views

Linear Regression formula and vector derivatives

I am currently following Andrew NG's Standford Online Machine Learning course and decided to prove the formulas on my own since it that's my method of understanding them better. I am a bit stuck on ...
Ghost's user avatar
  • 29
0 votes
0 answers
13 views

Multiplicative Variance Analysis

I come from a math/stats background and as I learned some stuff as a data analyst I learned some techniques that I have never seen academically and was hoping if anyone has any knowledge of the ...
hyg17's user avatar
  • 5,165
0 votes
0 answers
21 views

Proposed analysis techniques - optimal decision given expectation

I am going to conduct an analysis in order to "weight" different possibilities of actions in a given market. I have an overall level of effort that can be distributed accross the different ...
Mathman's user avatar
  • 11
0 votes
0 answers
13 views

What type of rotational invariance information can I get from interest points (coordinates) of corners from an image?

Assume that you are using the FAST algoritm for corner/feature detection. You pick an image and run the FAST algorithm. Question: All these green dots are actully coordinates in x and y direction. ...
euraad's user avatar
  • 2,962
0 votes
0 answers
13 views

Complex Probability with Dependent and Independent Events with conditional

Can I make the simplification: P(D | A ∩ B ∩ C) = P(D | A) ? And if yes, please may i have the proof please ? Or a book where I can find it ? When : D is independant of B and C but it is dependant of ...
Luis's user avatar
  • 11
0 votes
0 answers
17 views

Is this a useful dimensionality reduction on real projective space?

In the following, I will derive a dimensionality reduction for the real projective space. As I will use scalar product as a means of distance, I am unsure if the dimensionality reduction has a useful ...
Niklas Netter's user avatar
1 vote
1 answer
42 views

coefficients of the linear model $y=β_0+β_1x_1+β_2x_2$ that minimises the sum of squares error [closed]

Consider the following sample: $$(x_{11}, x_{12}, y_1) = (1, 3, 4), $$$$(x_{21}, x_{22}, y_2) = (2, 1, 5),$$$$ (x_{31}, x_{32}, y_3) = (3, 0, 7),$$$$ (x_{41}, x_{42}, y_4) = (4, −2, 6).$$ How can I ...
1Mathsss's user avatar
2 votes
1 answer
96 views

optimisation problem that minimises the sum of absolute errors of the simple linear model $y = a + bx$.

Consider the following sample: $(x_1, y_1) = (1, 1)$, $(x_2, y_2) = (2, 5)$, $(x_3, y_3) = (3, 8)$, $(x_4, y_4) = (4, 18)$. How can I write down an optimisation problem that minimises the sum of ...
1Mathsss's user avatar
0 votes
1 answer
57 views

Calculating the RMSE and $R^2$ with a linear model $y = a + bx$ so that the mean squared error of the model is minimised [closed]

Consider the following sample: $(x_1, y_1) = (1, 1)$, $(x_2, y_2) = (2, 5)$, $(x_3, y_3) = (3, 8)$, $(x_4, y_4) = (4, 18)$. Suppose you predict $y$ based on $x$ with a linear model $y = a + bx$ so ...
mathana's user avatar
  • 27
0 votes
2 answers
97 views

logistic regression model to calculate the log odds

Let’s say that you have estimated the following logistic regression model to calculate the log odds of a $4$ year old child being obese $(y = 1)$ using the independent variable weight in kgs $(x)$: ...
1Mathsss's user avatar
0 votes
0 answers
126 views

calculate all pairwise dot (inner) products between data points

Suppose we have $n$ data points in $\mathbb{R}^p$ and we would like to calculate all pairwise dot (inner) products between them. How can I write a single matrix multiplication that does this? (Some ...
user1052623's user avatar
0 votes
0 answers
14 views

Test normality from histogram data

I have histogram data that I'd like to formally test for normality but I don't have the actual data points that generated the histogram. The histogram looks something like this: Is there a standard ...
Matías Santurio's user avatar
1 vote
0 answers
55 views

Persistence Vector Spaces

I am currently reading Gunnar Carlsson's "Topological Pattern Recognition for Point Cloud Data", you can find it here: http://math.stanford.edu/~gunnar/actanumericathree.pdf I have a ...
Red Phoenix's user avatar
2 votes
0 answers
58 views

How to make the income distribution of a country follows the 80:20 rule?

My main question: Let's imagine a country with a population of $n$ people. Each person has a certain amount of income in a certain year. When we calculate the income distribution of this country, we ...
MonsterTruck123's user avatar
0 votes
1 answer
56 views

Need method to fit specific function to measured data

I have a function $$ f(x, a, b, c, d)=\frac{a+b \times x}{c+d \times x}$$ where for a set of different x there are measurement results corresponding to $f(x, a, b, c, d)$. It is the parameters $a, b, ...
user684566's user avatar
2 votes
0 answers
227 views

Proof of Eckart-Young Theorem (Mathematics for Machine Learning, Deisenroth)

I am trying to understand a proof of the Eckart-Young Theorem (source in title). Let me add the definition and proof they provided, afterward I'll say what I do not understand. Proof: Some points I ...
Aaron's user avatar
  • 67
0 votes
0 answers
24 views

Design system debt calculation

Thanks in advance for your help. I am a designer working on a software platform. I am working on a calculation that captures the amount of redundancy/design debt in our design system on a component ...
SamCC's user avatar
  • 1
1 vote
1 answer
38 views

Approximating Linear Equations from Measurements

I've been working on a fascinating problem involving measurements that reveal a relationship between two variables, $t$ and $y$. I have a set of four data points: $(-1, 0)$, $(0, 1)$, $(1, 2)$, and $(...
Bishop_1's user avatar
  • 379
1 vote
1 answer
33 views

Function form to fit a nonlinear trend

I'm trying to guess a function that would fit the non-oscillating part of the following curve: I tried something in the form of $y = 1 - e^{-ax}$ and $tan^{-1}(ax)$, but it didn't work since the ...
ValientProcess's user avatar
0 votes
0 answers
15 views

Modeling Regional Quality Based on SUM of GCL / SUM of TAP - Comparing Two Approaches

I'm learning Data Science and I'm currently working on analyzing loan data across different regions. My goal is to build a model that can assess the quality of a given region based on the ratio $$\...
WawMathematician's user avatar
0 votes
0 answers
15 views

Unifying multiple sample data

I would like to unify multiple sample data (of different sizes) into some "unified sample" to evaluate its collective variance. Is this something statisticians do? I thought of unifying ...
Elad Brod's user avatar
0 votes
0 answers
13 views

Modeling Fog Formation Percentage with Exponential Decay Based on Dew Point Depression and Wind Speed

I am working with a 15-minute time series dataset that includes columns for Dew Point Depression and Wind Speed. Utilizing specific conditions based on these columns, I've successfully categorized fog ...
Sariq Sahazada's user avatar
0 votes
0 answers
30 views

Exponential Power Fit

I have 3 data points for a material property (permeability of a gas through an solid elastomer) as a function of temperature. My goal is to extrapolate the property at a lower temperature. As a ...
RC_23's user avatar
  • 111

1
2 3 4 5
22