Questions tagged [data-analysis]
Questions on the use of mathematical techniques to extract properties from given data. Consider if your question might be more suited for Cross Validated (stats.SE) instead.
1,049
questions
-6
votes
0
answers
42
views
About quantitative analysis, plz help.. [closed]
I have questions below, which i cannot find the problem.
Please teach me to follow up the answer.
enter image description here
0
votes
0
answers
19
views
Proposed analysis techniques - optimal decision given expectation
I am going to conduct an analysis in order to "weight" different possibilities of actions in a given market. I have an overall level of effort that can be distributed accross the different ...
- 11
-2
votes
0
answers
20
views
PCA and multi dimensional dataset e.g. Hyperspectral photos [closed]
I do not know why my question was downvoted but let me try one more time.
I know the purpose of PCA is to reduce the number of dimension of vector. It finds the space with the highest variance and “...
- 17
0
votes
0
answers
38
views
Find formula for a known dataset [migrated]
I am trying to figure out how the computer in this game works.
In short: it's like normal monopoly, except instead of owning a property comletely everybody can buy shares for the property (max 9 ...
- 101
0
votes
0
answers
13
views
What type of rotational invariance information can I get from interest points (coordinates) of corners from an image?
Assume that you are using the FAST algoritm for corner/feature detection. You pick an image and run the FAST algorithm.
Question:
All these green dots are actully coordinates in x and y direction. ...
- 2,860
0
votes
0
answers
12
views
Complex Probability with Dependent and Independent Events with conditional
Can I make the simplification: P(D | A ∩ B ∩ C) = P(D | A) ? And if yes, please may i have the proof please ? Or a book where I can find it ?
When : D is independant of B and C but it is dependant of ...
- 11
-2
votes
0
answers
26
views
Linear vs nonlinear interpolation [closed]
I have two questions
How does one determine whether to use linear or nonlinear interpolation? Please give examples.
Edit: In the case of CDF, say you have CDF numerically and you want to find the ...
Time The Dream
0
votes
0
answers
14
views
Is this a useful dimensionality reduction on real projective space?
In the following, I will derive a dimensionality reduction for the real projective space. As I will use scalar product as a means of distance, I am unsure if the dimensionality reduction has a useful ...
1
vote
1
answer
39
views
coefficients of the linear model $y=β_0+β_1x_1+β_2x_2$ that minimises the sum of squares error [closed]
Consider the following sample:
$$(x_{11}, x_{12}, y_1) = (1, 3, 4), $$$$(x_{21}, x_{22}, y_2) = (2, 1, 5),$$$$ (x_{31}, x_{32}, y_3) = (3, 0, 7),$$$$ (x_{41}, x_{42}, y_4) = (4, −2, 6).$$
How can I ...
- 75
0
votes
0
answers
21
views
linear regression model that uses both input variables $x_1$ and $x_2$ to a given data
Considering the following sample:
$$(x_{11}, x_{12}, y_1) = (1, 3, 4), (x_{21}, x_{22}, y_2) = (2, 1, 5), (x_{31}, x_{32}, y_3) = (3, 0, 7).$$
If we fit a multiple linear regression model that uses ...
- 35
2
votes
1
answer
74
views
optimisation problem that minimises the sum of absolute errors of the simple linear model $y = a + bx$.
Consider the following sample:
$(x_1, y_1) = (1, 1)$, $(x_2, y_2) = (2, 5)$, $(x_3, y_3) = (3, 8)$, $(x_4, y_4) = (4, 18)$.
How can I write down an optimisation problem that minimises the sum of ...
- 75
0
votes
1
answer
52
views
Calculating the RMSE and $R^2$ with a linear model $y = a + bx$ so that the mean squared error of the model is minimised [closed]
Consider the following sample:
$(x_1, y_1) = (1, 1)$, $(x_2, y_2) = (2, 5)$, $(x_3, y_3) = (3, 8)$, $(x_4, y_4) = (4, 18)$.
Suppose you predict $y$ based on $x$ with a linear model $y = a + bx$ so ...
- 35
0
votes
2
answers
64
views
logistic regression model to calculate the log odds
Let’s say that you have estimated the following logistic regression model to calculate the log odds of a $4$ year old child being obese $(y = 1)$ using the independent variable weight in kgs $(x)$:
...
- 75
0
votes
0
answers
52
views
calculate all pairwise dot (inner) products between data points
Suppose we have $n$ data points in $\mathbb{R}^p$ and we would like to calculate all pairwise dot (inner) products between them.
How can I write a single matrix multiplication that does this?
(Some ...
- 71
0
votes
0
answers
12
views
Test normality from histogram data
I have histogram data that I'd like to formally test for normality but I don't have the actual data points that generated the histogram.
The histogram looks something like this:
Is there a standard ...
0
votes
0
answers
26
views
Using Bayesian statistics in time series forecasting
I would like to forecast demand count time series of taxi fleets at different locations on the map at different points in time. I.e. multivariate demand Time series forecasting.
Given hierarchinal ...
- 247
1
vote
0
answers
36
views
Persistence Vector Spaces
I am currently reading Gunnar Carlsson's "Topological Pattern Recognition for Point Cloud Data", you can find it here: http://math.stanford.edu/~gunnar/actanumericathree.pdf
I have a ...
- 629
2
votes
0
answers
56
views
How to make the income distribution of a country follows the 80:20 rule?
My main question:
Let's imagine a country with a population of $n$ people. Each person has a certain amount of income in a certain year. When we calculate the income distribution of this country, we ...
0
votes
1
answer
51
views
Need method to fit specific function to measured data
I have a function
$$ f(x, a, b, c, d)=\frac{a+b \times x}{c+d \times x}$$
where for a set of different x there are measurement results corresponding to $f(x, a, b, c, d)$. It is the parameters $a, b, ...
1
vote
0
answers
110
views
Proof of Eckart-Young Theorem (Mathematics for Machine Learning, Deisenroth)
I am trying to understand a proof of the Eckart-Young Theorem (source in title). Let me add the definition and proof they provided, afterward I'll say what I do not understand.
Proof:
Some points I ...
- 57
0
votes
0
answers
22
views
Design system debt calculation
Thanks in advance for your help.
I am a designer working on a software platform.
I am working on a calculation that captures the amount of redundancy/design debt in our design system on a component ...
- 1
0
votes
1
answer
34
views
Approximating Linear Equations from Measurements
I've been working on a fascinating problem involving measurements that reveal a relationship between two variables, $t$ and $y$. I have a set of four data points: $(-1, 0)$, $(0, 1)$, $(1, 2)$, and $(...
- 335
1
vote
1
answer
31
views
Function form to fit a nonlinear trend
I'm trying to guess a function that would fit the non-oscillating part of the following curve:
I tried something in the form of $y = 1 - e^{-ax}$ and $tan^{-1}(ax)$, but it didn't work since the ...
- 231
0
votes
0
answers
15
views
Modeling Regional Quality Based on SUM of GCL / SUM of TAP - Comparing Two Approaches
I'm learning Data Science and I'm currently working on analyzing loan data across different regions. My goal is to build a model that can assess the quality of a given region based on the ratio $$\...
0
votes
0
answers
14
views
Unifying multiple sample data
I would like to unify multiple sample data (of different sizes) into some "unified sample" to evaluate its collective variance. Is this something statisticians do?
I thought of unifying ...
0
votes
0
answers
12
views
Modeling Fog Formation Percentage with Exponential Decay Based on Dew Point Depression and Wind Speed
I am working with a 15-minute time series dataset that includes columns for Dew Point Depression and Wind Speed. Utilizing specific conditions based on these columns, I've successfully categorized fog ...
0
votes
0
answers
22
views
Exponential Power Fit
I have 3 data points for a material property (permeability of a gas through an solid elastomer) as a function of temperature. My goal is to extrapolate the property at a lower temperature.
As a ...
- 111
0
votes
0
answers
11
views
Ranking drivers of change in a time dependent weighted average
Let $C(t)$ be a discrete, finite set of Customer IDs with $|C(t)| =n(t) >0$ denoting the number of customers in the set at time $t \in \mathbb{N}$. Consider the following time-dependent weighted ...
0
votes
1
answer
55
views
Normalizing Euclidean distance by the length of the vectors
Suppose I have 4 vectors, the first 2 vectors are of length 4 and the last 2 vectors are of length 400. all values in the vectors range from 0.5 to 0.6.
The Euclidean distance between the last 2 ...
- 11
1
vote
0
answers
22
views
Standard deviation of standard deviation
Let's say $f=X1+X2$ and I am trying to find the standard deviation (SD) of $f$. Through error propagation the SD of $f$ is $\sigma_f=\sqrt{\Delta X1^2+\Delta X2^2}$, where $\Delta X1=\sqrt{X1}$ and $\...
- 11
2
votes
0
answers
60
views
Finding rational coefficients of a cubic polynomial that fits 4 data points that have been floored to an integer
I have 4 data points:
(204, 5422892)
(205, 5722486)
(207, 6343357)
(213, 8386502)
I have information that these data points were generated with a cubic polynomial
$y = ax ^ 3 + bx ^ 2 + cx + d$
with ...
- 119
0
votes
0
answers
39
views
Do Gaussian Processes not generalize well out of the training range?
I'm a beginner of the Gaussian Processes (GPs). I would like to know the generalizability of GPs. I trained a GP using training data generated by an objective function (the range $-5<x<5$, 10 ...
- 165
0
votes
0
answers
19
views
Calibration Resolution
I have a flow meter that I calibrate by doing timed catches. The best resolution I have on my scale is down to 0.01 lb. The meter I'm comparing against goes to 0.001 lb. Usually, I'll have something ...
0
votes
0
answers
28
views
How do I rotate a 2D catenary in 3D space?
I have data points of a catenary curve in 3D space (x,y,z co-ordinates). I am able to fit this data in 2D space properly with normal catenary equation y = a*cosh(x/a). However, on 3D space the ...
- 21
0
votes
0
answers
44
views
How does the Radial Basis Functions transform multimodal distributions to normal distributions?
In the book Hands-On Machine Learning by Aurélien Géron, in chapter two, the author states:
Another approach to transforming multimodal distributions is to add a feature for each of the modes (at ...
2
votes
0
answers
52
views
$[1]$ Does good $R^2$ necessarily tells us that the experiment went right? $[2]$ Is not extrapolation dangerous when used for a far point?
In an experiment, the relation between $x$ and $y$ is linear.
$x_\text{actual} = \alpha_x x_\text{observed} + \beta_x + \gamma_x$. [Here, $\alpha$ is the slope, $\beta$ is the intercept, and $\gamma$ ...
- 4,868
5
votes
1
answer
123
views
What is the exact function or algorithm Windows 10 is using to calculate taskbar underline color from Registry values, based on data I have collected?
In earlier versions of Windows, it was easy for users to set the exact colors of interface elements. For example, in Windows 7, you could go to [Control Panel -> Personalization -> Window Color],...
- 71
0
votes
1
answer
41
views
Fitting a line to the outermost plot on a graph
I am plotting the nonlinear regression on some fatigue data. My goal is to create a line that adheres to the slope line (in red). This line should sit beneath the lowest point on the left side of the ...
- 9
1
vote
0
answers
29
views
SIMPLS - partial least squares
I am currently learning partial least squares analysis. I understand the idea, and the method I know to get PLS is NIPALS. Since I am currently writing a paper about this subject, I want to learn ...
0
votes
0
answers
47
views
Estimating integer integrals based on other definite integrals
I'm working with a continous dataset that is incomplete and my task is to find the most likely values assuing normal distribution of all data.
I'm given a series of integrals such that $\int^5_{0}f(x)...
0
votes
0
answers
20
views
Center of Sums method - centroid of different shapes question
I'm doing fuzzy logic and I need to use the center of sums method to calculate a value.
I have the graph of functions here:
Sorry for the low quality, but I think this suffices.
The formula is with ...
- 893
-3
votes
1
answer
74
views
Best fit line (least squares method) giving garbage results
I don't know if I should post this in Mathematics or Stack Overflow. But its a math issue but its programming code.
Anyway, I went on this math website and put their method, it gave me total garbage ...
- 1
0
votes
1
answer
49
views
Finding the form of $z$ in terms of $x$ and $y$, letting $z(x,0)$ to be $0$ forcefully.
In a practical experiment, we have two independent variables, $x>0$ and $y \ge 0$, and one dependent variable, $z \ge 0$.
Theoretically, we know that at any fixed $x$, we have the relation between $...
- 4,868
0
votes
0
answers
19
views
Are there algorithms for 3d partial retrieval?
Suppose that $\{ P_i \}$ is a dataset of point clouds(PCDs), i.e., set of collections of 3-d real vectors. I have a query PCD $Q$ and I want to find all $P_i$ such that $P_i$ contains a subset (which ...
- 1
1
vote
0
answers
52
views
Low rank approximation with MSE approach
Suppose that we have a matrix $\mathbf{A}\in\mathbb{R}^{n\times p}$. If we want to find an approximation of $\mathbf{A}$ with maximum rank $k$, which is denoted as $\tilde{\mathbf{A}}$, is it possible ...
- 11
0
votes
0
answers
13
views
How to determine trend and uncertainty from dataset
I have a dataset with the cumulative ice mass change of the Greenland Ice Sheet with units Gt (Gigatonnes) and its uncertainty (error estimate).
...
3
votes
1
answer
297
views
What does the PDF of a CDF² mean?
I have empirical data representing a CDF. I fitted a normal/gaussian CDF (the line) to it, call it F(x)
I have found that squaring the data points gives a better fit; F(x)² = P(X ≤ x)²:
And here's ...
- 211
0
votes
0
answers
24
views
What are practical uses for the Triangle Matrix?
So I'm learning data science concepts for analysis of stock price data and I've gone into a rabbit hole. I first came across numpy.triu() in the Numpy library ...
- 101
0
votes
0
answers
25
views
Example for conditional Distribution and correlation between variables following normal distribution
A car company tests an upcoming car model for gas consumption $(x_1)$ and oil consumption $(x_2)$.From past tests it is known that the expected gas consumption is and the expected oil consumption is ...
- 49
1
vote
1
answer
124
views
A Bayesian network with 4 nodes question
Hello. I need to find the following probabilities:
$ \qquad a. P(\neg A, B, \neg C, D) \\
\qquad b. P(A|B,C,D) $
This is my solution:
I would like to get an opinion on my solution.
- 52