Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
22 views

How to fit sum of products of sine waves?

System Model: \begin{align} Y(t_1, t_2, t_3) = A \bigg[ & 2+ k_1\cos(w_1t_1+\phi_1) +k_2\cos\left(w_2t_2+\phi_2\right)+ \\[2ex] ...
0
votes
0answers
17 views

Maximum likelihood bivariate normal

I have the following model: $$x_{t+1} = x_t + a + \epsilon_{t+1}$$ $$y_{t+1} = b y_t + \delta_{t+1} $$ Where all the $x_t$ and $y_t$ are known and where the $(\epsilon_t,\delta_t)$ are independent ...
0
votes
0answers
6 views

Compute linear regression slope over $[x_i, y_i]$ when $x_i$ samples over $x$ are regularly distributed.

Context: To have an idea of the trend-line of a set of samples $[x_i, y_i]$, I usually compute the slope $a$ in the linear regression ($y = ax + b$) with a spreadsheet software and the formula: $$ a ...
-1
votes
0answers
9 views

Unit root test: Regression Analysis [on hold]

Null Hypothesis: SP500RLN has a unit root Exogenous: Constant, Linear Trend Lag Length: 1 (Automatic - based on SIC, maxlag=24) ...
1
vote
2answers
33 views

Power Regression $y=Ax^B+C$

I have to do reproduce a power regression but I don't have any experience in procedures like that. I read a little bit about power fit/power regression and that a formula like $y = ax^b$ is used for ...
0
votes
0answers
25 views

Kalman Filter and OLS Results Are Different

I read that Kalman Filters can be used for continuous / online linear regression and at the end of the regression its results and ordinary linear regression (OLS) results would be the same. I tried it ...
1
vote
1answer
38 views

When is $\mathbf{X}^{T}\mathbf{X}+\lambda\mathbf{I}$ invertible?

The question is quite simple: for a $N \times p$ matrix $\mathbf{X}$ with real entries, when is $\mathbf{X}^{T}\mathbf{X}+\lambda\mathbf{I}$ invertible (where $\mathbf{I}$ is the $p \times p$ identity ...
0
votes
1answer
28 views

Why does $E(C\cdot \epsilon\; \vert\; C\cdot X) = E(C\cdot \epsilon\; \vert\; X)$?

Let $C$ be an $n\times n$ matrix, $X$ is $n \times k$, $\epsilon$ is $n \times 1$ This is taken from a simply proof of strict exogeneity in an Econometrics textbook by Hayashi. The explanation he ...
0
votes
1answer
19 views

Simplification of a product of three matrices

Define $$\mathbf{c}_t = \begin{bmatrix} x_{1t} \\ x_{2t} \\ \vdots \\ x_{Nt} \end{bmatrix}\in \mathbb{R}^N$$ where all entries are in $\mathbb{R}$, $t = 1, 2, \dots, p+1$. I am trying to simplify ...
0
votes
1answer
28 views

Relating regression to projection?

I recently learned that one can think of regression as a projection of a vector in a high dimension space onto the other vector. I tried implementing this and got it to work: ...
0
votes
3answers
55 views

How to do a regression which includes reciprocals?

I'm trying to find an interpolating formula for a set of coefficients (I have $80$ at the moment). I tried first to find an interpolating polynomial, but that was not useful: using the first ...
1
vote
0answers
53 views
+100

Interpolation and mapping between scattered vectors in two unequally dimensioned spaces

Imagine two spaces: An ‘input’ space with dimension $m$. An ‘output’ space with dimension $n$. $m \geq n$ There are points in each of these spaces defined such that some characteristic is ...
0
votes
2answers
24 views
0
votes
2answers
34 views

Linear regression using gradient descent: is the whole weight vector updated with the same number?

I'm using gradient descent with mean squared error as error function to do linear regression. Take a look at the equations first. As you can see in eq.1, the prediction is done with a bias term b ...
1
vote
1answer
48 views

Why the identity $P_X=P_XZ(Z'P_XZ)^{-1}Z'P_X$ with $P_X=X(X'X)^{-1}X'$?

Suppose $X$ and $Z$ are matrices such that $(X,Z)$ and $P_XZ$ both have full column ranks. Here, $P_X=X(X'X)^{-1}X'$. Consider a regression model $$ P_Xy=P_XZ\zeta+v\tag{A} $$ where OLS is used ...
0
votes
1answer
22 views

Sum of component projection matrices

Show that if $X$ $=$ [$X_1$ $X_2]$ and $X_1'X_2 = 0$, then $P = P_1 + P_2$, where $P$ is defined as $X(X'X)^{-1}X'$, the projection matrix. Don't quite know where to start. I tried evaluating it by ...
0
votes
0answers
19 views

Variance in the sum of batch-correlated residuals in a regression

I am looking at a regression model of the following form: $Y=intercept+\beta_{Yf.n}X_f+\beta_{Yn.f}X_n +error$ where $X_f$ and $X_n$ are predictors. A value for $Y$ will be sampled from the ...
0
votes
0answers
5 views

To compute the total number of possible candidate basis functions of Kriging

How can I prove that the total number of possible candidate basis functions is $C_{m + P}^P$ in equation (12)? Thanks very much!
0
votes
0answers
8 views

Interpretation of diagonal detail in Haar Wavelet Transforms

I am a statistics grad student, and I have just begun exploring the topic of wavelet regression (specifically, Haar wavelets for discrete functions). I understand the generalization from a one ...
1
vote
2answers
16 views

Inverse of sum of matrices (SVD, ridge regression)

Looking at these slides, I've found the following: $X=UDV^T$, where $U$ and $V$ are orthogonal matrices, $V$ is a square matrix, and $D$ contains the singular values of $X$. The author then writes ...
0
votes
0answers
17 views

Relationship/correlation between data - does it exist?

Data I refer to in this question Some analysis has been conducted for my business by an external company. The data, as it stands, only really tells part of the story and doesn't provide any real ...
0
votes
2answers
23 views

exponential regression for bacteria growth

I'm studying regression lines and curves, and I've learn the methods for working with curves of the types $ax^2+bx+c$ and $ax+b$ as well as $a\sin(x)+b\cos(x)$. Now I'm asked this: $$(0,32), ...
0
votes
1answer
30 views

How do you calculate the correlation between the intercept's and beta's standard error in a univariate linear regression?

I am running a regression to predict a variable Y as follows: $Y=\alpha+\beta\times x+\epsilon$ I am trying to get a distribution of the expected value of Y given standard errors in the model ...
1
vote
1answer
47 views

What are the limitations of linear regression + feature / label transformation?

Regression Suppose I have data points in a matrix $X \in \mathbb{R}^{n \times m}$ as well as labels $\mathbb{R}^n$, where $n$ is the number of my data points and $m$ is the number of features per ...
1
vote
1answer
16 views

How to show $Y = \sum_{i=1}^n \frac{Y_i}{n-2}$ is a biased estimator of the mean?

Let $\{Y_1, \ldots, Y_n\}$ be a random sample with $E(Y_i)= \mu_Y$ and $\operatorname{var}(Y_i) = \sigma_Y^2$. Show that $Y = \sum_{i=1}^n \frac{Y_i}{n-2}$ is a biased estimator of the mean? This is ...
1
vote
1answer
18 views

Comparing a series expansion to polynomial regression

So I don't have a great background in mathematics but I have a quick and hopefully simple question for you guys. I'm a graduate student and I'm doing some polynomial regression on some thermodynamic ...
0
votes
0answers
24 views

Using least squares regression to apply nonlinear function to time series data

If you have a nonlinear function (see example), can you use a least squares regression approach to fit it to time series data ? Is this approach also valid for n variables? How many time points are ...
0
votes
0answers
13 views

Predict the probabilty of a value belonging to a particular class

I have got two classes : BG and FG and a set of values assigned to each of these classes. Given a new value how can I find ...
0
votes
0answers
47 views

Help Determining Gradient for Equation

I am writing an OO program with geometric objects. My Plane object is capable of taking a collection of 3d points and determining the plane of best fit. I'm using this popular document on it from ...
2
votes
1answer
47 views

Estimating landing position for a slowly falling object using latitude, longitude, and altitude.

I have a weather balloon project, in which I intend to use GPS to locate the payload when it finally comes down again. I will make the computer send coordinates to a server every minute or so, as ...
0
votes
1answer
12 views

Statistic test for comparing two regression models

I'm having two linear regression models as follows: $y = a_1x_1 + a_2x_2 + c$ and $y = b_1x_1 + b_2x_3 + c$. I'm looking for a statistical test for proving which model is better. I've obtained the ...
2
votes
0answers
15 views

Ridge Regression Centering Proof

This is a ridge regression problem. The following two problems are equivalent: $(w_t, b_\lambda ) = argmin_{w,b}\{\sum_{i=1}^m (y_i-b-w^Tx_i)^2+\lambda w^Tw\} $ $(w_t, b_\lambda )= ...
1
vote
1answer
50 views

Multivariational regression

I have been given following model $\ln(y)=\beta_0+\beta_1a+\beta_2a^2+\beta_3a^3+\beta_4a^4+\beta_5b$ and a set of observations that describe relation between $y$ and $\{a, b\}$. The goal is to ...
2
votes
1answer
69 views

Relationship between $L^1$ norm and sparsity

I'm doing some research in the field of sparse representation and sparse modeling. I have two variables and their $L^1$ norm is calculated to make comparisons. As I take it the smaller the value of ...
0
votes
0answers
6 views

Categorical vs Numerical covariate linear regression

We have started doing regression at my university, and I would like to model ACT scores of students given some covariates. My problem is I have a covariate grade with values "A", "B", "C",and "F". ...
0
votes
0answers
8 views

FWL Theorem: Regressing $X_1$ on $X_2$ (a matrix on a matrix)

I am confused regarding the FWL theorem. One step is to regress $X_1$ on $X_2$ **to get the residuals from this regression*. Note that $X_i = (Txk_i)$ matrix. $k_1$ not necessarily equal to $k_2$. ...
0
votes
0answers
17 views

Rewriting residuals transposed times estimates with Annihilator and Projection: $e'\hat{y} +0 = y'M_xP_xy =0$

I am looking at a proof that says $$e'\hat{y} +0 = y'M_xP_xy =0$$ where $P_x$ is the projection matrix, $M_x$ is annihilator. I don't see how they obtain this result. What I am wondering is they are ...
0
votes
0answers
23 views

How to scale set of numbers

I have set of numbers. Set can have very big numbers (1E+13) and small (1). I need to scale numbers to range from 1 to 500. The scale should save original proportions. Table below shows that formula ...
0
votes
0answers
18 views

Neural network for regression

The way I understand regression for neural networks is weights being added to each x-input from the dataset. I want something slightly different. I want weights ...
2
votes
1answer
32 views

Add weights to inputs of x-value function to optimize regression [closed]

Say I have $n$ functions (not the regression function) each with $n$ inputs. These functions compute the x-values. The function is a simple summation function where the input is multiplied by a ...
-1
votes
0answers
17 views

Can I/ How to conduct logistic regression on non-binary data?

I have collected data points from an experiment. The data' domain is [0,312] and range is [0,98]. I know a logistics function will fit the data very well. I performed the logistics regression on my ...
0
votes
0answers
12 views

estimating $\sigma$ of regression based on Brownian motion

Suppose I have a regression that looks like this: $x_t = \alpha + \beta t + \sigma W_t$. Also suppose, I want to estimate $\beta$ and $\alpha$ using say $n$ discrete observations by a regression. ...
1
vote
1answer
33 views

Relation between correlation and regression

Let $y\in \mathbb{R}$ be a random variable. Let $y$ be expressed as a linear combination of $x_i$ $i=1,2,\cdots,n$, as follows \begin{equation} y = \sum\limits_{i=1}^nw_ix_i + \epsilon \end{equation} ...
0
votes
1answer
36 views

How to calculate the linear regression model of function $y=\alpha + \beta k + \beta x$?

I have a linear function $y=\alpha + \beta k + \beta x$ and observation data that consist of pairs of $x$ and $y$. $\alpha$, $\beta$, and $k$ are unknown parameters. I want to estimate the value of ...
0
votes
0answers
28 views

Estimated simple linear regression model

Consider the simple linear regression model $y=50 + 10x + \varepsilon$ where $\varepsilon$ is $NID (0,16)$. Suppose that $n=20$ pairs of observations are used to fit this model. Generate $500$ samples ...
0
votes
1answer
30 views

Computing vector linear regression

In eye tracking we have to compute the linear regression for pupil and gaze. The formula is: $$\begin{bmatrix} gaze_x \\ gaze_y \end{bmatrix} = \begin{bmatrix} \theta_1 \\ \theta_2 \end{bmatrix} + ...
0
votes
1answer
44 views

Why are the two answers different?

I found exponential regression on Desmos for a few values: I want to find the X value when Y is 40,000. So, when zooming in on 40,000, I see that X is around 160. But, when putting the equation: ...
2
votes
0answers
37 views

Does linear regression form a subspace?

The author writes Given a vector of inputs $X^T = (X_1, \dots ,X_p)$, we can predict an output $Y$ via $$ \hat{Y} = \beta_0 + \sum_{j = 1}^p X_j \beta_j$$ He goes on to note that if we include a 1 in ...
1
vote
0answers
41 views

Multiple Regression Forecast

I'm going through a book called 'Regression Analysis by example by Hadi/Chaterjee and encountered a exercise(3.13) using a regression-output Part C: asks what salary would you forecast for a man ...
0
votes
0answers
25 views

Implementation of the LOWESS-algorithim (local regression data smoothing)

I need to implement the LOWESS-algorithm in a piece of software I am working on. The LOWESS-algorithm is a type of filter, which applies a locally weighted regression on each data point. In this ...