Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

0
votes
0answers
10 views

Functional data analysis: How to find the error term inorder to calculate the variance

Functional data analysis: How to find the error term in order to calculate the variance I am trying to do functional data analysis problem mentioned below: What I don't know is how to find the ...
0
votes
0answers
15 views

Conditional Interpretations of Linear Regression

We estimate a linear regressor in the 1 dimensional with x and y random variables with zero mean: y/x = $\alpha$ x We can rewrite this using the variance of the variables as: y/x = $\rho \frac{\...
0
votes
0answers
22 views

Fitting a quadratic model using simple linear regression

Suppose that the true model is $Y_i = \beta_0 + \beta_1X_i + \beta_2X_i^2 + \epsilon_i, 1 \leq i \leq n,$ where $ \{\epsilon_i\}$ are i.i.d normal random variables with mean zero and variance $ \...
0
votes
1answer
41 views

General Gauss-Markov theorem [closed]

$Y=XB+u$ where $X$ is a non random $n\times k$ Matrix, $\textrm{rank}(X)=k, E(u)=0, E(uu')=\sigma^2\Omega$, How to form $(1)$ How to proof $(2)$ the general Gauss-Markov theorem?
0
votes
0answers
12 views

Inverse of regression coefficient

rearranging the normal linear regression function with one variable yields $$-b_0/b_1+1/b_1*Y_i-1/b_1*u_i=X_i$$. The question for me is why the regression coefficient in front of Y, $1/b_1$, is not ...
0
votes
0answers
57 views

Proof in regression model

Im studying for an exam, i don't have the solution, so I hope some of you guys can help me. I have tried a lot but i can't do this proof. Here is the task: Suoppose we have the linear regression ...
0
votes
0answers
17 views

Valid multivariate Gaussian model

I wanted to know if someone could explain the reason why if we have $n < (1/2)p(p+1)$ the number of parameters $p$ is greater than the number of samples $n$. I just can't figure out where does this ...
0
votes
0answers
26 views

Optimal solution for linear regression problem

I have an assignment for university about the linear regression problem and the optimal solution. For the linear regression problem $\min_w ||Y - X\omega||^2$, $\omega^*$ is the optimal solution. ...
-2
votes
1answer
18 views

I can't find the product moment coefficient of correlation? [closed]

These are the equations of least square regression lines: $ Y = 20.8 - 0.219 X $ ($Y$ on $ X$) $X = 16.2 - 0.785 Y$ ($X$ on $Y$) Find the coefficient of correlation $r$.
0
votes
0answers
13 views

Variance of Residuals in Multiple Linear Regression

If I have $n$ variables $x_1,\dots,x_d$ in my dataset, and I regress the first one against all the others i.e. $x_1=a_{12}x_2 + a_{13}x_3 + \dots + a_{1d} x_d + \epsilon_1$, how can I calculate $\...
0
votes
0answers
6 views

Predictions from bivariate regression model

I have a bivariate (2 responses variables) regression model and I am stuck with the following predictions model and prediction Basically I need to predict Y_1 taking into account also the correlation ...
1
vote
1answer
98 views

Basic application of category theory to data science

Linear regression is the algorithm that, given a set of vectors ${\bf x}_i \in \mathbb{R}^p$, and a set of targets $y_i \in \mathbb{R}$, returns a vector ${\bf w} \in \mathbb{R}^p$ minimizing the ...
1
vote
0answers
27 views

Linear regression model

Assume the usual linear regression model with $Y = X \beta + \epsilon$, where $X$ is fixed and known and $E(\epsilon) = 0$ and $\operatorname{var}(\epsilon) = \sigma^2I$. Let $a$ be an $n$-...
0
votes
1answer
27 views

Regression: width of confidence interval vs t-statistic

My question is about confidence intervals of the slopes estimated in a multivariate regression. I would like to clear up something that is probably a fault somewhere in my understanding. As I ...
-1
votes
0answers
13 views

Angle between regression line and SD line when plots flipped

The question I am working on is: The scatterplot y* vs x* has a regression line that makes an angle with SD line. Then the corresponding angle for the scatterplot of x* vs y* is: I have generated ...
1
vote
0answers
22 views

Regression — gradient descent versus statistical methods

In a machine learning course, the professor described the gradient descent method for calculating the regression line. Shortly, we're looking for the $a$ and $b$ in $y=ax+b$ which describes the line. ...
0
votes
0answers
21 views

Error Term, Regression

Assume I have $Y_i=A+BX_i + \epsilon_i$ For a multiple linear regression I have, $Y_i=A+B_1X_1+B_2X_2+....+B_NX_N+\epsilon$ The question is, what is the distribution of $\epsilon$, knowing that $\...
1
vote
3answers
44 views

What method does a calculator use to calculate a linear regression line?

Take three coordinates $(1,1)$, $(3,2)$ and $(4,3)$. My calculator returns the linear regression line: $$y=0.6429x+0.2857$$ of the form $$y = ax +b$$ correct to four significant figures for constants ...
2
votes
1answer
27 views

Time-series modeling

I'm wondering what methods can be used to predict a future value using past values. I looked into linear regression modeling, but this doesn't allow for a time value. As an example, say I have an ...
0
votes
0answers
28 views

Proof for normal posterior distribution for a data for linear regression.

Let $D=(x_1,x_2,....X_N)$ be the data and $x_i$ is D-dimensional data. For linear regression the likelihood is as follows $$p(y|X,w, u, \sigma^2) = N(y|u + Xw, \sigma^2 I_N) \propto exp(-\frac{1}{2\...
0
votes
0answers
25 views

Interpret model estimates after log transformation

I sat up a mixed-effects linear model with the dependent variable log-transformed (in oder to get it normal distributed and as ist is common with this kind of data in other publications). All fixed ...
0
votes
0answers
21 views

find $\operatorname{Var}(\hat\epsilon_i)=\sigma^2({{n-1}\over n}-{{(x_i-\bar x)^2}\over{\sum_{i=1}^n}(x_i-\bar x)^2})$

I'm studying question 16.19 in the book Probability Essentials. The question is to show that: $\operatorname{Var}(\hat\epsilon_i)=\sigma^2({{n-1}\over n}-{{(x_i-\bar x)^2}\over{\sum_{i=1}^n}(x_i-\bar ...
0
votes
1answer
245 views

Correlation between Least Square Parameters obtained in Linear Regression

I am trying to solve Q 22 below: Upon minimizing the squared errors I get the following values of the parameters: $$\hat{\alpha} = \overline{y}$$ $$\hat{\beta} = \frac{\sum x_i y_i - \overline{x}\...
2
votes
1answer
49 views

Rank of sub-matrix of projection matrix

Consider the projection matrix in Linear Regression $P=X(X^TX)^{-1}X^T$. If we have $n$ points, $P$ is an $n$ x $n$ matrix. We also know it satisfies a number of properties, including that it's ...
1
vote
0answers
8 views

Regression task on desirable subsets with limited supervision

Suppose I have a set of n elements. I want to have a model for how "desirable" a subset of those elements is when evaluated by a person. The training/testing data is a number of given subsets, each ...
1
vote
1answer
19 views

How to interpret the vectors and design matrix in a linear model

In regression, linear models are of the form: $$y_i = \pmb z_i^T \pmb\beta_i + \epsilon_i$$ Or we can write this in a more general form with vectors and a design matrix: $$\pmb y = \pmb Z \pmb \beta ...
0
votes
1answer
36 views

What is weighted and unweighted linear regression in machine learning?

I'm taking Stanford's CS229 ML course and while studying about "parametric algorithms", Prof. Andrew Ng says that this class of algorithms has a fix number of parameters (parameters are also called as ...
2
votes
1answer
21 views

Bias and variance of IV estimation

NOTE: I have asked this question on stat.stackoverflow but got no answers/comments. Hence I decide to ask it on the math.stackoverflow platform as well. I'm studying IV estimation by myself and have ...
0
votes
0answers
28 views

what are $SSE$ and $S_{y,x}$?

I'm a BC student who is trying to solve some statistic quizzes. there is a multiple choice question that is this: In a simple regression model $y=a+bx+e$ our given data is this: $$ \bar{x} = 2, \...
0
votes
1answer
47 views

$Y_1,Y_2,Y_3$ are uncorrelated rvs such that $E(Y_1)=\beta_1+\beta_2$,$E(Y_2)=2\beta_1$ and $E(Y_3)=\beta_1-\beta_2$

Let $Y_1,Y_2,Y_3$ be uncorrelated random variables with common variance $\sigma^2 > 0 $ such that $E(Y_1)=\beta_1+\beta_2$,$E(Y_2)=2\beta_1$ and $E(Y_3)=\beta_1-\beta_2$ where $\beta_1$ and $\...
0
votes
2answers
28 views

Including the (0,0) point in regression

I have run a simple linear regression in Rstudio with two variables and got the following relation: y = 30000+1.95x Which is reasonably fair. My only concern is that, practically the (0,0) point ...
1
vote
1answer
26 views

Logistic Regression Explanation

I have two questions regarding logistic regression. 1) I understand that the results of a logistic regression model yield a table stating coefficients together with a p-statistic for each variable . ...
0
votes
2answers
25 views

Linear Regression Assumption

the following website states that linear regression assumes a linear relationship between dependent and independent variables: "First, linear regression needs the relationship between the independent ...
0
votes
1answer
33 views

Linear Regression Assumption: Normality of residual vs normality of variables

I have read in many places, including stack exchange, that in order to carry linear regression analysis the residuals have to be normal. This is required because most of the statistical results, ...
0
votes
0answers
17 views

Operating with penalized regression models

Assume we are working with a penalized linnear regression model. We have the following optimization problem: \begin{equation} \min_{\beta}\left\{\left\lVert y-X\beta\right\rVert_2^2+\lambda\sum_{j=1}^...
0
votes
1answer
26 views

Proof of Batch Gradient Descent's cost function gradient vector

In the book Hands-On Machine Learning with Scikit-Learn & TensorFlow, the author only showed the formula for the Batch Gradient Descent method, such as: $ \dfrac{\partial}{\partial \theta_{j}} ...
1
vote
1answer
94 views

Partial derivative of MSE cost function in Linear Regression?

I'm confused by multiple representations of the partial derivatives of Linear Regression cost function. This is the MSE cost function of Linear Regression. Here $h_\theta(x) = \theta_0+\theta_1x$ . \...
1
vote
0answers
24 views

What is saturated model used in glm in R?

So, I've read on stack exchange (glm summary explained) that residual deviance computed in the glm output is just the likelihood ratio chi-square stat comparing the saturated model to the reduced ...
1
vote
0answers
17 views

How to find the correspondence between ridge coefficients to modified ridge?

The ridge coefficients minimize a penalised residual sum of squares: $\hat{w}_{ridge}= \arg\min(\sum_{i=1}^{N}(y_{i}-w_{0}-\sum_{j=1}^{d}x_{ij}w_{j})^{2}+\lambda\sum_{j=1}^{d}w_{j}^{2}),\,\lambda\...
0
votes
1answer
12 views

Effect on $R^2$ squared of an additional regressor

I'm currently dealing with the simple linear regression model and the book I'm studying with says that, any time you add a regressor to your model, even if irrelevant, the coefficient of determination ...
0
votes
0answers
14 views

Multiple Regression Assumption

If I am running a multiple linear regression model with six independent variables against dependent variable, do the assumptions of multiple regression need to be satisfied? or does this only applies ...
0
votes
2answers
32 views

Formulating the polynomial regression

I'm trying to formulate a regression problem such that $y=ax^{b}$. Previously, I formulated the $y=ax+b$ like $y=Ac+e$ where $c= \begin{bmatrix} a\\ b \end{bmatrix}$ and $A= \begin{bmatrix} x_1 &...
0
votes
0answers
43 views

3d linear regression line equations

I have a bunch of points in 3D space (around to a survey line). Since only half of the point coordinates were measured, I want to compute a linear regression, from which I extrapolate the other points ...
1
vote
0answers
17 views

Regressions with two independent variables where one is part of another.

I am doing simple regression analysis where I use one-way fixed effect model to estimate the effects of two variables on dependent variable. The question I am asking is how to interpret these two ...
0
votes
0answers
41 views

Fitting a Straight Line to 3D Points, and then finding the nearest point to those lines.

I've been stuck on this problem for over a week now. Essentially, I have a group of 4 tracks of points from a bunch of detectors in 3 dimensions. I need to map these points to straight lines and then ...
1
vote
0answers
18 views

How to calculate $\text{cov}(\hat{Y}_{ij}, \hat{Y}_{kj})$ if $Y_{ij} = \mu + a_i + b_j + e_{ij}$?

Let's assume we have the model following Two-Factor model without replications : $$Y_{ij} = \mu + a_i + b_j + e_{ij}, \; i=1,\dots,p \; \text{and} \; j=1,\dots, q $$ I am interested in calculating ...
2
votes
0answers
20 views

Covariance of errors $\text{cov}(\hat{e_{ij}},\hat{e_{i\ell}})$ in Two-Way Anova model

Exercise : Consider the Two-Way Anova model $Y_{ij} = \mu + a_i + b_j + e_{ij}$ with $i = 1, \dots, p$ and $j=1,\dots,q$. Show that : $$\text{cov}(\hat{e_{ij}},\hat{e_{i\ell}}) = -\sigma^2\left(\...
1
vote
1answer
45 views

fitting triple exponential term function to data

The function, I am trying to fit to data is: $$y(x) = −(A+B)e^{−x/a_1} + 𝐴e^{−x/a_2} + Be^{−x/a_3}$$ this function is a little bit different to Is it possible to find initial parameters when fitting ...
0
votes
1answer
17 views

question about matrix equation for coefficients in linear regression

There is a matrix equation for solving a linear regression, $\vec{y}=X\vec{\beta}$ where $X$ is the matrix of features, $\vec{\beta}=[\beta_1,...,\beta_n]$ are the coefficients for each feature, and $\...
0
votes
1answer
14 views

Interpreting linear regression coefficients for a covariate that's correlated with other covariates

The interpretation of linear regression coefficients that I learned is that the coefficient is the change in outcome associated with a unit change in that covariate, assuming all other covariates stay ...