Questions tagged [linear-regression]
For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.
0
votes
0answers
10 views
Functional data analysis: How to find the error term inorder to calculate the variance
Functional data analysis: How to find the error term in order to calculate the variance
I am trying to do functional data analysis problem mentioned below:
What I don't know is how to find the ...
0
votes
0answers
15 views
Conditional Interpretations of Linear Regression
We estimate a linear regressor in the 1 dimensional with x and y random variables with zero mean:
y/x = $\alpha$ x
We can rewrite this using the variance of the variables as:
y/x = $\rho \frac{\...
0
votes
0answers
22 views
Fitting a quadratic model using simple linear regression
Suppose that the true model is
$Y_i = \beta_0 + \beta_1X_i + \beta_2X_i^2 + \epsilon_i, 1 \leq i \leq n,$
where $ \{\epsilon_i\}$ are i.i.d normal random variables with mean zero and variance $ \...
0
votes
1answer
41 views
General Gauss-Markov theorem [closed]
$Y=XB+u$ where $X$ is a non random $n\times k$ Matrix, $\textrm{rank}(X)=k, E(u)=0, E(uu')=\sigma^2\Omega$, How to form $(1)$ How to proof $(2)$ the general Gauss-Markov theorem?
0
votes
0answers
12 views
Inverse of regression coefficient
rearranging the normal linear regression function with one variable yields
$$-b_0/b_1+1/b_1*Y_i-1/b_1*u_i=X_i$$.
The question for me is why the regression coefficient in front of Y, $1/b_1$, is not ...
0
votes
0answers
57 views
Proof in regression model
Im studying for an exam, i don't have the solution, so I hope some of you guys can help me. I have tried a lot but i can't do this proof.
Here is the task:
Suoppose we have the linear regression ...
0
votes
0answers
17 views
Valid multivariate Gaussian model
I wanted to know if someone could explain the reason why if we have $n < (1/2)p(p+1)$ the number of parameters $p$ is greater than the number of samples $n$. I just can't figure out where does this ...
0
votes
0answers
26 views
Optimal solution for linear regression problem
I have an assignment for university about the linear regression problem and the optimal solution.
For the linear regression problem
$\min_w ||Y - X\omega||^2$,
$\omega^*$ is the optimal solution. ...
-2
votes
1answer
18 views
I can't find the product moment coefficient of correlation? [closed]
These are the equations of least square regression lines:
$ Y = 20.8 - 0.219 X $ ($Y$ on $ X$)
$X = 16.2 - 0.785 Y$ ($X$ on $Y$)
Find the coefficient of correlation $r$.
0
votes
0answers
13 views
Variance of Residuals in Multiple Linear Regression
If I have $n$ variables $x_1,\dots,x_d$ in my dataset, and I regress the first one against all the others i.e.
$x_1=a_{12}x_2 + a_{13}x_3 + \dots + a_{1d} x_d + \epsilon_1$,
how can I calculate $\...
0
votes
0answers
6 views
Predictions from bivariate regression model
I have a bivariate (2 responses variables) regression model and I am stuck with the following predictions model and prediction
Basically I need to predict Y_1 taking into account also the correlation ...
1
vote
1answer
98 views
Basic application of category theory to data science
Linear regression is the algorithm that, given a set of vectors ${\bf x}_i \in \mathbb{R}^p$, and a set of targets $y_i \in \mathbb{R}$, returns a vector ${\bf w} \in \mathbb{R}^p$ minimizing the ...
1
vote
0answers
27 views
Linear regression model
Assume the usual linear regression model with $Y = X \beta + \epsilon$, where $X$ is fixed and known and $E(\epsilon) = 0$ and $\operatorname{var}(\epsilon) = \sigma^2I$.
Let $a$ be an $n$-...
0
votes
1answer
27 views
Regression: width of confidence interval vs t-statistic
My question is about confidence intervals of the slopes estimated in a multivariate regression.
I would like to clear up something that is probably a fault somewhere in my understanding. As I ...
-1
votes
0answers
13 views
Angle between regression line and SD line when plots flipped
The question I am working on is:
The scatterplot y* vs x* has a regression line that makes an angle with SD line. Then the corresponding angle for the scatterplot of x* vs y* is:
I have generated ...
1
vote
0answers
22 views
Regression — gradient descent versus statistical methods
In a machine learning course, the professor described the gradient descent method for calculating the regression line. Shortly, we're looking for the $a$ and $b$ in $y=ax+b$ which describes the line. ...
0
votes
0answers
21 views
Error Term, Regression
Assume I have $Y_i=A+BX_i + \epsilon_i$
For a multiple linear regression I have,
$Y_i=A+B_1X_1+B_2X_2+....+B_NX_N+\epsilon$
The question is, what is the distribution of $\epsilon$, knowing that $\...
1
vote
3answers
44 views
What method does a calculator use to calculate a linear regression line?
Take three coordinates $(1,1)$, $(3,2)$ and $(4,3)$.
My calculator returns the linear regression line: $$y=0.6429x+0.2857$$ of the form $$y = ax +b$$ correct to four significant figures for constants ...
2
votes
1answer
27 views
Time-series modeling
I'm wondering what methods can be used to predict a future value using past values. I looked into linear regression modeling, but this doesn't allow for a time value.
As an example, say I have an ...
0
votes
0answers
28 views
Proof for normal posterior distribution for a data for linear regression.
Let $D=(x_1,x_2,....X_N)$ be the data and $x_i$ is D-dimensional data. For linear regression the likelihood is as follows
$$p(y|X,w, u, \sigma^2) = N(y|u + Xw, \sigma^2 I_N) \propto exp(-\frac{1}{2\...
0
votes
0answers
25 views
Interpret model estimates after log transformation
I sat up a mixed-effects linear model with the dependent variable log-transformed (in oder to get it normal distributed and as ist is common with this kind of data in other publications). All fixed ...
0
votes
0answers
21 views
find $\operatorname{Var}(\hat\epsilon_i)=\sigma^2({{n-1}\over n}-{{(x_i-\bar x)^2}\over{\sum_{i=1}^n}(x_i-\bar x)^2})$
I'm studying question 16.19 in the book Probability Essentials. The question is to show that: $\operatorname{Var}(\hat\epsilon_i)=\sigma^2({{n-1}\over n}-{{(x_i-\bar x)^2}\over{\sum_{i=1}^n}(x_i-\bar ...
0
votes
1answer
245 views
Correlation between Least Square Parameters obtained in Linear Regression
I am trying to solve Q 22 below:
Upon minimizing the squared errors I get the following values of the parameters:
$$\hat{\alpha} = \overline{y}$$
$$\hat{\beta} = \frac{\sum x_i y_i - \overline{x}\...
2
votes
1answer
49 views
Rank of sub-matrix of projection matrix
Consider the projection matrix in Linear Regression $P=X(X^TX)^{-1}X^T$. If we have $n$ points, $P$ is an $n$ x $n$ matrix. We also know it satisfies a number of properties, including that it's ...
1
vote
0answers
8 views
Regression task on desirable subsets with limited supervision
Suppose I have a set of n elements. I want to have a model for how "desirable" a subset of those elements is when evaluated by a person. The training/testing data is a number of given subsets, each ...
1
vote
1answer
19 views
How to interpret the vectors and design matrix in a linear model
In regression, linear models are of the form:
$$y_i = \pmb z_i^T \pmb\beta_i + \epsilon_i$$
Or we can write this in a more general form with vectors and a design matrix:
$$\pmb y = \pmb Z \pmb \beta ...
0
votes
1answer
36 views
What is weighted and unweighted linear regression in machine learning?
I'm taking Stanford's CS229 ML course and while studying about "parametric algorithms", Prof. Andrew Ng says that this class of algorithms has a fix number of parameters (parameters are also called as ...
2
votes
1answer
21 views
Bias and variance of IV estimation
NOTE: I have asked this question on stat.stackoverflow but got no answers/comments. Hence I decide to ask it on the math.stackoverflow platform as well.
I'm studying IV estimation by myself and have ...
0
votes
0answers
28 views
what are $SSE$ and $S_{y,x}$?
I'm a BC student who is trying to solve some statistic quizzes.
there is a multiple choice question that is this:
In a simple regression model $y=a+bx+e$ our given data is this:
$$ \bar{x} = 2, \...
0
votes
1answer
47 views
$Y_1,Y_2,Y_3$ are uncorrelated rvs such that $E(Y_1)=\beta_1+\beta_2$,$E(Y_2)=2\beta_1$ and $E(Y_3)=\beta_1-\beta_2$
Let $Y_1,Y_2,Y_3$ be uncorrelated random variables with common variance $\sigma^2 > 0 $ such that $E(Y_1)=\beta_1+\beta_2$,$E(Y_2)=2\beta_1$ and $E(Y_3)=\beta_1-\beta_2$
where $\beta_1$ and $\...
0
votes
2answers
28 views
Including the (0,0) point in regression
I have run a simple linear regression in Rstudio with two variables and got the following relation:
y = 30000+1.95x
Which is reasonably fair. My only concern is that, practically the (0,0) point ...
1
vote
1answer
26 views
Logistic Regression Explanation
I have two questions regarding logistic regression.
1) I understand that the results of a logistic regression model yield a table stating coefficients together with a p-statistic for each variable . ...
0
votes
2answers
25 views
Linear Regression Assumption
the following website states that linear regression assumes a linear relationship between dependent and independent variables:
"First, linear regression needs the relationship between the independent ...
0
votes
1answer
33 views
Linear Regression Assumption: Normality of residual vs normality of variables
I have read in many places, including stack exchange, that in order to carry linear regression analysis the residuals have to be normal. This is required because most of the statistical results, ...
0
votes
0answers
17 views
Operating with penalized regression models
Assume we are working with a penalized linnear regression model. We have the following optimization problem:
\begin{equation}
\min_{\beta}\left\{\left\lVert y-X\beta\right\rVert_2^2+\lambda\sum_{j=1}^...
0
votes
1answer
26 views
Proof of Batch Gradient Descent's cost function gradient vector
In the book Hands-On Machine Learning with Scikit-Learn & TensorFlow, the author only showed the formula for the Batch Gradient Descent method, such as:
$ \dfrac{\partial}{\partial \theta_{j}} ...
1
vote
1answer
94 views
Partial derivative of MSE cost function in Linear Regression?
I'm confused by multiple representations of the partial derivatives of Linear Regression cost function.
This is the MSE cost function of Linear Regression. Here $h_\theta(x) = \theta_0+\theta_1x$ .
\...
1
vote
0answers
24 views
What is saturated model used in glm in R?
So, I've read on stack exchange (glm summary explained) that residual deviance computed in the glm output is just the likelihood ratio chi-square stat comparing the saturated model to the reduced ...
1
vote
0answers
17 views
How to find the correspondence between ridge coefficients to modified ridge?
The ridge coefficients minimize a penalised residual sum of squares:
$\hat{w}_{ridge}= \arg\min(\sum_{i=1}^{N}(y_{i}-w_{0}-\sum_{j=1}^{d}x_{ij}w_{j})^{2}+\lambda\sum_{j=1}^{d}w_{j}^{2}),\,\lambda\...
0
votes
1answer
12 views
Effect on $R^2$ squared of an additional regressor
I'm currently dealing with the simple linear regression model and the book I'm studying with says that, any time you add a regressor to your model, even if irrelevant, the coefficient of determination
...
0
votes
0answers
14 views
Multiple Regression Assumption
If I am running a multiple linear regression model with six independent variables against dependent variable, do the assumptions of multiple regression need to be satisfied?
or does this only applies ...
0
votes
2answers
32 views
Formulating the polynomial regression
I'm trying to formulate a regression problem such that $y=ax^{b}$. Previously, I formulated the $y=ax+b$ like $y=Ac+e$ where $c=
\begin{bmatrix}
a\\
b
\end{bmatrix}$ and $A=
\begin{bmatrix}
x_1 &...
0
votes
0answers
43 views
3d linear regression line equations
I have a bunch of points in 3D space (around to a survey line). Since only half of the point coordinates were measured, I want to compute a linear regression, from which I extrapolate the other points ...
1
vote
0answers
17 views
Regressions with two independent variables where one is part of another.
I am doing simple regression analysis where I use one-way fixed effect model to estimate the effects of two variables on dependent variable. The question I am asking is how to interpret these two ...
0
votes
0answers
41 views
Fitting a Straight Line to 3D Points, and then finding the nearest point to those lines.
I've been stuck on this problem for over a week now.
Essentially, I have a group of 4 tracks of points from a bunch of detectors in 3 dimensions. I need to map these points to straight lines and then ...
1
vote
0answers
18 views
How to calculate $\text{cov}(\hat{Y}_{ij}, \hat{Y}_{kj})$ if $Y_{ij} = \mu + a_i + b_j + e_{ij}$?
Let's assume we have the model following Two-Factor model without replications :
$$Y_{ij} = \mu + a_i + b_j + e_{ij}, \; i=1,\dots,p \; \text{and} \; j=1,\dots, q $$
I am interested in calculating ...
2
votes
0answers
20 views
Covariance of errors $\text{cov}(\hat{e_{ij}},\hat{e_{i\ell}})$ in Two-Way Anova model
Exercise :
Consider the Two-Way Anova model $Y_{ij} = \mu + a_i + b_j + e_{ij}$ with $i = 1, \dots, p$ and $j=1,\dots,q$. Show that :
$$\text{cov}(\hat{e_{ij}},\hat{e_{i\ell}}) = -\sigma^2\left(\...
1
vote
1answer
45 views
fitting triple exponential term function to data
The function, I am trying to fit to data is:
$$y(x) = −(A+B)e^{−x/a_1} + 𝐴e^{−x/a_2} + Be^{−x/a_3}$$
this function is a little bit different to Is it possible to find initial parameters when fitting ...
0
votes
1answer
17 views
question about matrix equation for coefficients in linear regression
There is a matrix equation for solving a linear regression, $\vec{y}=X\vec{\beta}$ where $X$ is the matrix of features, $\vec{\beta}=[\beta_1,...,\beta_n]$ are the coefficients for each feature, and $\...
0
votes
1answer
14 views
Interpreting linear regression coefficients for a covariate that's correlated with other covariates
The interpretation of linear regression coefficients that I learned is that the coefficient is the change in outcome associated with a unit change in that covariate, assuming all other covariates stay ...
