Questions tagged [linear-regression]
For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.
0
votes
0answers
35 views
What does this absolute value notation mean?
In a regression model function, What does this absolute value notation mean?
0
votes
0answers
8 views
Minimum-volume confidence ellipsoid for regression with nuisance parameters
I have a linear regression problem with Gaussian errors, nuisance variables and the parameter of interest, $\alpha$. I want to find the smallest-volume region containing the true value of $\alpha$ ...
0
votes
0answers
4 views
Derivation for marginal effect sizes' distribution in linear model
Model:
$$Y_{N\times 1} = X_{N\times M} \beta_{M\times 1} + \epsilon_{N\times 1}$$
the design matrix $X_{N\times M}$ is known, each column of the design matrix has been standardized to have mean 0 and ...
0
votes
0answers
16 views
Ols estimator with the errors following a bernoulli distribution
I am having trouble understanding how i should approach the following problem:
Given 𝑦𝑖 = 𝛼 + 𝛽𝑥𝑖 + 𝜀𝑖 𝑖 = 1, … , N with 𝜀𝑖 𝑖 = 1,2 … , N being a succession of IID Bernoulli ...
0
votes
2answers
23 views
Derivation of Linear Regression using Normal Equations
I was going through Andrew Ng's course on ML and had a doubt regarding one of the steps while deriving the solution for linear regression using normal equations.
Normal equation: $\theta=(X^TX)^{-1}X^...
1
vote
0answers
17 views
Linear Regression: value for slope
I'm sorry, I'm new to linear regression so I had a very stupid question. The slope of the best-fit line is defined as the value that minimizes the sum of the squared deviations from each point. Is ...
1
vote
1answer
19 views
Conditional Expectation Decomposition in Regression Analysis
I am currently working on my understanding of regression fundamentals and I checked this source (one can find the (even exact) same statement in multiple sources).
In Theorem 3.1.1, the author claims ...
0
votes
0answers
14 views
How to reduce credible sets in over-specified linear regression while maintaining global coverage probability?
Vectors $A, B$ and covariance matrix $C$ are fixed and known. I have a vector of measurements, $Y\in\mathbb{R}^n$, sampled from
$$
M_1: Y \sim N(A\alpha_* + B\beta_*, C)
$$
My goal, roughly speaking,...
1
vote
0answers
23 views
Linear regression of basis functions for multivariate inputs
Background - My current understanding of linear regression of basis functions:
Given an input domain $\mathcal{X}$, target domain $\mathcal{Y}$, and a data set $S=\left\{ \left(x_{i},y_{i}\right)\...
0
votes
0answers
15 views
What is the formula of the linear regression with an error propagation
I am in Physics Licenciature and a day the teacher showed me a formula for the linear regression with error propagation, and time after, I was searching this formula and I didn't find it. Then I am ...
0
votes
1answer
25 views
What's the Most Appropriate Type of Regression for this Problem?
I have a data set from two groups: firms that use AI and their costs and firms that don't use AI and their costs. Within both groups I have data about their specific costs, e.g. fixed costs, variable ...
0
votes
0answers
8 views
Linear regression with dependent variables: express prediction with dot products
When dealing with Linear regressin with dependent variables, one can consider the optimization problem:
$$\arg \min_w 0.5\lVert{w}\rVert^2 \\ s.t. Xw=y$$
Where $X\in\mathbb{R}^{n,d}$ is the data ...
0
votes
0answers
36 views
ESL - Linear Model where output is a vector. Ch 2 Pg 11
In the terminology used in ESL, a vector is a column vector. Let output be a $k$-vector, i.e.$$Y=(Y_1,Y_2,\cdots,Y_K)^T$$Now please refer to following line on pg 12.
In general $\hat{Y}$ can be a $K$-...
0
votes
1answer
49 views
Linear regression analysis where the fit values must be greater than the observed values?
Long story short, I would like to efficiently:Minimize ||bX-y||2 subject to X ≥ 0 and bX ≥ y
I have an observation that is a single curve (y) in the form of signal intensity vs. frequency. I ...
0
votes
1answer
28 views
Equivalent formulation of LASSO?
I am currently trying to tell wheter or not those two problems are equivalent :
$$\min_x \|x\|_1 \text { s.t. } \|Ax-y\|^2_2 \le \varepsilon.$$
And
$$\min_x \|Ax-y\|^2_2 \text { s.t. } \|x\|_1\le ...
1
vote
1answer
17 views
How to approximate prediction interval in linear regression
Suppose we have a linear regression model of the following format :
$$ y(x) = \beta_0 + \beta_1 x_1+ \beta_2x_2+\beta_3x_3+\epsilon$$
We know that the prediction interval associated with a level $\...
1
vote
1answer
18 views
Linear regression model with 2 categorical variables
Let's consider the following problem : We want to predict a variable $y$ and we have two categorical variables : $A$ that can take 3 different values and $B$ than can take 2 different values.
A ...
1
vote
1answer
29 views
How to interpret regression equation?
I am trying to understand how to interpret the regression line given:
$y = -5.18 + 1.94x$ (regression line)
where $y$ is number of cold drinks sold and where $x$ is temperature
Interpret values of $...
0
votes
0answers
10 views
cross validation method in data set ,i want theoretical concept for selection of sample value by cross validation,
let say i have large sample size i divide it in two parts ,one is for learning and second is for validation purpose . in first step i estimate parameters from learning sample and then in second step i ...
0
votes
0answers
19 views
Total Least Square fitting
Say I want to fit a straight line using Total Least Square (as opposed to Least Square), which is to minimize the sum of (yi-k*xi-b)^2/(k^2+1) over all xi's and yi's, where xi's and yi's are training ...
0
votes
0answers
9 views
Two dimensional Linear Regression Hat Matrix
Let X=(X1, X2)nxp where X1 (nxq) with rank=q and X2 (nx(p-q)) with rank=(p-q).
Let H and H1 be hat matrix of X and X1.
1) Prove that HH1=H1 and H1H=H1
2) Prove that (H-H1) is idempotent
1
vote
1answer
14 views
What is J in while calculating SST in multiple regression?
I am little confused what actually is the J in the formula of the SST and SSR for multiple regression
SST= $Y^T\left[ 1-\frac{1}{n}J\right]Y$
SSR=$Y^T\left[ H-\frac{1}{n}J\right]Y$
1
vote
0answers
35 views
Elliptical Confidence Set Calculation
I got stuck in a homework question:
In Linear regression model with assumption $\varepsilon_{i} \sim \cal{N}(0, \sigma^{2})$, iid.
$$Y_{i} = X_{i}^{\intercal}\theta^{*} + \varepsilon_{i}, ~ i = 1, \...
0
votes
0answers
9 views
Why residuals are a good estimator of random disturbance?
Let a linear OLS model:
$$Y= X \beta + u$$
Where $u$ is a random disturbance.
If we define the residual of the regression as
$$e = Y - X \widehat{\beta}$$ where $\widehat{\beta}$ is the OLS vector of ...
0
votes
0answers
17 views
Standard Error Estimate for Beta Coefficients
Suppose I have a linear regression model consisting of $\beta$ estimates, relative to a reference term. Each of these $\beta$s has a $\bar{x}$, a $s_x$, and then $n$, with a calculated $\hat{\sigma}$ ...
0
votes
0answers
9 views
Find $V(\tilde\beta|X)$
In the linear regression $Y=X\beta+\epsilon$, with $E(\epsilon_i|x_i)=0$, it is known that the true $\beta$ satisfies the restriction $M\beta=0$, where $M$ is a $q \times k$ matrix with $q<k$. $\...
0
votes
1answer
20 views
Please correct my thinking about Ridge Regression
If ridge regression biases ALL beta coefficients of a regression model towards zero, wouldn't the model massively mispredict the y-variable?
I know my logic must be wrong here, but I'd appreciate if ...
0
votes
0answers
11 views
Error term too big/can in improve my multiple regression?
I have made regression selecting the best features (p-val<0.05). I now have the model with a 0.85 R-squared and residual of 2.64.
Well, sometimes when I try to predict some new instances, my ...
-3
votes
1answer
18 views
What is the t-score for the relationship between X & Y
The police chief is concerned about alarm systems degrading in city buildings and failing to operate. Using a sample of 100 alarms in operation the previous year, the police department regresses ...
0
votes
0answers
11 views
Multiple regression - positive/negative linear relationship
Is there a way other than from the x values to tell from the regression printout whether there is an evidence of a negative or positive linear relationship?
Would the values in the Coefficients ...
1
vote
1answer
42 views
L2-norm with estimated weights
Suppose I'm performing linear regression. My lecturer said the formula below can be used for estimating the weight vector that is passed to the L2-norm part of the loss function but he didn't ...
1
vote
0answers
33 views
What does it essentially mean if the neural network has convex error surface?
Suppose if I am building a Linear Regression model with one fully connected layer and a sigmoid with minimizing mean squared error as objective. I understand that this network has a convex error ...
1
vote
0answers
34 views
Linear Regression Diagnostics
I am trying to determine if there is a relationship between a dependent variable y and independent variable x by fitting a least squares regression model.
Scatterplot of data:
Diagnostic plots:
...
1
vote
0answers
64 views
Proving that $\mathbf{(H-\frac{1}{n}J_n)}$ is indempotent
I am trying to show that the matrix $\mathbf{(H-\frac{1}{n}J_n)}$ is idempotent where $\mathbf{H}$ is the Hat-matrix (Projection matrix) of linear regression and $J_n$ is the $n\times n$ matrix with $...
0
votes
0answers
22 views
basic, clear Book about linear regression with examples?
i'm taking a course about linear models , and the book used is Ravishanker "a first course of linear models", the problem is that the book its so theoretical and my background is so basic, so is very ...
0
votes
0answers
11 views
how affect the variance and the expected value add a variable in a linear model
if the correct model is $Y=X_{1}\beta_{1} + X_{2}\beta_{2} + \varepsilon$, with variance $Var(Y)=\sigma^{2}I$.how change the variance and the expected value of $\beta^{*}_{1}$, in $Y=X_{1}\beta^{*}_{1}...
0
votes
0answers
8 views
Variance vs. asymptotic variance of OLS estimators?
I immediately assumed that all three were true. Which one is false, and why?
1
vote
0answers
14 views
Confidence interval of dependant variable individual value by R-squared value
From the physical experiment I've got a sample collection of values (unique Y for each unique X). The problem is to find the ...
0
votes
2answers
22 views
Removing variable with big p-value?
I have made a regression with 2 explanatory variables. The summary of that regression shows that one of my variable has a big p-value (0.705). Should I include that variable when writing the the y hat ...
0
votes
0answers
6 views
find a Confidence interval in terms of sample correlation
i'm reading about linear models, and i tried to solve the next problem, but i'm really lost about how to start. i was reading in other books but i really have no idea how to tackle it. can you give me ...
0
votes
1answer
12 views
Linear regression model when intercept is known
I was given linear model:
$$Y_i = \beta_0 + \beta_1X_i + u_i$$
I know that $\beta_0 = 2$.
Now I should find out the unbiased estimate $\bar{β_1}$.
I know that $\beta_0 = 2$ represents the ...
0
votes
2answers
30 views
Confusion in Relationship between regression line slope and covariance
In simple linear regression model between RVs $(X,Y)$, the slope $\hat\beta_1$ is given as
$$
\hat\beta_1 = \dfrac{\sum_i^N(x-\overline{x})(y - \overline{y})}{\sum_i^N(x - \overline{x})^2} \tag{1}
$$
...
1
vote
0answers
42 views
Proof Οf Τhe Variance Οf $\hat y_{x_0}$
EXERCISE
Show in a simple linear regression model that the variance of $\hat y_{x_0}=\hat b_0+\hat b_1x$ is:
$$V(\hat y_{x_0})=σ^2 \dfrac {\sum_{i=1}^{n} x_i^2}{n \cdot \sum_{i=1}^{n} (x_i-\bar ...
0
votes
1answer
14 views
What are the possible effects of outliers on a regression analysis
What are the possible effects of outliers on a regression analysis?
I know that outliers can dramatically change the magnitude of the regression coefficients and even change the direction of the ...
0
votes
1answer
19 views
Estimate parameters through transformation and linear regression (statistics)
A model for a chemical process is
$$y_i = \frac{V_m}{k + x_i},$$
where $x_i$ and $y_i$, the predictor (independent) and response (dependent) variable, respectively, are obtained from data. $...
0
votes
1answer
22 views
What types of noise allow linear regression interpolation to be unbiased?
In a statistics book I learned that if there are $n$ random variables $X_{t(0)},\dots,X_{t(n-1)}$ (with $t(i)\in\mathbf{R}$) which are independently distributed with distribution $\mathcal{N}(at(i)+b,...
0
votes
0answers
37 views
Bayesian Interpretation for Ridge Regression and the Lasso
I'm learning the book "Introduction to Statistical Learning" and in the Chapter 6 about "Linear Model Selection and Regularization", there is a small part about "Bayesian Interpretation for Ridge ...
1
vote
0answers
27 views
Showing that $\operatorname{cov}(y_{x_0}, \hat{y_{x_0}}) = 0$
Exercise :
For the Simple Linear Model $\mathbb E[y_x] = b_0 + b_1x$, prove that for a newly given $x_0$ and $y_{x_0}$ a new observation while $\hat{y_{x_0}}$ its point estimate, it is :
$$\...
0
votes
1answer
9 views
response variable and explanatory variable?
This example relates to the percentage of expenditure, P, a publishing house spends on advertising and the change in revenue, R (expressed as a percentage) at the end of the following year.
What is ...
0
votes
1answer
39 views
How to prove that $\text{cov}(y_i,\hat{y_i}) = \sigma^2 h_{ii}$
In one part of our notes (and some books), the following expression is used :
For the generalized linear model $y=X\beta + \epsilon$, it is :
$$\text{cov}(y_i,\hat{y_i})=\sigma^2h_{ii}$$
...
