The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [linear-regression]

For questions about linear regressions, an approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables.

Filter by
Sorted by
Tagged with
1
vote
1answer
46 views

Transformation of independent variables in regression (Measurement Error)

Consider the model $𝑌_𝑖$=$𝛽_0$+$𝛽_1$$𝑋_𝑖∗$+$𝑢_𝑖$ In practice we measure $𝑋_𝑖∗$ by $𝑋_𝑖$ such that a) $𝑋_𝑖$=$𝑋_𝑖*+3$ b) $𝑋_𝑖$=$5𝑋_𝑖*$ What will be the effect of these ...
0
votes
0answers
166 views

How can I calculate the distribution of the least-squares estimator $\hat{\beta}$?

Let $Y = X \beta + \varepsilon$ with $Y \in \mathbb{R} \in \mathbb{R}^n$, $X \in \mathbb{R}^{n \times p}$, $\operatorname{rank}(X) = p$, $\beta \in \mathbb{R}^p$, $\varepsilon \sim \mathcal{N}_n(0, \...
0
votes
1answer
87 views

Given Least Squares of a set, compute Least Square of the set excluding a single element

I am trying to figure out how to implement a cyclic running Regression Line computing algorithm using Least Squares for streaming time series data in the most efficient way. In other words, having $LS(...
0
votes
1answer
119 views

Linear or quadratic best fit model

The statistics question asks which would be the best method to determine if your data would best be fit with a linear regression model or a quadratic model. I would think that the residual plot would ...
1
vote
0answers
21 views

Logistic regression and game winning probabilities

I'm researching how machine learning can be applied to predicting outcomes of games such as football and basketball with logistic regression. I was wondering if anyone had a brief overview of the ...
0
votes
2answers
11k views

1e+0.8= What? What does E mean? [duplicate]

Hello I came across a math equation and I was wondering what did the "e" stand for? the equation is : y=47931x-1E+0.8 Can someone please help me by showing what the E stands for and the answer for my ...
-1
votes
1answer
490 views

after normalize data, Using multiple regression analysis how to predict y?

I want to predict yield(y), I have independent variables are rain(x1) and Soil(x2) yield(y) : 25000, 26000, 27000, 28000, 29000 Rain_mm (x1) : 1000, 875, 852, 1005, ...
-3
votes
1answer
155 views

Conditional Expectation of Circle [closed]

$(X,Y)$ is uniform on the unit circle $(x^2 + y^2 = 1)$. How do I calculate the conditional mean as a function of X = x?
0
votes
1answer
193 views

Proving unbiassed estimators for Ordinary Least Squares

$\mathbf{\text {Show}}$ $\mathbf{E[\hat{\beta _1}]=\beta _1}$ I have already been able to prove $ \hat{\beta_1}= \frac{\sum_{i=1}^n (x_i-\bar{x})}{\sum_{i=1}^n (x_i- \bar{x})^2}$ I rewrite it as $ ...
1
vote
1answer
47 views

Linear estimator for bivariate distribution

I have a set of data $(x_i,y_i)$ for which i coumputed the Joint Probability Density Functions (see fig1). I would like to find the "best fitting line" that describes the distribution. In other words, ...
0
votes
1answer
2k views

Omitted Variable Bias for linear regression - Sign of coefficient changes.

My question relates to determining the direction of bias when the regression coefficient changes sign (from negative to positive) however the absolute value is smaller in the new estimate. The ...
0
votes
0answers
74 views

How would I solve a linear regression with SD on intercept and slope

I am sure that this is an easy question, but I am not able to figure it out. Given an equation, with ($SD$=standard deviation): $y=a(+/-SD)*x+b(+/- SD)$. Given a specific value for $y$, how would I ...
1
vote
1answer
390 views

Curve Fitting: Multidimensional input

According to Bishop's book, if your input is a multidimensional vector $x$, then non kernel based basis function models suffer from the curse of dimensionality: But the above picture is based on ...
1
vote
1answer
322 views

Finding a solution with minimal $\ell_2$ norm in linear regression with dependent variables

I am trying to find a linear regression for the problem: $$\displaystyle\arg\min_w\|y-Xw\|^2 $$ By finding the optimum of the above equation, I get $$\displaystyle X^TXw=X^Ty $$ In the case where $...
4
votes
2answers
863 views

Proving Convergence of Least Squares Regression with i.i.d. Gaussian Noise

I have a basic question that I can't seem to find an answer for -- perhaps I'm not wording it correctly. Suppose that we have an $n$-by-$d$ matrix, $X$ that represents input features, and we have a $n$...
0
votes
1answer
148 views

Experience of linear regression

As part of my work (programmer), I need to learn some linear regression. I have a degree in pure mathematics, but not in statistics. In fact, I have one course in statistics and two or three in ...
0
votes
1answer
478 views

Minimizing the sum of squared residuals

I have the equation $y=X\beta+u$, where $y \in \mathbb{R}^{n \ \times \ 1}$ , $X \in \mathbb{R}^{n \ \times (k+1)} $ , $\beta \in \mathbb{R}^{(k+1) \times \ 1 }$, and $u$ is the error term, and ...
0
votes
2answers
91 views

Theoretical regression line - how does it look like graphically

On the image there is the empirical regression line (the best fit by sum of min squares - $y_i - \hat{y}$). But how would the theoretical regression line look like? Does it join all the points? I ...
0
votes
1answer
94 views

Residual Plots - Banding

I came across a question on residual plots asking to consider the plot $(x_i,e_i)$ for $i=1,\cdots,n$. Discuss the conclusion that may be drawn if the plot $A)$ is approximately a horizontal band $B)...
1
vote
1answer
27 views

Showing that an increase in uncertainty is significant

I have a linear model $y = ax+b$ and I estimate the coefficients $a$ and $b$ in the ordinary way. I have found out that all of my values of $x$ were systematically overestimated, and also that they ...
0
votes
0answers
21 views

problem with Simple linear regression

I am solving a question with $$x_i = \{1.14, 2.23, 3.01, 4.55\}$$ $$y_i = \{10.77, 7.23, 5.31, 2.84\}$$ and to find out the best fit line of it. mean of $x_i = 2.73$ mean of $y_i = 6.54$ and ...
0
votes
1answer
30 views

Linear regression angle too large

Am trying to find linear regression slope (angle) of a line with the following set of coordinates. x axis y axis 123.4415, 5 123.4414, 4 123.4413, 3 123.4412, 2 123.4411, 1 the slope am ...
1
vote
1answer
584 views

Least-squares Projection

In least-squares, say we have $n$-points in 2-D space. Now, assume these points don't lie on a line(2-D hyperplane). Do we find $n$-dimensional hyperplane on which all these points lie? If yes, then ...
0
votes
1answer
41 views

Covariance between two variables when one of the variables only has two possible values

I have a data set which consists of the results of a study to determine if texting while driving has a statistically significant effect on participant’s driving quality. Participant's were divided ...
1
vote
2answers
2k views

Proof that sum of squares of error for simple linear regression follows chi-square distribution

I can understand that if Y1~Yn are random samples from N(μ,σ), then the sum of squares of difference between Yi and bar(Y) divided by sigma^2 follows chi-square distribution with n-1 degress of ...
1
vote
2answers
42 views

Definition of Linear

I'm not sure if this question is too broad, but here goes: What does the word "linear" mean in mathematical models? In my econometrics class, one of the Gauss Markov assumptions for running ...
1
vote
1answer
162 views

The compatibility constant

I am reading Lecture notes on sparsity by Sara van de Geer and trying to understand the idea behind the compatibility constant. Suppose that $\beta\in\mathbb R^p$. Let $S\subset\{1,\ldots,p\}$ be an ...
2
votes
2answers
2k views

Can a linear regression be quadratic?

The following is from a comp. sci. book that discusses regression. The passage seems to say that while a function fitted to a data set may be quadratic, it may yet be considered linear. This seems ...
1
vote
2answers
34 views

Question about intepretation of constants in a Linear Regression

Consider the number of passengers that, in 2009, used a given airport: $$D=[D_1, D_2, \ldots, D_m]^T$$ where $D_i$ represents the number of passengers that, in 2009, used the airport number $i$. I can ...
1
vote
1answer
78 views

Fisher F test - linear regression

We consider $Y=X\beta+\epsilon$, $\epsilon \sim N(0,\sigma^2 I)$. Let H$_0$ is $\beta_0=\dots\beta_k=0$ vs exist $i\in{0,1,\dots,k}: \beta_i\neq 0$ How to proof that if H$_0$ is true then $F=\frac{...
0
votes
2answers
123 views

Regression: Use derivative information for better estimate of parameters of model

If additional information of slopes are available in addition to observations; Can this information be used to improve fitting? Following could be considered as an example: Suppose I have $\mathbf{x}$...
0
votes
1answer
87 views

First detecting outliers or first transforming variables?

I have an, I think, rather basic question about linear regression to which I can not find a satisfactory answer. I am trying to apply linear regression on a certain data set. First of all, I have ...
1
vote
1answer
73 views

What is the meaning of null regression coefficients

I have real data $x(t)$ which I model using the linear regression model: $\displaystyle f(t)= \sum_{i=0}^n c_i \phi_i(t)$, where $c_i$ are the regression coefficients I'm trying to find and $\phi_i(t)$...
3
votes
2answers
2k views

Prove that $R^{2}$ cannot decrease when adding a variable

I know that in general this is true because the smaller model is nested within the larger model, so the larger model must have SSE at least as low as the smaller one, but I'm having a hard time ...
1
vote
2answers
58 views

Coefficient of determination is always 1 - high values

I've got some measurement on my abscissa reaching from about 7500 to 10300. On my ordinate my measurements reach from 10 to 90. Now, I'm doing linear regression and I'm also calculating the ...
0
votes
1answer
121 views

Proving $a$ (in $Y = aX + b + e$) satisfies $a = Cov(X, Y )/Var(X)$

In a linear regression model, we postulate that random variables $X$ and $Y$ are related by $$Y = aX + b + e$$ where a and b are constants (called the regression coefficients) and e (representing ...
0
votes
0answers
34 views

Best linear interpolation with expensive-to-compute function

I have a function $f(n)$ which is hard to evaluate, but which can be well-approximated as $an+b$ for unknown real constants $a,b.$ Say the time needed to evaluate $f(n)$ is $b(n)$ where $b$ is ...
0
votes
0answers
51 views

Convergence of gradient descent method with one parameter

We want to minimize the mean square error in a regression model that uses just one paremeter $w_0$. For convinience we take the mean square error to be $MSE(w_0) = \frac{1}{2N} \sum_{i=1}^{N} (y_n - ...
2
votes
1answer
58 views

Linear regression via the bi-variate normal distribution MLEs

I'm attempting to derive the maximum likelihood estimates for the parameters of the bi-variate normal distribution model of linear regression and I am well and truly stuck. Just looking for some ...
2
votes
0answers
59 views

Markov chains with nonlinear predictor variables

I have data tracking about 25,000 individuals as each one moves through a Markov chain. I want to know the shape of the relationship between a continuous 'covariate' (my independent variable of ...
1
vote
1answer
19 views

If the population estimators follow a given relationship , can we assume that the sample estimators would follow the same relationship as well?

Say in case of a standard CLRM ( classical linear regression model ) we are aware that the population estimaotrs $\beta_1$ , $\beta_2$ etc. satisfy the following relation that $f(\beta_1$,$\beta_2$,$...
1
vote
0answers
507 views

Complexity of least squares linear regression with multiple parameters

I'm doing a least squares linear regression using the numpy.linalg.lstsq with two parameters, i.e.: $y = \alpha x_1 + \beta x_2 + \gamma$ Where $x_1$ and $x_2$ are lists of size $N$, and the output $...
2
votes
1answer
358 views

Vector Space and Column Space of linear models?

I've started reading a book on linear models, and it turns out there's a lot of linear algebra (which I'm a bit weak in). There were two exercises that I think might be simple, but I don't know if I'm ...
1
vote
1answer
108 views

liner regression not using mean square error to estimate parameter

I am looking through some implementation of linear regression, I found it is not calculating parameter directly using formula like below, but calculate Pearson product-moment correlation coefficient ...
2
votes
1answer
164 views

Variance Estimate in linear regression

In a linear regression, $y=X\beta+\epsilon$, where $\epsilon\sim N(0, \sigma^2)$, $X\sim R^{N \times (p+1)}$. Assume the observations $y_i$ are uncorrelated and have constant variance $\sigma^2$, and ...
0
votes
1answer
62 views

Basic question on linear regression

I am trying to understand linear regression. The typical model takes form $$y_{i}=ax_{i} +b + \epsilon_{i}, \ \ \ i=1..N$$ where $\epsilon_{i}$, is an i.i.d Gaussian random variable. The objective ...
6
votes
0answers
309 views

Perturbation theory for least squares for very different A, b

Consider the least squares problem $f(x;A,b) = \|Ax-b\|_2^2$ and define $x^*$ the minimizer of $f(x;\hat A,\hat b)$, and $\hat x$ the minimizer of $f(x; A_2, b_2)$. I want to put some bound on $\|...
0
votes
1answer
67 views

Linear approximation to the infinite product $\prod_{k=0}^\infty \frac{1}{2} (1+x^{1/p^k})$

Let's consider the function defined by the infinite product: $$f_p(x)=\prod_{k=0}^\infty \frac{1}{2} (1+x^{1/p^k})$$ $$p \geq 2$$ The only closed form I know is for $p=2$. $$f_2(x)=\frac{x^2-1}{2 \...
1
vote
0answers
42 views

Why is Deming regression not defined for some sequences? What characterizes them?

In a nutshell, the purpose of Deming regression is to find a line such that the sum of square errors wrt to the points of a set of points is minimal. For a definition, please refer to Wikipedia. (I'm ...
0
votes
1answer
830 views

How to combine several linear regression function into one

I have the relationship between f and each of other five variables, the relationships are acquired by linear regression and are as follow $f = a{\tiny1}x{\tiny1} + b{\tiny1}$ $f = a{\tiny2}x{\tiny2} ...