Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

As a student learning Applied Regression Analysis, I come from a background with very little information about this topic.

I understand that given $y = \beta_0 + \beta_1x_1 + \epsilon$

$E(y\mid x) = \beta_0 + \beta_1x_1$ is an exact linear relationship.

However, if we use a function as log($x$), sin($x$), or cos($x$), will the relationship continue to be linear?

For instance, in the parameters $\beta$,

$E(Y\mid x_1,x_2) = \beta_1 x_1 + \beta_2 \log(x_2)$, is this linear?

Clearly, $\beta_1x_1$ is linear; however, is the part $\beta_2\log(x_2)$ also linear? From a calculus point of view, we know that the logarithm function isn't exactly linear in the sense it is a polynomial of degree one.

Any hints are much appreciated.

share|improve this question
    
Here is a related question: Why is polynomial regression considered a kind of linear regression?. –  Michael Hardy Dec 27 '13 at 17:56

2 Answers 2

By linear model we shall mean a mathematical equation that involves random variables , mathematical variables and parameters and that is linear in parameters.

Your $E(Y\mid x_1,x_2) = \beta_1x_1 + \beta_2 \log x_2$ is linear. Here your parameters are $\beta_1$ and $\beta_2$ and they are linear . Also note that $x_1$ and $x_2$ if given they are fixed and consequently $x_1$ and $\log x_2$ are fixed.

NB: $y=\beta_0 + \beta_1 \exp (-x_1) + \beta_2 \log ~x_2 +e$ ,where $e\sim N(0,\sigma^2)$, is Linear Model.

However $y=\beta_0 \exp(-x_1 \beta_1)+e$ ,where $e\sim N(0,\sigma^2)$, is not linear in parameters and is not a Linear Model.

share|improve this answer

Linear implies "linear in the parameters." For linear regression, the most important assumption is that the residuals $\epsilon_i$ are normally-distributed. Therefore, if your $x$-variables are skewed with right tails (log-normally-distributed), left tails, and basically not normally distributed, the residuals will not be normally distributed. Transforms on $x$ are only commonly performed in linear regression into order to obtain normally distributed residuals, which is an absolute requirement. If all of the $x$-variables used for a linear regression models are normally distributed with little skewness, then you commonly don't have to use transforms on $x$.

Please note: sometimes a regression model is presented in terms of a non-linear function for which non-linear regression is used (partial derivatives, gradient descent, etc.). However, in those cases, the problem statement is in terms of a non-linear function. When you are using linear regression, you are simply solving a system of linear equations.

Overall, when using linear regression, you must meet a number of assumptions such as:

  1. normally-distributed residuals
  2. homoscedasticity (variance of $y$ across values of $x$ must be equal)
  3. independence (each object or record must be independent of one another, i.e. zero correlation or zero covariance).
share|improve this answer
    
If normal distribution is weakened to an assumption of finite variance and expectation $0$, and independence is weakened to uncorrelatedness, and homoscedasticity is left as it is, then you have just the assumptions used in the Gauss--Markov theorem, whose conclusion is that the least-squares estimates are in a certain sense optimal. –  Michael Hardy Dec 27 '13 at 17:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.