Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
0answers
36 views

Smallest set of Liner equations, which exactly fit a set of points

I have a set of 2-d points,(it can be of any arbitrary dimension n). I want to find the minimum set of straight lines(linear equations) which exactly passes through the given 2-d points (unlike ...
0
votes
1answer
277 views

Fitting 2nd Order multivariate quadratic with matrices

Hopefully you at least entertain this question as it took forever to construct the below matrix using TeX. Any ways, so I have a list of data points ($X_1$,$X_2$,Y), with the X's being independent ...
0
votes
0answers
603 views

How to interpret coefficients in polynomial regression?

I am working on my thesis (study) about poverty incidence rate and its socio-economic factors using second-order polynomial regression without interaction. The final model in my study is ...
0
votes
2answers
30 views

Prove $\sum_i \frac{\bar{x}(x_i-\bar{x})}{nS_{xx}} =0$

Prove $$\sum_i \frac{\bar{x}(x_i-\bar{x})}{nS_{xx}} =0$$ That is one detail in a proof of the variance of the intercept $\alpha$ in the simple linear regression $Y_i=\alpha+\beta x_i$.
3
votes
2answers
5k views

Prove $SST=SSE+SSR$

Prove $$SST=SSE+SSR$$ I start with $$SST= \Sigma (y_i-\bar{y})^2=...=SSE+SSR+ \Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )$$ and I don't know how to prove that $\Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )=0$ a ...
1
vote
1answer
122 views

regression on circular data

How would one design a regression where the dependent variable is measured in degrees on a circle? The dependent variable is on the range [0, 360), and the independent variables are demographic ...
5
votes
1answer
2k views

What is the difference between Curve Fitting and Regression(Machine Learning)?

I know that Machine Learning regression algorithms try to find the function of the data. That is, if we have 1000 data points (x,y), to find a general continuous function that follows the trends of ...
1
vote
0answers
29 views

Regression factors and covariance matrix

I am trying to follow some notes. They have two matrices. One is called comfact (company factors). This is a $580 \times 5$ matrix. The $580$ rows represent $580$ different companies. The $5$ ...
1
vote
1answer
83 views

What's the “ridge” in Ridge Regression?

In normal least squares, we try to find $\hat\beta$ which minimizes $$\|y-X\beta\|^2$$ Ridge regression expands this to "penalize" certain values of $\beta$ via a matrix $\Gamma$: $$\|y-X\beta\|^2+\|\...
2
votes
0answers
285 views

A* vs D* vs Dijkstra [closed]

I understand the basis of A* as being a derivative of Dijkstra, however, I recently found out about D*. From wikipedia, I can understand the algorithm. What I do not understand is why I would use D* ...
0
votes
1answer
27 views

Linearilization of non-linear relationships (Linear Regression)

How do you linearise the following equation? $$ Y = \frac{\beta_{1}x}{\beta_{0} + x} + E. $$ $\beta_{0}$ and $\beta_{1}$ are the parameters and $E$ is the regression error term.
2
votes
1answer
236 views

Fitting a simple linear regression

A professor in the School of Business in a university polled a dozen colleagues about the number of professional meetings they attended in the past five years $x$ and the number of papers they ...
0
votes
1answer
85 views

Estimate correlation coefficient of unknown variable

Consider variable y depends on variable x and z linearly. I have $100$ sample values of $y$ and corresponding $x$ but don't have any values of $z$. The functional model is $$y = \alpha_1x + \alpha_2z ...
1
vote
0answers
42 views

Linear Regression Question (Linear Algebra) Help!!

Hey guys, I have a quick question. I am trying to prove that the squared sample correlation between fitted and observed values is equal to $R^2$ (coefficient of determination). I am having a lot of ...
1
vote
1answer
39 views

How to find restrictions of Autoregressive Lag Model (ADL)

All of my textbooks mention restrictions for AD models but don't explicitly say what they mean by "restrictions" and I'm having a hard time grasping what they mean. $y_t = \beta_0 +\phi y_{t-1} + ...
1
vote
1answer
118 views

Variance of Coefficients in a Simple Linear Regression

I have a linear regression model $\hat{y_i}=\hat{\beta_0}+\hat{\beta_1}x_i+\hat{\epsilon_i}$, where $\hat{\beta_0}$ and $\hat{\beta_1}$ are normally distributed unbiased estimators, and $\hat{\...
2
votes
2answers
942 views

connection between PCA and linear regression

Is there a formal link between linear regression and PCA? The goal of PCA is to decompose a matrix into a linear combination of variables that contain most of the information in the matrix. Suppose ...
1
vote
1answer
51 views

Regression towards the mean?

I am upset with what examples testing "regression to the mean" seem to allude to: People claiming to have "ESP" take a test, and A's score was 2 standard deviations above the mean, whereas B's score ...
1
vote
1answer
140 views

Robust Standard Errors

For OLS, my professor said that you should always test for heteroscedasticity first, rather than going straight to the adoption of robust standard errors. I didn't quite follow this and no ...
0
votes
1answer
23 views

which regression is better

suppose that we have two input vector and the variables in each vectors are independent and uncorrelated from each other,just only there is relationship between two vector,but not itself in ...
1
vote
2answers
2k views

“taking the derivative of both sides” ln(Y)=ln(x) (to interpret log transformed regression variables or better understand elasticities or % changes)

In a linear regression of the form Y=bX we often have ln transformed Y and X. ie., lnY=b*lnX This is interpreted as a 1% change in X resulting in a b% change in Y (approximately) The derivation of ...
2
votes
2answers
79 views

Find parameters for curve fitting (simple linear regression involved?)

I would like to fit data in g~t scatterplot, where ...
1
vote
1answer
411 views

Uncertainty in gradient of data

So I have a set of 9 x,y values, and I need to find the gradient/slope of the data, AND its associated error. Without the error, I would've used Excels LINEST function, but as the errors in my y ...
0
votes
2answers
859 views

Extrapolation with exponential curve

I would like to extrapolate time series using exponential curve while getting the parameters via linear regression. Exponential curve is given as $g=e^{~a + b \cdot t}$. Since I want to use linear ...
1
vote
0answers
39 views

How to find a function that can approximate another blackbox function programmaticly?

This question has been posted on http://stackoverflow.com/questions/21758016/how-to-find-a-function-that-can-approximate-another-blackbox-function-programmat I was suggested to post it here. I ...
3
votes
1answer
3k views

Why can we assume that the expected value of the error term is zero? [closed]

Why can we assume that the expected value of the error term in a linear regression model is zero? This is with regard to a simple linear regression.
1
vote
2answers
45 views

Would it be any concern if we find correlation between intercept and other regression coefficients?

During a multiple linear regression analysis, I found correlation between intercept (beta-0) and two of the other regression coefficients. Is there any problem or concern in this case? If no, please ...
0
votes
1answer
30 views

Setting up statistics problem

Assume $\beta_{U,T}$ is the underlying slope of straight line associating $U$ with $T$. We know that $X=U+f$ and $Z=T+e$ are measurable instead of $U$ and $T$, where $e$ and $f$ are uncorrelated ...
1
vote
3answers
48 views

Different Regression Lines?

Hi quick question with regression. If the coefficients of a simple regression line, B0 and B1, are the same then why are the regression lines of y on x and x on y different given the condition r^2 <...
1
vote
1answer
112 views

Combine linear models of different sets of data.

I'm working on a large data set D that can be partitioned into some disjoint subsets D1, D2, ..., Dn. For each subset Di, I have a linear model Mi that minimizes the residual error for data in Di. ...
0
votes
0answers
77 views

Question about sample variance with linear algebra

Given random variabes $Y_1,\dots,Y_n$ with mean $\mu$ and variance $\sigma^2$, I am supposed to prove that the sum of $(Y_i-(\text{mean of }y))^2$ can be expressed as $$y^T\left(I_{n\times n} - \...
1
vote
0answers
36 views

What is correlated with what in a linear regression?

I'm trying to make sure I understand the ins and outs of a linear regression and am making a table for what is correlated with what, so just want to see if I have everything included. I'm looking at ...
0
votes
1answer
80 views

Linear Regression Problem (“Regression Towards the Mean”)

I am having my mind turned upside down with a problem I am dealing with. So imagine we have a situation where we have pairs of points where x=heights of fathers and y=heights of the sons of these ...
2
votes
2answers
58 views

Merging Linear Regression

If I have built two linear regression models over sets $A$ and $B$, and now want a linear regression over set $A\cup{}B$. Is there a way to reuse what I already have?
1
vote
1answer
153 views

Calculating variance of estimated intercept parameter, $\hat\beta_0$

I have the following sample : $$ \begin{array}{c|lr} X&80&100&120&140&160&180&200&220&240&260\\ \hline Y & 70 &65&90&95&110&115&120&...
0
votes
1answer
267 views

Why is $(x'x)^{-1}x' = x(x'x)^{-1}$

If $(AB)'=B'A'$ then $(x'x)^{-1}x'$ should be equal to $x((x'x)^{-1})'$ . However most econometrics textbooks say that this is equal to $x(x'x)^{-1}$ . What happened to the transpose of $(x'x)^{-1}$? ...
0
votes
1answer
88 views

Finding estimates of a Linear Regression Equation - R

I'm new to Statistics and R. I'm currently looking through a book called "Discovering Statistics using R". Although the book implies you don't need any statistical background, some of the content isn'...
2
votes
2answers
40 views

Regression model when under-estimations costs us more than over-estimations

We have a factory and we are planning how many items produce in 2014. During the learning process we minimize the mean squared error. But, under-estimations costs us more than over-estimations. Let's ...
0
votes
1answer
75 views

Covariance of two random variables in a bivariate normal distribution

http://www.econstreams.com/bivariateproof.jpg Image uploaded to the link above. I'm just not seeing the connection between the 2nd equation on the left handside and the equation on the right. ...
1
vote
0answers
55 views

Solving a linear matrix equation with respect to the maximum of the euclidian distances between rows.

With $n>m$, real number matrices $A$, $B$, $C$ are shaped like: $$A=\left( \begin{array}{ccc} a_{1,1} & \cdots & a_{1,m} \\ \vdots & \ddots & \vdots \\ a_{n,m} & \cdots &...
1
vote
0answers
36 views

Adjusting regression for small sample bias

I have a set of data points $\{x_i\}$. These data points are grouped so that (say) $i\in\{1,2,3\}$ is group $A$, $i\in\{4,5,6,7\}$ is group $B$, etc. I would like to test the null hypothesis of no ...
0
votes
1answer
41 views

help in multiple linear regression

I am having a question in regression analysis in JMP or any other tool. I have one dependent variable $y$ and $2$ independent variables $x_1$ and $x_2$. For example: time $= y -$ per row time ( ...
1
vote
2answers
58 views

Polynomial best fit line for very large values

not only are the x values large, the difference between them and the y values is huge. My data points: 22353120,720 24448725,671.427053270323 26544330,634.312274868634 28639935,566.291966792026 ...
2
votes
1answer
113 views

How to calculate $\sum(X_i-\bar{X})^2$ in R

I'm trying to figure out how to calculate $\sum(X_i-\bar{X})^2$ in R, specifically identifying it in either the aov function or $\operatorname{lm}(y\sim x)$ function. I am trying to use it to ...
1
vote
0answers
41 views

Polynomial regression - differences between algorithms

I know that I can find a polynomial regression's coefficients doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose). This is a way of finding them; now, there is (as far as I know) at least one other ...
3
votes
4answers
908 views

Fitting curve for exponential: $y = A - B\mathrm{e}^{-t/\tau}$

I have some data that follows a saturation or charging profile such as $y = A - B\mathrm{e}^{-t/\tau}$. To begin with, is there a proper name for this function? I have seen it many times, including: ...
1
vote
1answer
40 views

Morphing $\beta_1$ into a different form (OLS question in Statistics)

I am currently studying Simple Linear Regression and I have successfully proven to myself how $\beta_1$ and $\beta_0$ are derived. However, I have been stuck on a seemingly simple problem (I'm sure ...
0
votes
2answers
46 views

Why is regression analysis also statistical test?

According to wikipedia, regression analysis is a statistical process for estimating the relationships among variables. Regression analysis is widely used for prediction and forecasting. So why is ...
1
vote
0answers
51 views

Linear Regression with multiple equations

I am trying to implement a linear regression algorithm to fit a set of "true" points with their "observed" location. The points are specified using spherical coordinates on a unit sphere. I have a ...
2
votes
2answers
197 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...