Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
2answers
367 views

Fitting a polynomial + exponential curve of a given form to data

I have got a number of data sets of some parameter $m_x$ against an independent variable $x$. Through each of the data sets I need to best fit a curve of the form $A + Bc^x$ such that $A$, $B$ and $c$ ...
0
votes
2answers
154 views

Programmatic Cubic Regression

All, Thanks in advance for your help. There're a lot of "low hanging fruit" problems at work I need to tackle as a tech-level employee. One of them is curve-fitting 6 data points to a cubic curve, ...
2
votes
0answers
793 views

What is the Moore-Penrose pseudoinverse for scaled linear regression?

The matrix equation for linear regression is: $$ \vec{y} = X\vec{\beta}+\vec{\epsilon} $$ The Least Square Error solution of this forms the normal equations: $$ ({\bf{X}}^T \bf{X}) \vec{\beta}= ...
2
votes
1answer
30 views

Determine which parameter has correlation with result and which is not

sorry for probably silly question, it's the first time when I need to do such work. I have large data set with regarding clicks on some element on web page. It contains some characteristics of such ...
0
votes
1answer
75 views

Regression vs. Normal Distribution

I have to estimate something using historical data. Should I find the equation of the curve of best fit to estimate? Or use a confidence interval, standard deviation, and a z-score to calculate it? ...
2
votes
1answer
1k views

Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t $$ Regression with ...
1
vote
0answers
58 views

Conditioning on $X$ equal to premultiplying by $X'$?

I am coming across similar thing in many problems in econometrics and I have not been able to figure out whether it is some general notion or only a "coincidence". To take two examples: Deriving ...
1
vote
1answer
119 views

Nonlinear regression analysis of a vector

I'm trying to get a nonlinear fit of a vector in Matlab with no success. Let's assume that I have a vector called data: data = [1,30,250,55,22,76] which can be ...
1
vote
0answers
216 views

Broken Line Regression

$X = $Lot & $Y = $Cost Give a broken line linear model with a breakpoint at $250$: $$Y = B_0 + B_1X_1 + B_2X_2 + B_3X_3 + e$$ where $X_2 = 0$ or $1$ depending on whether the lot size is $\geq ...
1
vote
1answer
449 views

How do I solve the weighted normal equations?

I am trying to solve the normal equations for a 3D LSE of a general quadric: $$ z = ax^2 + bx + cxy + dy^2 + ey + f$$ Write as a vector equation: $$ \vec{z}= \bf{X}\vec{\beta}$$ where the 'ith row ...
1
vote
1answer
543 views

Linear fit with horizontal and vertical error bars

I'm searching an equation to calculate the parameters for a linear fit. With parameters a and b, the $\chi ^{2}$ is used: $\chi ^{2} = \sum_{i=0}^N (y_{i}-a.x_{i}-b)^{2}$ And with errors: $\chi ...
2
votes
2answers
32 views

Linear regression. Lowering response maintaining equal independent variable.

I have put some data together and modelled the behaviour of the response ($y$) as function of three independent variables $x_1$, $x_2$ and $x_3$. A simple multi-linear regression. The model looks ...
1
vote
0answers
75 views

Residual Plot Analysis

I'm working on building a regression model for a large set of data (n>54000). Clearly a ton of assumptions are being violated that I have to try and adjust for. I'm all for transformations of data and ...
1
vote
0answers
87 views

Proving $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$

I have asked in another question how $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$. Note that $\hat{y}_h$ = $b_0 + b_1X_h$ which is a regression line ...
0
votes
1answer
61 views

Questions Regarding Linear Regression

Are the slope and intercept of a simple linear regression model always normally distributed? Is there ever a difference between the distribution of the estimated slope and intercept and the actual ...
0
votes
2answers
573 views

Linear Regression - Proof Sum Adds to Zero

In linear regression, why is $\sum(X_{i} - \mu_{x})$ = $0$? I understand that for ($\sum$ $Y_{i}$ minus the fitted value of Y) = $\sum$ $e_{i}$ this is true but why is this other fact true?
1
vote
1answer
299 views

Simple Linear Regression Question

Let $Y_{i} = \beta_{0} + \beta_{1}X_{i} + \epsilon_{i}$ be a simple linear regression model with independent errors and iid normal distribution. If $X_{i}$ are fixed what is the distribution of ...
2
votes
2answers
2k views

Linear Regression: Expectation Proof

I found the following proof in my notes: $E(Y_i) = E[\beta_0 + \beta X_i + \varepsilon_i] =\cdots= \beta_0 + \beta X_i$. This does not seem right to me, however. Why would $E(\beta_1 X_i) = \beta_1 ...
0
votes
2answers
227 views

Is a Relationship Quadratic?

I have a relationship $y=f(x)$ for which I can obtain data through simulation. I have good reason to suspect that this relationship is quadratic (rather than, say, exponential), and would like to ...
0
votes
0answers
248 views

Stata: “Between and fixed effect estimates” in a linear regression?

I'm working on a paper by B. H. Baltagi and I am trying to replicate the results. It can be found here, the data is here. I'm supposed to do a linear regression - sounds simple. The author uses Stata, ...
0
votes
1answer
26 views

Interpretation of regression formula returned by computer software

I have a dataset consisting of 744 records. Data exploring software generated an equation I don't know how to interpret in simple words. I really appreciate if you could help me about this matter. ...
0
votes
1answer
203 views

Multiple regression problems (restricted regression, dummy variables)

Q1. Model 1: $Y=X_1\beta_1+\varepsilon$ Model 2: $Y=X_1\beta_1+X_2\beta_2+\varepsilon$ (a) Suppose that Model 1 is true. If we estimates OLS estrimator $b_1$ for $\beta_1$ in Model 2, what will happen ...
1
vote
0answers
71 views

regression coefficient

Consider observations on three variables X1;X2 and X3: Suppose that X1 is regressed on X2: When the residual of the above regression is regressed on X3; the regression coefficient of X3 is b3: When X1 ...
0
votes
0answers
824 views

Generating an equation from an image I have

I am not exactly sure if this question belongs here but I could not think of a better place to ask. So I recently discovered that various people on the internet have created equations for rather ...
0
votes
1answer
338 views

Fast way of finding RSS of Multiple Linear Regression

Is there any smarter way to compute Residual Sum of Squares(RSS) in Multiple Linear Regression other then fitting the model -> find coefficients -> find fitted values -> find residuals -> find norm of ...
2
votes
1answer
240 views

Techniques to find regression parameters for multiple datasets where a subset of parameters should be the same for all datasets

I have five sets of observations of measured y as some function of measured $x_1, x_2, x_3,\ldots$ and I want to fit five functions to these observations. They have the form $$ y = f(x_1, x_2, ...
2
votes
1answer
480 views

Finding uncertainty in the slope/intercept for a non-linear least squares fit

I have the following function: $$M = a(\log_{10}W-2.5)+b$$ I also have a set of data with actual measured values of $W$ and $M$ (each have individual $\pm$ errors). Here's a small sampling of the ...
1
vote
1answer
34 views

Initializing Variables using Shrinkage

I have a user-user model which which users can rate their friendships(r) with others and also can have activities with them(a). I am using Matrix Factorization and Gradient Descent for updating the ...
1
vote
0answers
51 views

About the weights assigned in the linear regression

I have this confusion related to linear regression. Lets say I have two predictors $x_1$ and $x_2$ and the target is $y$. I learn a linear regression with $y \sim x_1,x_1 \cdot x_2,x_2$ with $x_1 ...
1
vote
1answer
1k views

Prediction Model for forecasting using Linear regression

I am very new to inferral statistics. I am trying to build a prediction model for forecasting the revenue for physicians based on some historical data. I was planning to use Multiple Linear Regression ...
1
vote
0answers
40 views

Sequential problem for n=1, non linear regression

I am trying to understand an example in my stats course notes, the example relates to calculating the best value for the next experiment. The function of the line is very simple: $$ln(Y_i) = ...
2
votes
1answer
142 views

Least Squares Regression To Half of a Parabola

I have a set of points in two dimensional space, and I know a priori that they approximate half of a parabola. I want to find the coefficients for a quadratic function where all of the points fall on ...
2
votes
4answers
100 views

Condition for $\det(A^{T}A)=0$

Is it always true that $\det(A^{T}A)=0$, $\hspace{0.5mm}$ for $A=n \times m$ matrix with $n<m$? From some notes I am reading on Regression analysis, and from some trials, it would appear this is ...
1
vote
1answer
317 views

Proof that a and b in linear regression are random variables

Does anyone know how to prove that the variables $a$ and $b$ that are used in linear regression are random variables? For me the assumption would be that these are dependent on the values of $x$ and ...
1
vote
3answers
1k views

Exponential extrapolation

Given a set of points on 2D surface $(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)$ and a function $f(x)=k+ab^x$, the task is to find values of $k,a$ and $b$ that minimize the following sum: $$\sum_{i=1}^n ...
3
votes
3answers
10k views

Fitting exponential curve to data

If I have a collection of data points that follow an exponential curve relationship, how can I manually construct the equation that defines the best-fit exponential curve for the data?
1
vote
1answer
300 views

How to find a line of best fit of the form $y=ax$?

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1)(5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3)$$ How can we find the best fitting line $y=ax$ ...
1
vote
0answers
367 views

Correlation coefficient.

A linear regression gives us a correlation coefficient $r=0$. What is the equation of the best fit line? Give an example of data with $r=0$ What is the value of the correlation coefficient of data ...
1
vote
1answer
125 views

How to gather useful information from a residue plot

You can usually see how good your linear regression line is by looking at the residue plot. If you see the points randomly distributed, you're good. But if you see a pattern, it means there is ...
1
vote
1answer
55 views

What is the way to determin how good a sequence will interpolate?

Say I have to sequences of numbers: $$[5, 10, 14, 21, 27, 31]$$ $$[1, 20, 21, 22, 30, 31]$$ Even though they both get to $31$ by the $6$th element, logic tells me that only the first one is a good ...
3
votes
1answer
77 views

Why is $\sum x^2 _t \times \text{Var}(\beta)=\frac{\sum x^2 _t \times \sigma^2}{ \sum x^2 _t} = \sigma^2$?

I do not get this connection. Is is reliable to divide this equation by $\sum x^2 _t$ to get just $\sigma^2$ ? $$\sum x^2 _t \times E(\hat \beta - \beta)^2=\sum x^2 _t \times ...
1
vote
1answer
31 views

How to interpret these regression values?

If GPA(gpa after fall semester in college) is the dependent variable and SAT (score on the SAT) is the independent variable and I have the following parameter estimates: Intercept: .66306 SAT: ...
2
votes
0answers
45 views

Coefficient of determination

$$ \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 = ( \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 - \displaystyle \sum^n_{i = 1} (y_i - \hat{y}_i)^2 ) + \displaystyle \sum^n_{i = 1} (y_i - ...
1
vote
2answers
133 views

Linear regression question

I don't understand the following derivation: $$ e_i = y_i - ax_i - b$$ $$ e_i = (y_i - \bar{y}) - a(x_i - \bar{x}) - (b - \bar{y} + a \bar{x}) $$ I don't really understand what they do and why they ...
0
votes
2answers
7k views

Unconditional expectation vs conditional expectation in regressions - does it really matter?

I refer here to a simple linear regression whose true representation is given by the equation: $y_i=x_i'\beta+u_i$, where as usual $x_i$ is a $Kx1$ vector of independent explanatory variables, ...
2
votes
1answer
484 views

Arriving at the Logistic function from a Binomial Distribution and Maximum Likelihood

I've been trying to understand the origin of the Logistic function in Logistic regression: $$\Pr(Y=1|x;\theta)=\frac{1}{1+e^{-\theta x}}$$ I was lead to beilive that one could somehow arrive at this ...
0
votes
1answer
152 views

How to solve multi-variate linear regression analytically?

We have $n$ variables $x_n$ and one stochastic function $y$ of these variables. We assume that function $y$ depends on variables in the following way: $y = c + \sum_{i=1}^n k_i x_i + \varepsilon_i$, ...
0
votes
1answer
172 views

Forecasting using multiple regression

I have data in the form given below, and I want to perform forecasting using multiple regression. I found definition of multiple regression from this link: http://otexts.com/fpp/5/1/ . I have these ...
0
votes
1answer
73 views

Non-Linear regression

Imagine that I have a function $ f(x,y) $ to model a physical phenomenon. I believe that functions is defined by $$ f(x,y) = A*x + B*y + C*x*y$$ I have many values for $ (x,y,f(x,y)) $, how can I ...
1
vote
1answer
50 views

Cointegration for Price levels Time Series

I don't understand why is the difference between price levels is a stationary process while the time series of price levels themselves is a non-stationary process. For example: ...