Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
0answers
274 views

What is the Moore-Penrose pseudoinverse for scaled linear regression?

The matrix equation for linear regression is: $$ \vec{y} = X\vec{\beta}+\vec{\epsilon} $$ The Least Square Error solution of this forms the normal equations: $$ ({\bf{X}}^T \bf{X}) \vec{\beta}= ...
2
votes
1answer
27 views

Determine which parameter has correlation with result and which is not

sorry for probably silly question, it's the first time when I need to do such work. I have large data set with regarding clicks on some element on web page. It contains some characteristics of such ...
0
votes
0answers
42 views

Which method to use for solving LSE

I am fitting elevation data $z(xi,yi)$ to a 2nd order polynomial: $z = ax^2 + by^2 + cxy + dx + ey + f$ using Least Squares Error. Which decomposition should I use to solve the normal equations? ...
0
votes
1answer
70 views

Regression vs. Normal Distribution

I have to estimate something using historical data. Should I find the equation of the curve of best fit to estimate? Or use a confidence interval, standard deviation, and a z-score to calculate it? ...
2
votes
1answer
350 views

Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t $$ Regression with ...
1
vote
0answers
50 views

Conditioning on $X$ equal to premultiplying by $X'$?

I am coming across similar thing in many problems in econometrics and I have not been able to figure out whether it is some general notion or only a "coincidence". To take two examples: Deriving ...
1
vote
1answer
73 views

Nonlinear regression analysis of a vector

I'm trying to get a nonlinear fit of a vector in Matlab with no success. Let's assume that I have a vector called data: data = [1,30,250,55,22,76] which can be ...
1
vote
0answers
97 views

Broken Line Regression

$X = $Lot & $Y = $Cost Give a broken line linear model with a breakpoint at $250$: $$Y = B_0 + B_1X_1 + B_2X_2 + B_3X_3 + e$$ where $X_2 = 0$ or $1$ depending on whether the lot size is $\geq ...
1
vote
1answer
124 views

How do I solve the weighted normal equations?

I am trying to solve the normal equations for a 3D LSE of a general quadric: $$ z = ax^2 + bx + cxy + dy^2 + ey + f$$ Write as a vector equation: $$ \vec{z}= \bf{X}\vec{\beta}$$ where the 'ith row ...
1
vote
1answer
115 views

Linear fit with horizontal and vertical error bars

I'm searching an equation to calculate the parameters for a linear fit. With parameters a and b, the $\chi ^{2}$ is used: $\chi ^{2} = \sum_{i=0}^N (y_{i}-a.x_{i}-b)^{2}$ And with errors: $\chi ...
2
votes
2answers
28 views

Linear regression. Lowering response maintaining equal independent variable.

I have put some data together and modelled the behaviour of the response ($y$) as function of three independent variables $x_1$, $x_2$ and $x_3$. A simple multi-linear regression. The model looks ...
1
vote
0answers
61 views

Residual Plot Analysis

I'm working on building a regression model for a large set of data (n>54000). Clearly a ton of assumptions are being violated that I have to try and adjust for. I'm all for transformations of data and ...
1
vote
0answers
62 views

Proving $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$

I have asked in another question how $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$. Note that $\hat{y}_h$ = $b_0 + b_1X_h$ which is a regression line ...
0
votes
0answers
21 views

Regression of a Multivariable, nonlinear System

$y =\beta_{1}ln(x_{1}) + \beta_{3} x_{2}x_{3} + \beta_{2}e^\ x_{4}$ How would one perform regression from a list of points to fit an equation of this form? $x_{1}$ through $x_{4}$ are explanatory ...
0
votes
1answer
39 views

Questions Regarding Linear Regression

Are the slope and intercept of a simple linear regression model always normally distributed? Is there ever a difference between the distribution of the estimated slope and intercept and the actual ...
0
votes
2answers
236 views

Linear Regression - Proof Sum Adds to Zero

In linear regression, why is $\sum(X_{i} - \mu_{x})$ = $0$? I understand that for ($\sum$ $Y_{i}$ minus the fitted value of Y) = $\sum$ $e_{i}$ this is true but why is this other fact true?
0
votes
0answers
54 views

Questions on Linear Regression

I had a few true / false questions on a practice test that I would like to discuss if possible. a) A value $R^2$ close to 1 indicates the linear regression is a good fit to data Yes, but I am not ...
1
vote
1answer
107 views

Simple Linear Regression Question

Let $Y_{i} = \beta_{0} + \beta_{1}X_{i} + \epsilon_{i}$ be a simple linear regression model with independent errors and iid normal distribution. If $X_{i}$ are fixed what is the distribution of ...
2
votes
2answers
397 views

Linear Regression: Expectation Proof

I found the following proof in my notes: $E(Y_i) = E[\beta_0 + \beta X_i + \varepsilon_i] =\cdots= \beta_0 + \beta X_i$. This does not seem right to me, however. Why would $E(\beta_1 X_i) = \beta_1 ...
0
votes
2answers
80 views

Is a Relationship Quadratic?

I have a relationship $y=f(x)$ for which I can obtain data through simulation. I have good reason to suspect that this relationship is quadratic (rather than, say, exponential), and would like to ...
1
vote
0answers
228 views

How to calculate probability with sigmoid output in feedforward neural network?

first of all I'm sorry for my not very skilled English, but I will do my best to explain my problem. I'm trying to create a feedforward neural network with one hidden layer (with probably arctan ...
0
votes
0answers
122 views

Stata: “Between and fixed effect estimates” in a linear regression?

I'm working on a paper by B. H. Baltagi and I am trying to replicate the results. It can be found here, the data is here. I'm supposed to do a linear regression - sounds simple. The author uses Stata, ...
0
votes
1answer
25 views

Interpretation of regression formula returned by computer software

I have a dataset consisting of 744 records. Data exploring software generated an equation I don't know how to interpret in simple words. I really appreciate if you could help me about this matter. ...
0
votes
1answer
115 views

Multiple regression problems (restricted regression, dummy variables)

Q1. Model 1: $Y=X_1\beta_1+\varepsilon$ Model 2: $Y=X_1\beta_1+X_2\beta_2+\varepsilon$ (a) Suppose that Model 1 is true. If we estimates OLS estrimator $b_1$ for $\beta_1$ in Model 2, what will happen ...
1
vote
0answers
45 views

regression coefficient

Consider observations on three variables X1;X2 and X3: Suppose that X1 is regressed on X2: When the residual of the above regression is regressed on X3; the regression coefficient of X3 is b3: When X1 ...
0
votes
0answers
203 views

Generating an equation from an image I have

I am not exactly sure if this question belongs here but I could not think of a better place to ask. So I recently discovered that various people on the internet have created equations for rather ...
0
votes
1answer
107 views

Fast way of finding RSS of Multiple Linear Regression

Is there any smarter way to compute Residual Sum of Squares(RSS) in Multiple Linear Regression other then fitting the model -> find coefficients -> find fitted values -> find residuals -> find norm of ...
0
votes
0answers
41 views

If $\underset{n \times n}{M}$ is a symmetric and idempotent matrix having rank $r$

If $\underset{n \times n}{M}$ is a symmetric and idempotent matrix having rank $r$ then $$w'Mw \sim \sigma^2 \chi^2_{(r)}$$ where $\underset {n \times 1}{W} \sim N(0,\sigma^2 I)$ that is, $w_i \sim ...
2
votes
1answer
124 views

Techniques to find regression parameters for multiple datasets where a subset of parameters should be the same for all datasets

I have five sets of observations of measured y as some function of measured $x_1, x_2, x_3,\ldots$ and I want to fit five functions to these observations. They have the form $$ y = f(x_1, x_2, ...
2
votes
1answer
203 views

Finding uncertainty in the slope/intercept for a non-linear least squares fit

I have the following function: $$M = a(\log_{10}W-2.5)+b$$ I also have a set of data with actual measured values of $W$ and $M$ (each have individual $\pm$ errors). Here's a small sampling of the ...
1
vote
1answer
21 views

Initializing Variables using Shrinkage

I have a user-user model which which users can rate their friendships(r) with others and also can have activities with them(a). I am using Matrix Factorization and Gradient Descent for updating the ...
1
vote
0answers
48 views

About the weights assigned in the linear regression

I have this confusion related to linear regression. Lets say I have two predictors $x_1$ and $x_2$ and the target is $y$. I learn a linear regression with $y \sim x_1,x_1 \cdot x_2,x_2$ with $x_1 ...
1
vote
1answer
620 views

Prediction Model for forecasting using Linear regression

I am very new to inferral statistics. I am trying to build a prediction model for forecasting the revenue for physicians based on some historical data. I was planning to use Multiple Linear Regression ...
1
vote
0answers
37 views

Sequential problem for n=1, non linear regression

I am trying to understand an example in my stats course notes, the example relates to calculating the best value for the next experiment. The function of the line is very simple: $$ln(Y_i) = ...
2
votes
1answer
108 views

Least Squares Regression To Half of a Parabola

I have a set of points in two dimensional space, and I know a priori that they approximate half of a parabola. I want to find the coefficients for a quadratic function where all of the points fall on ...
2
votes
4answers
94 views

Condition for $\det(A^{T}A)=0$

Is it always true that $\det(A^{T}A)=0$, $\hspace{0.5mm}$ for $A=n \times m$ matrix with $n<m$? From some notes I am reading on Regression analysis, and from some trials, it would appear this is ...
1
vote
1answer
120 views

Proof that a and b in linear regression are random variables

Does anyone know how to prove that the variables $a$ and $b$ that are used in linear regression are random variables? For me the assumption would be that these are dependent on the values of $x$ and ...
1
vote
3answers
525 views

Exponential extrapolation

Given a set of points on 2D surface $(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)$ and a function $f(x)=k+ab^x$, the task is to find values of $k,a$ and $b$ that minimize the following sum: $$\sum_{i=1}^n ...
0
votes
0answers
63 views

What is the difference between random and nonrandom?

In a simple regression model $Y_i=\beta_0+\beta_1 X_i+\epsilon_i$, $X_i$ is nonrandom. But we don't know $\beta_0, \beta_1$ value (we should estimate them in our model), $Y_i$ is random. I wonder what ...
1
vote
3answers
3k views

Fitting exponential curve to data

If I have a collection of data points that follow an exponential curve relationship, how can I manually construct the equation that defines the best-fit exponential curve for the data?
1
vote
1answer
163 views

How to find a line of best fit of the form $y=ax$?

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1)(5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3)$$ How can we find the best fitting line $y=ax$ ...
1
vote
0answers
105 views

Correlation coefficient.

A linear regression gives us a correlation coefficient $r=0$. What is the equation of the best fit line? Give an example of data with $r=0$ What is the value of the correlation coefficient of data ...
1
vote
1answer
83 views

How to gather useful information from a residue plot

You can usually see how good your linear regression line is by looking at the residue plot. If you see the points randomly distributed, you're good. But if you see a pattern, it means there is ...
1
vote
1answer
48 views

What is the way to determin how good a sequence will interpolate?

Say I have to sequences of numbers: $$[5, 10, 14, 21, 27, 31]$$ $$[1, 20, 21, 22, 30, 31]$$ Even though they both get to $31$ by the $6$th element, logic tells me that only the first one is a good ...
3
votes
1answer
72 views

Why is $\sum x^2 _t \times \text{Var}(\beta)=\frac{\sum x^2 _t \times \sigma^2}{ \sum x^2 _t} = \sigma^2$?

I do not get this connection. Is is reliable to divide this equation by $\sum x^2 _t$ to get just $\sigma^2$ ? $$\sum x^2 _t \times E(\hat \beta - \beta)^2=\sum x^2 _t \times ...
1
vote
1answer
26 views

How to interpret these regression values?

If GPA(gpa after fall semester in college) is the dependent variable and SAT (score on the SAT) is the independent variable and I have the following parameter estimates: Intercept: .66306 SAT: ...
2
votes
0answers
32 views

Coefficient of determination

$$ \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 = ( \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 - \displaystyle \sum^n_{i = 1} (y_i - \hat{y}_i)^2 ) + \displaystyle \sum^n_{i = 1} (y_i - ...
1
vote
2answers
120 views

Linear regression question

I don't understand the following derivation: $$ e_i = y_i - ax_i - b$$ $$ e_i = (y_i - \bar{y}) - a(x_i - \bar{x}) - (b - \bar{y} + a \bar{x}) $$ I don't really understand what they do and why they ...
0
votes
2answers
2k views

Unconditional expectation vs conditional expectation in regressions - does it really matter?

I refer here to a simple linear regression whose true representation is given by the equation: $y_i=x_i'\beta+u_i$, where as usual $x_i$ is a $Kx1$ vector of independent explanatory variables, ...
2
votes
1answer
260 views

Arriving at the Logistic function from a Binomial Distribution and Maximum Likelihood

I've been trying to understand the origin of the Logistic function in Logistic regression: $$\Pr(Y=1|x;\theta)=\frac{1}{1+e^{-\theta x}}$$ I was lead to beilive that one could somehow arrive at this ...