Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
1answer
61 views

Why does the regression line of $x$ on $y$ and $y$ on $x$ meet at $\bar{x}$ and $\bar{y}$?

Why does the least squares regression line of $x$ on $y$ and $y$ on $x$ intersect at $\bar{x}$ and $\bar{y}$? Also, why are the form of regression lines as they are? For the general form ...
2
votes
1answer
39 views

Probit model question (regression)

I'm reading a thesis and I need your help to understand the equation below. $$\Pr(\text{failure}=1 \mid X_1,X_3,X_3,X_4)=\int_{-\infty}^z \varphi(k) \, dk\tag{1}$$ $\varphi(k)$ is the standard ...
2
votes
0answers
80 views

Multilinear or Tensor Regression?

Given input data $x_t\in \mathbb{R}^n$ and output data $y_t\in\mathbb{R}^m$, the closed form solution to $\min_A \sum_t \|y_t - Ax_t\|^2_2$ is given by $A = (XX^T)^{-1}XY^T$ where $x_t$ form the ...
2
votes
0answers
42 views

Computing evidence for least-squares fit

I'm at a loss trying to implement Bayesian model selection for standard least-squares polynomials fits. I have three polynomials of order $1$, $2$, and $3$, and a sequence of $(x,y)$ data points. ...
2
votes
1answer
349 views

Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t $$ Regression with ...
2
votes
2answers
28 views

Linear regression. Lowering response maintaining equal independent variable.

I have put some data together and modelled the behaviour of the response ($y$) as function of three independent variables $x_1$, $x_2$ and $x_3$. A simple multi-linear regression. The model looks ...
2
votes
0answers
32 views

Coefficient of determination

$$ \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 = ( \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 - \displaystyle \sum^n_{i = 1} (y_i - \hat{y}_i)^2 ) + \displaystyle \sum^n_{i = 1} (y_i - ...
2
votes
0answers
106 views

Minimizing L4/ L6/ L2N norm for linear regression

OLS regression minimizes the sum of the squared errors. The normal equation for an OLS for $L_2$ minimization is as follows: $$b= (A'A)^{-1}A'y$$ What would be the equation to minimize the $L_4$ norm ...
2
votes
0answers
114 views

Orthonormal Matrix weighted regression

$Q$ is a rectangular matrix with orthonormal columns. A linear system composed of $$Qx= b$$ is really easy to solve as: $$Q'Q=I$$ hence: $$x=Q'b$$ Given that $Q$ is orthonormal can this be used to ...
2
votes
0answers
77 views

Regressing $Y$ back on the residuals

Suppose I have the linear regression model $ \hat{y_i} = a + b x_i $ for $a,b$ obtained via OLS. How does one regress $y$ back on the residuals $\hat{e}_i = y_i - \hat{y}_i$? If we write $ ...
2
votes
0answers
119 views

Effective model for calculating momentum or growth rate for a time series

I have a series of numbers tracking the performance of an entity on any given day. It's nothing but a simple integer for each date. For example, here's a series for Entity "X" ...
2
votes
0answers
191 views

Bare minimum of points in multiple polynomial regression

I have a question on multiple polynomial regression and the absolute minimum amount of points in the different terms. The minimum amount of points required for a second order polynomial would (in one ...
2
votes
0answers
78 views

Find $w$ as the minimizer of regularized logistic regression

Could someone point me to a reference on how to find $w$ as the minimizer of: $$ \frac{1}{2}\sum_{i=1}^{d}q_i(w_i-m_i)^2+\sum_{j=1}^{n}log(1+\exp(-y_jw^Tx_j)) $$ where $q_i$ (initialized with ...
2
votes
1answer
151 views

Projection Pursuit Regression

This is with reference to projection pursuit regressions. I kind of get the idea behind approximating a continuous function using weighted sums of ridge functions. I am not sure why ridge functions ...
2
votes
0answers
280 views

Logistic regression algorithm in Casio and Texas Instruments calculators

When using logistic regression on a Casio or Texas Instruments calculator, the output is of the form $$f(x) = \frac{c}{1+ae^{-bx}} $$ The problem I have (when teaching in a class where both types of ...
2
votes
2answers
468 views

Least squares estimator of mu

The question is: Assuming that $y_i = \mu + \epsilon_i $,$i = 1,\ldots,n$ with independent and identically distributed errors $\epsilon_i$ such that $E[\epsilon_i] = 0$ and $Var[\epsilon_i] = ...
2
votes
0answers
275 views

Finding a model for multiple non-linear regression

I want to implement a regression analysis, but I have problems with finding a model for the given data. There are $149$ $(x,y,z)$-values. $y$ values are all positive, $x$ is between $[-10, 10]$ and ...
2
votes
0answers
2k views

Derivation of standard error of beta in simple linear regression

Countless web pages show the equation for the standard error of the slope in a simple linear regression. For example: ...
2
votes
1answer
267 views

Multiple linear regression with interaction

I'm doing a multiple linear regression with interacting variables. I'll give you an example: y=value a1=material a2=weight a3=color a1 and a2 are interacting variables but a3 is not. Right now I'm ...
2
votes
0answers
85 views

Polynomial and exponential regression [duplicate]

Possible Duplicate: Determining computational complexity of stochastic processes I have some points $(x_i,y_i)$ generated by a program. These values are not exact, but are random ...
2
votes
0answers
2k views

Help with problem: Estimated Standard Deviation of Regression Equation (Simple Linear)

This is a practice problem. I've solved part (a). I have provided verified answers (from the published key) to all parts (a), (b) and & (c). I need help solving (b) and (c). Consider a simple ...
2
votes
0answers
226 views

Surface Function Fitting to Spherical Data

I have a set of geographic (longitude,latitude,value) data to which I would like to fit surface functions, specifically, the set of quadratic surfaces: $f(x,y)=Ax^2+Bx^2+Cxy+Dx+Ey+F$ At the moment, ...
2
votes
0answers
62 views

Accurate computation for Linear Regression case

I am writing a program that inputs a sequence of points $(x_i,y_i)$ based on the user clicking on certain pixels in an image shown. The program should then find the "best -fitting" line in the least ...
1
vote
5answers
476 views

Find square root approximation function (tool)

first I have to apologize for any uncorrect naming or categorisation of my question, as I am an electrical engineer rather than a mathematican. I try to find a simple solution for my problem: I have ...
1
vote
4answers
258 views

Parabola from 4 approximate points

I have calculated four approximate points from a sensors to get information. I would like to deduce the closest parabola to my points. The problem is that I can't solve it to get an appropriate ...
1
vote
3answers
497 views

Simple Least-Squares Regression Question

Given a set of 5 points (i.e. (1, 3), (2, 8) etc...), how can I get just the slope of the best fit line? I've been looking up least squares regression, but I'm rather statistics ignorant and don't ...
1
vote
1answer
85 views

Why can we assume that the expected value of the error term is zero? [closed]

Why can we assume that the expected value of the error term in a linear regression model is zero? This is with regard to a simple linear regression.
1
vote
2answers
44 views

Is it possible to fit any regression line to a set of data points?

If you have a set of data points (x1,y1), (x2,y2),...,(xn,yn) And you know it fits a trend y=f(x) where ...
1
vote
3answers
36 views

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and ...
1
vote
1answer
161 views

How to find a line of best fit of the form $y=ax$?

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1)(5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3)$$ How can we find the best fitting line $y=ax$ ...
1
vote
2answers
101 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...
1
vote
3answers
520 views

Exponential extrapolation

Given a set of points on 2D surface $(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)$ and a function $f(x)=k+ab^x$, the task is to find values of $k,a$ and $b$ that minimize the following sum: $$\sum_{i=1}^n ...
1
vote
3answers
190 views

Linear trend has to pass through a point

I need to interpolate a linear trend surface through a number of points but with the condition that the surface has to pass exactly through one of them. Can somebody give me any advice?
1
vote
2answers
82 views

Is there a way to fit an even function using only odd functions?

I was wondering if there is a way to make an infinite series of odd functions equal to an even function. For example, I would like to know if the next equation is valid ...
1
vote
2answers
632 views

Examples of logistic regression and multinomial logistic regression

I've been studying to understand the concept of logistic regression and I think I understand the idea more or less, but there are still some gaps to fill. What I'm looking for is an example of ...
1
vote
3answers
3k views

Fitting exponential curve to data

If I have a collection of data points that follow an exponential curve relationship, how can I manually construct the equation that defines the best-fit exponential curve for the data?
1
vote
1answer
56 views

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method?

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method? Link: http://www.youtube.com/watch?v=YwZYSTQs-Hk Thanks in advance!
1
vote
1answer
508 views

degrees of freedom for linear regression

If you have a reduced model with $H_0:\beta_1 = 1$, $H_a: \beta_1 \neq 1$, then the reduced model is: $$Y_i = 1X_i + \beta_0 + \epsilon_i$$ Are the degrees of freedom for the error term SSE $n-1$?
1
vote
2answers
388 views

Calculate Line Of Best Fit Using Exponential Weighting?

I know how to calculate a line of best fit with a set of data. I want to be able to exponentially weight the data that is more recent so that the more recent data has a greater effect on the line. ...
1
vote
3answers
764 views

Finding the change point in data from a piecewise linear function

Greetings, I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help! I have data conforming to a piecewise linear ...
1
vote
2answers
5k views

Find equation of curve fit programmatically in Matlab?

In MATLAB, when you plot something, there's a tool available which is called "curve fitting". And if you have a set of data points and a linear correlation, this tool will easily come up with an ...
1
vote
2answers
105 views

Prove $SST=SSE+SSR$

Prove $$SST=SSE+SSR$$ I start with $$SST= \Sigma (y_i-\bar{y})^2=...=SSE+SSR+ \Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )$$ and I don't know how to prove that $\Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )=0$ a ...
1
vote
2answers
22 views

Would it be any concern if we find correlation between intercept and other regression coefficients?

During a multiple linear regression analysis, I found correlation between intercept (beta-0) and two of the other regression coefficients. Is there any problem or concern in this case? If no, please ...
1
vote
2answers
36 views

Polynomial best fit line for very large values

not only are the x values large, the difference between them and the y values is huge. My data points: 22353120,720 24448725,671.427053270323 26544330,634.312274868634 28639935,566.291966792026 ...
1
vote
2answers
52 views

Multiple linear regression

For a homework we have to determine the effect of a predictor variable on an outcome variable using simple linear regression. We have lots of data (about 300 variables) and we may include some other ...
1
vote
1answer
118 views

Proof that a and b in linear regression are random variables

Does anyone know how to prove that the variables $a$ and $b$ that are used in linear regression are random variables? For me the assumption would be that these are dependent on the values of $x$ and ...
1
vote
2answers
98 views

Linear Regression to quadratic function

What is the optimal linear regression (w and w/o y-intercept) for a quadratic curve w.r.t. mean square error. Mathematically speaking: Given, $$y = x^2$$ for $$x = [-a,a]$$. What is the best ...
1
vote
1answer
311 views

What is the difference between a polynomial regression and a generalized linear model?

I have seen that a polynomial linear regression can have this form: $y = c_0 + c_1 x_1 + c_2 x_2 + \dots + c_k x_k $ but I have read that the general lineal model which is a form of the multiple ...
1
vote
2answers
83 views

What is the equation that fits this curve?

I have a curve that looks like this (it's cyclical): Curve I can get a partial fit by fitting a 3rd degree polynomial, but I have a feeling there must be a better fit (something that involves sin ...
1
vote
2answers
362 views

Normal Distribution from Standard Deviation?

So I have a data set $(x_{1},y_{1}), (x_{2},y_{2}),\dots,(x_{n},y_{n})$ and from it I have the values of $\sum x$, $\sum x^{2}$, $\sum y$, $\sum y^{2}$, $\sum xy$. My question is, how do I find a ...