Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
1answer
68 views

How to calculate $\sum(X_i-\bar{X})^2$ in R

I'm trying to figure out how to calculate $\sum(X_i-\bar{X})^2$ in R, specifically identifying it in either the aov function or $\operatorname{lm}(y\sim x)$ function. I am trying to use it to ...
2
votes
1answer
29 views

Aproximate data with this equation (or linearize the equation)

I have found an equation that describes the behaviour of a phisical system: $$ y=a_1e^{-a_2x} + a_3 + a_4x + a_5e^{{-a_6} / {(1-x)}}$$ Now I have data of that physical system and I want to ...
2
votes
0answers
16k views

Linear regression: degrees of freedom of SST, SSR, and RSS

I'm trying to understand the concept of degrees of freedom in the specific case of the three quantities involved in a linear regression solution, i.e. $SST=SSR+SSE, $ i.e. Total sum of squares = ...
2
votes
1answer
65 views

Best fit line using geometric distance (not vertical distance)

There must be a theory of finding the best fit line to a bunch of points in the plane, where "best fit" is defined by the geometric distance, not vertical distance. In other words, we are trying to ...
2
votes
1answer
55 views

Basic Multilinear regression question for finding examples or counterexamples.

Hello Wise mathematicians! I have few quenstions about Multi linear regresstion. I've been asked from my friend, but I have very weak knowledge background from that field. It seems my friend is in ...
2
votes
1answer
93 views

Why does the regression line of $x$ on $y$ and $y$ on $x$ meet at $\bar{x}$ and $\bar{y}$?

Why does the least squares regression line of $x$ on $y$ and $y$ on $x$ intersect at $\bar{x}$ and $\bar{y}$? Also, why are the form of regression lines as they are? For the general form ...
2
votes
1answer
50 views

Probit model question (regression)

I'm reading a thesis and I need your help to understand the equation below. $$\Pr(\text{failure}=1 \mid X_1,X_3,X_3,X_4)=\int_{-\infty}^z \varphi(k) \, dk\tag{1}$$ $\varphi(k)$ is the standard ...
2
votes
0answers
51 views

Computing evidence for least-squares fit

I'm at a loss trying to implement Bayesian model selection for standard least-squares polynomials fits. I have three polynomials of order $1$, $2$, and $3$, and a sequence of $(x,y)$ data points. ...
2
votes
0answers
573 views

What is the Moore-Penrose pseudoinverse for scaled linear regression?

The matrix equation for linear regression is: $$ \vec{y} = X\vec{\beta}+\vec{\epsilon} $$ The Least Square Error solution of this forms the normal equations: $$ ({\bf{X}}^T \bf{X}) \vec{\beta}= ...
2
votes
1answer
894 views

Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t $$ Regression with ...
2
votes
2answers
31 views

Linear regression. Lowering response maintaining equal independent variable.

I have put some data together and modelled the behaviour of the response ($y$) as function of three independent variables $x_1$, $x_2$ and $x_3$. A simple multi-linear regression. The model looks ...
2
votes
0answers
43 views

Coefficient of determination

$$ \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 = ( \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 - \displaystyle \sum^n_{i = 1} (y_i - \hat{y}_i)^2 ) + \displaystyle \sum^n_{i = 1} (y_i - ...
2
votes
0answers
150 views

Minimizing L4/ L6/ L2N norm for linear regression

OLS regression minimizes the sum of the squared errors. The normal equation for an OLS for $L_2$ minimization is as follows: $$b= (A'A)^{-1}A'y$$ What would be the equation to minimize the $L_4$ norm ...
2
votes
0answers
161 views

Orthonormal Matrix weighted regression

$Q$ is a rectangular matrix with orthonormal columns. A linear system composed of $$Qx= b$$ is really easy to solve as: $$Q'Q=I$$ hence: $$x=Q'b$$ Given that $Q$ is orthonormal can this be used to ...
2
votes
0answers
87 views

Regressing $Y$ back on the residuals

Suppose I have the linear regression model $ \hat{y_i} = a + b x_i $ for $a,b$ obtained via OLS. How does one regress $y$ back on the residuals $\hat{e}_i = y_i - \hat{y}_i$? If we write $ ...
2
votes
0answers
157 views

Effective model for calculating momentum or growth rate for a time series

I have a series of numbers tracking the performance of an entity on any given day. It's nothing but a simple integer for each date. For example, here's a series for Entity "X" ...
2
votes
0answers
328 views

Bare minimum of points in multiple polynomial regression

I have a question on multiple polynomial regression and the absolute minimum amount of points in the different terms. The minimum amount of points required for a second order polynomial would (in one ...
2
votes
0answers
92 views

Find $w$ as the minimizer of regularized logistic regression

Could someone point me to a reference on how to find $w$ as the minimizer of: $$ \frac{1}{2}\sum_{i=1}^{d}q_i(w_i-m_i)^2+\sum_{j=1}^{n}log(1+\exp(-y_jw^Tx_j)) $$ where $q_i$ (initialized with ...
2
votes
0answers
425 views

Logistic regression algorithm in Casio and Texas Instruments calculators

When using logistic regression on a Casio or Texas Instruments calculator, the output is of the form $$f(x) = \frac{c}{1+ae^{-bx}} $$ The problem I have (when teaching in a class where both types of ...
2
votes
2answers
860 views

Least squares estimator of mu

The question is: Assuming that $y_i = \mu + \epsilon_i $,$i = 1,\ldots,n$ with independent and identically distributed errors $\epsilon_i$ such that $E[\epsilon_i] = 0$ and $Var[\epsilon_i] = ...
2
votes
0answers
320 views

Finding a model for multiple non-linear regression

I want to implement a regression analysis, but I have problems with finding a model for the given data. There are $149$ $(x,y,z)$-values. $y$ values are all positive, $x$ is between $[-10, 10]$ and ...
2
votes
0answers
3k views

Derivation of standard error of beta in simple linear regression

Countless web pages show the equation for the standard error of the slope in a simple linear regression. For example: ...
2
votes
1answer
309 views

Multiple linear regression with interaction

I'm doing a multiple linear regression with interacting variables. I'll give you an example: $y$=value, $x_1$=material, $x_2$=weight, $x_3$=color $x_1$ and $x_2$ are interacting variables but $x_3$ ...
2
votes
0answers
85 views

Polynomial and exponential regression [duplicate]

Possible Duplicate: Determining computational complexity of stochastic processes I have some points $(x_i,y_i)$ generated by a program. These values are not exact, but are random ...
2
votes
0answers
3k views

Help with problem: Estimated Standard Deviation of Regression Equation (Simple Linear)

This is a practice problem. I've solved part (a). I have provided verified answers (from the published key) to all parts (a), (b) and & (c). I need help solving (b) and (c). Consider a simple ...
2
votes
0answers
272 views

Surface Function Fitting to Spherical Data

I have a set of geographic (longitude,latitude,value) data to which I would like to fit surface functions, specifically, the set of quadratic surfaces: $f(x,y)=Ax^2+Bx^2+Cxy+Dx+Ey+F$ At the moment, ...
2
votes
0answers
67 views

Accurate computation for Linear Regression case

I am writing a program that inputs a sequence of points $(x_i,y_i)$ based on the user clicking on certain pixels in an image shown. The program should then find the "best -fitting" line in the least ...
1
vote
5answers
579 views

Find square root approximation function (tool)

first I have to apologize for any uncorrect naming or categorisation of my question, as I am an electrical engineer rather than a mathematican. I try to find a simple solution for my problem: I have ...
1
vote
4answers
326 views

Parabola from 4 approximate points

I have calculated four approximate points from a sensors to get information. I would like to deduce the closest parabola to my points. The problem is that I can't solve it to get an appropriate ...
1
vote
3answers
601 views

Simple Least-Squares Regression Question

Given a set of 5 points (i.e. (1, 3), (2, 8) etc...), how can I get just the slope of the best fit line? I've been looking up least squares regression, but I'm rather statistics ignorant and don't ...
1
vote
2answers
67 views

Is it possible to fit any regression line to a set of data points?

If you have a set of data points (x1,y1), (x2,y2),...,(xn,yn) And you know it fits a trend y=f(x) where ...
1
vote
3answers
74 views

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and ...
1
vote
1answer
217 views

How to find a line of best fit of the form $y=ax$?

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1)(5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3)$$ How can we find the best fitting line $y=ax$ ...
1
vote
2answers
61 views

Using inverse of matrix A as approximate inverse of matrix that is very close to A

Say we have two matrices, $A$ and $A'$ so that $A\approx A'$, and we have the inverse of $A$, $B$, where $AB=I$, and the inverse of $A'$ where $A'B'=I$. If we have some guarantee about how big any ...
1
vote
2answers
46 views

Show $E\left(\mathbf{X}_i \otimes \mathbf{u}_i\right)=\mathbf{0}$ implies $E\left(\mathbf{X}_i^{\top}\mathbf{G}\mathbf{u}_i\right)=\mathbf{0}$

Let $\mathbf{X}_i$ be a $G \times K$ random matrix, and let $\mathbf{u}_i$ be a $G \times 1$ random vector, and suppose we have a sample of $i=1,\ldots,N$ of each. Suppose the following condition ...
1
vote
2answers
38 views

Would it be any concern if we find correlation between intercept and other regression coefficients?

During a multiple linear regression analysis, I found correlation between intercept (beta-0) and two of the other regression coefficients. Is there any problem or concern in this case? If no, please ...
1
vote
2answers
140 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...
1
vote
3answers
996 views

Exponential extrapolation

Given a set of points on 2D surface $(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)$ and a function $f(x)=k+ab^x$, the task is to find values of $k,a$ and $b$ that minimize the following sum: $$\sum_{i=1}^n ...
1
vote
2answers
324 views

Linear trend has to pass through a point

I need to interpolate a linear trend surface through a number of points but with the condition that the surface has to pass exactly through one of them. Can somebody give me any advice?
1
vote
2answers
128 views

Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

I know in general that if a matrix $A$ is positive definite, then there exists a (unique?) square root matrix $B$, which is also positive definite, such that $BB=A$. Therefore, suppose $A$ is ...
1
vote
1answer
46 views

Linear Regression with independent but non-identical noise

If I have this linear regression equation: $$y=X\beta+\epsilon $$ ($x$ and $\beta$ are vectors) The likelihood function can be written as $$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi ...
1
vote
1answer
94 views

Modeling non-linear data using least squares best fit

I have some data for liquid viscosity as a function of pressure and temperature. I would like to learn how come up with a single equation that would determine this fluid's viscosity with pressure and ...
1
vote
3answers
59 views

Linear regression throug the origin versus mean?

Assume that I have data that can be described by: $y_i = \beta x_i + \epsilon_i, \epsilon_i \sim (0,\sigma_{\epsilon})$, then the least squares estimator is given by $\hat{\beta_1} = ...
1
vote
1answer
123 views

Optimizing Independent Variables to Maximize Dependent Variable

I looked around online and couldn't find anything that was answering my question so I thought I would take to the stack! I'm interested in knowing if there is a statistical or mathematical way of ...
1
vote
1answer
134 views

Extremely poor polyfit, what am I doing wrong?

I have a dataset with me: http://pastebin.com/YZArky1j, which I am trying to polyfit. This is what I used to perform the polyfit: ...
1
vote
2answers
90 views

Is there a way to fit an even function using only odd functions?

I was wondering if there is a way to make an infinite series of odd functions equal to an even function. For example, I would like to know if the next equation is valid ...
1
vote
1answer
58 views

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method?

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method? Link: http://www.youtube.com/watch?v=YwZYSTQs-Hk Thanks in advance!
1
vote
1answer
583 views

degrees of freedom for linear regression

If you have a reduced model with $H_0:\beta_1 = 1$, $H_a: \beta_1 \neq 1$, then the reduced model is: $$Y_i = 1X_i + \beta_0 + \epsilon_i$$ Are the degrees of freedom for the error term SSE $n-1$?
1
vote
3answers
987 views

Finding the change point in data from a piecewise linear function

Greetings, I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help! I have data conforming to a piecewise linear ...
1
vote
1answer
9k views

Find equation of curve fit programmatically in Matlab?

In MATLAB, when you plot something, there's a tool available which is called "curve fitting". And if you have a set of data points and a linear correlation, this tool will easily come up with an ...