Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
0answers
49 views

Computing evidence for least-squares fit

I'm at a loss trying to implement Bayesian model selection for standard least-squares polynomials fits. I have three polynomials of order $1$, $2$, and $3$, and a sequence of $(x,y)$ data points. ...
2
votes
0answers
548 views

What is the Moore-Penrose pseudoinverse for scaled linear regression?

The matrix equation for linear regression is: $$ \vec{y} = X\vec{\beta}+\vec{\epsilon} $$ The Least Square Error solution of this forms the normal equations: $$ ({\bf{X}}^T \bf{X}) \vec{\beta}= ...
2
votes
1answer
836 views

Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t $$ Regression with ...
2
votes
2answers
30 views

Linear regression. Lowering response maintaining equal independent variable.

I have put some data together and modelled the behaviour of the response ($y$) as function of three independent variables $x_1$, $x_2$ and $x_3$. A simple multi-linear regression. The model looks ...
2
votes
0answers
43 views

Coefficient of determination

$$ \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 = ( \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 - \displaystyle \sum^n_{i = 1} (y_i - \hat{y}_i)^2 ) + \displaystyle \sum^n_{i = 1} (y_i - ...
2
votes
0answers
149 views

Minimizing L4/ L6/ L2N norm for linear regression

OLS regression minimizes the sum of the squared errors. The normal equation for an OLS for $L_2$ minimization is as follows: $$b= (A'A)^{-1}A'y$$ What would be the equation to minimize the $L_4$ norm ...
2
votes
0answers
161 views

Orthonormal Matrix weighted regression

$Q$ is a rectangular matrix with orthonormal columns. A linear system composed of $$Qx= b$$ is really easy to solve as: $$Q'Q=I$$ hence: $$x=Q'b$$ Given that $Q$ is orthonormal can this be used to ...
2
votes
0answers
86 views

Regressing $Y$ back on the residuals

Suppose I have the linear regression model $ \hat{y_i} = a + b x_i $ for $a,b$ obtained via OLS. How does one regress $y$ back on the residuals $\hat{e}_i = y_i - \hat{y}_i$? If we write $ ...
2
votes
0answers
152 views

Effective model for calculating momentum or growth rate for a time series

I have a series of numbers tracking the performance of an entity on any given day. It's nothing but a simple integer for each date. For example, here's a series for Entity "X" ...
2
votes
0answers
315 views

Bare minimum of points in multiple polynomial regression

I have a question on multiple polynomial regression and the absolute minimum amount of points in the different terms. The minimum amount of points required for a second order polynomial would (in one ...
2
votes
0answers
91 views

Find $w$ as the minimizer of regularized logistic regression

Could someone point me to a reference on how to find $w$ as the minimizer of: $$ \frac{1}{2}\sum_{i=1}^{d}q_i(w_i-m_i)^2+\sum_{j=1}^{n}log(1+\exp(-y_jw^Tx_j)) $$ where $q_i$ (initialized with ...
2
votes
1answer
217 views

Projection Pursuit Regression

This is with reference to projection pursuit regressions. I kind of get the idea behind approximating a continuous function using weighted sums of ridge functions. I am not sure why ridge functions ...
2
votes
0answers
414 views

Logistic regression algorithm in Casio and Texas Instruments calculators

When using logistic regression on a Casio or Texas Instruments calculator, the output is of the form $$f(x) = \frac{c}{1+ae^{-bx}} $$ The problem I have (when teaching in a class where both types of ...
2
votes
2answers
838 views

Least squares estimator of mu

The question is: Assuming that $y_i = \mu + \epsilon_i $,$i = 1,\ldots,n$ with independent and identically distributed errors $\epsilon_i$ such that $E[\epsilon_i] = 0$ and $Var[\epsilon_i] = ...
2
votes
0answers
316 views

Finding a model for multiple non-linear regression

I want to implement a regression analysis, but I have problems with finding a model for the given data. There are $149$ $(x,y,z)$-values. $y$ values are all positive, $x$ is between $[-10, 10]$ and ...
2
votes
0answers
3k views

Derivation of standard error of beta in simple linear regression

Countless web pages show the equation for the standard error of the slope in a simple linear regression. For example: ...
2
votes
1answer
308 views

Multiple linear regression with interaction

I'm doing a multiple linear regression with interacting variables. I'll give you an example: $y$=value, $x_1$=material, $x_2$=weight, $x_3$=color $x_1$ and $x_2$ are interacting variables but $x_3$ ...
2
votes
0answers
85 views

Polynomial and exponential regression [duplicate]

Possible Duplicate: Determining computational complexity of stochastic processes I have some points $(x_i,y_i)$ generated by a program. These values are not exact, but are random ...
2
votes
0answers
3k views

Help with problem: Estimated Standard Deviation of Regression Equation (Simple Linear)

This is a practice problem. I've solved part (a). I have provided verified answers (from the published key) to all parts (a), (b) and & (c). I need help solving (b) and (c). Consider a simple ...
2
votes
0answers
270 views

Surface Function Fitting to Spherical Data

I have a set of geographic (longitude,latitude,value) data to which I would like to fit surface functions, specifically, the set of quadratic surfaces: $f(x,y)=Ax^2+Bx^2+Cxy+Dx+Ey+F$ At the moment, ...
2
votes
0answers
67 views

Accurate computation for Linear Regression case

I am writing a program that inputs a sequence of points $(x_i,y_i)$ based on the user clicking on certain pixels in an image shown. The program should then find the "best -fitting" line in the least ...
1
vote
5answers
572 views

Find square root approximation function (tool)

first I have to apologize for any uncorrect naming or categorisation of my question, as I am an electrical engineer rather than a mathematican. I try to find a simple solution for my problem: I have ...
1
vote
4answers
318 views

Parabola from 4 approximate points

I have calculated four approximate points from a sensors to get information. I would like to deduce the closest parabola to my points. The problem is that I can't solve it to get an appropriate ...
1
vote
3answers
595 views

Simple Least-Squares Regression Question

Given a set of 5 points (i.e. (1, 3), (2, 8) etc...), how can I get just the slope of the best fit line? I've been looking up least squares regression, but I'm rather statistics ignorant and don't ...
1
vote
2answers
65 views

Is it possible to fit any regression line to a set of data points?

If you have a set of data points (x1,y1), (x2,y2),...,(xn,yn) And you know it fits a trend y=f(x) where ...
1
vote
3answers
68 views

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and ...
1
vote
1answer
209 views

How to find a line of best fit of the form $y=ax$?

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1)(5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3)$$ How can we find the best fitting line $y=ax$ ...
1
vote
2answers
60 views

Using inverse of matrix A as approximate inverse of matrix that is very close to A

Say we have two matrices, $A$ and $A'$ so that $A\approx A'$, and we have the inverse of $A$, $B$, where $AB=I$, and the inverse of $A'$ where $A'B'=I$. If we have some guarantee about how big any ...
1
vote
2answers
46 views

Show $E\left(\mathbf{X}_i \otimes \mathbf{u}_i\right)=\mathbf{0}$ implies $E\left(\mathbf{X}_i^{\top}\mathbf{G}\mathbf{u}_i\right)=\mathbf{0}$

Let $\mathbf{X}_i$ be a $G \times K$ random matrix, and let $\mathbf{u}_i$ be a $G \times 1$ random vector, and suppose we have a sample of $i=1,\ldots,N$ of each. Suppose the following condition ...
1
vote
2answers
38 views

Would it be any concern if we find correlation between intercept and other regression coefficients?

During a multiple linear regression analysis, I found correlation between intercept (beta-0) and two of the other regression coefficients. Is there any problem or concern in this case? If no, please ...
1
vote
2answers
135 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...
1
vote
3answers
961 views

Exponential extrapolation

Given a set of points on 2D surface $(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)$ and a function $f(x)=k+ab^x$, the task is to find values of $k,a$ and $b$ that minimize the following sum: $$\sum_{i=1}^n ...
1
vote
2answers
314 views

Linear trend has to pass through a point

I need to interpolate a linear trend surface through a number of points but with the condition that the surface has to pass exactly through one of them. Can somebody give me any advice?
1
vote
2answers
113 views

Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

I know in general that if a matrix $A$ is positive definite, then there exists a (unique?) square root matrix $B$, which is also positive definite, such that $BB=A$. Therefore, suppose $A$ is ...
1
vote
1answer
46 views

Linear Regression with independent but non-identical noise

If I have this linear regression equation: $$y=X\beta+\epsilon $$ ($x$ and $\beta$ are vectors) The likelihood function can be written as $$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi ...
1
vote
1answer
92 views

Modeling non-linear data using least squares best fit

I have some data for liquid viscosity as a function of pressure and temperature. I would like to learn how come up with a single equation that would determine this fluid's viscosity with pressure and ...
1
vote
3answers
59 views

Linear regression throug the origin versus mean?

Assume that I have data that can be described by: $y_i = \beta x_i + \epsilon_i, \epsilon_i \sim (0,\sigma_{\epsilon})$, then the least squares estimator is given by $\hat{\beta_1} = ...
1
vote
1answer
108 views

Optimizing Independent Variables to Maximize Dependent Variable

I looked around online and couldn't find anything that was answering my question so I thought I would take to the stack! I'm interested in knowing if there is a statistical or mathematical way of ...
1
vote
1answer
122 views

Extremely poor polyfit, what am I doing wrong?

I have a dataset with me: http://pastebin.com/YZArky1j, which I am trying to polyfit. This is what I used to perform the polyfit: ...
1
vote
2answers
90 views

Is there a way to fit an even function using only odd functions?

I was wondering if there is a way to make an infinite series of odd functions equal to an even function. For example, I would like to know if the next equation is valid ...
1
vote
1answer
58 views

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method?

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method? Link: http://www.youtube.com/watch?v=YwZYSTQs-Hk Thanks in advance!
1
vote
1answer
578 views

degrees of freedom for linear regression

If you have a reduced model with $H_0:\beta_1 = 1$, $H_a: \beta_1 \neq 1$, then the reduced model is: $$Y_i = 1X_i + \beta_0 + \epsilon_i$$ Are the degrees of freedom for the error term SSE $n-1$?
1
vote
3answers
970 views

Finding the change point in data from a piecewise linear function

Greetings, I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help! I have data conforming to a piecewise linear ...
1
vote
1answer
9k views

Find equation of curve fit programmatically in Matlab?

In MATLAB, when you plot something, there's a tool available which is called "curve fitting". And if you have a set of data points and a linear correlation, this tool will easily come up with an ...
1
vote
1answer
28 views

Multicollinearity: Why does highly correlated columns in the design matrix lead to high variance of the regression coefficient?

I came across the term "Multicollinearity" in statistics, particularly statistics. However, I never really understand mathematically why highly correlated (almost linearly dependent) columns in the ...
1
vote
1answer
59 views

How to apply non-linear regression to Logistic (sigmoid) curve

I've been looking at a useful way to represent Doppler shift from a satellite passing over a ground station. I've calculated the Doppler shift frequency values at 1-second interval for the duration of ...
1
vote
2answers
70 views

Maple: How do I type “solve” with an arrow under?

I am trying to learn using Maple 18 (Mac). I have defined a function with a list of X and Y values. f := x->LinReg(X, Y, x) Now I would like to output the unknown "x" value that correlates with ...
1
vote
2answers
149 views

How do I find equation of this curve?

I need to find equation of the curve as shown below, for which, I need to find equation for upper part. lower part is half circle. upper part is a constant distance from circle with line passing ...
1
vote
2answers
2k views

Prove $SST=SSE+SSR$

Prove $$SST=SSE+SSR$$ I start with $$SST= \Sigma (y_i-\bar{y})^2=...=SSE+SSR+ \Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )$$ and I don't know how to prove that $\Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )=0$ a ...
1
vote
1answer
81 views

Robust Standard Errors

For OLS, my professor said that you should always test for heteroscedasticity first, rather than going straight to the adoption of robust standard errors. I didn't quite follow this and no ...