Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
0answers
85 views

Polynomial and exponential regression [duplicate]

Possible Duplicate: Determining computational complexity of stochastic processes I have some points $(x_i,y_i)$ generated by a program. These values are not exact, but are random ...
2
votes
1answer
433 views

Fitting a sine function to data

I have a sequence of $n$ points $(x_i,y_i)$, for $i=1,\dots,n$. I would like to find the function, of the form $y=V\sin(x+\phi)$, which best fits the points. Which numerical method could I use? I have ...
2
votes
0answers
2k views

Help with problem: Estimated Standard Deviation of Regression Equation (Simple Linear)

This is a practice problem. I've solved part (a). I have provided verified answers (from the published key) to all parts (a), (b) and & (c). I need help solving (b) and (c). Consider a simple ...
2
votes
0answers
259 views

Surface Function Fitting to Spherical Data

I have a set of geographic (longitude,latitude,value) data to which I would like to fit surface functions, specifically, the set of quadratic surfaces: $f(x,y)=Ax^2+Bx^2+Cxy+Dx+Ey+F$ At the moment, ...
2
votes
0answers
65 views

Accurate computation for Linear Regression case

I am writing a program that inputs a sequence of points $(x_i,y_i)$ based on the user clicking on certain pixels in an image shown. The program should then find the "best -fitting" line in the least ...
1
vote
5answers
533 views

Find square root approximation function (tool)

first I have to apologize for any uncorrect naming or categorisation of my question, as I am an electrical engineer rather than a mathematican. I try to find a simple solution for my problem: I have ...
1
vote
4answers
297 views

Parabola from 4 approximate points

I have calculated four approximate points from a sensors to get information. I would like to deduce the closest parabola to my points. The problem is that I can't solve it to get an appropriate ...
1
vote
3answers
551 views

Simple Least-Squares Regression Question

Given a set of 5 points (i.e. (1, 3), (2, 8) etc...), how can I get just the slope of the best fit line? I've been looking up least squares regression, but I'm rather statistics ignorant and don't ...
1
vote
1answer
536 views

Why can we assume that the expected value of the error term is zero? [closed]

Why can we assume that the expected value of the error term in a linear regression model is zero? This is with regard to a simple linear regression.
1
vote
2answers
56 views

Is it possible to fit any regression line to a set of data points?

If you have a set of data points (x1,y1), (x2,y2),...,(xn,yn) And you know it fits a trend y=f(x) where ...
1
vote
3answers
58 views

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and ...
1
vote
1answer
196 views

How to find a line of best fit of the form $y=ax$?

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1)(5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3)$$ How can we find the best fitting line $y=ax$ ...
1
vote
2answers
40 views

Show $E\left(\mathbf{X}_i \otimes \mathbf{u}_i\right)=\mathbf{0}$ implies $E\left(\mathbf{X}_i^{\top}\mathbf{G}\mathbf{u}_i\right)=\mathbf{0}$

Let $\mathbf{X}_i$ be a $G \times K$ random matrix, and let $\mathbf{u}_i$ be a $G \times 1$ random vector, and suppose we have a sample of $i=1,\ldots,N$ of each. Suppose the following condition ...
1
vote
2answers
121 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...
1
vote
3answers
793 views

Exponential extrapolation

Given a set of points on 2D surface $(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)$ and a function $f(x)=k+ab^x$, the task is to find values of $k,a$ and $b$ that minimize the following sum: $$\sum_{i=1}^n ...
1
vote
3answers
263 views

Linear trend has to pass through a point

I need to interpolate a linear trend surface through a number of points but with the condition that the surface has to pass exactly through one of them. Can somebody give me any advice?
1
vote
2answers
86 views

Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

I know in general that if a matrix $A$ is positive definite, then there exists a (unique?) square root matrix $B$, which is also positive definite, such that $BB=A$. Therefore, suppose $A$ is ...
1
vote
1answer
36 views

Linear Regression with independent but non-identical noise

If I have this linear regression equation: $$y=X\beta+\epsilon $$ ($x$ and $\beta$ are vectors) The likelihood function can be written as $$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi ...
1
vote
1answer
78 views

Modeling non-linear data using least squares best fit

I have some data for liquid viscosity as a function of pressure and temperature. I would like to learn how come up with a single equation that would determine this fluid's viscosity with pressure and ...
1
vote
3answers
53 views

Linear regression throug the origin versus mean?

Assume that I have data that can be described by: $y_i = \beta x_i + \epsilon_i, \epsilon_i \sim (0,\sigma_{\epsilon})$, then the least squares estimator is given by $\hat{\beta_1} = ...
1
vote
1answer
88 views

Extremely poor polyfit, what am I doing wrong?

I have a dataset with me: http://pastebin.com/YZArky1j, which I am trying to polyfit. This is what I used to perform the polyfit: ...
1
vote
1answer
185 views

Intuition and the math behind normalization

What exactly is the purpose of normalization. From what I read, it is to adjust two different sets of values so you can compare them, but I don't understand why, nor the math behind it. Could anyone ...
1
vote
2answers
86 views

Is there a way to fit an even function using only odd functions?

I was wondering if there is a way to make an infinite series of odd functions equal to an even function. For example, I would like to know if the next equation is valid ...
1
vote
3answers
5k views

Fitting exponential curve to data

If I have a collection of data points that follow an exponential curve relationship, how can I manually construct the equation that defines the best-fit exponential curve for the data?
1
vote
1answer
57 views

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method?

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method? Link: http://www.youtube.com/watch?v=YwZYSTQs-Hk Thanks in advance!
1
vote
1answer
554 views

degrees of freedom for linear regression

If you have a reduced model with $H_0:\beta_1 = 1$, $H_a: \beta_1 \neq 1$, then the reduced model is: $$Y_i = 1X_i + \beta_0 + \epsilon_i$$ Are the degrees of freedom for the error term SSE $n-1$?
1
vote
3answers
903 views

Finding the change point in data from a piecewise linear function

Greetings, I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help! I have data conforming to a piecewise linear ...
1
vote
1answer
7k views

Find equation of curve fit programmatically in Matlab?

In MATLAB, when you plot something, there's a tool available which is called "curve fitting". And if you have a set of data points and a linear correlation, this tool will easily come up with an ...
1
vote
2answers
136 views

How do I find equation of this curve?

I need to find equation of the curve as shown below, for which, I need to find equation for upper part. lower part is half circle. upper part is a constant distance from circle with line passing ...
1
vote
1answer
57 views

Optimizing Independent Variables to Maximize Dependent Variable

I looked around online and couldn't find anything that was answering my question so I thought I would take to the stack! I'm interested in knowing if there is a statistical or mathematical way of ...
1
vote
2answers
1k views

Prove $SST=SSE+SSR$

Prove $$SST=SSE+SSR$$ I start with $$SST= \Sigma (y_i-\bar{y})^2=...=SSE+SSR+ \Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )$$ and I don't know how to prove that $\Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )=0$ a ...
1
vote
1answer
55 views

Robust Standard Errors

For OLS, my professor said that you should always test for heteroscedasticity first, rather than going straight to the adoption of robust standard errors. I didn't quite follow this and no ...
1
vote
2answers
31 views

Would it be any concern if we find correlation between intercept and other regression coefficients?

During a multiple linear regression analysis, I found correlation between intercept (beta-0) and two of the other regression coefficients. Is there any problem or concern in this case? If no, please ...
1
vote
2answers
48 views

Polynomial best fit line for very large values

not only are the x values large, the difference between them and the y values is huge. My data points: 22353120,720 24448725,671.427053270323 26544330,634.312274868634 28639935,566.291966792026 ...
1
vote
2answers
61 views

Multiple linear regression

For a homework we have to determine the effect of a predictor variable on an outcome variable using simple linear regression. We have lots of data (about 300 variables) and we may include some other ...
1
vote
1answer
183 views

Proof that a and b in linear regression are random variables

Does anyone know how to prove that the variables $a$ and $b$ that are used in linear regression are random variables? For me the assumption would be that these are dependent on the values of $x$ and ...
1
vote
2answers
110 views

Linear Regression to quadratic function

What is the optimal linear regression (w and w/o y-intercept) for a quadratic curve w.r.t. mean square error. Mathematically speaking: Given, $$y = x^2$$ for $$x = [-a,a]$$. What is the best ...
1
vote
1answer
299 views

QR factorization for ridge regression

I am solving an overdetermined system of equations: $$Ax= b$$ Using QR factorization, we can solve this system easily by posing it as: $$Rx= Q'b$$ I would like to regularize my estimate of $x$. I ...
1
vote
1answer
473 views

What is the difference between a polynomial regression and a generalized linear model?

I have seen that a polynomial linear regression can have this form: $y = c_0 + c_1 x_1 + c_2 x_2 + \dots + c_k x_k $ but I have read that the general lineal model which is a form of the multiple ...
1
vote
2answers
92 views

What is the equation that fits this curve?

I have a curve that looks like this (it's cyclical): Curve I can get a partial fit by fitting a 3rd degree polynomial, but I have a feeling there must be a better fit (something that involves sin ...
1
vote
2answers
395 views

Normal Distribution from Standard Deviation?

So I have a data set $(x_{1},y_{1}), (x_{2},y_{2}),\dots,(x_{n},y_{n})$ and from it I have the values of $\sum x$, $\sum x^{2}$, $\sum y$, $\sum y^{2}$, $\sum xy$. My question is, how do I find a ...
1
vote
1answer
87 views

$(X^tX)^{-1}$ when $p>>n$

For the $n\times p$ matrix $\mathbf{X}$, is there any use in approximating $(\mathbf{X}'\mathbf{X})^{-1}$ when $p>>n$? If so, what information might this tell us? I understand when $p<n$, ...
1
vote
3answers
129 views

Middle line between points

How can we calculate LINE that most fit the points T1(1,0) T2(2,0) T3(-1,1) T4(0,1) $x= (1,2,-1,0) $ $y= (0, 0, 1, 1) $ $1= (1, 1, 1, 1)$
1
vote
2answers
419 views

vertical vs. horizontal regression

A horizontal regression is defined as the following: $$m=\frac{\sum_{i=1}^n (x_i-\operatorname{average(x)})(y_i-\operatorname{average(y))}}{\sum_{i=1}^n (x_i-\operatorname{average(x)})^2}$$ whereas ...
1
vote
1answer
121 views

Calculating the regression equations

I have four data points $(1,2), (2,4), (3,5), (5,7)$ and Im looking for the least squares regression line that best fits them. I use the normal equation $A^tAx=A^tb$ in this form - ...
1
vote
1answer
2k views

Construct / find the simplest function based on data

Let's say I have these 7 natural numbers (all between 0 and 255): 255, 23, 45, 32, 87, 52, 146 How can I find a function F(x) that, once computed, gives me back ...
1
vote
1answer
728 views

Simple non linear fitting question(Least Squares Fitting--Exponential) [duplicate]

Possible Duplicate: easy to implement method to fit a power function (regression) I have the following simple function: $h = cV^n$ h and V being the variables and $c$ and $n$ are ...
1
vote
1answer
132 views

problem constructing an equation that connects variables

need a suggestion/advice/inputs from the mathematicians here. I have collected data of number of instructions/sec executed by a processor every second, number of loads/stores performed every second ...
1
vote
1answer
15 views

what is one basic/intermediate regression analysis standard textbook that is math intense

What is one basic/intermediate regression analysis standard textbook that is math intense with proofs/derivations? Also, i need that one to be comprehensive yet the diffculty is suitable for self ...
1
vote
1answer
23 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...