Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
2answers
115 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...
1
vote
3answers
679 views

Exponential extrapolation

Given a set of points on 2D surface $(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)$ and a function $f(x)=k+ab^x$, the task is to find values of $k,a$ and $b$ that minimize the following sum: $$\sum_{i=1}^n ...
1
vote
3answers
209 views

Linear trend has to pass through a point

I need to interpolate a linear trend surface through a number of points but with the condition that the surface has to pass exactly through one of them. Can somebody give me any advice?
1
vote
1answer
60 views

Modeling non-linear data using least squares best fit

I have some data for liquid viscosity as a function of pressure and temperature. I would like to learn how come up with a single equation that would determine this fluid's viscosity with pressure and ...
1
vote
3answers
48 views

Linear regression throug the origin versus mean?

Assume that I have data that can be described by: $y_i = \beta x_i + \epsilon_i, \epsilon_i \sim (0,\sigma_{\epsilon})$, then the least squares estimator is given by $\hat{\beta_1} = ...
1
vote
1answer
64 views

Extremely poor polyfit, what am I doing wrong?

I have a dataset with me: http://pastebin.com/YZArky1j, which I am trying to polyfit. This is what I used to perform the polyfit: ...
1
vote
1answer
84 views

Intuition and the math behind normalization

What exactly is the purpose of normalization. From what I read, it is to adjust two different sets of values so you can compare them, but I don't understand why, nor the math behind it. Could anyone ...
1
vote
2answers
85 views

Is there a way to fit an even function using only odd functions?

I was wondering if there is a way to make an infinite series of odd functions equal to an even function. For example, I would like to know if the next equation is valid ...
1
vote
3answers
4k views

Fitting exponential curve to data

If I have a collection of data points that follow an exponential curve relationship, how can I manually construct the equation that defines the best-fit exponential curve for the data?
1
vote
1answer
56 views

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method?

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method? Link: http://www.youtube.com/watch?v=YwZYSTQs-Hk Thanks in advance!
1
vote
1answer
527 views

degrees of freedom for linear regression

If you have a reduced model with $H_0:\beta_1 = 1$, $H_a: \beta_1 \neq 1$, then the reduced model is: $$Y_i = 1X_i + \beta_0 + \epsilon_i$$ Are the degrees of freedom for the error term SSE $n-1$?
1
vote
3answers
847 views

Finding the change point in data from a piecewise linear function

Greetings, I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help! I have data conforming to a piecewise linear ...
1
vote
1answer
6k views

Find equation of curve fit programmatically in Matlab?

In MATLAB, when you plot something, there's a tool available which is called "curve fitting". And if you have a set of data points and a linear correlation, this tool will easily come up with an ...
1
vote
2answers
124 views

How do I find equation of this curve?

I need to find equation of the curve as shown below, for which, I need to find equation for upper part. lower part is half circle. upper part is a constant distance from circle with line passing ...
1
vote
1answer
35 views

Optimizing Independent Variables to Maximize Dependent Variable

I looked around online and couldn't find anything that was answering my question so I thought I would take to the stack! I'm interested in knowing if there is a statistical or mathematical way of ...
1
vote
2answers
475 views

Prove $SST=SSE+SSR$

Prove $$SST=SSE+SSR$$ I start with $$SST= \Sigma (y_i-\bar{y})^2=...=SSE+SSR+ \Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )$$ and I don't know how to prove that $\Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )=0$ a ...
1
vote
2answers
30 views

Would it be any concern if we find correlation between intercept and other regression coefficients?

During a multiple linear regression analysis, I found correlation between intercept (beta-0) and two of the other regression coefficients. Is there any problem or concern in this case? If no, please ...
1
vote
2answers
47 views

Polynomial best fit line for very large values

not only are the x values large, the difference between them and the y values is huge. My data points: 22353120,720 24448725,671.427053270323 26544330,634.312274868634 28639935,566.291966792026 ...
1
vote
2answers
57 views

Multiple linear regression

For a homework we have to determine the effect of a predictor variable on an outcome variable using simple linear regression. We have lots of data (about 300 variables) and we may include some other ...
1
vote
1answer
146 views

Proof that a and b in linear regression are random variables

Does anyone know how to prove that the variables $a$ and $b$ that are used in linear regression are random variables? For me the assumption would be that these are dependent on the values of $x$ and ...
1
vote
2answers
107 views

Linear Regression to quadratic function

What is the optimal linear regression (w and w/o y-intercept) for a quadratic curve w.r.t. mean square error. Mathematically speaking: Given, $$y = x^2$$ for $$x = [-a,a]$$. What is the best ...
1
vote
1answer
408 views

What is the difference between a polynomial regression and a generalized linear model?

I have seen that a polynomial linear regression can have this form: $y = c_0 + c_1 x_1 + c_2 x_2 + \dots + c_k x_k $ but I have read that the general lineal model which is a form of the multiple ...
1
vote
2answers
88 views

What is the equation that fits this curve?

I have a curve that looks like this (it's cyclical): Curve I can get a partial fit by fitting a 3rd degree polynomial, but I have a feeling there must be a better fit (something that involves sin ...
1
vote
2answers
385 views

Normal Distribution from Standard Deviation?

So I have a data set $(x_{1},y_{1}), (x_{2},y_{2}),\dots,(x_{n},y_{n})$ and from it I have the values of $\sum x$, $\sum x^{2}$, $\sum y$, $\sum y^{2}$, $\sum xy$. My question is, how do I find a ...
1
vote
1answer
85 views

$(X^tX)^{-1}$ when $p>>n$

For the $n\times p$ matrix $\mathbf{X}$, is there any use in approximating $(\mathbf{X}'\mathbf{X})^{-1}$ when $p>>n$? If so, what information might this tell us? I understand when $p<n$, ...
1
vote
3answers
127 views

Middle line between points

How can we calculate LINE that most fit the points T1(1,0) T2(2,0) T3(-1,1) T4(0,1) $x= (1,2,-1,0) $ $y= (0, 0, 1, 1) $ $1= (1, 1, 1, 1)$
1
vote
1answer
118 views

Calculating the regression equations

I have four data points $(1,2), (2,4), (3,5), (5,7)$ and Im looking for the least squares regression line that best fits them. I use the normal equation $A^tAx=A^tb$ in this form - ...
1
vote
1answer
1k views

Construct / find the simplest function based on data

Let's say I have these 7 natural numbers (all between 0 and 255): 255, 23, 45, 32, 87, 52, 146 How can I find a function F(x) that, once computed, gives me back ...
1
vote
1answer
677 views

Simple non linear fitting question(Least Squares Fitting--Exponential) [duplicate]

Possible Duplicate: easy to implement method to fit a power function (regression) I have the following simple function: $h = cV^n$ h and V being the variables and $c$ and $n$ are ...
1
vote
1answer
132 views

problem constructing an equation that connects variables

need a suggestion/advice/inputs from the mathematicians here. I have collected data of number of instructions/sec executed by a processor every second, number of loads/stores performed every second ...
1
vote
1answer
26 views

time-series regression with missing data

I have a regression as follows for time-series data (e.g. stock prices versus other variables): $$ Y = b \cdot X + b_1 \cdot X_1 + e$$ where $X_1$ will be missing based on pre-determined dates ...
1
vote
1answer
24 views

Finding best predictors of a classification function

I have a large dataset where each element has a number of "input" categories that are either present or not (or if you like, true or false, 1 or 0 etc). Each one also has an output category, again a ...
1
vote
1answer
38 views

Variance- covariance matrix

Consider $H$ denotes hat matrix and $e$ denotes residual. In the book Applied regression Analysis by Draper/Smith, it is written that : $\mathbb V(e_i)$ is given ...
1
vote
1answer
15 views

Dummy recoding for more than two categorical variables

Say I am doing a study with 3 different types of fruit and I want to make a regression depending on the type that tries to predict the amount sold. I know that I could make 2 dummy variables: orange ...
1
vote
1answer
29 views

Is the Inverse of the Vectorised Solid Angle Equation for $n$ Circular Discs Continuous?

I have a continuous function$^{*1}$ that takes in 3 arguments, and returns 24 outputs. I want to know if the inverse of this function is continuous. The 3 input arguments are the x, y, and z position ...
1
vote
1answer
19 views

Linear regression with normalized variables

Suppose I have two variables X and Y such that mean(X) = 0 = mean(Y) and sd(X) = 1 = sd(Y). The slope of the linear regression line for Y vs X is cov(X,Y)/var(X) = corr(X,Y) since X and Y are ...
1
vote
2answers
70 views

Plotting an Ellipse after an Ellipse Fit

I wonder if someone can assist my understanding as I'm a bit stumped with this... I have taken the following (x,y) data which lies roughly on an ellipse: $$ \begin{pmatrix} 0.000234491 & 6855810 ...
1
vote
1answer
32 views

Calculating R-squared with duplicate data

I have the following question regarding the proper usage of R-squared value. Say I have an equation, that predicts energy consumption for the month of a building. One of the input variables accounts ...
1
vote
1answer
11 views

Meaning of $\mathcal{I}_t$ in assumption $E[u_t\mid\mathcal{I}_t]$ of distributed-lag model

When considering \begin{equation*} y_t = \beta_0 + \beta_1 x_t + \ldots + \beta_r x_{t-r} + u_t \end{equation*} an assumption made is \begin{equation*} E[u_t\mid\mathcal{I}_t] = 0 \end{equation*} ...
1
vote
1answer
37 views

Wolfram Exponential Fit not match as formulas

I am using WolframAlpha Exponential-Fit formulas to find equation of Exponential Regression http://mathworld.wolfram.com/LeastSquaresFittingExponential.html but after implementation, I tested with a ...
1
vote
1answer
28 views

clustering of singular values

let us consider following graph of singular values i want to make some kind of clustering of these data,namely to seperate main components from non main components,let say signal components ...
1
vote
1answer
44 views

Equations for Cubic Regression

So, I'm making a simple program for drawing graphs, and I'm looking at making some simple best-fit curves using some basic regression analysis. I've happily got linear and quadratic regression working ...
1
vote
1answer
59 views

regression on circular data

How would one design a regression where the dependent variable is measured in degrees on a circle? The dependent variable is on the range [0, 360), and the independent variables are demographic ...
1
vote
1answer
34 views

What's the “ridge” in Ridge Regression?

In normal least squares, we try to find $\hat\beta$ which minimizes $$\|y-X\beta\|^2$$ Ridge regression expands this to "penalize" certain values of $\beta$ via a matrix $\Gamma$: ...
1
vote
1answer
48 views

Variance of Coefficients in a Simple Linear Regression

I have a linear regression model $\hat{y_i}=\hat{\beta_0}+\hat{\beta_1}x_i+\hat{\epsilon_i}$, where $\hat{\beta_0}$ and $\hat{\beta_1}$ are normally distributed unbiased estimators, and ...
1
vote
1answer
28 views

Regression towards the mean?

I am upset with what examples testing "regression to the mean" seem to allude to: People claiming to have "ESP" take a test, and A's score was 2 standard deviations above the mean, whereas B's score ...
1
vote
3answers
37 views

Different Regression Lines?

Hi quick question with regression. If the coefficients of a simple regression line, B0 and B1, are the same then why are the regression lines of y on x and x on y different given the condition r^2 ...
1
vote
1answer
34 views

Combine linear models of different sets of data.

I'm working on a large data set D that can be partitioned into some disjoint subsets D1, D2, ..., Dn. For each subset Di, I have a linear model Mi that minimizes the residual error for data in Di. ...
1
vote
1answer
28 views

Morphing $\beta_1$ into a different form (OLS question in Statistics)

I am currently studying Simple Linear Regression and I have successfully proven to myself how $\beta_1$ and $\beta_0$ are derived. However, I have been stuck on a seemingly simple problem (I'm sure ...
1
vote
2answers
45 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...