Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

9
votes
2answers
1k views

Why is polynomial regression considered a kind of linear regression?

Why is polynomial regression considered a kind of linear regression? This is what I mean by polynomial regression. For example, the hypothesis function is $$h(x; t_0, t_1, t_2) = t_0 + t_1 x + t_2 ...
38
votes
9answers
11k views

Why do we use a Least Squares fit?

I've been wondering for a while now if there's any deep mathematical or statistical significance to finding the line that minimizes the square of the errors between the line and the data points. If ...
10
votes
1answer
4k views

easy to implement method to fit a power function (regression)

I want to fit to a dataset a power function ($y=Ax^B$). What is the best and easiest method to do this. I need the $A$ and $B$ parameters too. I'm using in general financial data in my project, which ...
7
votes
3answers
2k views

Derivation of the formula for Ordinary Least Squares Linear Regression

How was the formula for Ordinary Least Squares Linear Regression arrived at? Note I am not only looking for the proof, but also the derivation. Where did the formula come from?
24
votes
3answers
8k views

derivative of cost function for Logistic Regression

I am going over the lectures on Machine Learning at Coursera. I am struggling with the following. How can the partial derivative of ...
7
votes
5answers
15k views

Given a data set, how do you do a sinusoidal regression on paper? What are the equations, algorithms?

Most regressions are easy. Trivial once you know how to do it. Most of them involve substitutions which transform the data into a linear regression. But I have yet to figure out how to do a ...
15
votes
3answers
6k views

Finding the intersection point of many lines in 3D (point closest to all lines)

I have many lines (let's say 4) which are supposed to be intersected. (Please consider lines are represented as a pair of points). I want to find the point in space which minimizes the sum of the ...
2
votes
1answer
751 views

Polynomial fitting - how to fit and what is _polynomial fitting_

I don't understand what is polynomial fitting. Can anyone explain me how to fit a curve to given points?
6
votes
1answer
107 views

Formula for straight part of a slightly bumpy line

Given a straight line that deviates from the horizontal by at most 15 degrees. On this straight line there are bumps on top at random places on the line. The combined width of the bumps is at most ...
1
vote
1answer
52 views

Iterative Power Regression

If I have a set of data points that would fit inside a power equation of the form y = a*x^b, what is the best ITERATIVE method to find the values of 'a' and 'b'. I thought I could compute the error ...
1
vote
2answers
474 views

Linear trend has to pass through a point

I need to interpolate a linear trend surface through a number of points but with the condition that the surface has to pass exactly through one of them. Can somebody give me any advice?
-1
votes
2answers
178 views

Proving that the estimate of a mean is a least squares estimator?

I think this is a really simple question so please bear with me -- I just had my first class in regression and I'm a little confused about nomenclature/labeling. Does anyone recommend some good ...
11
votes
1answer
9k views

Computational complexity of least square regression operation

In a least square regression algorithm, I have to do the following operations to compute regression coefficients: Matrix multiplication, complexity: $O(C^2N)$ Matrix inversion, complexity: ...
11
votes
4answers
4k views

Polynomial fitting where polynomial must be monotonically increasing

Given a set of monotonically increasing data points (in 2D), I want to fit a polynomial to the data which is monotonically increasing over the domain of the data. If the highest x value is 100, I ...
2
votes
1answer
2k views

Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t $$ Regression with ...
12
votes
3answers
2k views

Best fit ellipsoid

Given a collection of points $P \subset \mathbb R^3$, a crude characterization of the "shape" of $P$ is sometimes given by the principal components. We construct a covariance matrix, e.g., if $P$ is ...
5
votes
1answer
38k views

How to find curve equation from data?

How do I find the formula when I only know some data points ? Usually I just use the Trendline option for diagrams in Excel, but this one eludes me. I expect it to be something like : ...
3
votes
1answer
4k views

Construct / find the simplest function based on data

Let's say I have these 7 natural numbers (all between 0 and 255): 255, 23, 45, 32, 87, 52, 146 How can I find a function F(x) that, once computed, gives me back ...
3
votes
3answers
693 views

Correlation between variables

I asked this question on stats SE but did not find a suitable answer so far. Maybe someone can help. Given n random variables x1,...,xn (one-dimensional). The following is known (corr() = Pearson ...
1
vote
1answer
173 views

Can an Artificial Neural Network with Only One Hidden Layer Suffice for all Applications?

I have heard that only a single layer is needed for an artificial neural network to fit any possible function (input to output). 1. Is this true and where is this shown? 2. If this is true, then ...
4
votes
3answers
11k views

Fitting exponential curve to data

If I have a collection of data points that follow an exponential curve relationship, how can I manually construct the equation that defines the best-fit exponential curve for the data?
4
votes
2answers
2k views

Fitting an exponential function to data

I have a noisy data set (the grey line in the graph below) that corresponds roughly to $y=m(1-2^{-x/k})$ where m and k are unknown constants. How can I determine the best-fit value of m and k? I ...
2
votes
2answers
142 views

derivation of simple linear regression parameters

I know there are some proof in the internet, but I attempted to proove the formulas for the intercept and the slope in simple linear regression using Least squares, some algebra, and partial ...
2
votes
1answer
6k views

Equations For Quadratic Regression

Does anyone know the specific equations for the three parameters in a least-squares quadratic regression? I'm looking for something like $\beta_1=,\beta_2=,\beta_3=$ for each of ...
1
vote
2answers
131 views

Fit exponential with constant

I have data whic would fit to an exponential function with a constant. So y=aexp(bt) + c Now I can solve an exponential without a constant using least square by taking log of y and making the ...
1
vote
1answer
35 views

Regression project in octave/matlab

I'm trying to establish a polynomial model to adjust the variation of the dollar throughout the year. Suppose hypothetically that I have the following data ...
0
votes
1answer
67 views

Non-Linear Model Transformation

I want to transform this Non-Linear Model $y= 8-ae^{bx}$ to Linear.And my issue is in this step $lny=ln(8-ae^{bx})$ how can simplify it to reach in a linear model which is like this $y*=b0+b1x$ ...
0
votes
1answer
228 views

Why is $E(u)=0$ when an intercept is included in OLS Estimation?

I am reading Wooldridge's graduate econometrics text. There he states that when estimating the equation $y=\mathbf{x\beta}+u$ by OLS, if an intercept (constant term) is included in your $\mathbf{x}$ ...
5
votes
2answers
269 views

Recommended Reading on Regression Analysis?

For a university project, I am implementing an automated regression analysis tool. However, I have very little background in statistics. So what books / articles / material would you suggest I could ...
3
votes
2answers
188 views

How many points to find a polynomial?

I would like to fit a formula $ax^b + cx^d+ e$ to a set of points. I have two questions. If my data were perfect, how many points do I need in the worst case to get $a,b,c,d,e$ exactly? If my data ...
3
votes
0answers
184 views

How to perform nonlinear regression with correlated errors?

I have a nonlinear least squares problem, but the errors are correlated. I could use R's nls function to do the regression if the errors were independent, but I don't know the right way to handle ...
3
votes
0answers
338 views

Least Square Method with Positive Parameters

this is my first post here in the Stack Exchange. A friend told me about this forum and I'm giving it a try. I searched a bit past threads, but couldn't find what I wanted, so I'm posting the problem ...
3
votes
2answers
758 views

Fitting a sine function to data

I have a sequence of $n$ points $(x_i,y_i)$, for $i=1,\dots,n$. I would like to find the function, of the form $y=V\sin(x+\phi)$, which best fits the points. Which numerical method could I use? I have ...
2
votes
1answer
97 views

Least-squares solution to a matrix equation?

Suppose I have $n$ observations of $m$ dependent variables $y_1,\dots,y_m$, and I believe they follow some model wherein they can all be written as linear combinations of some underlying variables ...
2
votes
1answer
265 views

Techniques to find regression parameters for multiple datasets where a subset of parameters should be the same for all datasets

I have five sets of observations of measured y as some function of measured $x_1, x_2, x_3,\ldots$ and I want to fit five functions to these observations. They have the form $$ y = f(x_1, x_2, ...
2
votes
2answers
581 views

Closed form for coefficients in Multiple Regression model

I want to find $\hat{\beta}$ in ordinary least squares s.t. $\hat{Y} = \hat{\beta}_0 + \hat{\beta}_1 X_1 + \cdots + \hat{\beta}_n X_n $. I know the way to do this is through the normal equation using ...
2
votes
0answers
361 views

Finding a model for multiple non-linear regression

I want to implement a regression analysis, but I have problems with finding a model for the given data. There are $149$ $(x,y,z)$-values. $y$ values are all positive, $x$ is between $[-10, 10]$ and ...
2
votes
0answers
88 views

Polynomial and exponential regression [duplicate]

Possible Duplicate: Determining computational complexity of stochastic processes I have some points $(x_i,y_i)$ generated by a program. These values are not exact, but are random ...
2
votes
1answer
274 views

How do I do a least squares fit of $a x + b y = 1$?

How do I do a least squares fit of the line equation $a x + b y = 1$, so that the points are as close to the line as possible? (Not just vertically close) If I use the matrices $$X = ...
2
votes
2answers
1k views

Fitting data to a portion of an ellipse or conic section

Is there a straightforward algorithm for fitting data to an ellipse or other conic section? The data generally only approximately fits a portion of the ellipse. I am looking for something that doesn't ...
1
vote
1answer
43 views

Regression: Service time based on number of copiers

For a random sample of 10 service calls, both the number of copiers and the total service time were recorded. Number of Copiers (x) : 4, 2, 5, 7, 1, 3, 4, 5, 2, 6 ...
1
vote
4answers
473 views

Program to find closest function to fit arbitrary data

I've wanted this for years, but have never come across anything; a program for Windows to find the closest function to fit arbitrary data. The data I feed it is simple: A table with two columns ...
1
vote
0answers
218 views

Matrix Decompositions: Difference between Cholesky Decomposition, Eigendecomposition and Jordan Normal Form Decomposition

I recently created a related topic about the square root matrix, in case you'd like to refer to that one. Here's what we want: Consider the matrix $\Omega=E(\mathbf{u}^{\top}\mathbf{u})$, where ...
1
vote
2answers
707 views

Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

I know in general that if a matrix $A$ is positive definite, then there exists a (unique?) square root matrix $B$, which is also positive definite, such that $BB=A$. Therefore, suppose $A$ is ...
1
vote
2answers
159 views

Plotting an Ellipse after an Ellipse Fit

I wonder if someone can assist my understanding as I'm a bit stumped with this... I have taken the following (x,y) data which lies roughly on an ellipse: $$ \begin{pmatrix} 0.000234491 & 6855810 ...
1
vote
3answers
973 views

Find parameters for exponential function fitting to datapoints

I have a set of datapoints, in this case the temperature of an object adjusting to the environment temperature over time. Because I know these kind of processes take the form of $$f(x)=Ae^{x/B}+C$$ I ...
1
vote
3answers
1k views

Finding the change point in data from a piecewise linear function

Greetings, I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help! I have data conforming to a piecewise linear ...
0
votes
1answer
53 views

How do I solve for vector $P$ in the matrix equation $s=A'B^{-1}A$?

I would like to rearrange the matrix equation $s=A'B^{-1}A$ into the form $A=f(s,B)$ (i.e., some function of $s$ and $B$), where s is scalar, $A$ is $n\times 1$, $A'$ is the transpose of $A$, and $B$ ...
0
votes
1answer
60 views

Advice to solve a system of 8th order univariate polynomials

I am struggling to solve a least square problem in which the tedious part is the initialization. Grid search methods are out of question. The initial problem I've stated my problem in a previous ...