Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
3answers
741 views

Find parameters for exponential function fitting to datapoints

I have a set of datapoints, in this case the temperature of an object adjusting to the environment temperature over time. Because I know these kind of processes take the form of $$f(x)=Ae^{x/B}+C$$ I ...
3
votes
3answers
600 views

Correlation between variables

I asked this question on stats SE but did not find a suitable answer so far. Maybe someone can help. Given n random variables x1,...,xn (one-dimensional). The following is known (corr() = Pearson ...
2
votes
0answers
85 views

Polynomial and exponential regression [duplicate]

Possible Duplicate: Determining computational complexity of stochastic processes I have some points $(x_i,y_i)$ generated by a program. These values are not exact, but are random ...
2
votes
3answers
378 views

Residuals of regression model

Let's suppose I do a regression between earnings and age (and suppose I do not know the distribution of earnings). Would it be possible for the residuals to be normally distributed? I am thinking it ...
0
votes
1answer
388 views

Strange behaviour with ode45 in matlab, probably some numerical error.

I'm having a problem with a parameter estimation in a non-linear model. I think the culprit is that ode45 (an ode solver in matlab) is not properly solving my ode. It's the in red highlighted part, ...
1
vote
0answers
407 views

Moore–Penrose pseudoinverse reference

Given the eigendecompositions $AA^{\top}=Q \Lambda Q^{\top}$ and $A^{\top}A=P \Lambda P^{\top}$, where $\Lambda$ is a diagonal matrix (of eigenvalues) and $P$ and $Q$ are unitary eigenvectors matrices ...
1
vote
4answers
240 views

How to fit a curve to my data

I have a datasheet. It looks like an hyperbola. How can I fit a curve to it? And how can I plot a curve of the first derivative? ...
3
votes
0answers
299 views

Least Square Method with Positive Parameters

this is my first post here in the Stack Exchange. A friend told me about this forum and I'm giving it a try. I searched a bit past threads, but couldn't find what I wanted, so I'm posting the problem ...
0
votes
1answer
138 views

Explain Least-squares plotting with Ones -matrix

I am stuck to this code and particularly the point F=[ones(size(x)) x x.^2]. What does it do mathematically? I cannot understand what the heck the matrix has to do ...
0
votes
1answer
119 views

Linear algebra with a linear model (Matlab)

Given the equation $$r = B + e(r\cos(\theta))$$ and the corresponding data: $\theta: 0.88; 1.1; 1.42; 1.77; 2.14$ and $r: 3; 2.4; 1.65; 1.25; 1.01$ How do you input these data for matlab to ...
3
votes
0answers
998 views

Correlation and Regression Question

Two separate tests are designed to measure a students ability to solve problems. Several students are randomly selected to take both tests and the results are: $$ \begin{matrix} \text{Test A}(x) ...
3
votes
2answers
565 views

Fitting a sine function to data

I have a sequence of $n$ points $(x_i,y_i)$, for $i=1,\dots,n$. I would like to find the function, of the form $y=V\sin(x+\phi)$, which best fits the points. Which numerical method could I use? I have ...
2
votes
1answer
420 views

Curve fitting with upper and lower bounds for derivatives

I compute (at a great cost) upper and lower bounds $f_u(x)$ and $f_l(x)$ of an unknown function $f(x)$ at points $x$ in $[0,1]$. Now I am interested in an estimation of the derivative $f'(x)$. I ...
9
votes
1answer
6k views

Computational complexity of least square regression operation

In a least square regression algorithm, I have to do the following operations to compute regression coefficients: Matrix multiplication, complexity: $O(C^2N)$ Matrix inversion, complexity: ...
1
vote
3answers
601 views

Simple Least-Squares Regression Question

Given a set of 5 points (i.e. (1, 3), (2, 8) etc...), how can I get just the slope of the best fit line? I've been looking up least squares regression, but I'm rather statistics ignorant and don't ...
1
vote
1answer
890 views

Simple non linear fitting question(Least Squares Fitting--Exponential) [duplicate]

Possible Duplicate: easy to implement method to fit a power function (regression) I have the following simple function: $h = cV^n$ h and V being the variables and $c$ and $n$ are ...
1
vote
0answers
888 views

Best-fitting plane

I need to implement the algorithm described below. Everything is fine until the eigenvalues computation. I'm completely new to them and I found a lot of very complicated paper on the net. Is it ...
3
votes
2answers
559 views

Confidence interval of a random variable for an ordinary linear regression

I have a small problem. With my limited stats background I am not sure I am getting this one right. After fitting an ordinary linear regression model I get ...
3
votes
1answer
179 views

Ridge Regression: $\hat{\beta} \rightarrow \beta$

I'm trying to find the probability limit of $$\hat{\beta} = \left( \sum x_i x'_i + \lambda I_k \right)^{-1} \left( \sum x_i y_i \right) $$ as $n \to \infty$, and $\lambda$ is some positive ...
8
votes
2answers
760 views

Why is polynomial regression considered a kind of linear regression?

Why is polynomial regression considered a kind of linear regression? This is what I mean by polynomial regression. For example, the hypothesis function is $$h(x; t_0, t_1, t_2) = t_0 + t_1 x + t_2 ...
1
vote
1answer
89 views

Measuring how monotonically “staircase-like” a set of values is

A bit of a bizarre question here -- I'm looking for assistance in generating a robust metric to measure how monotonically "step-wise" a series of values is. The set must not start or end at a specific ...
6
votes
1answer
242 views

How to find line that has least distance to all points?

I need to find the line having minimal distance to all points. I found linear regression and linear interpolation algorithms. But their minimal distance is only in y-axis: $D = y - f(x)$. But I need ...
1
vote
1answer
2k views

help with using the “simple regression (least squares) method” of forecasting

This problem is from an engineering management textbook (Morse & Babcock, 5th ed) : 2005 $48k 2006 $64k 2007 $67k 2008 $83k "What is the ...
2
votes
2answers
208 views

How to fit an equation to a curve with disturbances

For example, I have the following data: Y = 366 measured values X = 366 measured values t = [ 1 : 366 ], representing the days of the year (index) So at each t (day), we have value of Y and ...
2
votes
0answers
3k views

Help with problem: Estimated Standard Deviation of Regression Equation (Simple Linear)

This is a practice problem. I've solved part (a). I have provided verified answers (from the published key) to all parts (a), (b) and & (c). I need help solving (b) and (c). Consider a simple ...
3
votes
1answer
137 views

Bound of linear regression's object function

Randomly uniformly select $n$ numbers from a set $\{1,2,...,U\}$ with/without replacement, $y_i$ is the $i$th number selected, and $x_i$ is the rank of $y_i$ in the $n$ numbers. The rank is the order ...
30
votes
8answers
7k views

Why do we use a Least Squares fit?

I've been wondering for a while now if there's any deep mathematical or statistical significance to finding the line that minimizes the square of the errors between the line and the data points. If ...
1
vote
1answer
583 views

degrees of freedom for linear regression

If you have a reduced model with $H_0:\beta_1 = 1$, $H_a: \beta_1 \neq 1$, then the reduced model is: $$Y_i = 1X_i + \beta_0 + \epsilon_i$$ Are the degrees of freedom for the error term SSE $n-1$?
0
votes
1answer
142 views

Getting linear regression of huge numbers

I'm trying to get a linear regression slope and intercept for a large set of huge numbers. I'm doing this on a computer, but I keep getting overflow errors (attempting to calculate a number too large ...
11
votes
2answers
4k views

Finding the intersection point of many lines in 3D (point closest to all lines)

I have many lines (let's say 4) which are supposed to be intersected. (Please consider lines are represented as a pair of points). I want to find the point in space which minimizes the sum of the ...
4
votes
2answers
1k views

Fitting an exponential function to data

I have a noisy data set (the grey line in the graph below) that corresponds roughly to $y=m(1-2^{-x/k})$ where m and k are unknown constants. How can I determine the best-fit value of m and k? I ...
10
votes
4answers
3k views

Polynomial fitting where polynomial must be monotonically increasing

Given a set of monotonically increasing data points (in 2D), I want to fit a polynomial to the data which is monotonically increasing over the domain of the data. If the highest x value is 100, I ...
0
votes
1answer
97 views

Is this expression correct?

sorry I don't know how to post Latex here I have to implement in Matlab the following formula and I'd like to know if the expression given here really corresponds to the minimum indicated, and why: ...
0
votes
2answers
198 views

General nonlinear least squares?

I'm looking to solve some kind of generalized nonlinear least squares problem, I think. So for some background, lets say I have an ordinary nonlinear least squares problem. That is, a set of data ...
2
votes
0answers
272 views

Surface Function Fitting to Spherical Data

I have a set of geographic (longitude,latitude,value) data to which I would like to fit surface functions, specifically, the set of quadratic surfaces: $f(x,y)=Ax^2+Bx^2+Cxy+Dx+Ey+F$ At the moment, ...
1
vote
0answers
265 views

Point-wise error estimate in polynomial regression

In our application we wish to estimate the actual path of objects. We have a set of samples of object locations $(t_i, x_i, y_i, P_i)$ where $t_i$ is the sample time, $(x_i, y_i)$ is the 2D location, ...
2
votes
2answers
140 views

Choosing set of best estimators for linear least squares

I have a measured experimental dataset which is well approximated by the sum of several basis functions in linear combinations. Linear least squares of course gives me the optimal weight for each ...
2
votes
0answers
67 views

Accurate computation for Linear Regression case

I am writing a program that inputs a sequence of points $(x_i,y_i)$ based on the user clicking on certain pixels in an image shown. The program should then find the "best -fitting" line in the least ...
5
votes
2answers
3k views

Linear regression for minimizing the maximum of the residuals

We know that simple linear regression will do the following thing: Suppose there are $n$ data points $\{y_i,x_i\}$, where $i=1,2,\dots,n$. The goal is to find the equation of the straight line ...
4
votes
1answer
224 views

Who invented linearization of exponential datasets to find their approximating functions?

I just learned how to find the exponential function that approximates a dataset by taking the logarithm of the data points, doing a linear regression on that data, then working out the exponential ...
1
vote
1answer
367 views

bayesian networks for regression

Would it be possible to use bayesian network for regression and/or prediction? I understand that it is a tool one can use to compute probabilities, but I haven't found much material about possible ...
1
vote
1answer
54 views

Is there a name for this general problem (variation on least squares)?

The problem statement is as follows. Minimize $||g(X\beta)-y||^2$ with respect to $\beta$ where $g(\cdot)$ is some non-linear function, $y$ and $\beta$ are column vectors. General linear least ...
2
votes
2answers
435 views

Calculate Line Of Best Fit Using Exponential Weighting?

I know how to calculate a line of best fit with a set of data. I want to be able to exponentially weight the data that is more recent so that the more recent data has a greater effect on the line. ...
1
vote
1answer
339 views

Recursive coefficient of determination (R2)

Is there a way to compute the coefficient of determination $R^2$ in a recursive way? $R^2$ is defined as following: $$R^2 \equiv 1 - \frac{SS_{\rm err} }{ SS_{\rm tot}} = 1 - \frac{\sum_i (y_i - ...
0
votes
2answers
65 views

Greater precision with regression?

I used my TI-83 to find the quadratic regression of two data columns. The accuracy wasn't close at all. So I tried cubic and then finally quartic regression. The accuracy still isn't close enough. Is ...
1
vote
0answers
91 views

Variability of a curve with Vapnik-Chervonenkis dimension 4

Say I have 10,000 data in 2-D and I want to fit a curve to them. There are many functional forms this curve could take -- polynomial, B-spline, trigonometric, and so on. I've decided that I only want ...
1
vote
1answer
149 views

can an artificial neural network with only one hidden layer fit all purposes/applications/functions?

I have heard that only a single layer is needed for an ANN to fit any possible function (input to output). Is this true and where is this investigated/state/found? Then what is the advantage of having ...
0
votes
1answer
3k views

How different is Beta computation using Covariance and Linear Regression?

I wanted to compute Beta for a Stock against an Index (Say Stock X against S&P 500). I computed the daily returns for over one year applied the following logic : Beta = COVAR(X, S&P ...
2
votes
1answer
3k views

How to calculate hyperbola from data points?

I have 4 data points, from which I want to calculate a hyperbola. It seems that the Excel trendline feature can't do it for me, so how do I find the relationship? The points are: (x,y) (3, 0.008) ...
7
votes
1answer
550 views

Linear regression is wrong?

I tried asking this in electronics as a question related to oscillators, but I wasn't able to get a satisfactory answer. I think the more math-y types here may shed some additional light on the ...