Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

3
votes
1answer
137 views

Bound of linear regression's object function

Randomly uniformly select $n$ numbers from a set $\{1,2,...,U\}$ with/without replacement, $y_i$ is the $i$th number selected, and $x_i$ is the rank of $y_i$ in the $n$ numbers. The rank is the order ...
30
votes
8answers
6k views

Why do we use a Least Squares fit?

I've been wondering for a while now if there's any deep mathematical or statistical significance to finding the line that minimizes the square of the errors between the line and the data points. If ...
1
vote
1answer
578 views

degrees of freedom for linear regression

If you have a reduced model with $H_0:\beta_1 = 1$, $H_a: \beta_1 \neq 1$, then the reduced model is: $$Y_i = 1X_i + \beta_0 + \epsilon_i$$ Are the degrees of freedom for the error term SSE $n-1$?
0
votes
1answer
139 views

Getting linear regression of huge numbers

I'm trying to get a linear regression slope and intercept for a large set of huge numbers. I'm doing this on a computer, but I keep getting overflow errors (attempting to calculate a number too large ...
11
votes
2answers
4k views

Finding the intersection point of many lines in 3D (point closest to all lines)

I have many lines (let's say 4) which are supposed to be intersected. (Please consider lines are represented as a pair of points). I want to find the point in space which minimizes the sum of the ...
4
votes
2answers
1k views

Fitting an exponential function to data

I have a noisy data set (the grey line in the graph below) that corresponds roughly to $y=m(1-2^{-x/k})$ where m and k are unknown constants. How can I determine the best-fit value of m and k? I ...
10
votes
4answers
2k views

Polynomial fitting where polynomial must be monotonically increasing

Given a set of monotonically increasing data points (in 2D), I want to fit a polynomial to the data which is monotonically increasing over the domain of the data. If the highest x value is 100, I ...
0
votes
1answer
96 views

Is this expression correct?

sorry I don't know how to post Latex here I have to implement in Matlab the following formula and I'd like to know if the expression given here really corresponds to the minimum indicated, and why: ...
0
votes
2answers
196 views

General nonlinear least squares?

I'm looking to solve some kind of generalized nonlinear least squares problem, I think. So for some background, lets say I have an ordinary nonlinear least squares problem. That is, a set of data ...
2
votes
0answers
270 views

Surface Function Fitting to Spherical Data

I have a set of geographic (longitude,latitude,value) data to which I would like to fit surface functions, specifically, the set of quadratic surfaces: $f(x,y)=Ax^2+Bx^2+Cxy+Dx+Ey+F$ At the moment, ...
1
vote
0answers
258 views

Point-wise error estimate in polynomial regression

In our application we wish to estimate the actual path of objects. We have a set of samples of object locations $(t_i, x_i, y_i, P_i)$ where $t_i$ is the sample time, $(x_i, y_i)$ is the 2D location, ...
2
votes
2answers
140 views

Choosing set of best estimators for linear least squares

I have a measured experimental dataset which is well approximated by the sum of several basis functions in linear combinations. Linear least squares of course gives me the optimal weight for each ...
2
votes
0answers
67 views

Accurate computation for Linear Regression case

I am writing a program that inputs a sequence of points $(x_i,y_i)$ based on the user clicking on certain pixels in an image shown. The program should then find the "best -fitting" line in the least ...
5
votes
2answers
3k views

Linear regression for minimizing the maximum of the residuals

We know that simple linear regression will do the following thing: Suppose there are $n$ data points $\{y_i,x_i\}$, where $i=1,2,\dots,n$. The goal is to find the equation of the straight line ...
4
votes
1answer
216 views

Who invented linearization of exponential datasets to find their approximating functions?

I just learned how to find the exponential function that approximates a dataset by taking the logarithm of the data points, doing a linear regression on that data, then working out the exponential ...
1
vote
1answer
360 views

bayesian networks for regression

Would it be possible to use bayesian network for regression and/or prediction? I understand that it is a tool one can use to compute probabilities, but I haven't found much material about possible ...
1
vote
1answer
54 views

Is there a name for this general problem (variation on least squares)?

The problem statement is as follows. Minimize $||g(X\beta)-y||^2$ with respect to $\beta$ where $g(\cdot)$ is some non-linear function, $y$ and $\beta$ are column vectors. General linear least ...
2
votes
2answers
430 views

Calculate Line Of Best Fit Using Exponential Weighting?

I know how to calculate a line of best fit with a set of data. I want to be able to exponentially weight the data that is more recent so that the more recent data has a greater effect on the line. ...
1
vote
1answer
337 views

Recursive coefficient of determination (R2)

Is there a way to compute the coefficient of determination $R^2$ in a recursive way? $R^2$ is defined as following: $$R^2 \equiv 1 - \frac{SS_{\rm err} }{ SS_{\rm tot}} = 1 - \frac{\sum_i (y_i - ...
0
votes
2answers
65 views

Greater precision with regression?

I used my TI-83 to find the quadratic regression of two data columns. The accuracy wasn't close at all. So I tried cubic and then finally quartic regression. The accuracy still isn't close enough. Is ...
1
vote
0answers
91 views

Variability of a curve with Vapnik-Chervonenkis dimension 4

Say I have 10,000 data in 2-D and I want to fit a curve to them. There are many functional forms this curve could take -- polynomial, B-spline, trigonometric, and so on. I've decided that I only want ...
1
vote
1answer
147 views

can an artificial neural network with only one hidden layer fit all purposes/applications/functions?

I have heard that only a single layer is needed for an ANN to fit any possible function (input to output). Is this true and where is this investigated/state/found? Then what is the advantage of having ...
0
votes
1answer
3k views

How different is Beta computation using Covariance and Linear Regression?

I wanted to compute Beta for a Stock against an Index (Say Stock X against S&P 500). I computed the daily returns for over one year applied the following logic : Beta = COVAR(X, S&P ...
2
votes
1answer
3k views

How to calculate hyperbola from data points?

I have 4 data points, from which I want to calculate a hyperbola. It seems that the Excel trendline feature can't do it for me, so how do I find the relationship? The points are: (x,y) (3, 0.008) ...
7
votes
1answer
536 views

Linear regression is wrong?

I tried asking this in electronics as a question related to oscillators, but I wasn't able to get a satisfactory answer. I think the more math-y types here may shed some additional light on the ...
1
vote
0answers
46 views

Need practical help with a calculation

I'm sorry for a really basic question. I lack proper background in mathematics, but I have to calculate a list of values. I'm given a vector (list) of observations, and $\hat{Y}$, which is a list of ...
5
votes
2answers
7k views

How to fit a curve to a sinusoidal wave

I am wondering how to fit a sinusoidal wave (approximation). I would like to fit it in the form: $y = A\sin(Bx + C) + D$ where $A,\,B,\,C$ and $D$ are constants. The only constants I really care about ...
1
vote
1answer
134 views

problem constructing an equation that connects variables

need a suggestion/advice/inputs from the mathematicians here. I have collected data of number of instructions/sec executed by a processor every second, number of loads/stores performed every second ...
1
vote
0answers
48 views

Finding a confidence interval for a parameter of regression given two variable variances

Given an equation $Y_i=0.1+0.3x_i+e_i$, I am asked to calculate a 95% confidence interval for $Y_i$ when $x=0$. So I have an equation $Y_i=0.1+e_i$, and I know that standard error of 0.1 is 0.005, ...
0
votes
1answer
564 views

How do I fit a model with piecewise linear regression

I have a set of points in 3D (x,y,z). I ordered these points from the lowest to highest. So, I want to used linear regression to fit a line through these ordered points and then to find out a break ...
2
votes
1answer
664 views

Linear Regression with 3x3 Matrices

Here's my Homework Problem: We can generalize the least squares method to other polynomial curves. To find the quadratic equation $y=a x^2+b x+c$ that best fits the points $(-1, −3)$, $(0, 0)$, ...
0
votes
1answer
239 views

If the covariance of A, B if E(A) = 0, does convariance of A, B = 0?

So I have got 2 variables, $A$ and $B$. I know for a fact that $E(A)$ = 0. I don't know if they are independent. $$Cov(A, B) = E(A B) - E(B) E(A)$$ I know that $E(A) = 0$. Does that mean $E(A B) ...
6
votes
1answer
256 views

Can I build a program that will tell me if a real world data set looks linear, logarithmic, exponential etc?

I have a bunch of real world data sets and from manually plotting some of the data in graphs, I've discovered some data sets look pretty much logarithmic and some look linear, or exponential (and some ...
2
votes
1answer
259 views

How do I do a least squares fit of $a x + b y = 1$?

How do I do a least squares fit of the line equation $a x + b y = 1$, so that the points are as close to the line as possible? (Not just vertically close) If I use the matrices $$X = ...
2
votes
1answer
193 views

How to handle constant term in Least Squares Regression?

In the well known matrix form of a least squares regression where I am trying to solve for B in Y = B1X1 + B2X2 + B3 I might be given X and Y sample data as something like $X$ = $\begin{bmatrix} ...
2
votes
2answers
715 views

Fitting data to a portion of an ellipse or conic section

Is there a straightforward algorithm for fitting data to an ellipse or other conic section? The data generally only approximately fits a portion of the ellipse. I am looking for something that doesn't ...
0
votes
1answer
211 views

dependency between regression coefficients and probability distribution

let's consider regression problem. Given set of training data $\{(x_i,y_i)\}_{i=1}^N$, $x_i \in \mathbb{R}^n$ and $y_i \in \mathbb{R}$, find prediction function $y = f(x)$, e.g. in RBF regression case ...
0
votes
2answers
620 views

regression (using 3D points cloud dataset)

I have a dataset of trajectories. These trajectories are represented in 3D space (x,y,z). All trajectories of this dataset are similar in their shape, but they are not exactly the same, I mean, there ...
4
votes
4answers
337 views

How to model prices?

This is my first question here. As I'm not a matematician I thought I'd ask here for advice how to approach something I'm working on as a hobby project. A bit of context Let's say there is a ...
1
vote
3answers
970 views

Finding the change point in data from a piecewise linear function

Greetings, I'm performing research that will help determine the size of observed space and the time elapsed since the big bang. Hopefully you can help! I have data conforming to a piecewise linear ...
6
votes
2answers
318 views

Theoretical basis for overfitting

There are many examples in which making more "precise" predictions gives worse performance (e.g. Runge's phenomenon). My professor implied that there was a sound basis for choosing "simple" functions ...
12
votes
3answers
2k views

Best fit ellipsoid

Given a collection of points $P \subset \mathbb R^3$, a crude characterization of the "shape" of $P$ is sometimes given by the principal components. We construct a covariance matrix, e.g., if $P$ is ...
0
votes
1answer
63 views

Estimating the equation of a line from its rounded values

So, I'm playing a game where there is a certain percent value that changes over time. I look at it occasionally, it seems to be approximately linear in time. However, the change is slow enough that ...
5
votes
2answers
248 views

Recommended Reading on Regression Analysis?

For a university project, I am implementing an automated regression analysis tool. However, I have very little background in statistics. So what books / articles / material would you suggest I could ...
1
vote
1answer
9k views

Find equation of curve fit programmatically in Matlab?

In MATLAB, when you plot something, there's a tool available which is called "curve fitting". And if you have a set of data points and a linear correlation, this tool will easily come up with an ...
2
votes
1answer
639 views

Polynomial fitting - how to fit and what is _polynomial fitting_

I don't understand what is polynomial fitting. Can anyone explain me how to fit a curve to given points?
8
votes
1answer
2k views

easy to implement method to fit a power function (regression)

I want to fit to a dataset a power function ($y=Ax^B$). What is the best and easiest method to do this. I need the $A$ and $B$ parameters too. I'm using in general financial data in my project, which ...
6
votes
3answers
368 views

Are there variations on least-squares approximations?

In least-squares approximations the normal equations act to project a vector existing in N-dimensional space onto a lower dimensional space, where our problem actually lies, thus providing the "best" ...