Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.
1
vote
2answers
2k views
Find equation of curve fit programmatically in Matlab?
In MATLAB, when you plot something, there's a tool available which is called "curve fitting". And if you have a set of data points and a linear correlation, this tool will easily come up with an ...
1
vote
1answer
60 views
Proof that a and b in linear regression are random variables
Does anyone know how to prove that the variables $a$ and $b$ that are used in linear regression are random variables? For me the assumption would be that these are dependent on the values of $x$ and ...
1
vote
2answers
80 views
Fitting exponential curve to data
If I have a collection of data points that follow an exponential curve relationship, how can I manually construct the equation that defines the best-fit exponential curve for the data?
1
vote
2answers
57 views
Linear Regression to quadratic function
What is the optimal linear regression (w and w/o y-intercept) for a quadratic curve w.r.t. mean square error.
Mathematically speaking:
Given,
$$y = x^2$$
for
$$x = [-a,a]$$.
What is the best ...
1
vote
1answer
125 views
What is the difference between a polynomial regression and a generalized linear model?
I have seen that a polynomial linear regression can have this form:
$y = c_0 + c_1 x_1 + c_2 x_2 + \dots + c_k x_k $
but I have read that the general lineal model which is a form of the multiple ...
1
vote
1answer
65 views
$(X^tX)^{-1}$ when $p>>n$
For the $n\times p$ matrix $\mathbf{X}$, is there any use in approximating $(\mathbf{X}'\mathbf{X})^{-1}$ when $p>>n$? If so, what information might this tell us?
I understand when $p<n$, ...
1
vote
3answers
98 views
Middle line and parabola between points
How it is simplest equation for this
a)LINE that most fit the points T1(1,0) T2(2,0) T3(-1,1) T4(0,1)
x= (1,2,-1,0)
y= (0, 0, 1, 1)
1= (1, 1, 1, 1)
Solving this: you got sistem of 2 equations: ...
1
vote
1answer
87 views
Calculating the regression equations
I have four data points $(1,2), (2,4), (3,5), (5,7)$ and Im looking for the least squares regression line that best fits them.
I use the normal equation
$A^tAx=A^tb$
in this form -
...
1
vote
1answer
271 views
Construct / find the simplest function based on data
Let's say I have these 7 natural numbers (all between 0 and 255):
255, 23, 45, 32, 87, 52, 146
How can I find a function F(x) that, once computed, gives me back ...
1
vote
1answer
3k views
How to find curve equation from data?
How do I find the formula when I only know some data points ?
Usually I just use the Trendline option for diagrams in Excel, but this one eludes me.
I expect it to be something like : ...
1
vote
1answer
405 views
Simple non linear fitting question(Least Squares Fitting--Exponential) [duplicate]
Possible Duplicate:
easy to implement method to fit a power function (regression)
I have the following simple function:
$h = cV^n$
h and V being the variables and $c$ and $n$ are ...
1
vote
1answer
130 views
problem constructing an equation that connects variables
need a suggestion/advice/inputs from the mathematicians here.
I have collected data of number of instructions/sec executed by a processor every second, number of loads/stores performed every second ...
1
vote
1answer
36 views
How to gather useful information from a residue plot
You can usually see how good your linear regression line is by looking at the residue plot. If you see the points randomly distributed, you're good. But if you see a pattern, it means there is ...
1
vote
1answer
44 views
What is the way to determin how good a sequence will interpolate?
Say I have to sequences of numbers:
$$[5, 10, 14, 21, 27, 31]$$
$$[1, 20, 21, 22, 30, 31]$$
Even though they both get to $31$ by the $6$th element, logic tells me that only the first one is a good ...
1
vote
1answer
53 views
QR factorization for ridge regression
I am solving an overdetermined system of equations:
$$Ax= b$$
Using QR factorization, we can solve this system easily by posing it as:
$$Rx= Q'b$$
I would like to regularize my estimate of $x$. I ...
1
vote
2answers
62 views
What is the equation that fits this curve?
I have a curve that looks like this (it's cyclical):
Curve
I can get a partial fit by fitting a 3rd degree polynomial, but I have a feeling there must be a better fit (something that involves sin ...
1
vote
1answer
118 views
A couple of questions on the NURBS basis functions
I read a little about NURBS curves (specifically from http://www.cs.mtu.edu/~shene/COURSES/cs3621/NOTES/), and I have a couple of questions about the motivation behind the choices made in designing ...
1
vote
2answers
75 views
Least Squares Derivation
I was reading this to review the derivation of the ordinary least squares estimator but I'm having trouble differentiating (4). Can someone please help explain why
$ \dfrac{\partial ...
1
vote
2answers
349 views
Exponential Regression Model
I need to model my data ($(x,y)$ pairs) using the following exponential function:
$$f(x) = \exp((x + a)/b) - c$$
So, I need to find $a, b, c$ coefficients that are the best fit for my data. What is ...
1
vote
1answer
130 views
Approximation using Legendre polynomials
my aim is to fit data points by the use of Legendre Polynomials. Has anybody experience with this task? My final aim is to do this automatically with mathematica.
Thanks,
rainer
1
vote
2answers
116 views
How to do a regression with only integer values and a fixed intercept?
I need to write some code for an application that takes in a series of 2D points whose values are integers, and determines a polynomial regression that passes through the origin. I know how to do this ...
1
vote
1answer
64 views
Orthogonal fitted values
I have two regression models
$$Y=X\beta+\varepsilon,\quad \beta\in\mathbb{R}^k$$
$$Y=Z\alpha+u\quad \alpha\in\mathbb{R}^m$$ it is known that using OLS estimates $\hat{\beta},\hat{\alpha}$ fitted ...
1
vote
2answers
148 views
vertical vs. horizontal regression
A horizontal regression is defined as the following:
$$m=\frac{\sum_{i=1}^n (x_i-\operatorname{average(x)})(y_i-\operatorname{average(y))}}{\sum_{i=1}^n (x_i-\operatorname{average(x)})^2}$$
whereas ...
1
vote
2answers
162 views
Log-likelihood gradient and Hessian
Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions:
$f(x) = x^T \beta$
$p(x) = ...
1
vote
1answer
87 views
Econometrics OLS estimates
I cant seem to use the formula to calculate B1 without knowing xi and yi. Is it possible to calculate using just the variances and covariance? Please help!
The classical linear regression model
...
1
vote
1answer
37 views
Getting the formula of a live counter
I'm looking to replicate this greenhouse gases counter in my website. Poking around i found the initial data for the formula.
The counter use the following information:
Beginnig date: 2012/03/01
...
1
vote
2answers
174 views
Simple Least Squares Regression?
I have a vector X of 50 real numbers and a vector Y of 50 real numbers.
I want to model them as
y = ax + b
How do I determine a and b such that it minimizes the ...
1
vote
2answers
240 views
Residuals of regression model
Let's suppose I do a regression between earnings and age (and suppose I do not know the distribution of earnings). Would it be possible for the residuals to be normally distributed?
I am thinking it ...
1
vote
4answers
109 views
How to fit a curve to my data
I have a datasheet. It looks like an hyperbola. How can I fit a curve to it? And how can I plot a curve of the first derivative?
...
1
vote
1answer
146 views
Curve fitting with upper and lower bounds for derivatives
I compute (at a great cost) upper and lower bounds $f_u(x)$ and $f_l(x)$ of an unknown function $f(x)$ at points $x$ in $[0,1]$. Now I am interested in an estimation of the derivative $f'(x)$. I ...
1
vote
1answer
670 views
help with using the “simple regression (least squares) method” of forecasting
This problem is from an engineering management textbook (Morse & Babcock, 5th ed) :
2005 $48k
2006 $64k
2007 $67k
2008 $83k
"What is the ...
1
vote
1answer
160 views
bayesian networks for regression
Would it be possible to use bayesian network for regression and/or prediction? I understand that it is a tool one can use to compute probabilities, but I haven't found much material about possible ...
1
vote
1answer
254 views
Recursive coefficient of determination (R2)
Is there a way to compute the coefficient of determination $R^2$ in a recursive way?
$R^2$ is defined as following:
$$R^2 \equiv 1 - \frac{SS_{\rm err} }{ SS_{\rm tot}} = 1 -
\frac{\sum_i (y_i - ...
1
vote
1answer
104 views
can an artificial neural network with only one hidden layer fit all purposes/applications/functions?
I have heard that only a single layer is needed for an ANN to fit any possible function (input to output). Is this true and where is this investigated/state/found? Then what is the advantage of having ...
1
vote
2answers
309 views
Fitting data to a portion of an ellipse or conic section
Is there a straightforward algorithm for fitting data to an ellipse or other conic section? The data generally only approximately fits a portion of the ellipse. I am looking for something that doesn't ...
1
vote
0answers
60 views
How to calculate probability with sigmoid output in feedforward neural network?
first of all I'm sorry for my not very skilled English, but I will do my best to explain my problem.
I'm trying to create a feedforward neural network with one hidden layer (with probably arctan ...
1
vote
0answers
20 views
regression coefficient
Consider observations on three variables X1;X2 and X3: Suppose that
X1 is regressed on X2: When the residual of the above regression is
regressed on X3; the regression coefficient of X3 is b3: When X1 ...
1
vote
0answers
21 views
Nesterovs third method - implementation in python [migrated]
I am looking at implementing Nesterov's method for my algorithm being written in python. Can anyone please point me to docs which can help me get started in terms of implementation of this method? I ...
1
vote
1answer
13 views
Initializing Variables using Shrinkage
I have a user-user model which which users can rate their friendships(r) with others and also can have activities with them(a). I am using Matrix Factorization and Gradient Descent for updating the ...
1
vote
0answers
25 views
About the weights assigned in the linear regression
I have this confusion related to linear regression. Lets say I have two predictors $x_1$ and $x_2$ and the target is $y$. I learn a linear regression with $y \sim x_1,x_1 \cdot x_2,x_2$ with $x_1 ...
1
vote
0answers
31 views
Sequential problem for n=1, non linear regression
I am trying to understand an example in my stats course notes, the example relates to calculating the best value for the next experiment.
The function of the line is very simple:
$$ln(Y_i) = ...
1
vote
0answers
39 views
Correlation coefficient.
A linear regression gives us a correlation coefficient $r=0$.
What is the equation of the best fit line?
Give an example of data with $r=0$
What is the value of the correlation coefficient of data ...
1
vote
1answer
20 views
How to interpret these regression values?
If GPA(gpa after fall semester in college) is the dependent variable and SAT (score on the SAT) is the independent variable and I have the following parameter estimates:
Intercept: .66306
SAT: ...
1
vote
0answers
20 views
Coefficient of determination
$$ \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 = ( \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 - \displaystyle \sum^n_{i = 1} (y_i - \hat{y}_i)^2 ) + \displaystyle \sum^n_{i = 1} (y_i - ...
1
vote
2answers
74 views
Linear regression question
I don't understand the following derivation:
$$ e_i = y_i - ax_i - b$$
$$ e_i = (y_i - \bar{y}) - a(x_i - \bar{x}) - (b - \bar{y} + a \bar{x}) $$
I don't really understand what they do and why they ...
1
vote
1answer
32 views
Cointegration for Price levels Time Series
I don't understand why is the difference between price levels is a stationary process while the time series of price levels themselves is a non-stationary process.
For example:
...
1
vote
0answers
38 views
Show that $Y$ has constant regression with respect to $X$ and/but that $X$ and $Y$ are not independent.
The joint density of $X$ and $Y$ is given by
$$ f(x,y) = \left\{
\begin{array}{l l}
1/2 & \quad \text{$|x|+|y|\leq 1$}\\
0 & \quad \text{otherwise}
\end{array} \right.$$
Show ...
1
vote
0answers
53 views
Formula for confidence interval in multi-variable regression
What is the formula for calculating the confidence interval for the expected value of $\hat{y}$ in a multi-variable regression model.
In other words, I'm looking for the following formula just for ...
1
vote
1answer
73 views
Question about logistic regression
A logistic regression is meant for a binary/categorical variable. Sort of like age vs baldness.
1) So, does the "S-curve" regression equation output give the odds of having that condition for a ...
1
vote
1answer
66 views
Linear regression: b1 has the minimum variance among all unbiased linear estimators of beta1
There is a proof provided in Applied Linear Regression Models (1983) by Kutner et al. (Page 64), which is quite clear and easy to understand, except one point, namely, it assumes that $\sum k_i d_i = ...
