Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
55 views

Variance of Coefficients in a Simple Linear Regression

I have a linear regression model $\hat{y_i}=\hat{\beta_0}+\hat{\beta_1}x_i+\hat{\epsilon_i}$, where $\hat{\beta_0}$ and $\hat{\beta_1}$ are normally distributed unbiased estimators, and ...
1
vote
1answer
28 views

Regression towards the mean?

I am upset with what examples testing "regression to the mean" seem to allude to: People claiming to have "ESP" take a test, and A's score was 2 standard deviations above the mean, whereas B's score ...
1
vote
3answers
37 views

Different Regression Lines?

Hi quick question with regression. If the coefficients of a simple regression line, B0 and B1, are the same then why are the regression lines of y on x and x on y different given the condition r^2 ...
1
vote
1answer
35 views

Combine linear models of different sets of data.

I'm working on a large data set D that can be partitioned into some disjoint subsets D1, D2, ..., Dn. For each subset Di, I have a linear model Mi that minimizes the residual error for data in Di. ...
1
vote
1answer
29 views

Morphing $\beta_1$ into a different form (OLS question in Statistics)

I am currently studying Simple Linear Regression and I have successfully proven to myself how $\beta_1$ and $\beta_0$ are derived. However, I have been stuck on a seemingly simple problem (I'm sure ...
1
vote
2answers
45 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
1
vote
1answer
27 views

Unexplainable determination coëfficiënt

I have a series of data (more specific, they are coördinates of a package attached to the end of a mini level-luffing crane. The "flightpath" is linear and horizontal.) Now when I plot the data: ...
1
vote
1answer
673 views

Convert nonlinear regression equation to a linear regression equation

The question is: "Show how the nonlinear regression equation y=aX^B can be converted to a linear regression equation solvable by the method of least ...
1
vote
1answer
67 views

Weird formula for linear regression

I'll try to make the matter as clear as possible given the circumstances. My boss asked me to look at an old report a former employee wrote around a couple of months ago. Apparently the report ...
1
vote
1answer
125 views

Simple linear regression - understanding given

The question is to fill out the missing numbers (A-L) of a simple linear regression model. I am having problems with converting and interpreting the given table in terms of variables. Would it be ...
1
vote
1answer
71 views

Logistic regression with interactions

I'm supposed to do a model using logistic regression. So I have a series of $N$ observed data points each of which consists of $m$ explanatory variables $x = (x_{1,i}, ... , x_{m,i})$ and associated ...
1
vote
1answer
524 views

Mean response in linear regression

What does mean response in linear regession mean? I don't understand the definition given in wikipedia. This is the definition: Mean response is an estimate of the mean of the $y$ population ...
1
vote
1answer
152 views

Machine Learning, why not use matrix multiplication instead of gradient descent?

If we want to minimize our Cost function for a given set of data, why do we use gradient descent and continually guess values until we find a min value for theta when when can just use matrix ...
1
vote
1answer
161 views

How do I solve the weighted normal equations?

I am trying to solve the normal equations for a 3D LSE of a general quadric: $$ z = ax^2 + bx + cxy + dy^2 + ey + f$$ Write as a vector equation: $$ \vec{z}= \bf{X}\vec{\beta}$$ where the 'ith row ...
1
vote
1answer
205 views

Linear fit with horizontal and vertical error bars

I'm searching an equation to calculate the parameters for a linear fit. With parameters a and b, the $\chi ^{2}$ is used: $\chi ^{2} = \sum_{i=0}^N (y_{i}-a.x_{i}-b)^{2}$ And with errors: $\chi ...
1
vote
1answer
116 views

Simple Linear Regression Question

Let $Y_{i} = \beta_{0} + \beta_{1}X_{i} + \epsilon_{i}$ be a simple linear regression model with independent errors and iid normal distribution. If $X_{i}$ are fixed what is the distribution of ...
1
vote
1answer
708 views

Prediction Model for forecasting using Linear regression

I am very new to inferral statistics. I am trying to build a prediction model for forecasting the revenue for physicians based on some historical data. I was planning to use Multiple Linear Regression ...
1
vote
1answer
95 views

How to gather useful information from a residue plot

You can usually see how good your linear regression line is by looking at the residue plot. If you see the points randomly distributed, you're good. But if you see a pattern, it means there is ...
1
vote
1answer
50 views

What is the way to determin how good a sequence will interpolate?

Say I have to sequences of numbers: $$[5, 10, 14, 21, 27, 31]$$ $$[1, 20, 21, 22, 30, 31]$$ Even though they both get to $31$ by the $6$th element, logic tells me that only the first one is a good ...
1
vote
1answer
44 views

Cointegration for Price levels Time Series

I don't understand why is the difference between price levels is a stationary process while the time series of price levels themselves is a non-stationary process. For example: ...
1
vote
2answers
271 views

Question about logistic regression

A logistic regression is meant for a binary/categorical variable. Sort of like age vs baldness. 1) So, does the "S-curve" regression equation output give the odds of having that condition for a ...
1
vote
1answer
251 views

Linear regression: b1 has the minimum variance among all unbiased linear estimators of beta1

There is a proof provided in Applied Linear Regression Models (1983) by Kutner et al. (Page 64), which is quite clear and easy to understand, except one point, namely, it assumes that $\sum k_i d_i = ...
1
vote
1answer
264 views

QR factorization for ridge regression

I am solving an overdetermined system of equations: $$Ax= b$$ Using QR factorization, we can solve this system easily by posing it as: $$Rx= Q'b$$ I would like to regularize my estimate of $x$. I ...
1
vote
1answer
349 views

A couple of questions on the NURBS basis functions

I read a little about NURBS curves (specifically from http://www.cs.mtu.edu/~shene/COURSES/cs3621/NOTES/), and I have a couple of questions about the motivation behind the choices made in designing ...
1
vote
2answers
132 views

Least Squares Derivation

I was reading this to review the derivation of the ordinary least squares estimator but I'm having trouble differentiating (4). Can someone please help explain why $ \dfrac{\partial ...
1
vote
2answers
494 views

Exponential Regression Model

I need to model my data ($(x,y)$ pairs) using the following exponential function: $$f(x) = \exp((x + a)/b) - c$$ So, I need to find $a, b, c$ coefficients that are the best fit for my data. What is ...
1
vote
1answer
221 views

Approximation using Legendre polynomials

my aim is to fit data points by the use of Legendre Polynomials. Has anybody experience with this task? My final aim is to do this automatically with mathematica. Thanks, rainer
1
vote
2answers
423 views

How to do a regression with only integer values and a fixed intercept?

I need to write some code for an application that takes in a series of 2D points whose values are integers, and determines a polynomial regression that passes through the origin. I know how to do this ...
1
vote
1answer
94 views

Orthogonal fitted values

I have two regression models $$Y=X\beta+\varepsilon,\quad \beta\in\mathbb{R}^k$$ $$Y=Z\alpha+u\quad \alpha\in\mathbb{R}^m$$ it is known that using OLS estimates $\hat{\beta},\hat{\alpha}$ fitted ...
1
vote
2answers
386 views

vertical vs. horizontal regression

A horizontal regression is defined as the following: $$m=\frac{\sum_{i=1}^n (x_i-\operatorname{average(x)})(y_i-\operatorname{average(y))}}{\sum_{i=1}^n (x_i-\operatorname{average(x)})^2}$$ whereas ...
1
vote
2answers
793 views

Log-likelihood gradient and Hessian

Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions: $f(x) = x^T \beta$ $p(x) = ...
1
vote
1answer
116 views

Econometrics OLS estimates

I cant seem to use the formula to calculate B1 without knowing xi and yi. Is it possible to calculate using just the variances and covariance? Please help! The classical linear regression model ...
1
vote
1answer
39 views

Getting the formula of a live counter

I'm looking to replicate this greenhouse gases counter in my website. Poking around i found the initial data for the formula. The counter use the following information: Beginnig date: 2012/03/01 ...
1
vote
2answers
369 views

Simple Least Squares Regression?

I have a vector X of 50 real numbers and a vector Y of 50 real numbers. I want to model them as y = ax + b How do I determine a and b such that it minimizes the ...
1
vote
3answers
630 views

Find parameters for exponential function fitting to datapoints

I have a set of datapoints, in this case the temperature of an object adjusting to the environment temperature over time. Because I know these kind of processes take the form of $$f(x)=Ae^{x/B}+C$$ I ...
1
vote
4answers
205 views

How to fit a curve to my data

I have a datasheet. It looks like an hyperbola. How can I fit a curve to it? And how can I plot a curve of the first derivative? ...
1
vote
1answer
349 views

Curve fitting with upper and lower bounds for derivatives

I compute (at a great cost) upper and lower bounds $f_u(x)$ and $f_l(x)$ of an unknown function $f(x)$ at points $x$ in $[0,1]$. Now I am interested in an estimation of the derivative $f'(x)$. I ...
1
vote
1answer
84 views

Measuring how monotonically “staircase-like” a set of values is

A bit of a bizarre question here -- I'm looking for assistance in generating a robust metric to measure how monotonically "step-wise" a series of values is. The set must not start or end at a specific ...
1
vote
1answer
1k views

help with using the “simple regression (least squares) method” of forecasting

This problem is from an engineering management textbook (Morse & Babcock, 5th ed) : 2005 $48k 2006 $64k 2007 $67k 2008 $83k "What is the ...
1
vote
1answer
304 views

bayesian networks for regression

Would it be possible to use bayesian network for regression and/or prediction? I understand that it is a tool one can use to compute probabilities, but I haven't found much material about possible ...
1
vote
1answer
326 views

Recursive coefficient of determination (R2)

Is there a way to compute the coefficient of determination $R^2$ in a recursive way? $R^2$ is defined as following: $$R^2 \equiv 1 - \frac{SS_{\rm err} }{ SS_{\rm tot}} = 1 - \frac{\sum_i (y_i - ...
1
vote
1answer
130 views

can an artificial neural network with only one hidden layer fit all purposes/applications/functions?

I have heard that only a single layer is needed for an ANN to fit any possible function (input to output). Is this true and where is this investigated/state/found? Then what is the advantage of having ...
1
vote
0answers
20 views

Least square / Linear regression over a simplex

I have to solve the following least square problem: $$\hat{x} = \arg \min_{x \in S} \|Ax - b\|^2$$ If $S = \mathbb{R}^n$, then the solution is given by $$\hat{x} = (A^TA)^{-1}A^Tb$$ having posed ...
1
vote
1answer
32 views

Prove that the the variance estimator $\widehat{\sigma}^2=MSE/(n-2)$ is biased is the simple linear regression model

This is in scope of the simple linear model. Im trying to prove that $\mathbb{E}\left(\widehat{\sigma}^2\right) = \sigma^2$ for $$\widehat{\sigma}^2 = \frac{1}{n-2}\sum^n_{i=1} ...
1
vote
1answer
23 views

Least Squares Regression Matrix for Rational Functions

So first off no, this isn't a homework problem. Second, I'm trying to understand how this works, NOT find a program that will do it for me. Okay so I've known for a while how to use Gaussian-Jordan ...
1
vote
1answer
19 views

Regression Model with even function?

Is there any method to test if the mean function, $f(x)$, of a regression model $y=f(x)+\epsilon$ is even or not?
1
vote
0answers
16 views

Orthogonalization of Variables

Let's assume we have three collinear variables/factors, $X_1$, $X_2$, $X_3$. Would there be a method to orthogonalize these variables in a simultaneous way: in other word, orthogonalizing them in such ...
1
vote
2answers
32 views

Errors and Residual

Why are errors independent but residuals dependent? As far i know the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. But also ...
1
vote
2answers
26 views

Simple linear regression seems off

I have the following datapoints: $$p1(52,730)$$ $$p2(53,409)$$ $$p3(52,250)$$ $$p4(52,90)$$ Now I want to find the best fitting line between these points. When I use simple linear regression I get $$y ...
1
vote
2answers
32 views

orthogonal matrices vs. orthogonal columns

I'm just reading a book on econometrics and now I'm stuck with a problem: There is a Theorem on "Orthogonal Partitioned Regression" which says: "In the multiple linear least squares regression of ...