Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
2answers
2k views

“taking the derivative of both sides” ln(Y)=ln(x) (to interpret log transformed regression variables or better understand elasticities or % changes)

In a linear regression of the form Y=bX we often have ln transformed Y and X. ie., lnY=b*lnX This is interpreted as a 1% change in X resulting in a b% change in Y (approximately) The derivation of ...
2
votes
2answers
79 views

Find parameters for curve fitting (simple linear regression involved?)

I would like to fit data in g~t scatterplot, where ...
1
vote
1answer
410 views

Uncertainty in gradient of data

So I have a set of 9 x,y values, and I need to find the gradient/slope of the data, AND its associated error. Without the error, I would've used Excels LINEST function, but as the errors in my y ...
0
votes
2answers
813 views

Extrapolation with exponential curve

I would like to extrapolate time series using exponential curve while getting the parameters via linear regression. Exponential curve is given as $g=e^{~a + b \cdot t}$. Since I want to use linear ...
1
vote
0answers
38 views

How to find a function that can approximate another blackbox function programmaticly?

This question has been posted on http://stackoverflow.com/questions/21758016/how-to-find-a-function-that-can-approximate-another-blackbox-function-programmat I was suggested to post it here. I ...
3
votes
1answer
3k views

Why can we assume that the expected value of the error term is zero? [closed]

Why can we assume that the expected value of the error term in a linear regression model is zero? This is with regard to a simple linear regression.
1
vote
2answers
45 views

Would it be any concern if we find correlation between intercept and other regression coefficients?

During a multiple linear regression analysis, I found correlation between intercept (beta-0) and two of the other regression coefficients. Is there any problem or concern in this case? If no, please ...
0
votes
1answer
30 views

Setting up statistics problem

Assume $\beta_{U,T}$ is the underlying slope of straight line associating $U$ with $T$. We know that $X=U+f$ and $Z=T+e$ are measurable instead of $U$ and $T$, where $e$ and $f$ are uncorrelated ...
1
vote
3answers
48 views

Different Regression Lines?

Hi quick question with regression. If the coefficients of a simple regression line, B0 and B1, are the same then why are the regression lines of y on x and x on y different given the condition r^2 <...
1
vote
1answer
106 views

Combine linear models of different sets of data.

I'm working on a large data set D that can be partitioned into some disjoint subsets D1, D2, ..., Dn. For each subset Di, I have a linear model Mi that minimizes the residual error for data in Di. ...
0
votes
0answers
77 views

Question about sample variance with linear algebra

Given random variabes $Y_1,\dots,Y_n$ with mean $\mu$ and variance $\sigma^2$, I am supposed to prove that the sum of $(Y_i-(\text{mean of }y))^2$ can be expressed as $$y^T\left(I_{n\times n} - \...
1
vote
0answers
36 views

What is correlated with what in a linear regression?

I'm trying to make sure I understand the ins and outs of a linear regression and am making a table for what is correlated with what, so just want to see if I have everything included. I'm looking at ...
0
votes
1answer
78 views

Linear Regression Problem (“Regression Towards the Mean”)

I am having my mind turned upside down with a problem I am dealing with. So imagine we have a situation where we have pairs of points where x=heights of fathers and y=heights of the sons of these ...
2
votes
2answers
57 views

Merging Linear Regression

If I have built two linear regression models over sets $A$ and $B$, and now want a linear regression over set $A\cup{}B$. Is there a way to reuse what I already have?
1
vote
1answer
149 views

Calculating variance of estimated intercept parameter, $\hat\beta_0$

I have the following sample : $$ \begin{array}{c|lr} X&80&100&120&140&160&180&200&220&240&260\\ \hline Y & 70 &65&90&95&110&115&120&...
0
votes
1answer
261 views

Why is $(x'x)^{-1}x' = x(x'x)^{-1}$

If $(AB)'=B'A'$ then $(x'x)^{-1}x'$ should be equal to $x((x'x)^{-1})'$ . However most econometrics textbooks say that this is equal to $x(x'x)^{-1}$ . What happened to the transpose of $(x'x)^{-1}$? ...
0
votes
1answer
86 views

Finding estimates of a Linear Regression Equation - R

I'm new to Statistics and R. I'm currently looking through a book called "Discovering Statistics using R". Although the book implies you don't need any statistical background, some of the content isn'...
2
votes
2answers
40 views

Regression model when under-estimations costs us more than over-estimations

We have a factory and we are planning how many items produce in 2014. During the learning process we minimize the mean squared error. But, under-estimations costs us more than over-estimations. Let's ...
0
votes
1answer
75 views

Covariance of two random variables in a bivariate normal distribution

http://www.econstreams.com/bivariateproof.jpg Image uploaded to the link above. I'm just not seeing the connection between the 2nd equation on the left handside and the equation on the right. ...
1
vote
0answers
54 views

Solving a linear matrix equation with respect to the maximum of the euclidian distances between rows.

With $n>m$, real number matrices $A$, $B$, $C$ are shaped like: $$A=\left( \begin{array}{ccc} a_{1,1} & \cdots & a_{1,m} \\ \vdots & \ddots & \vdots \\ a_{n,m} & \cdots &...
1
vote
0answers
36 views

Adjusting regression for small sample bias

I have a set of data points $\{x_i\}$. These data points are grouped so that (say) $i\in\{1,2,3\}$ is group $A$, $i\in\{4,5,6,7\}$ is group $B$, etc. I would like to test the null hypothesis of no ...
0
votes
1answer
41 views

help in multiple linear regression

I am having a question in regression analysis in JMP or any other tool. I have one dependent variable $y$ and $2$ independent variables $x_1$ and $x_2$. For example: time $= y -$ per row time ( ...
1
vote
2answers
58 views

Polynomial best fit line for very large values

not only are the x values large, the difference between them and the y values is huge. My data points: 22353120,720 24448725,671.427053270323 26544330,634.312274868634 28639935,566.291966792026 ...
2
votes
1answer
111 views

How to calculate $\sum(X_i-\bar{X})^2$ in R

I'm trying to figure out how to calculate $\sum(X_i-\bar{X})^2$ in R, specifically identifying it in either the aov function or $\operatorname{lm}(y\sim x)$ function. I am trying to use it to ...
1
vote
0answers
41 views

Polynomial regression - differences between algorithms

I know that I can find a polynomial regression's coefficients doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose). This is a way of finding them; now, there is (as far as I know) at least one other ...
3
votes
4answers
883 views

Fitting curve for exponential: $y = A - B\mathrm{e}^{-t/\tau}$

I have some data that follows a saturation or charging profile such as $y = A - B\mathrm{e}^{-t/\tau}$. To begin with, is there a proper name for this function? I have seen it many times, including: ...
1
vote
1answer
39 views

Morphing $\beta_1$ into a different form (OLS question in Statistics)

I am currently studying Simple Linear Regression and I have successfully proven to myself how $\beta_1$ and $\beta_0$ are derived. However, I have been stuck on a seemingly simple problem (I'm sure ...
0
votes
2answers
46 views

Why is regression analysis also statistical test?

According to wikipedia, regression analysis is a statistical process for estimating the relationships among variables. Regression analysis is widely used for prediction and forecasting. So why is ...
1
vote
0answers
50 views

Linear Regression with multiple equations

I am trying to implement a linear regression algorithm to fit a set of "true" points with their "observed" location. The points are specified using spherical coordinates on a unit sphere. I have a ...
2
votes
2answers
193 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...
0
votes
2answers
161 views

Error analysis in “linearized” regression

I am currently taking an experimental chemistry course where we need to fit data to an equation of the form $y=a\exp(bx)$. They recommend "linearizing" this equation by taking the logarithms of both ...
3
votes
0answers
392 views

Can the sigmoid function approximate any function (or relation) where 0<y<1

I'm studying Machine Learning and Artificial Neural Networks. Some basic principles of Machine Learning are linear regression, multivariate linear regression, and nonlinear regression. The last of ...
1
vote
2answers
52 views

adapting a function by a linear regression

I am wondering if it is possible to adapt the function $$y=a\cdot \ln(x)+\frac{b}{x}+x$$ by a linear regression to fit experimental data? If yes, how could this be done? Thank you!
-1
votes
2answers
181 views

Proving that the estimate of a mean is a least squares estimator?

I think this is a really simple question so please bear with me -- I just had my first class in regression and I'm a little confused about nomenclature/labeling. Does anyone recommend some good ...
0
votes
1answer
52 views

Probability, Linear Models, Expectation

I'm trying to find a way of predicting various models from one "perfect model", using EXCEL. i.e. If I assume that all models should behave like the original one, for which I have all the data, ...
2
votes
1answer
31 views

Aproximate data with this equation (or linearize the equation)

I have found an equation that describes the behaviour of a phisical system: $$ y=a_1e^{-a_2x} + a_3 + a_4x + a_5e^{{-a_6} / {(1-x)}}$$ Now I have data of that physical system and I want to ...
0
votes
1answer
87 views

Contrary interpretations of Least Squares for Regression

According to the original thought, our goal is to minimize the quadratic error $$\min\{\frac{1}{2}(Ax-b)^2 \}$$ Then, we search the extremum by the derivation of $x$ $$A^T(Ax-b)=0$$ $$A^TAx=A^Tb$$ $$...
6
votes
2answers
36k views

Linear regression: degrees of freedom of SST, SSR, and RSS

I'm trying to understand the concept of degrees of freedom in the specific case of the three quantities involved in a linear regression solution, i.e. $SST=SSR+SSE, $ i.e. Total sum of squares = ...
0
votes
2answers
111 views

How to fit a power function to data with noise

I have multiple data-sets from a Fourier series of a function $f(t,x)$ (the data-sets where obtained by varying $x$), so $A_n=\frac{2}{T}\int_0^T{f(t,x)\sin{\frac{2\pi nt}{T}}dt}$, which seems to ...
3
votes
1answer
159 views

Minimum required data for cosine fit

With a minimum amount of (noisy) data-points, I need to find the amplitude of a simple cosine y=A*cos(x), where x is an angle from 0:2pi. I know how to fit data to the function, and I know how to ...
2
votes
2answers
735 views

How to fit a sinusoidal function through 2 points with known slopes?

I can define my sinusoidal function as $y(x) = A\sin(B x+c) + D$ or as $y(x) = A \sin(B x) + C \cos(B x) + D$ Now, I have two points with known slopes that I must fit this sine wave to, thus my ...
0
votes
2answers
81 views

Regression Analysis

When I have a table of values like \begin{array}{c|ccccc} x & 1 & 2 & 3 & 4 & 5 \\ y & 3 & 6 & 8 & 9 & 0 \\ y & 4 & 6 & 1 & 2 & 4 \end{array}...
1
vote
2answers
49 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
0
votes
1answer
45 views

regression on principal component analysis

I have done a PCA to get my principal components and now do a principal component regression. In the PCR the 1., 2. and 4. component are significant and the 3. component is insignificant. Can anyone ...
1
vote
1answer
35 views

Unexplainable determination coëfficiënt

I have a series of data (more specific, they are coördinates of a package attached to the end of a mini level-luffing crane. The "flightpath" is linear and horizontal.) Now when I plot the data: ...
1
vote
3answers
193 views

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and ...
2
votes
1answer
112 views

Creating a lift chart for a classification tree

This is likely a simple question but I'm new to data mining techniques and am trying to compare two different predictive models. I've created a logistic regression and a classification tree and would ...
0
votes
1answer
193 views

Regression with equally spaced set

I'm working on an algorithm (written in Python/Cython, but it reads like pseudo-code) that estimates the gradient of each point in noisy data, using a variable window size. It's working very well, but ...
0
votes
1answer
142 views

Jacobian in Levenberg-Marquardt for 4-Parameter equation

I am trying to fully understand how I can use Levenberg-Marquardt to minimise a 4 parameter equation. There are lots of fancy programs to do this but the documentation about the mathematics is ...
0
votes
1answer
69 views

How to check linear independence

How can I check the linear independence of my variables? I have this system $Ax=b$ where $A$ is a $N \times 4$ matrix. I want to check the linear independence between the 4 variables in matrix $A$.