Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
0answers
16 views

Regression factors and covariance matrix

I am trying to follow some notes. They have two matrices. One is called comfact (company factors). This is a $580 \times 5$ matrix. The $580$ rows represent $580$ different companies. The $5$ ...
0
votes
0answers
26 views

Multivariate Linear Regression for a System of Linear Equations

I have a system of linear equations in the form of $A\vec{x}=\vec{y}$, where $A$ is an $n\times n$ matrix and $\vec{x}$ and $\vec{y}$ are $n\times 1$ matrices. Suppose $\vec{x}$ and $\vec{y}$ ...
1
vote
1answer
34 views

What's the “ridge” in Ridge Regression?

In normal least squares, we try to find $\hat\beta$ which minimizes $$\|y-X\beta\|^2$$ Ridge regression expands this to "penalize" certain values of $\beta$ via a matrix $\Gamma$: ...
0
votes
0answers
20 views

linear regression optimization

If I have this equation to be minimized (it is from linear regression) and the example is: or in its complete form: so it says that one needs to minimize the function J(theta). The parts of ...
0
votes
0answers
17 views

Multinomial Logistic Regression with an Unknown Number of Categories

The problem I'm facing involves determining the number of categories in a data set in a principled way. I feel that logistic regression may be the right tool for this project, but is it possible to do ...
2
votes
0answers
53 views

A* vs D* vs Dijkstra [closed]

I understand the basis of A* as being a derivative of Dijkstra, however, I recently found out about D*. From wikipedia, I can understand the algorithm. What I do not understand is why I would use D* ...
1
vote
1answer
17 views

Regression analysis question

I do writing that involves correlation studies, but I am not a mathematician. I am considering extending my research to golf, but wonder whether the nature of golf scoring makes that unfeasible. In ...
0
votes
1answer
16 views

Linearilization of non-linear relationships (Linear Regression)

How do you linearise the following equation? $$ Y = \frac{\beta_{1}x}{\beta_{0} + x} + E. $$ $\beta_{0}$ and $\beta_{1}$ are the parameters and $E$ is the regression error term.
2
votes
1answer
57 views

Fitting a simple linear regression

A professor in the School of Business in a university polled a dozen colleagues about the number of professional meetings they attended in the past five years $x$ and the number of papers they ...
0
votes
1answer
32 views

Estimate correlation coefficient of unknown variable

Consider variable y depends on variable x and z linearly. I have $100$ sample values of $y$ and corresponding $x$ but don't have any values of $z$. The functional model is $$y = \alpha_1x + \alpha_2z ...
1
vote
0answers
36 views

Linear Regression Question (Linear Algebra) Help!!

Hey guys, I have a quick question. I am trying to prove that the squared sample correlation between fitted and observed values is equal to $R^2$ (coefficient of determination). I am having a lot of ...
0
votes
1answer
25 views

How to find restrictions of Autoregressive Lag Model (ADL)

All of my textbooks mention restrictions for AD models but don't explicitly say what they mean by "restrictions" and I'm having a hard time grasping what they mean. $y_t = \beta_0 +\phi y_{t-1} + ...
1
vote
1answer
48 views

Variance of Coefficients in a Simple Linear Regression

I have a linear regression model $\hat{y_i}=\hat{\beta_0}+\hat{\beta_1}x_i+\hat{\epsilon_i}$, where $\hat{\beta_0}$ and $\hat{\beta_1}$ are normally distributed unbiased estimators, and ...
2
votes
2answers
84 views

connection between PCA and linear regression

Is there a formal link between linear regression and PCA? The goal of PCA is to decompose a matrix into a linear combination of variables that contain most of the information in the matrix. Suppose ...
1
vote
1answer
28 views

Regression towards the mean?

I am upset with what examples testing "regression to the mean" seem to allude to: People claiming to have "ESP" take a test, and A's score was 2 standard deviations above the mean, whereas B's score ...
0
votes
1answer
37 views

Robust Standard Errors

For OLS, my professor said that you should always test for heteroscedasticity first, rather than going straight to the adoption of robust standard errors. I didn't quite follow this and no ...
0
votes
1answer
19 views

which regression is better

suppose that we have two input vector and the variables in each vectors are independent and uncorrelated from each other,just only there is relationship between two vector,but not itself in ...
-1
votes
2answers
270 views

“taking the derivative of both sides” ln(Y)=ln(x) (to interpret log transformed regression variables or better understand elasticities or % changes)

In a linear regression of the form Y=bX we often have ln transformed Y and X. ie., lnY=b*lnX This is interpreted as a 1% change in X resulting in a b% change in Y (approximately) The derivation of ...
1
vote
2answers
55 views

Find parameters for curve fitting (simple linear regression involved?)

I would like to fit data in g~t scatterplot, where ...
0
votes
1answer
49 views

Uncertainty in gradient of data

So I have a set of 9 x,y values, and I need to find the gradient/slope of the data, AND its associated error. Without the error, I would've used Excels LINEST function, but as the errors in my y ...
0
votes
2answers
78 views

Extrapolation with exponential curve

I would like to extrapolate time series using exponential curve while getting the parameters via linear regression. Exponential curve is given as $g=e^{~a + b \cdot t}$. Since I want to use linear ...
0
votes
0answers
35 views

Finding the error on the gradient

I have a set of data $(x,y)$ with errors on the $y$-values (the errors for each point are different). The data is linear and I can compute a least squares line of best fit in the form $y=mx+c$. How ...
1
vote
0answers
25 views

How to find a function that can approximate another blackbox function programmaticly?

This question has been posted on http://stackoverflow.com/questions/21758016/how-to-find-a-function-that-can-approximate-another-blackbox-function-programmat I was suggested to post it here. I ...
0
votes
0answers
72 views

Example of Logarithmic and/or logistic growth? Flappy Bird?

Trying to think of an interesting example to play with. Textbooks mainly have "boring" examples like bacteria growth. Either log growth (flattening out) or logistic (S-curve) Adoption of a certain ...
1
vote
1answer
213 views

Why can we assume that the expected value of the error term is zero? [closed]

Why can we assume that the expected value of the error term in a linear regression model is zero? This is with regard to a simple linear regression.
1
vote
2answers
30 views

Would it be any concern if we find correlation between intercept and other regression coefficients?

During a multiple linear regression analysis, I found correlation between intercept (beta-0) and two of the other regression coefficients. Is there any problem or concern in this case? If no, please ...
0
votes
1answer
24 views

Setting up statistics problem

Assume $\beta_{U,T}$ is the underlying slope of straight line associating $U$ with $T$. We know that $X=U+f$ and $Z=T+e$ are measurable instead of $U$ and $T$, where $e$ and $f$ are uncorrelated ...
1
vote
3answers
37 views

Different Regression Lines?

Hi quick question with regression. If the coefficients of a simple regression line, B0 and B1, are the same then why are the regression lines of y on x and x on y different given the condition r^2 ...
1
vote
1answer
34 views

Combine linear models of different sets of data.

I'm working on a large data set D that can be partitioned into some disjoint subsets D1, D2, ..., Dn. For each subset Di, I have a linear model Mi that minimizes the residual error for data in Di. ...
0
votes
0answers
35 views

Question about sample variance with linear algebra

Given random variabes $Y_1,\dots,Y_n$ with mean $\mu$ and variance $\sigma^2$, I am supposed to prove that the sum of $(Y_i-(\text{mean of }y))^2$ can be expressed as $$y^T\left(I_{n\times n} - ...
1
vote
0answers
20 views

What is correlated with what in a linear regression?

I'm trying to make sure I understand the ins and outs of a linear regression and am making a table for what is correlated with what, so just want to see if I have everything included. I'm looking at ...
0
votes
1answer
37 views

Linear Regression Problem (“Regression Towards the Mean”)

I am having my mind turned upside down with a problem I am dealing with. So imagine we have a situation where we have pairs of points where x=heights of fathers and y=heights of the sons of these ...
0
votes
0answers
43 views

Find the asymptote of an experimental dataset

I have an experimental dataset that looks roughly like this (this figure is a stand-in I created based on a function with a random part): Just looking at the dataset it seems to have a horizontal ...
2
votes
2answers
36 views

Merging Linear Regression

If I have built two linear regression models over sets $A$ and $B$, and now want a linear regression over set $A\cup{}B$. Is there a way to reuse what I already have?
1
vote
1answer
49 views

Calculating variance of estimated intercept parameter, $\hat\beta_0$

I have the following sample : $$ \begin{array}{c|lr} X&80&100&120&140&160&180&200&220&240&260\\ \hline Y & 70 ...
-1
votes
1answer
73 views

Why is $(x'x)^{-1}x' = x(x'x)^{-1}$

If $(AB)'=B'A'$ then $(x'x)^{-1}x'$ should be equal to $x((x'x)^{-1})'$ . However most econometrics textbooks say that this is equal to $x(x'x)^{-1}$ . What happened to the transpose of $(x'x)^{-1}$? ...
0
votes
1answer
46 views

Finding estimates of a Linear Regression Equation - R

I'm new to Statistics and R. I'm currently looking through a book called "Discovering Statistics using R". Although the book implies you don't need any statistical background, some of the content ...
2
votes
2answers
37 views

Regression model when under-estimations costs us more than over-estimations

We have a factory and we are planning how many items produce in 2014. During the learning process we minimize the mean squared error. But, under-estimations costs us more than over-estimations. Let's ...
0
votes
1answer
30 views

Covariance of two random variables in a bivariate normal distribution

http://www.econstreams.com/bivariateproof.jpg Image uploaded to the link above. I'm just not seeing the connection between the 2nd equation on the left handside and the equation on the right. ...
0
votes
0answers
18 views

Principal Component Regression

Suppose that Z1, Z2 and Z3 are the principal components of a data set and Y is a vector of the response variable. The correlation coecients between Y and Z1, Z2 and Z3 are 0.25, -0.4 and 0.7, ...
1
vote
0answers
38 views

Solving a linear matrix equation with respect to the maximum of the euclidian distances between rows.

With $n>m$, real number matrices $A$, $B$, $C$ are shaped like: $$A=\left( \begin{array}{ccc} a_{1,1} & \cdots & a_{1,m} \\ \vdots & \ddots & \vdots \\ a_{n,m} & \cdots ...
1
vote
0answers
27 views

Adjusting regression for small sample bias

I have a set of data points $\{x_i\}$. These data points are grouped so that (say) $i\in\{1,2,3\}$ is group $A$, $i\in\{4,5,6,7\}$ is group $B$, etc. I would like to test the null hypothesis of no ...
0
votes
1answer
33 views

help in multiple linear regression

I am having a question in regression analysis in JMP or any other tool. I have one dependent variable $y$ and $2$ independent variables $x_1$ and $x_2$. For example: time $= y -$ per row time ( ...
1
vote
2answers
47 views

Polynomial best fit line for very large values

not only are the x values large, the difference between them and the y values is huge. My data points: 22353120,720 24448725,671.427053270323 26544330,634.312274868634 28639935,566.291966792026 ...
0
votes
0answers
41 views

standard deviation of residuals: PCR vs MLR

Does anyone know, why the standard deviation of the residuals of a PCR (principal component regression) is greater than the one of a "normal" MLR (multivariate linear regression)? Thanks!
2
votes
1answer
44 views

How to calculate $\sum(X_i-\bar{X})^2$ in R

I'm trying to figure out how to calculate $\sum(X_i-\bar{X})^2$ in R, specifically identifying it in either the aov function or $\operatorname{lm}(y\sim x)$ function. I am trying to use it to ...
1
vote
0answers
26 views

Polynomial regression - differences between algorithms

I know that I can find a polynomial regression's coefficients doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose). This is a way of finding them; now, there is (as far as I know) at least one other ...
2
votes
4answers
185 views

Fitting curve for exponential: $y = A - B\mathrm{e}^{-t/\tau}$

I have some data that follows a saturation or charging profile such as $y = A - B\mathrm{e}^{-t/\tau}$. To begin with, is there a proper name for this function? I have seen it many times, including: ...
1
vote
1answer
28 views

Morphing $\beta_1$ into a different form (OLS question in Statistics)

I am currently studying Simple Linear Regression and I have successfully proven to myself how $\beta_1$ and $\beta_0$ are derived. However, I have been stuck on a seemingly simple problem (I'm sure ...
0
votes
2answers
43 views

Why is regression analysis also statistical test?

According to wikipedia, regression analysis is a statistical process for estimating the relationships among variables. Regression analysis is widely used for prediction and forecasting. So why is ...