Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
36 views

finding column vectors - linear transformations

$L:\mathbb{R}^3\rightarrow \mathbb{R}^2$ with bases $\mathcal{S}=\left\{\left(-1,1,0\right),\left(0,1,1\right),\left(1,0,0\right)\right\} \: \text{for} \:\mathbb{R}^3 \:\text{and} \\ ...
1
vote
1answer
29 views

Multicollinearity: Why does highly correlated columns in the design matrix lead to high variance of the regression coefficient?

I came across the term "Multicollinearity" in statistics, particularly statistics. However, I never really understand mathematically why highly correlated (almost linearly dependent) columns in the ...
1
vote
1answer
71 views

How to apply non-linear regression to Logistic (sigmoid) curve

I've been looking at a useful way to represent Doppler shift from a satellite passing over a ground station. I've calculated the Doppler shift frequency values at 1-second interval for the duration of ...
1
vote
2answers
76 views

Maple: How do I type “solve” with an arrow under?

I am trying to learn using Maple 18 (Mac). I have defined a function with a list of X and Y values. f := x->LinReg(X, Y, x) Now I would like to output the unknown "x" value that correlates with ...
1
vote
2answers
152 views

How do I find equation of this curve?

I need to find equation of the curve as shown below, for which, I need to find equation for upper part. lower part is half circle. upper part is a constant distance from circle with line passing ...
1
vote
2answers
2k views

Prove $SST=SSE+SSR$

Prove $$SST=SSE+SSR$$ I start with $$SST= \Sigma (y_i-\bar{y})^2=...=SSE+SSR+ \Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )$$ and I don't know how to prove that $\Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )=0$ a ...
1
vote
1answer
82 views

Robust Standard Errors

For OLS, my professor said that you should always test for heteroscedasticity first, rather than going straight to the adoption of robust standard errors. I didn't quite follow this and no ...
1
vote
2answers
50 views

Polynomial best fit line for very large values

not only are the x values large, the difference between them and the y values is huge. My data points: 22353120,720 24448725,671.427053270323 26544330,634.312274868634 28639935,566.291966792026 ...
1
vote
2answers
66 views

Multiple linear regression

For a homework we have to determine the effect of a predictor variable on an outcome variable using simple linear regression. We have lots of data (about 300 variables) and we may include some other ...
1
vote
1answer
222 views

Proof that a and b in linear regression are random variables

Does anyone know how to prove that the variables $a$ and $b$ that are used in linear regression are random variables? For me the assumption would be that these are dependent on the values of $x$ and ...
1
vote
2answers
114 views

Linear Regression to quadratic function

What is the optimal linear regression (w and w/o y-intercept) for a quadratic curve w.r.t. mean square error. Mathematically speaking: Given, $$y = x^2$$ for $$x = [-a,a]$$. What is the best ...
1
vote
1answer
395 views

QR factorization for ridge regression

I am solving an overdetermined system of equations: $$Ax= b$$ Using QR factorization, we can solve this system easily by posing it as: $$Rx= Q'b$$ I would like to regularize my estimate of $x$. I ...
1
vote
1answer
632 views

What is the difference between a polynomial regression and a generalized linear model?

I have seen that a polynomial linear regression can have this form: $y = c_0 + c_1 x_1 + c_2 x_2 + \dots + c_k x_k $ but I have read that the general lineal model which is a form of the multiple ...
1
vote
2answers
95 views

What is the equation that fits this curve?

I have a curve that looks like this (it's cyclical): Curve I can get a partial fit by fitting a 3rd degree polynomial, but I have a feeling there must be a better fit (something that involves sin ...
1
vote
2answers
419 views

Normal Distribution from Standard Deviation?

So I have a data set $(x_{1},y_{1}), (x_{2},y_{2}),\dots,(x_{n},y_{n})$ and from it I have the values of $\sum x$, $\sum x^{2}$, $\sum y$, $\sum y^{2}$, $\sum xy$. My question is, how do I find a ...
1
vote
1answer
94 views

$(X^tX)^{-1}$ when $p>>n$

For the $n\times p$ matrix $\mathbf{X}$, is there any use in approximating $(\mathbf{X}'\mathbf{X})^{-1}$ when $p>>n$? If so, what information might this tell us? I understand when $p<n$, ...
1
vote
3answers
132 views

Middle line between points

How can we calculate LINE that most fit the points T1(1,0) T2(2,0) T3(-1,1) T4(0,1) $x= (1,2,-1,0) $ $y= (0, 0, 1, 1) $ $1= (1, 1, 1, 1)$
1
vote
1answer
4k views

Can someone explain what plim is?

In my Introductory Econometrics class we discussed a concept of "plim" or "probability limit. I'm not sure what this means though and my professor doesn't explain it well at all. Can someone tell me ...
1
vote
2answers
521 views

vertical vs. horizontal regression

A horizontal regression is defined as the following: $$m=\frac{\sum_{i=1}^n (x_i-\operatorname{average(x)})(y_i-\operatorname{average(y))}}{\sum_{i=1}^n (x_i-\operatorname{average(x)})^2}$$ whereas ...
1
vote
1answer
124 views

Calculating the regression equations

I have four data points $(1,2), (2,4), (3,5), (5,7)$ and Im looking for the least squares regression line that best fits them. I use the normal equation $A^tAx=A^tb$ in this form - ...
1
vote
1answer
890 views

Simple non linear fitting question(Least Squares Fitting--Exponential) [duplicate]

Possible Duplicate: easy to implement method to fit a power function (regression) I have the following simple function: $h = cV^n$ h and V being the variables and $c$ and $n$ are ...
1
vote
1answer
89 views

Measuring how monotonically “staircase-like” a set of values is

A bit of a bizarre question here -- I'm looking for assistance in generating a robust metric to measure how monotonically "step-wise" a series of values is. The set must not start or end at a specific ...
1
vote
1answer
134 views

problem constructing an equation that connects variables

need a suggestion/advice/inputs from the mathematicians here. I have collected data of number of instructions/sec executed by a processor every second, number of loads/stores performed every second ...
1
vote
1answer
20 views

Gauss-Markov Theorem: How can you show that $\Lambda^T (X^TX)^gX^TX(X^TX)^{g^T} \Lambda$ = $\Lambda^T(X^TX)^g\Lambda$?

I'm stuck on how to go from the first line to the second line in this equation related to the Gauss-Markov model where $\mathbf{y}=X\mathbf{b}+\mathbf{e}$, $E(\mathbf{e})=0$, and ...
1
vote
1answer
22 views

Linear Regression quadratic terms

I have a hard time understanding the term 'linear regression'. For what I know, linear means polynomial of degree 1. But then, I found that in one of my lectures, the lecturers are saying that this ...
1
vote
1answer
47 views

Significance level for a hypothesis test for a linear regression

Consider linear regression model $Y_i=a+b\cdot x_i+\epsilon_i$, $i=1,2,3,4,5$, where $a,b\in\mathbb{R}$ are unknown and $x_1=x_2=1,x_3=3,x_4=x_5=5$, $\epsilon_i$ are iid, normally distributed with ...
1
vote
1answer
15 views

contour plot in multiple linear regression

I have recently saw some examples about contour plots and multiple linear regression, for what I know a countour plot is obtained for having a graphical view of how the weights in a linear regression ...
1
vote
1answer
84 views

Using the observation vector $ \vec{y}$ instead of the centered observation vector $ \vec{y_{d}} $ doesn't change the projection $\vec{\hat y}$

I'm wondering why the two statements below are equal regardless of using $\vec{y}$ in deviation form/mean-deviaton/centered form or not. In other words, why isn't the result changed when you use the ...
1
vote
1answer
379 views

MATLAB curve fitting - least squares method - wrong “fit” using high degrees

Anyone here that could help me with the following problem? The following code calculates the best polynomial fit to a given data-set, that is; a polynomial of a specified degree. Unfortunately, ...
1
vote
1answer
32 views

Question on regression

So I've been given this formula For regression $R^2=1 - \sum \frac{{(y_i - \hat{y}_i)}^2}{(y_1-\bar{y})^2}$ Now an obvious question that has come to me is why $R^2$ stays the same in certain ...
1
vote
1answer
19 views

what is one basic/intermediate regression analysis standard textbook that is math intense

What is one basic/intermediate regression analysis standard textbook that is math intense with proofs/derivations? Also, i need that one to be comprehensive yet the diffculty is suitable for self ...
1
vote
1answer
40 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
1
vote
1answer
40 views

How to Change Summation Expression $\sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$ into Matrix Expression

Let $\mathbf{X}_i$ be a $G \times K$ matrix, and suppose are $i=1,...,N$ of these matrices. Note that \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{X}_i &= \begin{bmatrix} ...
1
vote
1answer
73 views

Low Leverage in Residuals, Logistic Regression

I am doing an interpretation of logistic regression and I have an observation withh high residuals but low leverage. I thought that means that it is an outlier(bad prediction) but not ...
1
vote
1answer
52 views

Prove $\operatorname{Var}(\hat{e}_{ij}) = \sigma^2 \left(\frac{n_i-1}{n_i}\right)$

$\newcommand{\Var}{\operatorname{Var}}$ Let $y_{ij}$ denote the observed response of the $j$th experimental unit in the $i$th treatment group, and the $e_{ij}$ are i.i.d. $N(0,\sigma^2)$ experimental ...
1
vote
1answer
57 views

Understanding polynomial regression

I'm looking for a good tutorial on how to calculate a "line of best fit" for non-linear data. I found this site: http://easycalculation.com/statistics/learn-regression.php which gives a very good ...
1
vote
1answer
37 views

time-series regression with missing data

I have a regression as follows for time-series data (e.g. stock prices versus other variables): $$ Y = b \cdot X + b_1 \cdot X_1 + e$$ where $X_1$ will be missing based on pre-determined dates ...
1
vote
1answer
27 views

Finding best predictors of a classification function

I have a large dataset where each element has a number of "input" categories that are either present or not (or if you like, true or false, 1 or 0 etc). Each one also has an output category, again a ...
1
vote
1answer
59 views

Variance- covariance matrix

Consider $H$ denotes hat matrix and $e$ denotes residual. In the book Applied regression Analysis by Draper/Smith, it is written that : $\mathbb V(e_i)$ is given ...
1
vote
1answer
22 views

Dummy recoding for more than two categorical variables

Say I am doing a study with 3 different types of fruit and I want to make a regression depending on the type that tries to predict the amount sold. I know that I could make 2 dummy variables: orange ...
1
vote
2answers
60 views

Conditional Expectation, Orthogonality, and Correlation

I know that if $\epsilon$ and $x$ are independent, then $E[\epsilon|x]=E[\epsilon]$ and Cov$(\epsilon,x)=0$. However, $E[\epsilon|x]=E[\epsilon]=0$ implies Cov$(\epsilon,x)=0$ iff $\epsilon$ and $x$ ...
1
vote
1answer
36 views

Is the Inverse of the Vectorised Solid Angle Equation for $n$ Circular Discs Continuous?

I have a continuous function$^{*1}$ that takes in 3 arguments, and returns 24 outputs. I want to know if the inverse of this function is continuous. The 3 input arguments are the x, y, and z position ...
1
vote
1answer
51 views

Linear regression with normalized variables

Suppose I have two variables X and Y such that mean(X) = 0 = mean(Y) and sd(X) = 1 = sd(Y). The slope of the linear regression line for Y vs X is cov(X,Y)/var(X) = corr(X,Y) since X and Y are ...
1
vote
2answers
98 views

Plotting an Ellipse after an Ellipse Fit

I wonder if someone can assist my understanding as I'm a bit stumped with this... I have taken the following (x,y) data which lies roughly on an ellipse: $$ \begin{pmatrix} 0.000234491 & 6855810 ...
1
vote
1answer
44 views

Calculating R-squared with duplicate data

I have the following question regarding the proper usage of R-squared value. Say I have an equation, that predicts energy consumption for the month of a building. One of the input variables accounts ...
1
vote
1answer
13 views

Meaning of $\mathcal{I}_t$ in assumption $E[u_t\mid\mathcal{I}_t]$ of distributed-lag model

When considering \begin{equation*} y_t = \beta_0 + \beta_1 x_t + \ldots + \beta_r x_{t-r} + u_t \end{equation*} an assumption made is \begin{equation*} E[u_t\mid\mathcal{I}_t] = 0 \end{equation*} ...
1
vote
1answer
50 views

Wolfram Exponential Fit not match as formulas

I am using WolframAlpha Exponential-Fit formulas to find equation of Exponential Regression http://mathworld.wolfram.com/LeastSquaresFittingExponential.html but after implementation, I tested with a ...
1
vote
1answer
32 views

clustering of singular values

let us consider following graph of singular values i want to make some kind of clustering of these data,namely to seperate main components from non main components,let say signal components ...
1
vote
1answer
103 views

Equations for Cubic Regression

So, I'm making a simple program for drawing graphs, and I'm looking at making some simple best-fit curves using some basic regression analysis. I've happily got linear and quadratic regression working ...
1
vote
1answer
79 views

regression on circular data

How would one design a regression where the dependent variable is measured in degrees on a circle? The dependent variable is on the range [0, 360), and the independent variables are demographic ...