Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
32 views

How to Change Summation Expression $\sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$ into Matrix Expression

Let $\mathbf{X}_i$ be a $G \times K$ matrix, and suppose are $i=1,...,N$ of these matrices. Note that \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{X}_i &= \begin{bmatrix} ...
1
vote
1answer
30 views

Low Leverage in Residuals, Logistic Regression

I am doing an interpretation of logistic regression and I have an observation withh high residuals but low leverage. I thought that means that it is an outlier(bad prediction) but not ...
1
vote
1answer
50 views

Prove $\operatorname{Var}(\hat{e}_{ij}) = \sigma^2 \left(\frac{n_i-1}{n_i}\right)$

$\newcommand{\Var}{\operatorname{Var}}$ Let $y_{ij}$ denote the observed response of the $j$th experimental unit in the $i$th treatment group, and the $e_{ij}$ are i.i.d. $N(0,\sigma^2)$ experimental ...
1
vote
1answer
33 views

Understanding polynomial regression

I'm looking for a good tutorial on how to calculate a "line of best fit" for non-linear data. I found this site: http://easycalculation.com/statistics/learn-regression.php which gives a very good ...
1
vote
1answer
30 views

time-series regression with missing data

I have a regression as follows for time-series data (e.g. stock prices versus other variables): $$ Y = b \cdot X + b_1 \cdot X_1 + e$$ where $X_1$ will be missing based on pre-determined dates ...
1
vote
1answer
25 views

Finding best predictors of a classification function

I have a large dataset where each element has a number of "input" categories that are either present or not (or if you like, true or false, 1 or 0 etc). Each one also has an output category, again a ...
1
vote
1answer
49 views

Variance- covariance matrix

Consider $H$ denotes hat matrix and $e$ denotes residual. In the book Applied regression Analysis by Draper/Smith, it is written that : $\mathbb V(e_i)$ is given ...
1
vote
1answer
17 views

Dummy recoding for more than two categorical variables

Say I am doing a study with 3 different types of fruit and I want to make a regression depending on the type that tries to predict the amount sold. I know that I could make 2 dummy variables: orange ...
1
vote
2answers
45 views

Conditional Expectation, Orthogonality, and Correlation

I know that if $\epsilon$ and $x$ are independent, then $E[\epsilon|x]=E[\epsilon]$ and Cov$(\epsilon,x)=0$. However, $E[\epsilon|x]=E[\epsilon]=0$ implies Cov$(\epsilon,x)=0$ iff $\epsilon$ and $x$ ...
1
vote
1answer
31 views

Is the Inverse of the Vectorised Solid Angle Equation for $n$ Circular Discs Continuous?

I have a continuous function$^{*1}$ that takes in 3 arguments, and returns 24 outputs. I want to know if the inverse of this function is continuous. The 3 input arguments are the x, y, and z position ...
1
vote
1answer
36 views

Linear regression with normalized variables

Suppose I have two variables X and Y such that mean(X) = 0 = mean(Y) and sd(X) = 1 = sd(Y). The slope of the linear regression line for Y vs X is cov(X,Y)/var(X) = corr(X,Y) since X and Y are ...
1
vote
2answers
80 views

Plotting an Ellipse after an Ellipse Fit

I wonder if someone can assist my understanding as I'm a bit stumped with this... I have taken the following (x,y) data which lies roughly on an ellipse: $$ \begin{pmatrix} 0.000234491 & 6855810 ...
1
vote
1answer
34 views

Calculating R-squared with duplicate data

I have the following question regarding the proper usage of R-squared value. Say I have an equation, that predicts energy consumption for the month of a building. One of the input variables accounts ...
1
vote
1answer
12 views

Meaning of $\mathcal{I}_t$ in assumption $E[u_t\mid\mathcal{I}_t]$ of distributed-lag model

When considering \begin{equation*} y_t = \beta_0 + \beta_1 x_t + \ldots + \beta_r x_{t-r} + u_t \end{equation*} an assumption made is \begin{equation*} E[u_t\mid\mathcal{I}_t] = 0 \end{equation*} ...
1
vote
1answer
38 views

Wolfram Exponential Fit not match as formulas

I am using WolframAlpha Exponential-Fit formulas to find equation of Exponential Regression http://mathworld.wolfram.com/LeastSquaresFittingExponential.html but after implementation, I tested with a ...
1
vote
1answer
30 views

clustering of singular values

let us consider following graph of singular values i want to make some kind of clustering of these data,namely to seperate main components from non main components,let say signal components ...
1
vote
1answer
54 views

Equations for Cubic Regression

So, I'm making a simple program for drawing graphs, and I'm looking at making some simple best-fit curves using some basic regression analysis. I've happily got linear and quadratic regression working ...
1
vote
1answer
63 views

regression on circular data

How would one design a regression where the dependent variable is measured in degrees on a circle? The dependent variable is on the range [0, 360), and the independent variables are demographic ...
1
vote
1answer
37 views

What's the “ridge” in Ridge Regression?

In normal least squares, we try to find $\hat\beta$ which minimizes $$\|y-X\beta\|^2$$ Ridge regression expands this to "penalize" certain values of $\beta$ via a matrix $\Gamma$: ...
1
vote
1answer
61 views

Variance of Coefficients in a Simple Linear Regression

I have a linear regression model $\hat{y_i}=\hat{\beta_0}+\hat{\beta_1}x_i+\hat{\epsilon_i}$, where $\hat{\beta_0}$ and $\hat{\beta_1}$ are normally distributed unbiased estimators, and ...
1
vote
1answer
29 views

Regression towards the mean?

I am upset with what examples testing "regression to the mean" seem to allude to: People claiming to have "ESP" take a test, and A's score was 2 standard deviations above the mean, whereas B's score ...
1
vote
3answers
37 views

Different Regression Lines?

Hi quick question with regression. If the coefficients of a simple regression line, B0 and B1, are the same then why are the regression lines of y on x and x on y different given the condition r^2 ...
1
vote
1answer
38 views

Combine linear models of different sets of data.

I'm working on a large data set D that can be partitioned into some disjoint subsets D1, D2, ..., Dn. For each subset Di, I have a linear model Mi that minimizes the residual error for data in Di. ...
1
vote
1answer
30 views

Morphing $\beta_1$ into a different form (OLS question in Statistics)

I am currently studying Simple Linear Regression and I have successfully proven to myself how $\beta_1$ and $\beta_0$ are derived. However, I have been stuck on a seemingly simple problem (I'm sure ...
1
vote
2answers
45 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
1
vote
1answer
29 views

Unexplainable determination coëfficiënt

I have a series of data (more specific, they are coördinates of a package attached to the end of a mini level-luffing crane. The "flightpath" is linear and horizontal.) Now when I plot the data: ...
1
vote
1answer
834 views

Convert nonlinear regression equation to a linear regression equation

The question is: "Show how the nonlinear regression equation y=aX^B can be converted to a linear regression equation solvable by the method of least ...
1
vote
1answer
71 views

Weird formula for linear regression

I'll try to make the matter as clear as possible given the circumstances. My boss asked me to look at an old report a former employee wrote around a couple of months ago. Apparently the report ...
1
vote
1answer
138 views

Simple linear regression - understanding given

The question is to fill out the missing numbers (A-L) of a simple linear regression model. I am having problems with converting and interpreting the given table in terms of variables. Would it be ...
1
vote
1answer
73 views

Logistic regression with interactions

I'm supposed to do a model using logistic regression. So I have a series of $N$ observed data points each of which consists of $m$ explanatory variables $x = (x_{1,i}, ... , x_{m,i})$ and associated ...
1
vote
1answer
645 views

Mean response in linear regression

What does mean response in linear regession mean? I don't understand the definition given in wikipedia. This is the definition: Mean response is an estimate of the mean of the $y$ population ...
1
vote
1answer
165 views

Machine Learning, why not use matrix multiplication instead of gradient descent?

If we want to minimize our Cost function for a given set of data, why do we use gradient descent and continually guess values until we find a min value for theta when when can just use matrix ...
1
vote
1answer
187 views

How do I solve the weighted normal equations?

I am trying to solve the normal equations for a 3D LSE of a general quadric: $$ z = ax^2 + bx + cxy + dy^2 + ey + f$$ Write as a vector equation: $$ \vec{z}= \bf{X}\vec{\beta}$$ where the 'ith row ...
1
vote
1answer
238 views

Linear fit with horizontal and vertical error bars

I'm searching an equation to calculate the parameters for a linear fit. With parameters a and b, the $\chi ^{2}$ is used: $\chi ^{2} = \sum_{i=0}^N (y_{i}-a.x_{i}-b)^{2}$ And with errors: $\chi ...
1
vote
1answer
136 views

Simple Linear Regression Question

Let $Y_{i} = \beta_{0} + \beta_{1}X_{i} + \epsilon_{i}$ be a simple linear regression model with independent errors and iid normal distribution. If $X_{i}$ are fixed what is the distribution of ...
1
vote
1answer
758 views

Prediction Model for forecasting using Linear regression

I am very new to inferral statistics. I am trying to build a prediction model for forecasting the revenue for physicians based on some historical data. I was planning to use Multiple Linear Regression ...
1
vote
1answer
99 views

How to gather useful information from a residue plot

You can usually see how good your linear regression line is by looking at the residue plot. If you see the points randomly distributed, you're good. But if you see a pattern, it means there is ...
1
vote
1answer
50 views

What is the way to determin how good a sequence will interpolate?

Say I have to sequences of numbers: $$[5, 10, 14, 21, 27, 31]$$ $$[1, 20, 21, 22, 30, 31]$$ Even though they both get to $31$ by the $6$th element, logic tells me that only the first one is a good ...
1
vote
1answer
44 views

Cointegration for Price levels Time Series

I don't understand why is the difference between price levels is a stationary process while the time series of price levels themselves is a non-stationary process. For example: ...
1
vote
2answers
296 views

Question about logistic regression

A logistic regression is meant for a binary/categorical variable. Sort of like age vs baldness. 1) So, does the "S-curve" regression equation output give the odds of having that condition for a ...
1
vote
1answer
273 views

Linear regression: b1 has the minimum variance among all unbiased linear estimators of beta1

There is a proof provided in Applied Linear Regression Models (1983) by Kutner et al. (Page 64), which is quite clear and easy to understand, except one point, namely, it assumes that $\sum k_i d_i = ...
1
vote
1answer
365 views

A couple of questions on the NURBS basis functions

I read a little about NURBS curves (specifically from http://www.cs.mtu.edu/~shene/COURSES/cs3621/NOTES/), and I have a couple of questions about the motivation behind the choices made in designing ...
1
vote
2answers
134 views

Least Squares Derivation

I was reading this to review the derivation of the ordinary least squares estimator but I'm having trouble differentiating (4). Can someone please help explain why $ \dfrac{\partial ...
1
vote
2answers
508 views

Exponential Regression Model

I need to model my data ($(x,y)$ pairs) using the following exponential function: $$f(x) = \exp((x + a)/b) - c$$ So, I need to find $a, b, c$ coefficients that are the best fit for my data. What is ...
1
vote
1answer
225 views

Approximation using Legendre polynomials

my aim is to fit data points by the use of Legendre Polynomials. Has anybody experience with this task? My final aim is to do this automatically with mathematica. Thanks, rainer
1
vote
2answers
468 views

How to do a regression with only integer values and a fixed intercept?

I need to write some code for an application that takes in a series of 2D points whose values are integers, and determines a polynomial regression that passes through the origin. I know how to do this ...
1
vote
2answers
84 views

Multiple regression with model $Y = (1 + c_1X_1)(1 + c_2X_2)\ldots(1 + c_nX_n)$

I'm currently working with data contained in $Y, X_1, X_2, \ldots, X_n$ and wish to fit it to the model: $Y = (1 + c_1X_1)(1 + c_2X_2)\ldots(1 + c_nX_n)$ where the $c_i$ are coefficients to be ...
1
vote
1answer
95 views

Orthogonal fitted values

I have two regression models $$Y=X\beta+\varepsilon,\quad \beta\in\mathbb{R}^k$$ $$Y=Z\alpha+u\quad \alpha\in\mathbb{R}^m$$ it is known that using OLS estimates $\hat{\beta},\hat{\alpha}$ fitted ...
1
vote
2answers
907 views

Log-likelihood gradient and Hessian

Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions: $f(x) = x^T \beta$ $p(x) = ...
1
vote
1answer
118 views

Econometrics OLS estimates

I cant seem to use the formula to calculate B1 without knowing xi and yi. Is it possible to calculate using just the variances and covariance? Please help! The classical linear regression model ...