Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
45 views

Questions about derivation of linear regression.

I had few questions about linear regression derivation. SSE = Sum i=1toN (yi - bo - b1xi)^2 In above example, i simply found values bo and b1 where SSE is minimum by finding partial derivates of ...
1
vote
0answers
33 views

How is the sum of squares of residuals divided by variance has a chi-square distribution with n-2 degrees of freedom?

I came across this while reading the Linear Regression chapter in Sheldon Ross's Book: So my doubt is that How can I prove that it is a chi-square distribution with degree n-2. I looked up a bit ...
2
votes
1answer
28 views

Linear regression and standardization

I am trying to use a linear regression to model an expected value Y for an input X. X and Y have a large difference between them, so I was converting to standard (z) score, doing my calculation ...
1
vote
1answer
33 views

Linear regression with a given (non zero) intercept

If I have a simple linear regression model with the intercept $\beta_0$ known, would the least squares estimator of $\beta_1$ be $\frac{\sum(y_ix_i)}{\sum({x_i}^2)} - ...
0
votes
1answer
36 views

E(b1)=beta1 regression

In the prove of $E[b_1]=\beta_1$ I saw this steps : $$ E[b_1]=E[\frac{S_{xy}}{S_{xx}}]=E[\sum[\frac{[(x_i - \bar{X})*y_i]}{\sum(x_i-\bar{X})}]]=\frac{\sum[(x_i-\bar{x})*E(y_i)]}{\sum(x_i-\bar{X})} ...
0
votes
1answer
69 views

When fitting a polynomial to data points, how to determine the reasonable degree to use?

I have wondered the following: Suppose that there is a set of data points $(x_i,y_i)$. Then I would like to know if it is more reasonable to assume if there is a polynomial relation of degree $m$ ...
1
vote
0answers
18 views

Penalty function of multi-peak fit?

The question I have is about the answer from here by @Silvia: http://mathematica.stackexchange.com/questions/26336/how-to-perform-a-multi-peak-fitting I can only understand some of the code but the ...
1
vote
2answers
191 views

Sxx in linear regression

What is the meaning of 'Sxx' and 'Sxy' in simple linear regression? I know the formula but what is the meaning of those formulas?
0
votes
0answers
14 views

What method to apply to center value in multiple linear regression?

This a multiple linear regression question, an approach for modeling the relationship between a scalar dependent variable $Y$ and several explanatory variables (or independent variables) denoted $X$. ...
0
votes
1answer
86 views

How to prove that non-diagonal elements of hat matrix (from regression) are limited?

I want to prove this inequality: $$h_{ij}^2 \le 0.25$$ where $h_{ij}$ is an element of hat matrix $H = X(X'X)^{-1}X'$ from multiple linear regression model ($ Y = X\beta + \epsilon$, $X$ is a $n\times ...
0
votes
0answers
38 views

When the predictor variable is so coded that $\bar X = 0$ and the normal error regression model applies, are $b_0$ and $b_1$ independent?

The Statement of the Problem: When the predictor variable is so coded that $\bar X = 0$ and the normal error regression model applies, are $b_0$ and $b_1$ independent? Are the joint confidence ...
1
vote
1answer
18 views

How to compute F test in a NLM?

I tried to do the F test for NLMs by myself and run into a dead-end. I have a linear model with normal distributed error $$Y = X\beta + e$$ I know that the test statistic $$F = ...
2
votes
2answers
41 views

Interpreting OLS Regression Coefficients with High Multicolinearity

I am having trouble understanding the interpretation of OLS coefficients when predictors are highly correlated. My understanding of OLS coefficients is that they estimate a change in the expected ...
0
votes
1answer
27 views

standardised random variable least square regression $X$ against $Y$, $Y$ against $X$

Let $X$ and $Y$ be mean 0 and variance 1 random variables such that we choose $\alpha$ and $\beta$ to minimise $$\mathbb{E}(X-\beta Y)^2$$ and $$\mathbb{E}(Y-\alpha X)^2$$ after not so difficult ...
1
vote
0answers
34 views

Please show that $f(\beta_0,\beta_1)=\log(1+\operatorname{exp}(-y_1(\beta_0+\beta_1 x_1)))+\log(1+\operatorname{exp}(-y_2(\beta_0+\beta_1 x_2)))$

I would like to show that the following result is indeed true. I am very new with this subject, so I ask for a hint to get me started please. Please show that ...
0
votes
0answers
40 views

Estimating a probability distribution by fitting a function to a frequency histogram

If I want to estimate a probability distribution, is it common practice to simply fit a function to a frequency histogram? So, I am training a classifier, the performance of which is evaluated by its ...
1
vote
2answers
85 views

Why is $R^2=\rho^2$

Considering $y_i=\beta_1+\beta_2x_i+\epsilon_i$ $\bar y_i=\hat\beta_1+\hat\beta_2\bar x_i+\bar\epsilon_i$ a linear equation of least square used when it seems that there is a like between two data, ...
1
vote
0answers
12 views

How to get the relative contribution of each variable in a difference that forms the numerator?

I am facing a problem that may seem simple at first but with which I struggle. The question relates to economics where I try to see the effect of deficit at time t ($D_t$) onto Output at time t+h ...
0
votes
0answers
31 views

variances of the slope and intersect of an orthogonal / Deming) linear regression

I am a humble tinkerer who tries to get a rover to run a SLAM (simultaneous localization and mapping) process in his house. I equipped the rover with a laser rangefinder which collect distance and ...
2
votes
1answer
41 views

Discrete version of continuous SIR model

I'm working with a SIR infection model, which is $$\begin{array}{rcl} \frac{dS}{dt} & = & -\beta IS\\ \frac{dI}{dt} & = & \beta IS-\gamma I\\ \frac{dR}{dt} & = & \gamma I ...
1
vote
0answers
55 views

Least-squares fit of a nonlinear (polar) system

I want to determine the six unknown coefficients (uppercase letters) of the model $$x=X_c+(Au+B)\cos(Cv+D),\\y=Y_c+(Au+B)\sin(Cv+D)$$ given a set of data $(x_k,y_k,u_k,v_k)$, by least-squares. As ...
0
votes
2answers
93 views

Nonlinear LS regression

• Problem formulation I have to fit the following nonlinear model to a dataset: $$f(x)=\frac{C_1 \cdot a}{a^2 + C_2 \cdot x^2}$$ $a$: fitting parameter $C_1, C_2$: Given constants I can't apply ...
0
votes
0answers
34 views

Hypothesis testing involving regression variable

Consider the regression model $Y = \textbf{X}\beta + \epsilon$, where $\epsilon \sim N(0, \sigma^2I_n)$, $\beta = (\beta_0, \beta_1, \dots, \beta_{10})^{T}$ Construct a test with significance level ...
0
votes
1answer
38 views

For linear regression: compute $\Theta T X$

I have started learning linear regression and the equation $h(X) = \Theta T X$ has puzzled me. Let's say we have a training set of $m$ and $n$ features such that $X$ is a $m \times n$ matrix. ...
1
vote
1answer
37 views

small sample (1 point) MLE estimation

I'm trying to estimate $m$ in $y = mx+e$ from a single $(x,y)$ observation. $m,x,e$ are all $N(0,1)$ and independent. If I do a Monte-Carlo estimation on say $(1,3)$ I get about $1.5$. This seems ...
0
votes
0answers
37 views

Linear regression where explanatory variable of 0 has no meaning

I want to build a predictive model, where given a few numeric explanatory variable n1, n2, ...
1
vote
1answer
53 views

Intuition for least square regression line involving joint distribution

Let $X$ and $Y$ be random variables of the continuous type having the joint pdf $f(x,y) = 8xy$, with $0 \leq x \leq y \leq 1$. Determine the equation of the least square regression line. Does the ...
0
votes
0answers
36 views

Prove multiple OLS t-test follows t-distribution

I'm trying to prove the multiple regression test has a t distribution, i.e.: $\frac{\hat B_j - Bj}{se(\hat B_j)} \sim t (df=n-k-1)$ I was able to prove $\frac{\hat B_j - B_j}{sd(\hat B_j)} \sim ...
0
votes
0answers
11 views

Problems getting the covariance matrix of the ressiduals

In order to get the variance-covariance matrix of the residuals of a linear regression model, I do the following: Considering that the residual vector $e$ is: $e = Y - \hat{y} = XB+\epsilon - Xb$ ...
1
vote
0answers
21 views

Comparing an isotonic model to an additive model

Say I have a dataset in $x,y$, and say I fit a few different models to the dataset. Examples could be an isotonic regression, a smoothing spline and a simple linear regression. What are some ways I ...
0
votes
0answers
24 views

Why is $\hat y_i=\hat\beta_1-\hat\beta_2x_i$?

I'm trying to show that $\hat\sigma^2 =\frac{\sum\hat\epsilon^2}{n-2}$ is an estimator without biais and I started with: $$\hat\epsilon_i=y_i-\hat y_i$$ and my teacher suggested me to use the ...
1
vote
1answer
142 views

Expected mean squared error and MSR

In a small-scale regression study, five observations on $Y$ were obtained corresponding to $X = 1,4,10, 11$, and $14$. Assume that $\sigma=0.6,B_0=5,B_1=3$ a. What are the expected values ...
0
votes
0answers
73 views

Prove that OLS estimator of the intercept has minimum variance

Let $$y_i=B_0+B_1X_i+\epsilon_i$$ where $\epsilon_i\sim > N(0,\sigma^2)$. Find the least squares estimator of $B_0$ and show that it is unbiased and has minimum variance. I will not write in ...
1
vote
0answers
46 views

Linear Least Squares vs. Ordinary Least Squares

My understanding is that Ordinary Least Squares (Usually taught in Statistics classes) uses the vertical distance only when minimizing error/residuals (see Wikipedia for Ordinary Least Squares) with a ...
0
votes
1answer
102 views

polynomial curve fitting: higher order models' root mean square error does not decrease

I am trying to fit a curve for 15 data points. I started by creating a linear model and observing the root mean square difference, followed by quadratic, cubic and increasing the degree of polynomial ...
0
votes
0answers
18 views

What is the algorithm for training and testing a logistic regression model using Newton's method and l2 regularization?

I have a spam dataset with 57 features and 3065 data points. I also have test data with around 1500 data points. The classes are spam/non-spam. I have to fit a logistic regression model on MatLab ...
0
votes
0answers
25 views

A property of Piece-wise continuous simple linear regression model

My fellow members I attempted to model the growth of capital of a small business person who does business with the aim of just raising his or her capital, as follows: Assumptions A fixed capital ...
1
vote
1answer
80 views

Statistical property proof

Consider the multiple linear regression model $y = Xβ + ε$, where $X$ is the $n × p$ design matrix, $y$ is the $n × 1$ dimensional vector of response and $ε ∼ N(0,σ^2I)$. The vector consisting of the ...
0
votes
0answers
20 views

Piecewise linear regression with a given cost function

If we're given a cluster of $(x, y)$ values that appear non-linear [1](example image), we wish to partition the set of points into $r$ sets of continuous points [2] and then find regression lines on ...
0
votes
2answers
27 views

How to understand this multiple regression question without having an example in the textbook?

How to understand this multiple regression question without having an example in the textbook? Please briefly show how to do the final question and give the answer to the final question.
0
votes
0answers
17 views

Proof reviewing: If z¹ and z² two random variables get by two $x_i$ centered and reducted, what may be $ρ_z¹_z²$ in terms of $ρ_x¹_x²$.

Let be z¹ and z² two random variables get by x¹ and x² centered and scaled, give the expression of $ρ_z¹_z²$ in terms of $ρ_x¹_x²$. what I know: the Matrix $R$ of correlation is $$\left\{ ...
1
vote
1answer
51 views

Difference between $\sigma^2\{\text{pred}\}$ and $\sigma^2\{\hat Y_h\}$?

Can someone explain this to me? I've read the relevant section of the text about a million times, and it was even explained in class, but I can't seem to wrap my head around it. The Statement of the ...
0
votes
0answers
24 views

Comparing ridge regressions

I am applying ridge regression to biological problem and using the ridge coefficients as measure of explanatory power of each predictor. I have 2 sets of reponse variables with 2 matching sets of ...
0
votes
0answers
21 views

Explanatory power of a multiple regression function

What is the explanatory power of a multiple regression model? How is the explanatory power measured?
0
votes
0answers
12 views

Is division by $\sum x_i-\bar{x}$ actually null?

I'm trying to find out what are $\hat{β_1}, \hat{β_2}$ $ \left \{ \begin{array}{c @{=} c} \frac{∂S( \hat{β_1}, \hat{β_2})}{∂S \hat{β_1}} =-2\sum(yi − \hat{β_1} − \hat{β_2}xi) = 0, \\ ...
0
votes
0answers
22 views

Showing that a linear regression by method of leasts squares exists for any set of $n \geq 2$ points

As part of a project for school I am conducting a statistical study that involves the use of linear regressions on many sets of points. Before being able to apply my results using a computer programme ...
1
vote
0answers
29 views

what does is mean by 'overfitting' of data?

I have the following equation that is to be estimated: y = a + bA + cB + dC + eF + dG + e and i got 2 other additional variables, fH and gI, that i do not wish to add in.. can i reason this out by ...
2
votes
2answers
55 views

Fitting two parallel lines to a set of points

In two dimension I have a set of points X = $\{x_1,..., x_N\}$. I want to fit two parallel lines to these points like $l_1$ and $l_2$ $$l_1 = p_1 + \lambda n^\perp$$ $$l_2 = p_2 + \lambda n^\perp$$ ...
0
votes
0answers
19 views

distribution of linear regression predicted values

Let $\tilde{y}_*$ the prediction for a new observation at level $X = x_*$. Assume that $\sigma^2$ and $\tilde{y}_*$ are independent. Show that $$\frac{\tilde{y}_* - y_*}{se(\textrm{pred})(\tilde{y}_* ...
0
votes
0answers
23 views

3 variables 'linked' to one outcome (for lack of better words)

Paper route salary formula (reverse engineer...again for lack of a better word;) Ok suppose I have these variables: Pieces of mail : 302 Number of stops: 177 TotalWeight* : 272 Hours allocated: ...