0
votes
1answer
32 views

Derivative of an exponentially weighted moving average

It has been a while since my university math courses, so let me apologize right off the bat... I'm using GSL to perform non-linear regression analysis and am mostly happy with the outcome, however, ...
5
votes
2answers
54 views

Update a regression on the fly?

Say I have 100 people each with a height, weight, and age. I make a regression that predicts age based on height and weight. Now, I would like to update that model when I meet someone new. I don't ...
0
votes
0answers
18 views

R squared (Proportion of variance explained) in terms of conditional variance?

My question concerns a comparison between 2 models in terms of proportion of variance explained. Let $y_{t+1}$ denotes the variable I want to explain or predict and $\mathcal{F}_t$ the information ...
0
votes
0answers
22 views

Unconditional sampling distribution of regression coefficients

I am trying to find the (unconditional) sampling distribution of a regression coefficient in a simple linear regression. The linear regression $Y = \beta_0 + X\beta_1 + \epsilon$ is conducted for $N$ ...
0
votes
2answers
35 views

Testing if $X_1$ has an influence of $Y$

Consider you have the suspicion that $Y$ is influenced by two attributes $X_1$ and $X_2$: $$ Y=\theta_0+\theta_1X_1+\theta_2X_2+\theta_3 X_1X_2+U $$ The following data are given. Test ...
0
votes
1answer
19 views

When residual standard error is equal to standard deviation of dependent variable in linear regression?

I wonder when residual standard error is equal to standard deviation of dependent variable in linear regression? Could someone provide some information on this topic and explanation?
0
votes
1answer
29 views

Is the sum of predicted y values equal to the sum of actual y values?

Say I have a set of points Y and I want to accuratly predict the values of Y by using three variables X1,X2,X3. Hence my equation is Y=intercept + C1*X1 + C2*X2 + C3*X3 After performing linear ...
0
votes
0answers
34 views

interpretation of ANOVA using RStudio

This is my first post so go easy on me... So I have this table as 'tomato' and I have come up with some linear models to try out which are given below... however I don't know how to interpret the ...
0
votes
0answers
13 views

Conditional Expectation, Orthogonality, and Correlation

I know that if $\epsilon$ and $x$ are independent, then $E[\epsilon|x]=E[\epsilon]$ and Cov$(\epsilon,x)=0$. However, $E[\epsilon|x]=E[\epsilon]=0$ implies Cov$(\epsilon,x)=0$ iff $\epsilon$ and $x$ ...
2
votes
1answer
44 views

Trigonometric regression

What methods are performed for regression with trigonometric functions? E.g. : Sequence: $$-1, 0, 1, -1, 0, 1, \text{.....}$$ Regression: ...
3
votes
2answers
63 views

Why Linear Regression

First i will like to say that i am not a statistician nor am i good in the field. I have been collecting data for over a period of e.g 100 days and each day has a varying amount of data that i can ...
0
votes
2answers
21 views

Strong vs weak relationship in this correlation

I produced this plot and regression line in R and I thought my results were quite odd. Is the relationship of the correlation determined by how steep the regression line is? So in this case it isn't ...
0
votes
1answer
22 views

Logistic Regression derivation

From the Wikipedia article http://en.wikipedia.org/wiki/Multinomial_logistic_regression: $ln \frac{\Pr(Y_i=1)}{\Pr(Y_i=K)} = \beta_1 \cdot \mathbf{X}_i $ $ln \frac{\Pr(Y_i=2)}{\Pr(Y_i=K)} = ...
2
votes
0answers
90 views

Is it compulsory to make transformation to the econometric model in order to have only diagonal elements on variance-covariance matrix of errors?

I need some sharped and advanced advices for the following issue ... Model and its assumptions I'm working on the methodology of a two-way error component model. Here is the model: $y_{jis} = ...
0
votes
0answers
19 views

Expected in-sample error of linear regression with respect to a dataset D

In my textbook, there is a statement mentioned on the topic of linear regression/machine learning, and a question, which is simply quoted as, Consider a noisy target, $ y = (w^{*})^T \textbf{x} + ...
0
votes
1answer
21 views

Expected error of best possible linear fit?

I asked the following question on stat SE, but I could not get a mathematically rigorous answer, and I have decided to ask here again. In my textbook, there is a statement mentioned on the topic of ...
0
votes
1answer
21 views

Is it a wrong expression for the local log-likelihood of logistic regression?

In page 206 of the book 'Elements of statistical learning', the author wrote: The local log-likelihood for this $J$ class model can be written $\sum_{i=1}^NK_\lambda (x_0, ...
0
votes
0answers
9 views

Find sample size underlying these regression results.

I have been trying to work out the answer to this question and have been having no luck, so hopefully you can help. The questions asks you to find sample size from two regression results as below: ...
1
vote
1answer
34 views

Does more data give you a better forecast?

Say I have a large set of data. Each data point corresponds to a particular day in the year, so for 1 year I will have 365 points. Say I have collected this sort of data for 5 years. Now, I want to ...
0
votes
0answers
17 views

stats project - good model, what to do with it?

I've recently been working on a stats project for school. I have been comparing a country's 'quality of life index' with 'moral' opinions survey to see if there are relations. Here's some example ...
0
votes
0answers
42 views

Help larry water his tomato plants with math

I have a bit of a real world problem that I believe Math can help me solve. I think it might be easiest to phrase in a manor similar to that of high school textbook. Larry has a device that can ...
0
votes
1answer
14 views

Variance of Estimated Coefficients in Logistic Regression

I have a logistic regression model with a binary variable as the response and a categorical variable with 3 categories as a predictor. The fitted model is: logit(P(Y=1)) = intercept -0.19*C2 + ...
1
vote
1answer
31 views

Calculating R-squared with duplicate data

I have the following question regarding the proper usage of R-squared value. Say I have an equation, that predicts energy consumption for the month of a building. One of the input variables accounts ...
3
votes
0answers
36 views

Determine whether ARMA(p,q) is stationary and/or invertible?

Determine whether an ARMA(p,q) process is stationary and invertible such that $y_t = \sum_{i=1}^{p} \phi_i y_{t-i} + \sum_{i=1}^{p} \theta_{i} \epsilon_{t-i}$ with the restriction that ...
0
votes
0answers
17 views

multicollinearity with intervals

You have multicollinearity when you have 2 variables (X1,X2) that have a relationship, X1=a+X2 where a is constant. My question is: is there still a multicollinearity issue if a is not constant, ...
1
vote
1answer
53 views

Intuition and the math behind normalization

What exactly is the purpose of normalization. From what I read, it is to adjust two different sets of values so you can compare them, but I don't understand why, nor the math behind it. Could anyone ...
1
vote
0answers
11 views

Proofs on regression analysis

How can I prove: 1) estimating population variance $\hat\sigma^2={1 \over n-2}[S_{YY}-{S^2_{XY} \over S_{XX}}]$. 2)expected value of error mean square=$E(EMS)=\sigma^2$ To prove (2): I showed that ...
2
votes
0answers
26 views

What is ${\rm cov}(e_i, \hat y_i)$ in simple linear regression?

The model is $y_i = \beta_0 + \beta_1x_i + \epsilon_i$ What is ${\rm cov}(e_i, \hat y_i)$? What is ${\rm cov}(\epsilon_i, \hat \beta_1)$? What is ${\rm cov}(e_i, \epsilon_i)$? For 1, I am writing ...
0
votes
0answers
20 views

Estimating variance in least squares in regression

How to show $\hat\sigma^2=$$1\over n-2$$[S_{yy}-$$S^2_{xy}\over S_{xx}$]. I don't know the matrix form in regression.I don't even understand how to begin this
1
vote
1answer
26 views

properties of least square estimators in regression

$Y_i=\beta_0+\beta_1 X_i+\epsilon_i$ where $\epsilon_i$ is normally distributed with mean $0$ and variance $\sigma^2$ . The least square estimators of this model are $\hat\beta_0$ and $\hat\beta_1$. ...
0
votes
3answers
62 views

Reliability of linear regression to predict future

When we have a set of data, where X is the cause, and Y is the effect, we can use linear regression to predict values for Y, based on values of X. I have learned that you may only safely apply this ...
0
votes
3answers
26 views

Scatter plot : Are the two observed data related

I have a regression question that ask to draw the scattered plot graph and then conclude if the two data lists are related. The two data lists are years of people and their cholesterol level. I went ...
0
votes
0answers
24 views

Identification of linear regression function under $\ell_1$-norm criterion.

Consider a linear system \begin{align*} y = \theta^Tx + e, \end{align*} where $x\in\mathbb{R}^n$ and $e\in\mathbb{R}$ are independent Gaussian random variables with distribution $\mathcal{N}(0,I_n)$ ...
0
votes
0answers
19 views

Deming estimator

Can someone help me prove that Deming estimator $\hat{\beta_{1}}$ is monotone in $\delta$: (See https://en.wikipedia.org/wiki/Deming_regression). When deriving the estimator w.r.t $\delta$, I find ...
1
vote
0answers
18 views

Logistic regression eye treacting data (need model)

I have two sets of time course data, they are for an eye-tracking study. The data is 20 100ms chunks, one category being percent fixations for canonical sentences, and the other being percent looks ...
1
vote
2answers
73 views

derivation of simple linear regression parameters

I know there are some proof in the internet, but I attempted to proove the formulas for the intercept and the slope in simple linear regression using Least squares, some algebra, and partial ...
2
votes
0answers
30 views

Smallest set of Liner equations, which exactly fit a set of points

I have a set of 2-d points,(it can be of any arbitrary dimension n). I want to find the minimum set of straight lines(linear equations) which exactly passes through the given 2-d points (unlike ...
0
votes
1answer
53 views

How to interpret coefficients in polynomial regression?

I am working on my thesis (study) about poverty incidence rate and its socio-economic factors using second-order polynomial regression without interaction. The final model in my study is ...
0
votes
2answers
27 views

Prove $\sum_i \frac{\bar{x}(x_i-\bar{x})}{nS_{xx}} =0$

Prove $$\sum_i \frac{\bar{x}(x_i-\bar{x})}{nS_{xx}} =0$$ That is one detail in a proof of the variance of the intercept $\alpha$ in the simple linear regression $Y_i=\alpha+\beta x_i$.
1
vote
2answers
396 views

Prove $SST=SSE+SSR$

Prove $$SST=SSE+SSR$$ I start with $$SST= \Sigma (y_i-\bar{y})^2=...=SSE+SSR+ \Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )$$ and I don't know how to prove that $\Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )=0$ a ...
1
vote
1answer
53 views

regression on circular data

How would one design a regression where the dependent variable is measured in degrees on a circle? The dependent variable is on the range [0, 360), and the independent variables are demographic ...
0
votes
1answer
16 views

Regression concepts clarified.

I want to understand regression in a clearer way. Suppose I assume the relationship between $Y$ and $X$ is linear. The regression equation I posit is: $Y_{i} = b_{0} +b_{1}X + e_{i}$. This means that ...
0
votes
0answers
26 views

Multivariate Linear Regression for a System of Linear Equations

I have a system of linear equations in the form of $A\vec{x}=\vec{y}$, where $A$ is an $n\times n$ matrix and $\vec{x}$ and $\vec{y}$ are $n\times 1$ matrices. Suppose $\vec{x}$ and $\vec{y}$ ...
1
vote
1answer
32 views

What's the “ridge” in Ridge Regression?

In normal least squares, we try to find $\hat\beta$ which minimizes $$\|y-X\beta\|^2$$ Ridge regression expands this to "penalize" certain values of $\beta$ via a matrix $\Gamma$: ...
0
votes
0answers
17 views

Multinomial Logistic Regression with an Unknown Number of Categories

The problem I'm facing involves determining the number of categories in a data set in a principled way. I feel that logistic regression may be the right tool for this project, but is it possible to do ...
2
votes
1answer
48 views

Fitting a simple linear regression

A professor in the School of Business in a university polled a dozen colleagues about the number of professional meetings they attended in the past five years $x$ and the number of papers they ...
0
votes
1answer
30 views

Estimate correlation coefficient of unknown variable

Consider variable y depends on variable x and z linearly. I have $100$ sample values of $y$ and corresponding $x$ but don't have any values of $z$. The functional model is $$y = \alpha_1x + \alpha_2z ...
1
vote
0answers
36 views

Linear Regression Question (Linear Algebra) Help!!

Hey guys, I have a quick question. I am trying to prove that the squared sample correlation between fitted and observed values is equal to $R^2$ (coefficient of determination). I am having a lot of ...
1
vote
1answer
48 views

Variance of Coefficients in a Simple Linear Regression

I have a linear regression model $\hat{y_i}=\hat{\beta_0}+\hat{\beta_1}x_i+\hat{\epsilon_i}$, where $\hat{\beta_0}$ and $\hat{\beta_1}$ are normally distributed unbiased estimators, and ...
2
votes
2answers
81 views

connection between PCA and linear regression

Is there a formal link between linear regression and PCA? The goal of PCA is to decompose a matrix into a linear combination of variables that contain most of the information in the matrix. Suppose ...