Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
40 views

Size of sample and correlation coefficient

$X$ and $Y$'s correlation coefficient is $r=0.5$. What is the size of sample when the correlation is significant at $\alpha=0.05$ with two sided test? Is there a more "formal" way to solve this ...
1
vote
1answer
19 views

what is one basic/intermediate regression analysis standard textbook that is math intense

What is one basic/intermediate regression analysis standard textbook that is math intense with proofs/derivations? Also, i need that one to be comprehensive yet the diffculty is suitable for self ...
0
votes
1answer
40 views

Matrix form for Weighted Least Squares

If we have the following weighted least-squares regression, with $\hat{\beta} = (X'WX)^{-1}X'WY$ How can we express the squared errors, MSE and the fitted values in matrix form? These are the OLS ...
0
votes
1answer
33 views

How to find the deviated form of beta 1 in OLS

How to find the deviated form of beta 1 in OLS Y=β1+β2X+u estimated β1=(ΣX^2ΣY-ΣXΣXY)/(nΣX^2-(ΣX)^2) I do not know how to turn this part (ΣX^2ΣY-ΣXΣXY)into deviated form. I found that estimated ...
0
votes
0answers
37 views

If we make the lambda very small in ridge regression, why do the beta hats still decrease?

I understand that the betas estimated from minimizing the function (y hat - y)^2 + beta' beta decrease as lambda increase. However, looking at the picture below, why cannot we choose a lambda so ...
1
vote
1answer
25 views

Help Linear Regression

It would be really nice if someone could help me out and suggest me with his expert opinion. Going through the table and looking at Part (a) of this question. I believe I should use Scatter plot to ...
2
votes
1answer
44 views

Standard Error in OLS Regression

Assuming I have the following linear regression set-up: $y_i = \alpha + x_i * \beta + \epsilon_i$ for $i = 1,2,..., n$. When I run the regession, I get a $\beta$ and $\alpha$ estimates, along ...
0
votes
1answer
39 views

Multiple Regression with Categorical Predictor Variables of More than Two Levels

I'm planning on running a hierarchical multiple regression in SPSS. In the first step, I would like to enter demographic characteristics, second step continuous predictor variables of interest, and ...
0
votes
0answers
17 views

Simplifying $\sum_i[z_i'(\delta-\hat{\delta})]^2x_ix_i'$ to apply a law of large numbers

I'm in the context of linear regressions. Let $n$ be the sample size and for $i=1,\ldots,n$, let $$ \underbrace{x_i}_{K\times 1},\quad \underbrace{z_i}_{L\times 1}\quad(n>K\geq L) $$ be column ...
0
votes
0answers
25 views

multiple linear regression analysis and case of p>>n

In the case for simple linear regression and multiple linear regression, if we have p >> n, is it undefined? And in the case for ridge regression, p>>n requires $\lambda$ to increase b/c more ...
1
vote
0answers
29 views

Outlier Contained in Prediction Interval (Tme series Forecasting Problem)

In my stats class today, the professor was showing us some output from MINITAB on a prediction interval that was calculated (from time series data using standard linear regression). For one of the ...
0
votes
0answers
147 views

Inclined Elliptic Tank Volume Calculation

Can someone help me determine an equation for calculating the volume of an elliptical cylinder on it's side and inclined 5 degrees from horizontal? The tank has flat ends. I have found several ...
0
votes
1answer
44 views

For a general linear regression , are Y and Y hat independent?

For a general linear regression , are $Y$ and $\hat{Y}$ independent? $Y$=XB+e $\hat{Y}$=X$\hat{B}$ I think they are dependent, because if they rely on the same data, they should have some sort of ...
1
vote
1answer
40 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
0
votes
1answer
25 views

What is a common name for the resulting function?

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
1
vote
0answers
14 views

derive the distribution(a multiple regression problem)

(Multiple regression model with p's predictor variables.) Derive the distribution of $$\frac{(b-\beta)X'X(b-\beta)}{MSE\cdot p}$$ As far as I know, $b\sim N(\beta,\sigma^2 (X'X)^{-1})$ $b-\beta ...
0
votes
1answer
40 views

Proving an implication in a linear regression

Suppose we have a linear regression: $$ y_i=X_i'\beta+u_i,\quad i=1,\ldots,T. $$ Here $y_i$ and $u_i$ are scalars, $X_i$ and $\beta$ $k\times 1$. $\beta$ is a (non-stochastic) vector of ...
0
votes
1answer
16 views

Conditional expectation; regression

Assume that E[$y_{it}|c_i, x_{it}$] = $c_i$ + $x'_{it}\beta$ Eliminate $c_i$ by taking the expectation with respect to $c_i$, leading to E[$y_{it}|x_{it}$] = E[$c_i|x_{it}$] + $x'_{it}\beta$ ...
0
votes
0answers
23 views

Question about regression model

Suppose you fit (estimate the parameters of) a regression model, obtaining $\hat{Y}$, $\hat{B}$, and $\hat{E}$. And you fit a second regression model , using $\hat{Y}$ x matrix from previous model ...
0
votes
1answer
31 views

how to show for a simple regression with an intercept and one independent variable$ R^2 = r ^2$ , where $r$ is the ordinary correlation coefficient.

how to show for a simple regression with an intercept and one independent variable $R^2 = r ^2$, where $r$ is the ordinary correlation coefficient. Here is where I'm at. $R^2= ...
1
vote
0answers
19 views

Regression on even function?

Is there any test for whether or not the regression function is even? Suppose we have a model: $Y=g(X, \epsilon)$, where $Y, X$ are both one dimensional. My questions is how do we test for $g$ is an ...
1
vote
0answers
45 views

The Hessian Matrix I calculate is twice as much as it should be. Why?

I have a function "fkt." In this example, let it be as simple as $y=a \cdot x+b$. I have a real dataset with values obeying to the model. After regression of the points to the model, I find the ...
3
votes
1answer
85 views

When is Block-Partitioned Matrix Invertible?

Suppose I have a block partitioned matrix \begin{equation} \begin{bmatrix} \mathbf{X}_1^{\top}\mathbf{X}_1 & \mathbf{X}_1^{\top}\mathbf{X}_2 \\ \mathbf{X}_2^{\top}\mathbf{X}_1 & ...
0
votes
0answers
10 views

Multivariate regression analyisis with grouped data. How do I 'un-group'

I am trying to determine if either of two non-numeric variables effect the percentage of people who take a specific action. There is another variable that needs to be controlled for. In real world ...
0
votes
1answer
9 views

Do multiclass logistic regressions obey Kolmogorov's second axiom?

Logistic regressions were taught to me using the intuition that they approximate $\mathbb{P}(Y=y|x;\theta)$. Multiclass regressions use one-vs-all classification, selecting one $y$ and classifying ...
1
vote
2answers
46 views

Show $E\left(\mathbf{X}_i \otimes \mathbf{u}_i\right)=\mathbf{0}$ implies $E\left(\mathbf{X}_i^{\top}\mathbf{G}\mathbf{u}_i\right)=\mathbf{0}$

Let $\mathbf{X}_i$ be a $G \times K$ random matrix, and let $\mathbf{u}_i$ be a $G \times 1$ random vector, and suppose we have a sample of $i=1,\ldots,N$ of each. Suppose the following condition ...
1
vote
1answer
40 views

How to Change Summation Expression $\sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$ into Matrix Expression

Let $\mathbf{X}_i$ be a $G \times K$ matrix, and suppose are $i=1,...,N$ of these matrices. Note that \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{X}_i &= \begin{bmatrix} ...
1
vote
0answers
46 views

Which projection, in $L_\infty$ norm or $L_2$ norm, is non-expansion?

I am just wondering which projection is non-expansion? Basically, I am wondering if $F$ is a projection operator then which norm would satisfy the following non-expansion property, where for a given ...
1
vote
1answer
86 views

what is the difference between 'estimate of residual standard error' and 'residual standard error'?

What is the difference between 'estimate of residual standard error' and 'residual standard error'? Can someone please provide the formulas? Thanks!
1
vote
0answers
107 views

Matrix Decompositions: Difference between Cholesky Decomposition, Eigendecomposition and Jordan Normal Form Decomposition

I recently created a related topic about the square root matrix, in case you'd like to refer to that one. Here's what we want: Consider the matrix $\Omega=E(\mathbf{u}^{\top}\mathbf{u})$, where ...
0
votes
0answers
19 views

Matrix problem in Mixed Regression

the background is $y= X \beta +e$ y=n*1 X=n*p $\beta=p*1$ e=n*1 take singular value decomposition of X $X=P \Delta Q$ $\beta=QKP'y$ K is a digaonal matrix and depending on its form can represent ...
0
votes
0answers
18 views

Multivariate Regression

Suppose there are $n$ variables that map through a function to a single output variable $r$. Given a set of 50-100 data sets with accepted input and output values that satisfy this relation, is it ...
1
vote
2answers
120 views

Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

I know in general that if a matrix $A$ is positive definite, then there exists a (unique?) square root matrix $B$, which is also positive definite, such that $BB=A$. Therefore, suppose $A$ is ...
0
votes
0answers
6 views

2d spatio-time regression

I have this question which intuitively appears to be simple, but I couldn't find a solution to it. Imagine you have a ball moving in one direction (x) and that the measurements of the movement are ...
0
votes
1answer
52 views

linear regression, expectation and mean squared error

Let us assume that data is generated according to a true model $$y_i = \beta_{true}x_i + \epsilon_i$$ for $i = 1, ..., n$ Assume that $x_i$ are fixed, and $\epsilon_i$~ N(0, $\sigma^2$) ...
1
vote
0answers
19 views

Multiple regression model

I have a multiple regression equation which as four quarters (maybe called them as parameters) ...
2
votes
0answers
51 views

Predicting profit with price variation

I am currently working on a high school project that aims to predict profit from X amount of items to Y amount of profit based off a deviated sale price. For instance: I sale 10 cookies for 10 ...
1
vote
0answers
28 views

Round robin logistic regression

I have a poll with four answers (A,B,C,D) and response information about people who have taken that poll. I have created four models (one for each of the answers) in a one vs all. i.e. the model for ...
1
vote
0answers
51 views

Uni-variate Moving Average Theta coefficients

Consider the Uni-variate Moving Average Models (MA models) MA(1) $$x_t = \mu + w_t +\theta_1w_{t-1}$$ or the second order moving average MA(2) $$x_t = \mu + w_t +\theta_1w_{t-1}+\theta_2w_{t-2}$$ ...
-1
votes
1answer
19 views

Linear Regression question 1

I would really be grateful if someone could let me know how to answer Part (a) of Question 1. I believe i should scatter plot both x and y values separately for year 2000 and year 2001 on same graph ...
1
vote
0answers
12 views

Proving $Corr(\hat{e}_{ij}, \hat{e}_{jk}) = \frac{-1}{n_i-1}$ for $ j \neq k$

For the model of a single factor experiment: $y_{ij}= \mu + \alpha_i + e_{ij}$, $(1 \leq i \leq a, 1 \leq j \leq n_i)$, where a = the number of treatments, $n_i$ = the number of experimental units ...
1
vote
0answers
17 views

Principal Components vs Principal Directions

I'm trying to do statistical downscaling of some climate data and there is a module of principal component analysis by regression method required. I am confused with the different terms here. What is ...
0
votes
0answers
12 views

Inverting the inner product of a matrix with highly correlated columns

If matrix $X$ has highly correlated (but not exactly linearly dependent) columns, $X'X$ is still invertible but $(X'X)^{-1}$ will be large. Why is that the case? The motivation for this question ...
0
votes
1answer
25 views

Interpreting confidence interval of regression coefficient.

In a Simple Linear Regression analysis, independent variable is weekly income and dependent variable is weekly consumption expenditure. Here $95$% confidence interval of regression coefficient, ...
0
votes
1answer
25 views

linear regression model beta estimate

Suppose we want to estimate $\beta$ by minimizing $L(\beta)=\sum_{i=1}^n(y_i-\beta x_i)^2+\lambda|\beta|$, where $\lambda$ is a fixed positive constant. Calculate the estimate. How would I ...
0
votes
1answer
34 views

Linearize non linear function

Is it possible to linearize the function $f(x) = 1-exp(\frac{x}{b})$ so that one could use it in a linear regression?
0
votes
0answers
26 views

Question regarding Balanced Incomplete Block Design

Question: Consider a BIB design with a treatments, b blocks and c < a number of plots in each block where a,b,c ≥ 2. Let $n_{ij} = 1$ if an observation is made on the ith treatment in the jth ...
12
votes
6answers
1k views

Why get the sum of squares instead of the sum of absolute values?

I'm self-studying machine learning and getting into the basics of linear regression models. From what I understand so far, a good regression model minimizes the sum of the squared differences between ...
0
votes
2answers
36 views

When does Mean Square Error increase?

As far i know, we want the model to include as few regressors as possible because the variance of the prediction $\hat y$ increases as the number of regressor increases. But from the hald cement ...
2
votes
0answers
168 views

Normal equations for minimization of Frobenius norm least squares error

I'm having a hard time understanding the most efficient sequence of steps for deriving the normal equations for Frobenius norm least squares minimization. Here I want to minimize the norm of a matrix ...