Tagged Questions

Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
0answers
5 views

Regression question (details inside). Measuring the incremental impact on the dependent variable of one category over other categories

Formulate a regression equation you would use to test for the differences in ROE between firms that used tier 1 investment banks as their advisors and those that used tier 2 or tier 3 banks (note: ...
0
votes
1answer
30 views

Optimization Problem (Linear Algebra)

I am not trying to cheat or anything, so any reference to online literature or MOOCs, that teach this stuff, will be highly appreciated. The problem is to prove that the following optimization ...
1
vote
0answers
12 views

Statistics linear modelling in R

Suppose I have a date set of the form: Test Subject/Sex/No. of mistakes made in the morning/ No. of mistakes made in the afternoon A / M / 2 / 5 B / F / 1 / 4 C / M / 3 / 5 D / F / 1 / 5 Suppose ...
0
votes
0answers
22 views

$E(\bar{Y})=\bar{Y}$? [Linear Regression]

A book, I am reading, derives the covariance between $b_0$ and $b_1$ as follows: By definition, $$Cov(b_0,b_1)=E[(b_0-Eb_0)(b_1-Eb_1)]$$ $$=E[(b_0-\beta_0)(b_1-\beta_1)]$$ ...
1
vote
0answers
11 views

Cannot figure out autocavariance

The moving average model of order q has the form $$Y_t =β_0 +e_t +b_1e_{t−1} +b_2e_{t−2} +...+b_qe_{t−q}$$ where $e_t$ is a serially uncorrelated random variable with mean $0$ and variance $σ^2_e$. ...
1
vote
0answers
14 views

Time series regression

What if I have a stationary independent variable and 2 non-stationary dependent variables, and I want to run a regression on them, what model is the most appropriate?
-2
votes
2answers
27 views

Correlation coefficient from regression line [on hold]

I encountered the below question in one of the interviews. Question : Suppose you have a random variable P and you define a new random variable $Q$ such that $Q = 2 - 3 P$. Calculate the value of ...
0
votes
0answers
20 views

Deriving single linear regression parameters in terms of multiple linear regression parameters

Suppose the true population model is ln(wage) = B0 + B1(education) + B2(experience) + v (v is error term) Suppose the model is estimated as ln(wage) = B3 + B4(education) + u How do I calculate ...
0
votes
1answer
25 views

Is there such a thing as a combination of linear and non-linear regression in one form?

Let's say I have a dataset D with many variables. I can get a multiple linear regression from that in the form ...
2
votes
1answer
32 views

How to take the derivative of Matrices

I was browsing the derivation of the Least Squares estimates and stumbled about this problem. It said that: $$E = (Y + XB)^2$$ $$\frac{dE}{dB} = -X^TY + X^TXB$$ It is to my understanding that the ...
1
vote
0answers
15 views

Derive the F-statistic for this model

Problem: Consider the model $y=\beta+\varepsilon$, where $\varepsilon\sim N_4(0,\sigma^2I)$, and $\sum_{i=1}^4\beta_i=0$. Derive the F-statistic for testing $H_0:\beta_1=\beta_2$. I tried to write the ...
1
vote
1answer
36 views

Size of sample and correlation coefficient

$X$ and $Y$'s correlation coefficient is $r=0.5$. What is the size of sample when the correlation is significant at $\alpha=0.05$ with two sided test? Is there a more "formal" way to solve this ...
0
votes
0answers
6 views

variance of regression coefficient of residual vector

y=ax+bz+u where x,y,and z are n*1 observation vector and u is an error vector.(x and z are independent.) Assume that x and z are non-random and that u has zero mean and covariance matrix σ^2In Let ...
1
vote
1answer
15 views

what is one basic/intermediate regression analysis standard textbook that is math intense

What is one basic/intermediate regression analysis standard textbook that is math intense with proofs/derivations? Also, i need that one to be comprehensive yet the diffculty is suitable for self ...
0
votes
1answer
24 views

Matrix form for Weighted Least Squares

If we have the following weighted least-squares regression, with $\hat{\beta} = (X'WX)^{-1}X'WY$ How can we express the squared errors, MSE and the fitted values in matrix form? These are the OLS ...
0
votes
1answer
29 views

How to find the deviated form of beta 1 in OLS

How to find the deviated form of beta 1 in OLS Y=β1+β2X+u estimated β1=(ΣX^2ΣY-ΣXΣXY)/(nΣX^2-(ΣX)^2) I do not know how to turn this part (ΣX^2ΣY-ΣXΣXY)into deviated form. I found that estimated ...
0
votes
0answers
9 views

If we make the lambda very small in ridge regression, why do the beta hats still decrease?

I understand that the betas estimated from minimizing the function (y hat - y)^2 + beta' beta decrease as lambda increase. However, looking at the picture below, why cannot we choose a lambda so ...
1
vote
1answer
19 views

Help Linear Regression

It would be really nice if someone could help me out and suggest me with his expert opinion. Going through the table and looking at Part (a) of this question. I believe I should use Scatter plot to ...
-1
votes
0answers
17 views

Understanding Linear Regression

Hello all. It's really good to be in here and I appreciate greatly to all the gurus who help people with their knowledge. It would be really nice if someone could help me out and suggest me with ...
2
votes
1answer
31 views

Standard Error in OLS Regression

Assuming I have the following linear regression set-up: $y_i = \alpha + x_i * \beta + \epsilon_i$ for $i = 1,2,..., n$. When I run the regession, I get a $\beta$ and $\alpha$ estimates, along ...
0
votes
1answer
28 views

Multiple Regression with Categorical Predictor Variables of More than Two Levels

I'm planning on running a hierarchical multiple regression in SPSS. In the first step, I would like to enter demographic characteristics, second step continuous predictor variables of interest, and ...
0
votes
0answers
12 views

Simplifying $\sum_i[z_i'(\delta-\hat{\delta})]^2x_ix_i'$ to apply a law of large numbers

I'm in the context of linear regressions. Let $n$ be the sample size and for $i=1,\ldots,n$, let $$ \underbrace{x_i}_{K\times 1},\quad \underbrace{z_i}_{L\times 1}\quad(n>K\geq L) $$ be column ...
0
votes
0answers
13 views

multiple linear regression analysis and case of p>>n

In the case for simple linear regression and multiple linear regression, if we have p >> n, is it undefined? And in the case for ridge regression, p>>n requires $\lambda$ to increase b/c more ...
1
vote
0answers
16 views

Outlier Contained in Prediction Interval (Tme series Forecasting Problem)

In my stats class today, the professor was showing us some output from MINITAB on a prediction interval that was calculated (from time series data using standard linear regression). For one of the ...
0
votes
0answers
10 views

Algorithm does not converge in R [migrated]

I am doing a logistic regression in R, where I am modeling how potholes and weather correlate to accidents. When I run a logistic regression, I get the message "Algorithm does not converge" The ...
0
votes
0answers
33 views

Inclined Elliptic Tank Volume Calculation

Can someone help me determine an equation for calculating the volume of an elliptical cylinder on it's side and inclined 5 degrees from horizontal? The tank has flat ends. I have found several ...
0
votes
0answers
18 views

If you control some variable in a regression , how would you find what proportion of the remaining variation?

If you control some variable in a regression , how would you find what proportion of the remaining variation that is explained by the controlled variable? so that is to say , we have variables x1 x2 ...
0
votes
1answer
36 views

For a general linear regression , are Y and Y hat independent?

For a general linear regression , are $Y$ and $\hat{Y}$ independent? $Y$=XB+e $\hat{Y}$=X$\hat{B}$ I think they are dependent, because if they rely on the same data, they should have some sort of ...
1
vote
1answer
23 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
0
votes
1answer
23 views

What is a common name for the resulting function?

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
1
vote
0answers
9 views

derive the distribution(a multiple regression problem)

(Multiple regression model with p's predictor variables.) Derive the distribution of $$\frac{(b-\beta)X'X(b-\beta)}{MSE\cdot p}$$ As far as I know, $b\sim N(\beta,\sigma^2 (X'X)^{-1})$ $b-\beta ...
0
votes
1answer
36 views

Proving an implication in a linear regression

Suppose we have a linear regression: $$ y_i=X_i'\beta+u_i,\quad i=1,\ldots,T. $$ Here $y_i$ and $u_i$ are scalars, $X_i$ and $\beta$ $k\times 1$. $\beta$ is a (non-stochastic) vector of ...
0
votes
1answer
11 views

Conditional expectation; regression

Assume that E[$y_{it}|c_i, x_{it}$] = $c_i$ + $x'_{it}\beta$ Eliminate $c_i$ by taking the expectation with respect to $c_i$, leading to E[$y_{it}|x_{it}$] = E[$c_i|x_{it}$] + $x'_{it}\beta$ ...
0
votes
0answers
10 views

For the general linear model, what is the distribution of $C\hat{B}$? Note $C$ is $q \times (k+1)$.

For the general linear model, what is the distribution of $C\hat{B}$? note $C$ is $q \times (k+1)$. So as far as I know, the general linear model is $\hat{Y}=X\hat{B}+e$. I don't understand why the ...
0
votes
0answers
19 views

Question about regression model

Suppose you fit (estimate the parameters of) a regression model, obtaining $\hat{Y}$, $\hat{B}$, and $\hat{E}$. And you fit a second regression model , using $\hat{Y}$ x matrix from previous model ...
0
votes
1answer
28 views

how to show for a simple regression with an intercept and one independent variable$ R^2 = r ^2$ , where $r$ is the ordinary correlation coefficient.

how to show for a simple regression with an intercept and one independent variable $R^2 = r ^2$, where $r$ is the ordinary correlation coefficient. Here is where I'm at. $R^2= ...
1
vote
0answers
17 views

Regression on even function?

Is there any test for whether or not the regression function is even? Suppose we have a model: $Y=g(X, \epsilon)$, where $Y, X$ are both one dimensional. My questions is how do we test for $g$ is an ...
1
vote
0answers
31 views

The Hessian Matrix I calculate is twice as much as it should be. Why?

I have a function "fkt." In this example, let it be as simple as $y=a \cdot x+b$. I have a real dataset with values obeying to the model. After regression of the points to the model, I find the ...
3
votes
1answer
29 views

When is Block-Partitioned Matrix Invertible?

Suppose I have a block partitioned matrix \begin{equation} \begin{bmatrix} \mathbf{X}_1^{\top}\mathbf{X}_1 & \mathbf{X}_1^{\top}\mathbf{X}_2 \\ \mathbf{X}_2^{\top}\mathbf{X}_1 & ...
0
votes
0answers
4 views

Multivariate regression analyisis with grouped data. How do I 'un-group'

I am trying to determine if either of two non-numeric variables effect the percentage of people who take a specific action. There is another variable that needs to be controlled for. In real world ...
0
votes
1answer
5 views

Do multiclass logistic regressions obey Kolmogorov's second axiom?

Logistic regressions were taught to me using the intuition that they approximate $\mathbb{P}(Y=y|x;\theta)$. Multiclass regressions use one-vs-all classification, selecting one $y$ and classifying ...
1
vote
2answers
40 views

Show $E\left(\mathbf{X}_i \otimes \mathbf{u}_i\right)=\mathbf{0}$ implies $E\left(\mathbf{X}_i^{\top}\mathbf{G}\mathbf{u}_i\right)=\mathbf{0}$

Let $\mathbf{X}_i$ be a $G \times K$ random matrix, and let $\mathbf{u}_i$ be a $G \times 1$ random vector, and suppose we have a sample of $i=1,\ldots,N$ of each. Suppose the following condition ...
1
vote
1answer
32 views

How to Change Summation Expression $\sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$ into Matrix Expression

Let $\mathbf{X}_i$ be a $G \times K$ matrix, and suppose are $i=1,...,N$ of these matrices. Note that \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{X}_i &= \begin{bmatrix} ...
1
vote
0answers
35 views

Which projection, in $L_\infty$ norm or $L_2$ norm, is non-expansion?

I am just wondering which projection is non-expansion? Basically, I am wondering if $F$ is a projection operator then which norm would satisfy the following non-expansion property, where for a given ...
1
vote
1answer
19 views

what is the difference between 'estimate of residual standard error' and 'residual standard error'?

What is the difference between 'estimate of residual standard error' and 'residual standard error'? Can someone please provide the formulas? Thanks!
1
vote
0answers
40 views

Matrix Decompositions: Difference between Cholesky Decomposition, Eigendecomposition and Jordan Normal Form Decomposition

I recently created a related topic about the square root matrix, in case you'd like to refer to that one. Here's what we want: Consider the matrix $\Omega=E(\mathbf{u}^{\top}\mathbf{u})$, where ...
0
votes
0answers
16 views

Matrix problem in Mixed Regression

the background is $y= X \beta +e$ y=n*1 X=n*p $\beta=p*1$ e=n*1 take singular value decomposition of X $X=P \Delta Q$ $\beta=QKP'y$ K is a digaonal matrix and depending on its form can represent ...
0
votes
0answers
13 views

Multivariate Regression

Suppose there are $n$ variables that map through a function to a single output variable $r$. Given a set of 50-100 data sets with accepted input and output values that satisfy this relation, is it ...
1
vote
2answers
86 views

Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

I know in general that if a matrix $A$ is positive definite, then there exists a (unique?) square root matrix $B$, which is also positive definite, such that $BB=A$. Therefore, suppose $A$ is ...
0
votes
0answers
3 views

2d spatio-time regression

I have this question which intuitively appears to be simple, but I couldn't find a solution to it. Imagine you have a ball moving in one direction (x) and that the measurements of the movement are ...