Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
16 views

MSE in case of log-transformed dependent variable

Let's consider the following log-linear model: $log(Y_i) = \alpha + X_i\beta + \epsilon_i$ for i = 1, ..., N The fitted value is: $\widehat{log(Y)} = \hat{\alpha} + X\hat{\beta}$ Assuming ...
0
votes
0answers
17 views

Curve fitting using a graph extracted from an article, is it possible?

I want to curve fit a graph from an article which I can only extract from the pdf file as a screenshot. Therefore, I do not have the coordinates of the data points explicitly, yet I know that the ...
1
vote
0answers
20 views

Regression question (details inside). Measuring the incremental impact on the dependent variable of one category over other categories

Formulate a regression equation you would use to test for the differences in ROE between firms that used tier 1 investment banks as their advisors and those that used tier 2 or tier 3 banks (note: ...
0
votes
1answer
48 views

Optimization Problem (Linear Algebra)

I am not trying to cheat or anything, so any reference to online literature or MOOCs, that teach this stuff, will be highly appreciated. The problem is to prove that the following optimization ...
1
vote
0answers
16 views

Statistics linear modelling in R

Suppose I have a date set of the form: Test Subject/Sex/No. of mistakes made in the morning/ No. of mistakes made in the afternoon A / M / 2 / 5 B / F / 1 / 4 C / M / 3 / 5 D / F / 1 / 5 Suppose ...
1
vote
0answers
19 views

Cannot figure out autocavariance

The moving average model of order q has the form $$Y_t =β_0 +e_t +b_1e_{t−1} +b_2e_{t−2} +...+b_qe_{t−q}$$ where $e_t$ is a serially uncorrelated random variable with mean $0$ and variance $σ^2_e$. ...
1
vote
0answers
19 views

Time series regression

What if I have a stationary independent variable and 2 non-stationary dependent variables, and I want to run a regression on them, what model is the most appropriate?
0
votes
1answer
40 views

Is there such a thing as a combination of linear and non-linear regression in one form?

Let's say I have a dataset D with many variables. I can get a multiple linear regression from that in the form ...
2
votes
1answer
38 views

How to take the derivative of Matrices

I was browsing the derivation of the Least Squares estimates and stumbled about this problem. It said that: $$E = (Y + XB)^2$$ $$\frac{dE}{dB} = -X^TY + X^TXB$$ It is to my understanding that the ...
1
vote
1answer
40 views

Size of sample and correlation coefficient

$X$ and $Y$'s correlation coefficient is $r=0.5$. What is the size of sample when the correlation is significant at $\alpha=0.05$ with two sided test? Is there a more "formal" way to solve this ...
1
vote
1answer
21 views

what is one basic/intermediate regression analysis standard textbook that is math intense

What is one basic/intermediate regression analysis standard textbook that is math intense with proofs/derivations? Also, i need that one to be comprehensive yet the diffculty is suitable for self ...
0
votes
1answer
47 views

Matrix form for Weighted Least Squares

If we have the following weighted least-squares regression, with $\hat{\beta} = (X'WX)^{-1}X'WY$ How can we express the squared errors, MSE and the fitted values in matrix form? These are the OLS ...
0
votes
1answer
33 views

How to find the deviated form of beta 1 in OLS

How to find the deviated form of beta 1 in OLS Y=β1+β2X+u estimated β1=(ΣX^2ΣY-ΣXΣXY)/(nΣX^2-(ΣX)^2) I do not know how to turn this part (ΣX^2ΣY-ΣXΣXY)into deviated form. I found that estimated ...
0
votes
0answers
39 views

If we make the lambda very small in ridge regression, why do the beta hats still decrease?

I understand that the betas estimated from minimizing the function (y hat - y)^2 + beta' beta decrease as lambda increase. However, looking at the picture below, why cannot we choose a lambda so ...
1
vote
1answer
25 views

Help Linear Regression

It would be really nice if someone could help me out and suggest me with his expert opinion. Going through the table and looking at Part (a) of this question. I believe I should use Scatter plot to ...
2
votes
1answer
44 views

Standard Error in OLS Regression

Assuming I have the following linear regression set-up: $y_i = \alpha + x_i * \beta + \epsilon_i$ for $i = 1,2,..., n$. When I run the regession, I get a $\beta$ and $\alpha$ estimates, along ...
0
votes
1answer
42 views

Multiple Regression with Categorical Predictor Variables of More than Two Levels

I'm planning on running a hierarchical multiple regression in SPSS. In the first step, I would like to enter demographic characteristics, second step continuous predictor variables of interest, and ...
0
votes
0answers
18 views

Simplifying $\sum_i[z_i'(\delta-\hat{\delta})]^2x_ix_i'$ to apply a law of large numbers

I'm in the context of linear regressions. Let $n$ be the sample size and for $i=1,\ldots,n$, let $$ \underbrace{x_i}_{K\times 1},\quad \underbrace{z_i}_{L\times 1}\quad(n>K\geq L) $$ be column ...
0
votes
0answers
27 views

multiple linear regression analysis and case of p>>n

In the case for simple linear regression and multiple linear regression, if we have p >> n, is it undefined? And in the case for ridge regression, p>>n requires $\lambda$ to increase b/c more ...
1
vote
0answers
31 views

Outlier Contained in Prediction Interval (Tme series Forecasting Problem)

In my stats class today, the professor was showing us some output from MINITAB on a prediction interval that was calculated (from time series data using standard linear regression). For one of the ...
0
votes
0answers
161 views

Inclined Elliptic Tank Volume Calculation

Can someone help me determine an equation for calculating the volume of an elliptical cylinder on it's side and inclined 5 degrees from horizontal? The tank has flat ends. I have found several ...
0
votes
1answer
45 views

For a general linear regression , are Y and Y hat independent?

For a general linear regression , are $Y$ and $\hat{Y}$ independent? $Y$=XB+e $\hat{Y}$=X$\hat{B}$ I think they are dependent, because if they rely on the same data, they should have some sort of ...
1
vote
1answer
41 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
0
votes
1answer
26 views

What is a common name for the resulting function?

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
1
vote
0answers
16 views

derive the distribution(a multiple regression problem)

(Multiple regression model with p's predictor variables.) Derive the distribution of $$\frac{(b-\beta)X'X(b-\beta)}{MSE\cdot p}$$ As far as I know, $b\sim N(\beta,\sigma^2 (X'X)^{-1})$ $b-\beta ...
0
votes
1answer
40 views

Proving an implication in a linear regression

Suppose we have a linear regression: $$ y_i=X_i'\beta+u_i,\quad i=1,\ldots,T. $$ Here $y_i$ and $u_i$ are scalars, $X_i$ and $\beta$ $k\times 1$. $\beta$ is a (non-stochastic) vector of ...
0
votes
1answer
17 views

Conditional expectation; regression

Assume that E[$y_{it}|c_i, x_{it}$] = $c_i$ + $x'_{it}\beta$ Eliminate $c_i$ by taking the expectation with respect to $c_i$, leading to E[$y_{it}|x_{it}$] = E[$c_i|x_{it}$] + $x'_{it}\beta$ ...
0
votes
0answers
24 views

Question about regression model

Suppose you fit (estimate the parameters of) a regression model, obtaining $\hat{Y}$, $\hat{B}$, and $\hat{E}$. And you fit a second regression model , using $\hat{Y}$ x matrix from previous model ...
0
votes
1answer
31 views

how to show for a simple regression with an intercept and one independent variable$ R^2 = r ^2$ , where $r$ is the ordinary correlation coefficient.

how to show for a simple regression with an intercept and one independent variable $R^2 = r ^2$, where $r$ is the ordinary correlation coefficient. Here is where I'm at. $R^2= ...
1
vote
0answers
19 views

Regression on even function?

Is there any test for whether or not the regression function is even? Suppose we have a model: $Y=g(X, \epsilon)$, where $Y, X$ are both one dimensional. My questions is how do we test for $g$ is an ...
1
vote
0answers
46 views

The Hessian Matrix I calculate is twice as much as it should be. Why?

I have a function "fkt." In this example, let it be as simple as $y=a \cdot x+b$. I have a real dataset with values obeying to the model. After regression of the points to the model, I find the ...
3
votes
1answer
107 views

When is Block-Partitioned Matrix Invertible?

Suppose I have a block partitioned matrix \begin{equation} \begin{bmatrix} \mathbf{X}_1^{\top}\mathbf{X}_1 & \mathbf{X}_1^{\top}\mathbf{X}_2 \\ \mathbf{X}_2^{\top}\mathbf{X}_1 & ...
0
votes
0answers
10 views

Multivariate regression analyisis with grouped data. How do I 'un-group'

I am trying to determine if either of two non-numeric variables effect the percentage of people who take a specific action. There is another variable that needs to be controlled for. In real world ...
0
votes
1answer
10 views

Do multiclass logistic regressions obey Kolmogorov's second axiom?

Logistic regressions were taught to me using the intuition that they approximate $\mathbb{P}(Y=y|x;\theta)$. Multiclass regressions use one-vs-all classification, selecting one $y$ and classifying ...
1
vote
2answers
46 views

Show $E\left(\mathbf{X}_i \otimes \mathbf{u}_i\right)=\mathbf{0}$ implies $E\left(\mathbf{X}_i^{\top}\mathbf{G}\mathbf{u}_i\right)=\mathbf{0}$

Let $\mathbf{X}_i$ be a $G \times K$ random matrix, and let $\mathbf{u}_i$ be a $G \times 1$ random vector, and suppose we have a sample of $i=1,\ldots,N$ of each. Suppose the following condition ...
1
vote
1answer
45 views

How to Change Summation Expression $\sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$ into Matrix Expression

Let $\mathbf{X}_i$ be a $G \times K$ matrix, and suppose are $i=1,...,N$ of these matrices. Note that \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{X}_i &= \begin{bmatrix} ...
1
vote
0answers
51 views

Which projection, in $L_\infty$ norm or $L_2$ norm, is non-expansion?

I am just wondering which projection is non-expansion? Basically, I am wondering if $F$ is a projection operator then which norm would satisfy the following non-expansion property, where for a given ...
1
vote
1answer
110 views

what is the difference between 'estimate of residual standard error' and 'residual standard error'?

What is the difference between 'estimate of residual standard error' and 'residual standard error'? Can someone please provide the formulas? Thanks!
1
vote
0answers
119 views

Matrix Decompositions: Difference between Cholesky Decomposition, Eigendecomposition and Jordan Normal Form Decomposition

I recently created a related topic about the square root matrix, in case you'd like to refer to that one. Here's what we want: Consider the matrix $\Omega=E(\mathbf{u}^{\top}\mathbf{u})$, where ...
0
votes
0answers
19 views

Matrix problem in Mixed Regression

the background is $y= X \beta +e$ y=n*1 X=n*p $\beta=p*1$ e=n*1 take singular value decomposition of X $X=P \Delta Q$ $\beta=QKP'y$ K is a digaonal matrix and depending on its form can represent ...
0
votes
0answers
20 views

Multivariate Regression

Suppose there are $n$ variables that map through a function to a single output variable $r$. Given a set of 50-100 data sets with accepted input and output values that satisfy this relation, is it ...
1
vote
2answers
140 views

Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

I know in general that if a matrix $A$ is positive definite, then there exists a (unique?) square root matrix $B$, which is also positive definite, such that $BB=A$. Therefore, suppose $A$ is ...
0
votes
0answers
7 views

2d spatio-time regression

I have this question which intuitively appears to be simple, but I couldn't find a solution to it. Imagine you have a ball moving in one direction (x) and that the measurements of the movement are ...
0
votes
1answer
54 views

linear regression, expectation and mean squared error

Let us assume that data is generated according to a true model $$y_i = \beta_{true}x_i + \epsilon_i$$ for $i = 1, ..., n$ Assume that $x_i$ are fixed, and $\epsilon_i$~ N(0, $\sigma^2$) ...
1
vote
0answers
19 views

Multiple regression model

I have a multiple regression equation which as four quarters (maybe called them as parameters) ...
2
votes
0answers
58 views

Predicting profit with price variation

I am currently working on a high school project that aims to predict profit from X amount of items to Y amount of profit based off a deviated sale price. For instance: I sale 10 cookies for 10 ...
1
vote
0answers
28 views

Round robin logistic regression

I have a poll with four answers (A,B,C,D) and response information about people who have taken that poll. I have created four models (one for each of the answers) in a one vs all. i.e. the model for ...
1
vote
0answers
59 views

Uni-variate Moving Average Theta coefficients

Consider the Uni-variate Moving Average Models (MA models) MA(1) $$x_t = \mu + w_t +\theta_1w_{t-1}$$ or the second order moving average MA(2) $$x_t = \mu + w_t +\theta_1w_{t-1}+\theta_2w_{t-2}$$ ...
-1
votes
1answer
20 views

Linear Regression question 1

I would really be grateful if someone could let me know how to answer Part (a) of Question 1. I believe i should scatter plot both x and y values separately for year 2000 and year 2001 on same graph ...
1
vote
0answers
12 views

Proving $Corr(\hat{e}_{ij}, \hat{e}_{jk}) = \frac{-1}{n_i-1}$ for $ j \neq k$

For the model of a single factor experiment: $y_{ij}= \mu + \alpha_i + e_{ij}$, $(1 \leq i \leq a, 1 \leq j \leq n_i)$, where a = the number of treatments, $n_i$ = the number of experimental units ...