Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
1answer
21 views

Analytic solution for matrix factorization using alternating least squares

The standard form for ridge regression aims to minimize the following cost function. $$ \min\ \ \sum_i(y_i-x_i^T\beta)^2 + \lambda\sum_j\beta^2_j $$ As described here, it's possible to differentiate ...
0
votes
0answers
22 views

standard deviation and adjusted R-squared for simultaneous regressions

I am conducting a study that requires two steps of statistical estimation. First, I run a regular OLS regression, from which I gather three outputs that I need: coefficient values standard ...
1
vote
1answer
39 views

MATLAB curve fitting - least squares method - wrong “fit” using high degrees

Anyone here that could help me with the following problem? The following code calculates the best polynomial fit to a given data-set, that is; a polynomial of a specified degree. Unfortunately, ...
-5
votes
1answer
30 views

How to fit a function to experimental data? [on hold]

I have an experimental data say $y=[1,2,3,4,5]$ for given $x$ values and I have a set of equations model fit as $$\sum (y -ax +cx)=0 \text{ and } \ \sum (y+ax - cx)=0$$ How to solve this for $a$ ...
0
votes
0answers
11 views

Multivariate regression with nonindependent variables

I'm trying to run a multivariate regression in which not all variables are independent, and an not sure if this is possible. The reason is as follows: Let's say we have a large number of contracts, ...
0
votes
1answer
36 views

adjusted R squared with multiple dependent varialbles

A question about regression in statistics. What is the formula for adjusted R squared if there are multiple dependent variables
0
votes
1answer
25 views

How to proof that least square estimator $\hat{B}$ doesnt exist when $x$ is linearly dependent?

For the linear regression model $Y=xB+e$, prove that if the columns of $X$ are linearly dependent, the least square estimator $\hat{B}$ does not exist I know that since $\hat{B}$ is an unbiased ...
0
votes
0answers
12 views

questions about distribution of multivariate normal

I'm looking at this past exam question, For A) Cbhat~N(CU,C(summation)C') B)I have very faint idea of what to do, I tried finding some theroems about distribution but couldn't find any that ...
0
votes
1answer
26 views

Linear regression involving angles in a triangle.

In a survey experiment, three independent measurements $29.5^{\circ}$, $30.5^{\circ}$, $120.5^{\circ}$ are obtained from the three angles $\alpha,\beta,\gamma$ of a triangle. Formulate the ...
1
vote
1answer
28 views

Question on regression

So I've been given this formula For regression $R^2=1 - \sum \frac{{(y_i - \hat{y}_i)}^2}{(y_1-\bar{y})^2}$ Now an obvious question that has come to me is why $R^2$ stays the same in certain ...
0
votes
0answers
17 views

Trying to find similarity between collection of points

This is a kind of weird problem, and I'm not sure what the best Stack Exchange to post this on is, but I assume Mathematics could help the most. I have many sets of points in 3D space (xyz ...
0
votes
1answer
15 views

how binary quantile regression divides the dependent variable into quantiles

I am not very clear with binary quantile regression. As if it was ordinary quantile regression, it would divide the dependent variable's value by its ascending value into quantiles. But I cannot ...
0
votes
1answer
31 views

Recalculate R^2 deleting 1 point

Is there a way to recalculate $R^2$ of a regression that I delete a point (for example an outlier point)? The idea is to get the $R^2$ without a point but without recalculating all the regression. ...
0
votes
2answers
50 views

Parameter optimization using a regression model.

I am working on an optimization problem. I build a regression model to understand the behavior of a system which depends on two variables which are functions of another two variables. My regression ...
0
votes
1answer
21 views

Proof of Correlation Coefficients

Good evening, I have a problem with an exercise: Let $X$ and $Y$ be two real square integrable random variables with var$X>0$, var$Y>0$. The correlation Corr$(X,Y)$ quantifices how far $X$ and ...
1
vote
2answers
36 views

Assumption of a Random error term in a regression

In one of my recent statistics courses, our teacher introduced the linear regression model. The typical $y=\alpha + \beta X + \epsilon$, where $\epsilon$ is a "random" error term. The teacher then ...
0
votes
0answers
8 views

How to interpret Standard Residual Deviation?

have fitted regression lines for 200 data sets where each data set contains 40 data points.As a result of that now I have 200 Standard Residual Deviation values and need to interpret that results. How ...
0
votes
0answers
25 views

Using Mean vs Median for Rapidly Changing Data

I am writing a report on global GDP per capita trends. As many of you know, were large shifts in the growth rates of these figures before and after the Great Recession. There has been some analysis on ...
1
vote
1answer
83 views

Incorrect angle detected between two planes

I want to calculate the angle between 2 planes, Reference plane and Plane1. When I feed the X,Y,Z co-ordinates of pointCloud to the function plane_fit.m (by Kevin Mattheus Moerman), I get the output ...
0
votes
0answers
18 views

Algebraic expression of a regression matrix

Let's say I'm doing a multivariate regression between a set of input $n$-dimension vectors (noted by the matrix $X=\{X_1,X_2,...X_m\}$) and a transformed version (noted by the matrix ...
0
votes
0answers
38 views

Hypothesis testing - Beta coefficient

Apologies if this is trivial, just linke me somewhere. I'm currently taking statistics 101, I can't wrap my head around the hypothesis testing of coefficients. As follows, the t-test reads $$T=\frac ...
0
votes
0answers
10 views

MSE in case of log-transformed dependent variable

Let's consider the following log-linear model: $log(Y_i) = \alpha + X_i\beta + \epsilon_i$ for i = 1, ..., N The fitted value is: $\widehat{log(Y)} = \hat{\alpha} + X\hat{\beta}$ Assuming ...
0
votes
0answers
14 views

Curve fitting using a graph extracted from an article, is it possible?

I want to curve fit a graph from an article which I can only extract from the pdf file as a screenshot. Therefore, I do not have the coordinates of the data points explicitly, yet I know that the ...
0
votes
0answers
15 views

How to compute confidence bound in linear regression

In a simple linear regression problem, let $A$ be an $m\times n$ matrix of samples, $A=[x^T_1; x^T_2; ...;x^T_m]$, $w$ is the $n\times 1$ parameter vector, and $b$ is $m\times 1$ response vector. The ...
1
vote
0answers
9 views

Regression question (details inside). Measuring the incremental impact on the dependent variable of one category over other categories

Formulate a regression equation you would use to test for the differences in ROE between firms that used tier 1 investment banks as their advisors and those that used tier 2 or tier 3 banks (note: ...
0
votes
1answer
42 views

Optimization Problem (Linear Algebra)

I am not trying to cheat or anything, so any reference to online literature or MOOCs, that teach this stuff, will be highly appreciated. The problem is to prove that the following optimization ...
1
vote
0answers
13 views

Statistics linear modelling in R

Suppose I have a date set of the form: Test Subject/Sex/No. of mistakes made in the morning/ No. of mistakes made in the afternoon A / M / 2 / 5 B / F / 1 / 4 C / M / 3 / 5 D / F / 1 / 5 Suppose ...
0
votes
0answers
41 views

$E(\bar{Y})=\bar{Y}$? [Linear Regression]

A book, I am reading, derives the covariance between $b_0$ and $b_1$ as follows: By definition, $$Cov(b_0,b_1)=E[(b_0-Eb_0)(b_1-Eb_1)]$$ $$=E[(b_0-\beta_0)(b_1-\beta_1)]$$ ...
1
vote
0answers
12 views

Cannot figure out autocavariance

The moving average model of order q has the form $$Y_t =β_0 +e_t +b_1e_{t−1} +b_2e_{t−2} +...+b_qe_{t−q}$$ where $e_t$ is a serially uncorrelated random variable with mean $0$ and variance $σ^2_e$. ...
1
vote
0answers
15 views

Time series regression

What if I have a stationary independent variable and 2 non-stationary dependent variables, and I want to run a regression on them, what model is the most appropriate?
0
votes
0answers
24 views

Deriving single linear regression parameters in terms of multiple linear regression parameters

Suppose the true population model is ln(wage) = B0 + B1(education) + B2(experience) + v (v is error term) Suppose the model is estimated as ln(wage) = B3 + B4(education) + u How do I calculate ...
0
votes
1answer
38 views

Is there such a thing as a combination of linear and non-linear regression in one form?

Let's say I have a dataset D with many variables. I can get a multiple linear regression from that in the form ...
2
votes
1answer
33 views

How to take the derivative of Matrices

I was browsing the derivation of the Least Squares estimates and stumbled about this problem. It said that: $$E = (Y + XB)^2$$ $$\frac{dE}{dB} = -X^TY + X^TXB$$ It is to my understanding that the ...
1
vote
0answers
18 views

Derive the F-statistic for this model

Problem: Consider the model $y=\beta+\varepsilon$, where $\varepsilon\sim N_4(0,\sigma^2I)$, and $\sum_{i=1}^4\beta_i=0$. Derive the F-statistic for testing $H_0:\beta_1=\beta_2$. I tried to write the ...
1
vote
1answer
38 views

Size of sample and correlation coefficient

$X$ and $Y$'s correlation coefficient is $r=0.5$. What is the size of sample when the correlation is significant at $\alpha=0.05$ with two sided test? Is there a more "formal" way to solve this ...
0
votes
0answers
7 views

variance of regression coefficient of residual vector

y=ax+bz+u where x,y,and z are n*1 observation vector and u is an error vector.(x and z are independent.) Assume that x and z are non-random and that u has zero mean and covariance matrix σ^2In Let ...
1
vote
1answer
16 views

what is one basic/intermediate regression analysis standard textbook that is math intense

What is one basic/intermediate regression analysis standard textbook that is math intense with proofs/derivations? Also, i need that one to be comprehensive yet the diffculty is suitable for self ...
0
votes
1answer
26 views

Matrix form for Weighted Least Squares

If we have the following weighted least-squares regression, with $\hat{\beta} = (X'WX)^{-1}X'WY$ How can we express the squared errors, MSE and the fitted values in matrix form? These are the OLS ...
0
votes
1answer
32 views

How to find the deviated form of beta 1 in OLS

How to find the deviated form of beta 1 in OLS Y=β1+β2X+u estimated β1=(ΣX^2ΣY-ΣXΣXY)/(nΣX^2-(ΣX)^2) I do not know how to turn this part (ΣX^2ΣY-ΣXΣXY)into deviated form. I found that estimated ...
0
votes
0answers
11 views

If we make the lambda very small in ridge regression, why do the beta hats still decrease?

I understand that the betas estimated from minimizing the function (y hat - y)^2 + beta' beta decrease as lambda increase. However, looking at the picture below, why cannot we choose a lambda so ...
1
vote
1answer
19 views

Help Linear Regression

It would be really nice if someone could help me out and suggest me with his expert opinion. Going through the table and looking at Part (a) of this question. I believe I should use Scatter plot to ...
2
votes
1answer
34 views

Standard Error in OLS Regression

Assuming I have the following linear regression set-up: $y_i = \alpha + x_i * \beta + \epsilon_i$ for $i = 1,2,..., n$. When I run the regession, I get a $\beta$ and $\alpha$ estimates, along ...
0
votes
1answer
32 views

Multiple Regression with Categorical Predictor Variables of More than Two Levels

I'm planning on running a hierarchical multiple regression in SPSS. In the first step, I would like to enter demographic characteristics, second step continuous predictor variables of interest, and ...
0
votes
0answers
12 views

Simplifying $\sum_i[z_i'(\delta-\hat{\delta})]^2x_ix_i'$ to apply a law of large numbers

I'm in the context of linear regressions. Let $n$ be the sample size and for $i=1,\ldots,n$, let $$ \underbrace{x_i}_{K\times 1},\quad \underbrace{z_i}_{L\times 1}\quad(n>K\geq L) $$ be column ...
0
votes
0answers
14 views

multiple linear regression analysis and case of p>>n

In the case for simple linear regression and multiple linear regression, if we have p >> n, is it undefined? And in the case for ridge regression, p>>n requires $\lambda$ to increase b/c more ...
1
vote
0answers
19 views

Outlier Contained in Prediction Interval (Tme series Forecasting Problem)

In my stats class today, the professor was showing us some output from MINITAB on a prediction interval that was calculated (from time series data using standard linear regression). For one of the ...
0
votes
0answers
53 views

Inclined Elliptic Tank Volume Calculation

Can someone help me determine an equation for calculating the volume of an elliptical cylinder on it's side and inclined 5 degrees from horizontal? The tank has flat ends. I have found several ...
0
votes
0answers
18 views

If you control some variable in a regression , how would you find what proportion of the remaining variation?

If you control some variable in a regression , how would you find what proportion of the remaining variation that is explained by the controlled variable? so that is to say , we have variables x1 x2 ...
0
votes
1answer
39 views

For a general linear regression , are Y and Y hat independent?

For a general linear regression , are $Y$ and $\hat{Y}$ independent? $Y$=XB+e $\hat{Y}$=X$\hat{B}$ I think they are dependent, because if they rely on the same data, they should have some sort of ...
1
vote
1answer
28 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...