Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
21 views

L1 regression statistics

Consider fitting the below dataset using L1 regression: x: 8.3 8.3 12.1 12.1 17.0 17.0 17.0 24.3 24.3 24.3 33.6 y: 224 312 362 521 640 539 728 945 738 759 663 Why do the regression estimates ...
0
votes
0answers
460 views

How do I find Sxx in a Simple linear regression model?

In a Simple linear regression model, I have only Sxy and Syy data with me. How shall I derive Sxx, linking Sxy and Syy based on first principles? I know the formulas separately. I want to find Sxx, ...
0
votes
0answers
27 views

Classical Regression Model: Combining linearity and strict exogeneity

I am studying the Classical Regression Model for random samples. Hence consider the random sample $(y_i,\mathbf{x_i})$ Where: ...
0
votes
0answers
16 views

How to derive F statistic for general linear hypothesis

I want to derive the F statistic for general linear hypothesis $H_0$ : T$\beta$ = c vs $H_1$ : T$\beta$ $\not =$ c where T is $p * q$ matrix with rank q I have tried to express $SS_{Res} (RM)$ - ...
0
votes
1answer
35 views

how to apply weighting factor to linear regression

Say if I have two sets of data, x and y. And I am required to apply a weighting factor,1/x, to the regression line. Does that mean I should plot 1/y versus 1/x and then get the regression? Could ...
0
votes
0answers
24 views

Difference in coefficient estimates

I have simple regression $y_i=\beta_0+\beta_1x+\epsilon$. Then I have estimates $\hat{\beta_0},\hat{\beta_1}$ computed from n observation and estimates $\beta_0',\beta_1'$ computed from n+1 ...
0
votes
1answer
26 views

Questions about Multi linear regression model.

I have two questions about multi linear regression model. First question. Suppose 2 independent samples Sample1 : $y_1$, ... $y_{n_1}$ and $x_1$, ..., $x_{n_1}$ Sample2 : $y_{n_1 +1}$, ... $y_{n_1 ...
0
votes
0answers
12 views

relation within Gauss-Newton method for minimization

If we study model fit on a nonlinear regression model $Y_i=f(z_i,\theta)+\epsilon_i$, $i=1,...,n$, and in the Gauss-Newton method, the update on the parameter $\theta$ from step $t$ to $t+1$ is to ...
1
vote
1answer
98 views

Using the observation vector $ \vec{y}$ instead of the centered observation vector $ \vec{y_{d}} $ doesn't change the projection $\vec{\hat y}$

I'm wondering why the two statements below are equal regardless of using $\vec{y}$ in deviation form/mean-deviaton/centered form or not. In other words, why isn't the result changed when you use the ...
0
votes
0answers
20 views

Poisson distribution with normal informative priors

I'm Jia, a student of economics and finance. I was wondering if someone could help in understanding this problem. I've just started to attend a new course "Financial and nonlinear econometrics" and ...
0
votes
0answers
40 views

$\left(\sum_{i=1}^n1\right)\left(\sum_{i=1}^n x_i^2\right)-\left(\sum_{i=1}^n x_i\right)^2=\frac{1}{2}\sum_{i=1}^n\sum_{k=1}^n\left(x_i-x_j\right)^2$?

I am reading Widder's Advanced Calculus and on page 130 he states that \begin{align}\left(\sum_{i=1}^n 1\right)\left(\sum_{i=1}^n x_i^2\right)-\left(\sum_{i=1}^n ...
1
vote
1answer
21 views

An algebraic identity: $M(X_2)-M(X)=P(M(X_2)X_1)$?

Notation: for a matrix $Z$ of full column rank, we define $P(Z)=Z(Z'Z)^{-1}Z'$ and $M(Z)=I-P(Z)$. ($I$ is the identity of matrix with as many rows as $Z$.) Let's consider the linear regression model ...
0
votes
1answer
37 views

Advice to solve a system of 8th order univariate polynomials

I am struggling to solve a least square problem in which the tedious part is the initialization. Grid search methods are out of question. The initial problem I've stated my problem in a previous ...
0
votes
0answers
16 views

Fit derivative to a set of points

Let's say I have a set of discrete values $X = {x_1, x_2, x_3, ..., x_n}$ from the sampling at a rate $f_s$ of a continuous function. I scale some values in $X$ (in a different manner for each one), ...
0
votes
0answers
14 views

Determine the multi-dimensional relationship given the data

I have a dependent variable - A and 3 independent variables, H,V and N I have a data for all the variables and dependency relationship is based on my operational knowledge. I'd like to know what ...
1
vote
0answers
33 views

Estimate b when $\hat{y}$ and x are given

Using linear algebra for solving this linear regression problem, I've got the equation $\hat{y}$ = xb. Finding the projection $\hat{y}$ of y onto the columns of the matrix X containing the ...
0
votes
1answer
45 views

Linear Regression and finding Correlation Coefficient

In a Simple Linear Regression $y= \alpha + \beta x + \epsilon $, we gather this information: $S_y=20, S_x=5, \widehat{\beta} = 0.2 $ how I could find Instance Correlation Coefficient between x and ...
0
votes
0answers
21 views

How to calculate expected value (expected lifetime) when process partitioned into 2 distributions?

I have a cohort with data showing the number of users in some application. I see (and I know) that the process should be modelled: in its first $k$ time units with in the uniform distribution in ...
0
votes
1answer
14 views

Express in the form of general linear regression

I have to apply a transformation to the following to fit it to the general linear regression model $Y_{i} = \beta_{0} + \beta_{1}X_{i1} + \beta_{2}X_{i2} + ... + \beta_{p}X_{i(p-1)} + \epsilon_{i}$ ...
0
votes
1answer
28 views

Regression coefficient in simple regression

Let's say we have two random variables $Y$ and $X$ used to form regression model $$Y=\alpha+\beta X+\mu$$ It also holds that $E(\mu)=0$, $\text{Var}(\mu)=\sigma_{\mu}^2$, $\text{Var}(X)=\sigma_{X}^2$, ...
0
votes
0answers
17 views

Relationship among parameters from models with different link function and scaled response variable

Given the model, $\log(A_i) = \alpha + \beta \, covar_i$, with $i=1,\dots,1000$, $\alpha=4$, $\beta=0.2$, and covariate $covar \sim U(-1,1)$, I derived $\log(A)$ values (in $\texttt{R}$) as: ...
1
vote
0answers
19 views

Appropriate regression test apart from MLR for crime data?

thanks in advance. I'm looking to run some statistical methods to find the correlation of crime rates to crime factors. I know about MLR, which is pretty simple to run in SPSS, but what are the other ...
0
votes
0answers
18 views

curve fittin with non-gaussian noise

Fitting with the least squares method results in the ML fit assuming the given points have a gaussian distributed noise. What methods are there for non-gaussian noise distributions, especially ...
0
votes
2answers
32 views

How to start using R (or free alternatives)?

How to start using software R to make regression analysis and forecasting? Are there any other free software to work with this kind of analysis?
0
votes
1answer
40 views

Linear Regression Application

I have a linear equation as follows: $B_0*x_0 + B_1*x_1 + ... + B_8*x_8 = result$ And i have about 200 different situations that are categorized into two different groups, depending on whether ...
1
vote
1answer
34 views

Multicollinearity: Why does highly correlated columns in the design matrix lead to high variance of the regression coefficient?

I came across the term "Multicollinearity" in statistics, particularly statistics. However, I never really understand mathematically why highly correlated (almost linearly dependent) columns in the ...
0
votes
1answer
34 views

Relationship between Kriging and Gaussian process regression models

In the science of Bayesian modeling one method involves using Gaussian processes to derive regression functions on data. I notice in looking at the plots for such regressions that they resemble ...
0
votes
2answers
28 views

Expected Value of an inefficient estimator of the $\beta$ parameter of a simple linear regression

A simple linear regression is defined as follow: $$y_i=\alpha+\beta x_i+\epsilon_i \qquad i=1,...,n$$ An inefficient way of estimating $\beta$ is defined as follow: ...
0
votes
1answer
22 views

Is the regression model Identified? (is it possible to obtain a least squares estimator of the parameters?)

$y_t$ is the dependent variable, $x_t$ and $z_t$ are explanatory variables, and $α$, $β$ and $γ$ are unknown parameters. $y_t$ = $α$ + $β$$x^3_t$ + $γ$/$log$($x_t$) + $u_t$ $y_t$ = $α$ + $β$$x_t$ + ...
0
votes
2answers
27 views

Calculate $m_1, m_2 $ for $y = m_1x_1 + m_2x_2$

Given these values: $$x_1 = \left\{1, 3, 6, 8\right\}$$ $$x_2 = \left\{2, 8, 5, 10\right\}$$ $$y = \left\{8.6, 30.8, 34.1, 53.8\right\}$$ And this formula $$y = m_1x_1 + m_2x_2$$ How do I ...
0
votes
0answers
19 views

Can we see the beta coefficients in OLS as mean values?

Can we see the beta coefficients in OLS as mean values? I mean the estimator β alone. y=Xβ+ε
1
vote
1answer
19 views

Find the OLS estimator $β_1$ when a new variable is added to the regression

Suppose $y_t$ = $β$$x_t$ + $u_t$ , where t = 1, 2, ..., n. We know, in this case, the OLS estimator is $\hatβ$ = ∑$x_t$$y_t$ / ∑$x_t^2$ . Now suppose one more observation $x_{n+1}$ is added. At the ...
0
votes
0answers
18 views

Regression model, find $Var(y_i-\hat y_i)$

For the model of $y_i = \beta_0 + \beta_1x_{i1} + e_i$ for $i = 1,...,n$, where $e_i \sim N(0,\sigma^2)$ Find $E(\hat y_i)$ and $Var(\hat y_i)$. Hence or otherwise, find $E(y_i-\hat y_i)$ and ...
1
vote
2answers
69 views

Factoring a multivariate linear polynomial

I'm a computer programmer trying to solve a particular toy problem, and my understanding of linear algebra is far too lacking to solve it! I have a data set that can be modeled using this function: ...
0
votes
1answer
33 views

Multilinear fit vs Polynomial fit

I have a program that generates some physics data in 1D and 2D functions. In this program, the user defines a number of models that are used to compute a 2D function. That 2D function, and it's ...
1
vote
0answers
28 views

Predictive models

Given a set of temperatures of different cities for a month, which prediction model should I use for a two day look ahead prediction? Regression models or Time series?
0
votes
1answer
98 views

The $\alpha$ estimation for the model $x_i = \xi_i \cdot \alpha$

We have $n$ sensors $X_i$ which estimate the scalar value $\alpha$ with different relative accuracies $\delta_i \ll 1$: $$ x_i = X_i(\alpha) = \xi_i \cdot \alpha, \ \ \ \xi_i \sim N(1, \delta_i) $$ ...
0
votes
0answers
23 views

Show that $E[g(X)u]=0$ in a standard regression model where $u = \hat{\beta}-E(\hat{\beta})$

Consider the standard regression model $y = X\beta + \epsilon$ where $y$ and $\epsilon$ are $(n \times 1)$ vectors and $X$ a $(n \times K)$ matrix. Let $\beta$ be any estimator of $\beta$. Let $u = ...
1
vote
0answers
19 views

Population versions of multiple correlation coefficients and least squares estimates

I'm reading an old paper (Wold and Faxer (1957)) which considers the theoretical relation $$ y=\beta_1x_1+\cdots+\beta_hx_h+\zeta $$ where $y,x_1,\ldots,x_h,\zeta$ are (scalar) random variables ...
0
votes
1answer
19 views

rotating and exchanging x for y's in regression

I was just wondering what happens generally if i send all my x points to y's and y's to x's (i.e reflect along the y=x line) - if I change the x's and y's will my old error minimizing line still be ...
1
vote
0answers
35 views

var(AB) when A,B not independent

I need to find the variance of $\hat\beta_1 * \bar X_1 / \bar Y$ , where we have the regression equation Y= $\beta_0 + \beta_1* X_1 +…+ \beta_j* X_j$ I initially was thinking the answer is simply ...
0
votes
0answers
51 views

Fitting a equation to a spiral curve

I am completely new to this forum and also to this type of mathematical modeling. I am interested to fit the following equation to the points obtained from experimental data. I am looking for an ...
0
votes
1answer
39 views

Critical points of quadratic forms

Let $A$ be an $n\times n$ symmetric matrix, let $b$ be an $n$-vector, let $c \in \mathbb{R}$ and set $Q(x) = 1/2 x^T Ax-x^T b+c$. Prove that $x_0$, defined as a solution to $Ax_0=b$ is a critical ...
0
votes
0answers
30 views

t test vs f test

For conducting statistical tests concerning the parameter $\beta_1$ (the slope of the estimated linear regression function), why is the $t$ test more versatile than the $F$ test? This is a question ...
0
votes
0answers
12 views

Analysis of block-greedy algorithms for function approximation?

I consider the problem of selecting a final basis set $\{\phi_{c_j}\}_{c_1}^{c_n}$ approximation of function $f \in \cal{H}$ in a Hilbert space that minimizes $L_2$ error. One can use a greedy ...
2
votes
1answer
20 views

Three-Perpendicular Theorem for linear regressions

For a random vector $X=(X_1,\ldots,X_p)'$, we define $$ \mathcal{L}(X)=\{b_0+b_1X_1+\cdots+b_pX_p,b_0,\ldots,b_p\in\mathbb{R}\}. $$ The linear regression of the $q$-dimensional random vector ...
-1
votes
1answer
15 views

For the three measurements b=0, 3, 12 at times t=0, 1, 2 find the best parabola y=C+Dt+E$t^2$

So I know how to do least squares regression using matrices to solve for Ax=b. I simply do $A^TAx=A^Tb$. However I don't really know how to account for the second power in a typical parabola ...
0
votes
1answer
25 views

What is the least squares solution given a line passes through original and following points?

So I am looking for the line y=Dt through the origin that fits the data y=4 at t=1, y=5 at t=2 and y=8 at t=3. This is what I have done so far. I know the three equations that are supposed to be ...
2
votes
0answers
21 views

Proof – OLS estimator regression [closed]

I am having trouble figuring out how I need to form and present an answer to a question. I completely understand the concepts of the math and analysis, I just don't understand how to give an answer ...
1
vote
0answers
37 views

consistency of OLS on misspecified AR(1) process

Suppose the true relationship in data is driven by AR(1) process as follows: $$X_t=\rho X_{t-1}+\epsilon_t\hbox{ , }|\rho|<1$$ and $\epsilon$ is a white noise of $(0,1)$ expectation and variance. ...