Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
16 views

variance of residual in use of uncertainty in gradient and intercept

For a real and differentiable function $z\left ( v_{j=1},\cdot \cdot \cdot ,v_{j=N} \right )$ the estimated uncertainty in z is $\sigma^{2}_{z}=\sum_{i=1}^{N}\left [ \left ( \frac{\partial z}{\...
0
votes
1answer
68 views

Non-Linear Model Transformation

I want to transform this Non-Linear Model $y= 8-ae^{bx}$ to Linear.And my issue is in this step $lny=ln(8-ae^{bx})$ how can simplify it to reach in a linear model which is like this $y*=b0+b1x$ ...
0
votes
0answers
13 views

Likelihood of an autoregressive model

I have the following autoregressive model: $Y_I=\lambda_t + \alpha_t(Y_t-\lambda_{t-1}) + \epsilon_t$ where $\lambda_t=\beta_1+\beta_2cos(\pi t/6)+\beta_3sin(\pi t/6)$ and $\epsilon$ has a ...
0
votes
1answer
27 views

How to compute Simple Linear Regression equation not using Least Squares Estimators?

I know how to compute the Simple Linear Regression (SLR) equation using Least Squares Estimators, $b_0$ and $b_1$. But I was given the following table: $$ \begin{array}{c|lcr} & \text{mean} &...
0
votes
0answers
13 views

Plotting data with error bars with gaussian distributed error

I want to fit a straight line model of the form $y_i = a+ b\ x_i$ to the list of $(x, y)$ pairs given below. How can I plot the data with error bars in both coordinates? ...
1
vote
0answers
53 views

Formula for the error terms squared of a linear regression model

I need to prove that the sum of the error terms squared $\sum\epsilon^2$ of a linear model is equal to a given formula: $\sum_{i=1}^{n}(Y_i-\alpha-\beta(x_i-\bar{x}))^2=n(\hat{\alpha}-\alpha)^2+(\hat{...
-1
votes
1answer
79 views

linear regression-slope

Your friend in the U.S. gives you a simple regression fit for predicting house prices from square feet. The estimated intercept is -44850 and the estimated slope is 280.76. You believe that your ...
1
vote
1answer
30 views

R Squared in polynomial multipleregrssion

In a linear regression, we can use R-Squared to check if a model fits But what if I have a polynomial regression with to variable $var_1$ and $var_2$ and a model that goes like $$y=x_0+ x_1\cdot ...
0
votes
0answers
20 views

Retrieve inputs for second pass (cross section) regression from Time series linear regression

I will explain my question through simple example best demonstrating the issue. Basically I work with Matlab, so if there is already implemented lib functions for this - That's can be the answer. And ...
1
vote
1answer
47 views

Regression: Service time based on number of copiers

For a random sample of 10 service calls, both the number of copiers and the total service time were recorded. Number of Copiers (x) : 4, 2, 5, 7, 1, 3, 4, 5, 2, 6 ...
0
votes
0answers
13 views

Prove information matrix equality for binary Logit model

I need to prove the information matrix equality for a binary logit model. I know that in general the information matrix equality means the following: "The Information Matrix Equality (IME), which is ...
0
votes
1answer
36 views

Calculated product of two projection matrices

how do i show that $$P_MP_m=P_mP_M=P_m$$ where $$P_M=X_M(X'_MX_M)^{-1}X'_M$$ $$P_M=X_m(X'_mX_m)^{-1}X'_m$$ and $X_m$ is embedded into the larger matrix with the same number of columns $X_M$.
1
vote
0answers
14 views

given a set of points and vectors, expand the set

I have a set of $n$ points: $(Sx, Sy, Sz)$ and $n$ vectors: $(Vx,Vy,Vz)$, each uniquely mapped to one another. The set also exists inside of an $X*Y*Z$ size rectangular space. I want to ensure a ...
0
votes
0answers
24 views

Parameterizing conditional expectations in terms of regression coefficients (gaussian case)

Consider three jointly normally distributed random variables $X,Y$ and $Z$. I know that in the Gaussian case $$E[Z\mid X, Y]=\beta_{ZX;Y}X +\beta_{ZY;X} Y$$ where $\beta_{ZX;Y}$ notes the ...
0
votes
0answers
5 views

using simple regression to capture a “normal range” for a dependent variable against a factor?

The question reads as follows My JMP output of the simple regression of the data is as follows: Is this all I need? The question phrasing is somewhat throwing me for a loop. Is there something ...
2
votes
1answer
203 views

Simple Regression ~ House Price Prediction

I am stuck with this question. "You have a data set consisting of the sales prices of houses in your neighborhood, with each sale time-stamped by the month and year in which the house sold. You ...
2
votes
0answers
31 views

What does it mean to regress out current features?

First of all, I'd like to say that this is the intro to a homework problem. Please do not post any answers, I am only looking for clarification on some terminology in the setup. I am trying to ...
1
vote
0answers
18 views

How to regress certain non-linear data

How can I perform a regression onto data of that follows this shape: \begin{equation} U(x):=\sum_{i=1}^N\, a_ix^ie^{-b_ix} \end{equation} where the $a_i\in \mathbb{R}$ and the $b_i \in (0,\infty)$ ...
2
votes
1answer
9 views

Inverse function for non-linear regression purpouses

The Setting: I want to perform a regression onto data of that follows this shape: \begin{equation} U(x):=\sum_{i=1}^N\, a_ix^ie^{-b_ix} \end{equation} where the $a_i\in \mathbb{R}$ and the $b_i \in (...
2
votes
1answer
50 views

How do I calculate regression line using a data set with repeated values indicated as frequencies?

I have a data set that comprises of Independent Variable $(X)$ and Dependent Variable $(Y)$ values with a certain frequency $(F)$. I know that I have to find $x^2$ and $xy$ but how do I factor in the ...
1
vote
1answer
39 views

How to find equation of a curve from points

I have got a curved line that I would like to find the equation of using Microsoft Excel. The curve seems to be either a polynomial or some sort of a trig graph. I've done some research and looked at ...
0
votes
1answer
22 views

How to interpret parameter in proportional hazard model?

How can I show that $b$ in this proportional hazard model $y=1-x^{e^{bz}}$ is the percent change in $y$ with a unit change in $z$? For brevity, I have neglected to show the functional form of $x$, ...
1
vote
0answers
15 views

Imposing non-negativity constraint on a linear regression function

Suppose I am interested in estimating the linear regression model $$ Y_i = g(X_i)^T\beta + \epsilon_i $$ where $Y_i$ is a scalar outcome of interest, $X_i$ is a scalar covariate with support on the ...
0
votes
1answer
28 views

How to interpret b in $y=x^{e^{bz}}$ in nonlinear regression?

What is the correct way to interpret b in this nonlinear equation $y=x^{e^{bz}}$? I've estimated the model and b seems to be the percent change in y with a unit change in z, but I am unsure how to ...
0
votes
0answers
10 views

Hosmer–Lemeshow test - Best Model

I have 3 different models and I do the Hosmer–Lemeshow test. I have a p-value and a Chi2 value. How can I know which model fits the best my data? Khi-2 || Pr > Khi-2 12.04 || 0.19 7....
0
votes
0answers
9 views

Why can I plug the roots of a partial derivative of a linear optimization objective E into E without changing it?

As an example, to fit a line to 2D data $\boldsymbol x_i$ with the parameters $\theta = (a\;\;b\;\;c)^T$ with the normal equation $\langle \boldsymbol x, \left(\begin{smallmatrix}a\\b\end{smallmatrix}\...
0
votes
0answers
32 views

How to fit sum of products of sine waves?

System Model: \begin{align} Y(t_1, t_2, t_3) = A \bigg[ & 2+ k_1\cos(w_1t_1+\phi_1) +k_2\cos\left(w_2t_2+\phi_2\right)+ \\[2ex] &4k_3\cos\left(\dfrac{w_1t_1+\phi_1}{2}\right)\cos\left(\dfrac{...
0
votes
0answers
13 views

Compute linear regression slope over $[x_i, y_i]$ when $x_i$ samples over $x$ are regularly distributed.

Context: To have an idea of the trend-line of a set of samples $[x_i, y_i]$, I usually compute the slope $a$ in the linear regression ($y = ax + b$) with a spreadsheet software and the formula: $$ a ...
1
vote
2answers
55 views

Power Regression $y=Ax^B+C$

I have to do reproduce a power regression but I don't have any experience in procedures like that. I read a little bit about power fit/power regression and that a formula like $y = ax^b$ is used for ...
0
votes
0answers
44 views

Kalman Filter and OLS Results Are Different

I read that Kalman Filters can be used for continuous / online linear regression and at the end of the regression its results and ordinary linear regression (OLS) results would be the same. I tried it ...
2
votes
1answer
64 views

When is $\mathbf{X}^{T}\mathbf{X}+\lambda\mathbf{I}$ invertible?

The question is quite simple: for a $N \times p$ matrix $\mathbf{X}$ with real entries, when is $\mathbf{X}^{T}\mathbf{X}+\lambda\mathbf{I}$ invertible (where $\mathbf{I}$ is the $p \times p$ identity ...
0
votes
1answer
32 views

Why does $E(C\cdot \epsilon\; \vert\; C\cdot X) = E(C\cdot \epsilon\; \vert\; X)$?

Let $C$ be an $n\times n$ matrix, $X$ is $n \times k$, $\epsilon$ is $n \times 1$ This is taken from a simply proof of strict exogeneity in an Econometrics textbook by Hayashi. The explanation he ...
0
votes
1answer
29 views

Simplification of a product of three matrices

Define $$\mathbf{c}_t = \begin{bmatrix} x_{1t} \\ x_{2t} \\ \vdots \\ x_{Nt} \end{bmatrix}\in \mathbb{R}^N$$ where all entries are in $\mathbb{R}$, $t = 1, 2, \dots, p+1$. I am trying to simplify $$\...
0
votes
1answer
41 views

Relating regression to projection?

I recently learned that one can think of regression as a projection of a vector in a high dimension space onto the other vector. I tried implementing this and got it to work: ...
0
votes
3answers
61 views

How to do a regression which includes reciprocals?

I'm trying to find an interpolating formula for a set of coefficients (I have $80$ at the moment). I tried first to find an interpolating polynomial, but that was not useful: using the first $...
3
votes
1answer
74 views

Interpolation and mapping between scattered vectors in two unequally dimensioned spaces

Imagine two spaces: An ‘input’ space with dimension $m$. An ‘output’ space with dimension $n$. $m \geq n$ There are points in each of these spaces defined such that some characteristic is defined....
0
votes
2answers
29 views
0
votes
2answers
44 views

Linear regression using gradient descent: is the whole weight vector updated with the same number?

I'm using gradient descent with mean squared error as error function to do linear regression. Take a look at the equations first. As you can see in eq.1, the prediction is done with a bias term b ...
1
vote
1answer
52 views

Why the identity $P_X=P_XZ(Z'P_XZ)^{-1}Z'P_X$ with $P_X=X(X'X)^{-1}X'$?

Suppose $X$ and $Z$ are matrices such that $(X,Z)$ and $P_XZ$ both have full column ranks. Here, $P_X=X(X'X)^{-1}X'$. Consider a regression model $$ P_Xy=P_XZ\zeta+v\tag{A} $$ where OLS is used to ...
0
votes
1answer
25 views

Sum of component projection matrices

Show that if $X$ $=$ [$X_1$ $X_2]$ and $X_1'X_2 = 0$, then $P = P_1 + P_2$, where $P$ is defined as $X(X'X)^{-1}X'$, the projection matrix. Don't quite know where to start. I tried evaluating it by ...
0
votes
0answers
22 views

Variance in the sum of batch-correlated residuals in a regression

I am looking at a regression model of the following form: $Y=intercept+\beta_{Yf.n}X_f+\beta_{Yn.f}X_n +error$ where $X_f$ and $X_n$ are predictors. A value for $Y$ will be sampled from the ...
0
votes
0answers
14 views

Interpretation of diagonal detail in Haar Wavelet Transforms

I am a statistics grad student, and I have just begun exploring the topic of wavelet regression (specifically, Haar wavelets for discrete functions). I understand the generalization from a one ...
1
vote
2answers
24 views

Inverse of sum of matrices (SVD, ridge regression)

Looking at these slides, I've found the following: $X=UDV^T$, where $U$ and $V$ are orthogonal matrices, $V$ is a square matrix, and $D$ contains the singular values of $X$. The author then writes $...
0
votes
0answers
20 views

Relationship/correlation between data - does it exist?

Data I refer to in this question Some analysis has been conducted for my business by an external company. The data, as it stands, only really tells part of the story and doesn't provide any real ...
0
votes
2answers
41 views

exponential regression for bacteria growth

I'm studying regression lines and curves, and I've learn the methods for working with curves of the types $ax^2+bx+c$ and $ax+b$ as well as $a\sin(x)+b\cos(x)$. Now I'm asked this: $$(0,32), (2,65),(...
0
votes
1answer
41 views

How do you calculate the correlation between the intercept's and beta's standard error in a univariate linear regression?

I am running a regression to predict a variable Y as follows: $Y=\alpha+\beta\times x+\epsilon$ I am trying to get a distribution of the expected value of Y given standard errors in the model ...
1
vote
1answer
73 views

What are the limitations of linear regression + feature / label transformation?

Regression Suppose I have data points in a matrix $X \in \mathbb{R}^{n \times m}$ as well as labels $\mathbb{R}^n$, where $n$ is the number of my data points and $m$ is the number of features per ...
1
vote
1answer
16 views

How to show $Y = \sum_{i=1}^n \frac{Y_i}{n-2}$ is a biased estimator of the mean?

Let $\{Y_1, \ldots, Y_n\}$ be a random sample with $E(Y_i)= \mu_Y$ and $\operatorname{var}(Y_i) = \sigma_Y^2$. Show that $Y = \sum_{i=1}^n \frac{Y_i}{n-2}$ is a biased estimator of the mean? This is ...
1
vote
1answer
20 views

Comparing a series expansion to polynomial regression

So I don't have a great background in mathematics but I have a quick and hopefully simple question for you guys. I'm a graduate student and I'm doing some polynomial regression on some thermodynamic ...
0
votes
0answers
26 views

Using least squares regression to apply nonlinear function to time series data

If you have a nonlinear function (see example), can you use a least squares regression approach to fit it to time series data ? Is this approach also valid for n variables? How many time points are ...