Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
1answer
34 views

Regression maximum likelihood

Given this regression model: $y_{i}=\beta_{0}+\beta_{1}x_{i}+E_{i}$. All the assumptions are valid except that now: $E_{i}\sim N(0,x_{i}\sigma^{2})$ Find Maximum likelihood parameters for $\beta_0$,$...
1
vote
2answers
34 views

The four assumptions on linear regression

It is clear that the four assumptions of a linear regression model are: Linearity, Independence of error, Homoscedasticity and Normality of error distribution. My question is does any of these four ...
2
votes
0answers
27 views

Proof that correlation coefficient squared equals the coefficient of determination

Hi I as the title says I'm looking at the proof that $r^2$ = $R^2$ in the case of simple linear regression, but I don't understand one part. There are different versions of the proof, but in most of ...
0
votes
0answers
14 views

Combining predicted responses from two regressions

I am trying to find info related to combining confidence levels & intervals on predicted response variables. To explain my problem, considering the following: A and B are two different values ...
1
vote
1answer
23 views

Interpretation of PCA

I am wondering if there is a practical interpretation of a principal component analysis: Consider you have a data matrix $X\in\mathbb{R}^{N\times p}$ and you perform a principal component analysis ...
0
votes
0answers
19 views

Logistic Regression with some other Sigmoid function

I have been using Logistic regression with Newton's method where the update rule is the following: $$ W^\intercal = W^\intercal + (X^\intercal P (I - P) X)^{-1} X^\intercal(y - p) $$ Here $P$ is ...
0
votes
0answers
8 views

Is there any correlation between approximation trendline parameters?

Let's say I have two data sets $(x,y)$ and $(p,q)$ and two approximation trendlines: Logarithmic: $y = b·ln(x) + a$ Linear: $y = bx + a$ Let's say I applied logarithmic approximation to both data ...
1
vote
1answer
26 views

Understanding convergence of OLS estimator

From a linear regression with one explanatory variable, $ y = \beta_0 + \beta_1x+e$, the OLS estimator can be written as \begin{equation} \hat{\beta}_1 = \frac{\widehat{cov(y,x)}}{\widehat{var(x)}}. \...
0
votes
1answer
46 views

Variant of linear regression using perpendicular distance instead of vertical

Normally, linear regression asks for a pair of parameters m,b such that for a set of given points $\{x_i,y_i\}$ the variance of $y-m\cdot x-b$ is minimized (this minimizes the distance in y-direction ...
2
votes
1answer
17 views

linear modeling question: How can I find variance for vector Y?

Linear models are $$Y_1= 2\theta_1+3\theta_2 +\epsilon_1$$ $$Y_2= -2\theta_1+\theta_2 +\epsilon_2$$ and $\epsilon_1=3z_1-z_2$ and $\epsilon_2=4z_1+z_2$, where $z_1, z_2$ are two random variance such ...
0
votes
0answers
20 views

Compute mean of posterior distribution given prior

Is there a formula to compute the mean/expectation of a posterior distribution given the prior? In the ridge regression context, for example. I have $Y=X\theta + \epsilon$ where $\epsilon$ ~ $N(0, \...
1
vote
2answers
25 views

Estimator for linear regression where data points have different variances

So in the case where data points have the same variance $\sigma^2$, the estimator (in normal equation form) can be written as $$\theta=(X^TX)^{-1}X^TY$$ I'm not sure how to derive a similar formula ...
0
votes
0answers
9 views

Derivation of log maximum likelihood for multiple target variables

So I'm working through Pattern Recognition and Machine Learning from Bishop. In chapter 3.1.5 he generalizes fitting a function to multiple target variables. Ultimately, we have the function: $\...
0
votes
2answers
46 views

How to calculate parameters of a logarithmic approximation trendline?

I have a set of (Y) data $\left\{y_1, y_2, ..., y_n \right\}$ and a set of (X) $\left\{x_1, x_2, ..., x_n \right\}$ which I use to build a graph. I need to place a logarithmic trendline over the graph,...
0
votes
1answer
35 views

How to find the point after which a discrete function follows a linear and steady trend

I have many discrete functions that follow the same trend. An example of discrete function is shown in the figure below. At each step, represented on x-axis, we reduce a given area, represented on ...
0
votes
0answers
26 views

Weighted average of slope

I'm trying to find the weighted average of a temperature increase in a vessel. So, essentially the weighted average of a temperature over time. I'd like some input if this equation gets me what I'm ...
0
votes
0answers
15 views

P value graph meaning

What is the meaning of this graph? Why there is a p-value/2 in the right and not in the left?
0
votes
0answers
15 views

Given a confidence interval for B1, determine a confidence interval for the change in mean of Y

I have a simple linear regression model: $Y = B_{0} + B_{1} * X + e$ I determined a 95% confidence interval for $B_{1}$: $[-1.873 * 10^{-5} , -1.095 * 10^{-5}]$ And then the second part of the ...
0
votes
1answer
35 views

Derive estimator for weighted linear regression

I can't figure out to derive estimator for normal equations for weighted linear regression. (Supposed to be similar to normal equations.) I set up problem as $W(y-XB)^T(y-xB)$ My Steps: $W(y^Ty - ...
5
votes
0answers
173 views

Prove the estimator $\hat{B}$ of ridge regression = mean of the posterior distribution under a Gaussian prior

I want to prove that the estimator of ridge regression is the mean of the posterior distribution under Gaussian prior. $$y \sim N(X\beta,\sigma^2I),\quad \text{prior }\beta \sim N(0,\gamma^2 I).$$ $...
0
votes
2answers
52 views

How to apply the method least squares polynomial of single degree?

Now I am making Almon model. Lag is 3, and polynomial of 2 degree, so I have following linear regression equation $y_{t}$ = $a$ + $c_{0}$$z_{0}$+ $c_{1}$$z_{1}$+$c_{2}$$z_{2}$. I have a list of $y_{t}$...
1
vote
1answer
41 views

writing a piecewise regression model as a linear model

lets write the following piecewise regression model $$y= \alpha_0 + \alpha_1 x +\epsilon ;\ \ x\le x_0 $$ $$ y=\beta_0 +\beta_1 x + \epsilon \ \ x\gt x_0$$ according to the variable $x_0$ is known,...
2
votes
1answer
38 views

Kalman filter using regressed model

I'm currently polishing flight control system for KSP, and I'm fightinng high-frequency noise in state vector measurements right now. I want to try to apply Kalman filter to provide more smooth values ...
1
vote
0answers
16 views

Estimating required data in Gaussian Process regression

This question concerns determining a function, $f(x)$ say, based on noisy measurements $y = f(x) + \xi$ (where $\xi$ is IID gaussian noise) using Gaussian process machine learning with likelihood ...
1
vote
1answer
27 views

Variance of residuals from simple linear regression

I am trying to compute $Var(e_i)$. So far I have $Var(e_i)=Var(y_i-\hat y_i)=Var(y_i)+Var(\hat y_i)-2cov(y_i,\hat y_i)$ Now, I know that $Cov(y_i,\hat y_i)=var(\hat y_i)$ but how do I prove ...
0
votes
0answers
19 views

Do Monte Carlo perturbations capture all the uncertainty in prediction?

I have a model $M$ that I use to predict a value $y = M(\vec x)$. I have known one-$\sigma$ error bars on each input $x_i \in \vec x$. I want to know the one-$\sigma$ error bar on my prediction $y$. ...
0
votes
0answers
16 views

multiple regression Brown-Forsythe

I have a multiple regression model with three variables: $Y$= Labor Hours $X_1$= Cases Shipped $X_2$= Costs $X_3$= dummy variable which gives a $1$ if it falls on a holiday and $0$ if it does not. ...
0
votes
1answer
28 views

Least Square Estimators of a Linear Regression Model

A linear regression model may be written either: $Y_i$ = $\beta_0$ + $\beta_1X_i$ + $\epsilon_i$ Or $Y_i$ = $\alpha_0$ + $\alpha_1(X_i + \bar x)$ + $\epsilon_i$ Use the method of least square to ...
0
votes
1answer
19 views

How to find the least square estimators for linear regression model.

I have the linear regression model: $Y_i= \alpha_0 + \alpha_1(X_i - \overline{X})$ Anyway I got through the method for find the least square estimator for $\alpha_0$ and end up with $\sum_{i=1}^n \...
1
vote
1answer
46 views

Questions about derivation of linear regression.

I had few questions about linear regression derivation. SSE = Sum i=1toN (yi - bo - b1xi)^2 In above example, i simply found values bo and b1 where SSE is minimum by finding partial derivates of 'bo'...
1
vote
0answers
39 views

How is the sum of squares of residuals divided by variance has a chi-square distribution with n-2 degrees of freedom?

I came across this while reading the Linear Regression chapter in Sheldon Ross's Book: So my doubt is that How can I prove that it is a chi-square distribution with degree n-2. I looked up a bit ...
2
votes
1answer
28 views

Linear regression and standardization

I am trying to use a linear regression to model an expected value Y for an input X. X and Y have a large difference between them, so I was converting to standard (z) score, doing my calculation (...
1
vote
1answer
34 views

Linear regression with a given (non zero) intercept

If I have a simple linear regression model with the intercept $\beta_0$ known, would the least squares estimator of $\beta_1$ be $\frac{\sum(y_ix_i)}{\sum({x_i}^2)} - \frac{\beta_0*\sum(x_i)}{\sum(...
0
votes
1answer
36 views

E(b1)=beta1 regression

In the prove of $E[b_1]=\beta_1$ I saw this steps : $$ E[b_1]=E[\frac{S_{xy}}{S_{xx}}]=E[\sum[\frac{[(x_i - \bar{X})*y_i]}{\sum(x_i-\bar{X})}]]=\frac{\sum[(x_i-\bar{x})*E(y_i)]}{\sum(x_i-\bar{X})} ...$...
0
votes
1answer
70 views

When fitting a polynomial to data points, how to determine the reasonable degree to use?

I have wondered the following: Suppose that there is a set of data points $(x_i,y_i)$. Then I would like to know if it is more reasonable to assume if there is a polynomial relation of degree $m$ ...
1
vote
0answers
20 views

Penalty function of multi-peak fit?

The question I have is about the answer from here by @Silvia: http://mathematica.stackexchange.com/questions/26336/how-to-perform-a-multi-peak-fitting I can only understand some of the code but the ...
1
vote
2answers
301 views

Sxx in linear regression

What is the meaning of 'Sxx' and 'Sxy' in simple linear regression? I know the formula but what is the meaning of those formulas?
0
votes
0answers
14 views

What method to apply to center value in multiple linear regression?

This a multiple linear regression question, an approach for modeling the relationship between a scalar dependent variable $Y$ and several explanatory variables (or independent variables) denoted $X$. ...
0
votes
1answer
86 views

How to prove that non-diagonal elements of hat matrix (from regression) are limited?

I want to prove this inequality: $$h_{ij}^2 \le 0.25$$ where $h_{ij}$ is an element of hat matrix $H = X(X'X)^{-1}X'$ from multiple linear regression model ($ Y = X\beta + \epsilon$, $X$ is a $n\times ...
0
votes
0answers
38 views

When the predictor variable is so coded that $\bar X = 0$ and the normal error regression model applies, are $b_0$ and $b_1$ independent?

The Statement of the Problem: When the predictor variable is so coded that $\bar X = 0$ and the normal error regression model applies, are $b_0$ and $b_1$ independent? Are the joint confidence ...
1
vote
1answer
19 views

How to compute F test in a NLM?

I tried to do the F test for NLMs by myself and run into a dead-end. I have a linear model with normal distributed error $$Y = X\beta + e$$ I know that the test statistic $$F = \frac{n-r}{q}\frac{\|Q_{...
2
votes
2answers
44 views

Interpreting OLS Regression Coefficients with High Multicolinearity

I am having trouble understanding the interpretation of OLS coefficients when predictors are highly correlated. My understanding of OLS coefficients is that they estimate a change in the expected ...
0
votes
1answer
27 views

standardised random variable least square regression $X$ against $Y$, $Y$ against $X$

Let $X$ and $Y$ be mean 0 and variance 1 random variables such that we choose $\alpha$ and $\beta$ to minimise $$\mathbb{E}(X-\beta Y)^2$$ and $$\mathbb{E}(Y-\alpha X)^2$$ after not so difficult ...
1
vote
0answers
34 views

Please show that $f(\beta_0,\beta_1)=\log(1+\operatorname{exp}(-y_1(\beta_0+\beta_1 x_1)))+\log(1+\operatorname{exp}(-y_2(\beta_0+\beta_1 x_2)))$

I would like to show that the following result is indeed true. I am very new with this subject, so I ask for a hint to get me started please. Please show that $f(\beta_0,\beta_1)=\log(1+\...
0
votes
0answers
42 views

Estimating a probability distribution by fitting a function to a frequency histogram

If I want to estimate a probability distribution, is it common practice to simply fit a function to a frequency histogram? So, I am training a classifier, the performance of which is evaluated by its ...
1
vote
2answers
85 views

Why is $R^2=\rho^2$

Considering $y_i=\beta_1+\beta_2x_i+\epsilon_i$ $\bar y_i=\hat\beta_1+\hat\beta_2\bar x_i+\bar\epsilon_i$ a linear equation of least square used when it seems that there is a like between two data, $...
1
vote
0answers
13 views

How to get the relative contribution of each variable in a difference that forms the numerator?

I am facing a problem that may seem simple at first but with which I struggle. The question relates to economics where I try to see the effect of deficit at time t ($D_t$) onto Output at time t+h ($...
0
votes
0answers
31 views

variances of the slope and intersect of an orthogonal / Deming) linear regression

I am a humble tinkerer who tries to get a rover to run a SLAM (simultaneous localization and mapping) process in his house. I equipped the rover with a laser rangefinder which collect distance and ...
2
votes
1answer
44 views

Discrete version of continuous SIR model

I'm working with a SIR infection model, which is $$\begin{array}{rcl} \frac{dS}{dt} & = & -\beta IS\\ \frac{dI}{dt} & = & \beta IS-\gamma I\\ \frac{dR}{dt} & = & \gamma I \end{...
1
vote
0answers
57 views

Least-squares fit of a nonlinear (polar) system

I want to determine the six unknown coefficients (uppercase letters) of the model $$x=X_c+(Au+B)\cos(Cv+D),\\y=Y_c+(Au+B)\sin(Cv+D)$$ given a set of data $(x_k,y_k,u_k,v_k)$, by least-squares. As ...