Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression]

Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

1
vote
2answers
18 views

Showing that $\hat \beta_1 = S_{xy}/S_{xx}$ for a simple linear regression

To show $\hat \beta_1 = S_{xy}/S_{xx}$ I know I can use $\mathbf{\hat\beta_{2x1} = (X'X)^{-1} X' y}$ However, when I do this problem I only get to this step and I'm unsure if I'm even taking the ...
-1
votes
0answers
15 views

GARCH(1,1) Stochastic Process

I have seen textbooks state the following are equivalent formulations for GARCH(1,1): $dV = (1-\alpha-\beta) (V_L - V) dt + \alpha V \sqrt{2} dW_{t}$ $\sigma_n^2 = (1-\alpha-\beta) \cdot V_L + \...
0
votes
1answer
22 views

Deriving max. likelihood estimate of β for a logistic model of two classes with a single binary regressor

I have the log-likelihood function: $$l(\overrightarrow\beta)=\sum_{i=1}^n [y_i log(p(\overrightarrow x_i;\overrightarrow\beta))+(1-y_i)log(1-p(\overrightarrow x_i;\overrightarrow\beta)] $$ where $...
1
vote
0answers
20 views

How to prove this (alternative) solution of ridge regression?

The ridge regression optimization formula for $X\in\mathbb{R}^{n\times m}$, $y\in\mathbb{R}^n$ with $n>m$, $$ \arg \min_{w\in \mathbb{R}^n} ||y-Xw||_2^2 + \lambda ||w||_2^2 $$ I know that the ...
0
votes
0answers
22 views

log transformation in Regression and correlation coefficient [closed]

Let μX and μY be respective means of X and Y , let σ2 be the common variance of X and Y , and let ρ be the correlation coefficient of X and Y . Also, let β be the slope of regressing log Y on log X ...
0
votes
0answers
18 views

For science and tech, is there really a practical reason to linearize equations before plotting to find constants?

I understand that if you have a lot of equations that it will speed up computation time. But I often see in papers someone will linearize something like the Michaelis Menten equation to then solve ...
1
vote
0answers
43 views

Minimizing Frobenius Norm subject to a constraint

I have the following problem: $ \min_{A} || A - X||_{F}\\s.t. \theta = (A^TA)^{-1}A^Ty$ where $A$ and $X$ are $n\space x \space p$ matrices and $y$ is a $n\space x \space 1$ vector. $\theta$ is the ...
0
votes
1answer
33 views

How can I prove $\hat\beta_0$ and $\hat\beta_1$ are linear in $\hat Y_i$?

A fitted regression line of a linear model is given by : $$ \hat Y = \hat\beta_0 + \hat\beta_1X $$ How can I prove $\hat\beta_0$ and $\hat\beta_1$ are linear in $\hat Y_i$ ? I'm unsure where to ...
0
votes
0answers
13 views

Formula for finding the slope of a regression line

I want to calculate the gradient of a regression line and have been looking at various websites and books to find the formula for it. My school textbook writes that this is: $$m={S_{xy}\over(S_x)^2}$$...
13
votes
1answer
239 views

On the integral $\int_{-\frac\pi2}^{\frac\pi2}\sin(x/\sin(x/\sin(x/\sin\cdots)))dx$

This question is the final one out of the set (see I and II), I promise! Consider $f_1(x)=\sin(x)$ and $f_2(x)=\sin\left(\frac x{f_1(x)}\right)$ such that $f_n$ satisfies the relation $$f_n(x)=\sin\...
0
votes
0answers
12 views

Alternative to linear regression to predict values over time?

I have a model which consists of (date, amount). Currently I'm using linear regression to predict values for the next 2 weeks as follows: ...
1
vote
1answer
22 views

Proof Verification: $\tilde{\beta_1}$ is an unbiased estimator of $\beta_1$ obtained by assuming intercept is zero

Consider the standard simple regression model $y= \beta_o + \beta_1 x +u$ under the Gauss-Markov Assumptions SLR.1 through SLR.5. Let $\tilde{\beta_1}$ be the estimator for $\beta_1$ obtained by ...
0
votes
0answers
6 views

Smoothing algorithm for unequal variances in data

I have data in the form of a histogram with bins that each have their own error bars. I'm interested in finding a smoothing algorithm that fits the data while taking the error bars into account (e.g. ...
1
vote
4answers
34 views

Regression of Exponential Ramp

Given a set of measured data ( temperature ), I need to estimate parameters of the exponential function which I suppose be the best fit: $y=A+C(1-e^{-t/\tau}$) From an operative point of view, given ...
0
votes
1answer
29 views

Least Squares estimator of $\beta$ using matrices

Show, using matrix notation and staring with the principle of least squares, that the least squares estimator of $\beta$ is given by: $\hat\beta =$ $\frac{\sum_{i=0}^nx_iy_i}{\sum_{i=0}^n x_i^2}$ I'...
1
vote
1answer
16 views

Simple Linear Regression problem involving its design matrix

Problem states: Consider the simple linear regression model without intercept, i.e. $y_i = \beta x_i + \epsilon_i, i = 1, 2, ..., n$ Write down your design matrix, $\mathbf X$. So then $\mathbf X$ $...
0
votes
1answer
41 views

Least Squares Criterion Problem

Problem: Show that the least squares criterion applied to the "intercept-only" model, i.e. $y_i = \beta_0 + \epsilon_i$, $i = 1, 2, ..., n$ results in the least squares estimator of $\beta_0: \hat \...
0
votes
2answers
35 views

Forcing Quadratic Fit to Origin or to a Point

I want to fit some data using quadratic fit. However, I need to force the curve to go through the origin point, or any other point. How can I solve it mathematically?
2
votes
1answer
28 views

Sharper definition of Least Squares Criterion?

For one of my questions on my test, we were to define the least squares criterion. I said: When we create a linear model we are attempting to minimize the distance between each of the data points ...
1
vote
0answers
19 views

Why can't we interpret the meaning of intercept in the sample regression function by putting x=0?

In the book Woolridge, this is written, "The intercept, $\beta_o$, is the predicted value of y when x = 0, although in some cases it will not make sense to set x = 0. In those situations, $\beta_o$ is ...
1
vote
0answers
24 views

What is the difference between a Savitzky-Golay filter and LOESS?

I don't fully understand the difference between these two smoothing algorithms. It seems like they both take a window, fit a polynomial, sample from the fit, and move on. I would guess maybe the ...
-1
votes
2answers
25 views

Regression - slope doubt

The gradient of the regression line $x$ on $y$ is $-0.2$ and the line passes through $(0,3)$. If the equation of the line is $x = c + dy$, find the value of $c$ and $d$. What I did: As per my ...
0
votes
1answer
17 views

Statistics (Regression Analysis): Show that the residuals from a linear regression model can be expressed as $e=(I-H) \epsilon$

Statistics (Regression Analysis): Show that the residuals from a linear regression model can be expressed as $\mathbf{e} = (\mathbf{I}-\mathbf{H})\mathbf{\epsilon}$ The bold represents vectors or ...
1
vote
0answers
18 views

Regression Model Sensitivity - show that when coefficient small, prediction not very sensitive with respect to change in feature vector

Quote Consider the regression model $\hat{y} = x^T\beta + v$, where $\hat{y}$ is the prediction, $x$ is a feature vector, $\beta$ is a coefficient vector, and $v$ is the offset term. If $x$ and $\...
3
votes
1answer
124 views

Matrix Regression for linear ODE system

Background I have the following homogeneous ODE system as an Initial Value Problem: $$ y'=A\cdot y\quad\wedge\quad y(0)=y_0 $$ where $y\in\mathbb{R}^{N\times 1}$ is the unknown vector and $A\in\...
2
votes
2answers
82 views

How to feed data into a polynomial basis function regression (unregularized) for degree n?

We know that polynomial base function Models is: $$t = \sum_{i=0}^n w^T\phi_j(x) = w_0*\phi_0(x) + w_1*\phi_1(x) + w_2*\phi_2(x)+ ....$$ $$\phi_j(x) = x^j$$ Problem: I am not sure how to pass the ...
0
votes
0answers
12 views

Considering heteroskedasticity in Cp approach to adjusting training error rate in regression

I have been introduced to $C_p$ as a way to adjust the training error rate to account for bias due to overfitting regression models. $C_p$ is defined as such: $C_p = \frac{1}{n} (RSS + 2d\hat{\sigma}...
0
votes
0answers
13 views

find a confidence interval in a linear model problem

i'm trying to solve a problem that involve a linear model given its normal equations, and the errors have a normal distribution but i'm a little lost. the problem is about construct a 95% ...
0
votes
0answers
15 views

how to optimize reduced rank regression with constant diagnoal constraint?

I am trying to optimize a panel regression $G=\beta G+e$. $G$ is $N \times T$ matrix. $\beta$ is $N\times N$ unknown coefficient matrix. Each column of G represents the growth of all stocks at period $...
0
votes
2answers
22 views

Does $E(\sum e_i^2) = \sum E(y_i^2) - E(\sum \hat y_i^2)$ hold true?

This was posted as a practice proof for a regressions class. I've worked through it from the perspective of $SSE = SST - SSR$, but I cannot reduce to the given equation. There were other mistakes made ...
1
vote
1answer
15 views

estimate of a slope in the simple linear regression model $y=\beta_0+\beta_1 x+\epsilon$

I have two formulas for estimate of a slope in the simple linear regression model $y=\beta_0+\beta_1 x+\epsilon$: $\hat{\beta_1}=\frac{\sum_{i=1}^N(x_i-\bar{x})(y_i-\bar{y})}{\sum^N_{i=1}(x_i-\bar{x}...
0
votes
1answer
33 views

Find X in a 4PL Curve Regression

I'm working on a project where I need to replicate a calculation that is currently done by a legacy system, let's call this legacy system of 4P. In the company nobody knows how the calculation is done,...
0
votes
0answers
14 views

How do I optimally sample to fit a function to data?

Setup I have categorical (true/false) data $H$ of whether a neuron fires an action potential (spike) under various stimulation conditions $z = [S, D]$ (strength, duration). The neuron is described ...
1
vote
2answers
54 views

How do the normal equations *always* have a solution?

My professor says that "the normal equations always have a solution", even when $A$ is not full rank. HOwever, this does not make sense to me. The normal equations are $$A^\dagger=(A^TA)^{-1}A^T$$ ...
0
votes
1answer
29 views

Introduction to Regression Analysis Proof

Show that: $\sum_{i=1}^n(x_i-\bar x)^2 = $ $\sum_{i=1}^nx_i^2$ $-$$(\frac{\sum_{i=1}^n(x_i)^2}{n})$ I know $n$ is a positive number. I think I should start with $\bar x$ $=$ $\frac{\sum_{i=1}^n(x_i)}{...
0
votes
0answers
24 views

Fitting a curve to $N$ dimensional data

I have a data set with $N$ independent variables and one dependent variable(function of all the $N$ independent variables). The dependent variable is either $0$ or $1$ (like a step function). I am ...
0
votes
1answer
37 views

How do I construct a regression model for some data which is of hyperbolic form?

I have some data for an object moving under constant acceleration. Velocity $(m/s)$ $[0.84,1.58,2.32,3.06,3.80,4.54,5.28]$ Time $(s)$ $[0,1.5,3.0,4.5,6.0,7.5,9.0]$ The usual approach would be to ...
0
votes
0answers
10 views

Decrease Beta bewteen to variables

I have two time series of return data. One is global equity and one is a seperate stock. I would like to know if there is a theoretical way of decreasing the EQ Beta of the stock? To be more clear: ...
2
votes
0answers
31 views

From Ng video: Using feature normalization with polynomial regression

In this video on machine learning by Andrew Ng, called "Features and Polynomial Regression", at time 4:34, he mentions the possibility of feature normalization in polynomial regression. By which he ...
0
votes
0answers
22 views

What is copula regression basicly?

Iam learning copula regression. Could you help me? What is copula regression basicly? Which studies I can read for learning this? What is your simple definition for copula regression?
1
vote
1answer
21 views

Fitting probability distribution connection to regression

My question is whether there is some connection between fitting probability distribution on some data set and linear regression? Or this two tools are for different problems? By fitting probability ...
0
votes
0answers
32 views

Correlation coefficient in terms of standard units: intuition

Correlation coefficient $$r = \frac{1}{n}\sum_{i=1}^n\frac{(x_i-\bar x)(y_i-\bar y)}{\sigma_x\cdot\sigma_y}$$ But for a given data point $x_i$ and predicted value $y_p$, $$\frac{(y_p-\bar y)}{\...
0
votes
0answers
28 views

Probabilistically predicting the output of an algorithm

I have an (optimization) algorithm that, on every iteration, outputs an integer number. Output is non-increasing; plotting the output wrt. cycle index shows that the relation from output to cycles ...
2
votes
1answer
19 views

How to show $\operatorname{Cov}(b_0,b_1)=-\frac{\sigma\bar{x}}{S_{xx}}$

Consider the equation $y_i=\beta_0+\beta_1x_i+\epsilon_i$ for $i=1, \dotsc, n$. We have unbiased estimators $b_0$ and $b_1$ for $\beta_0$ and $\beta_1$ respectively, where $b_0=\bar{y}-b_1\bar{x}$ ...
3
votes
1answer
55 views

Correlation coefficient and regression line : Geometric intuition

correlation coefficient $$r = \frac{1}{n}\sum_{i=1}^n\frac{(x_i-\bar x)(y_i-\bar y)}{\sigma_x\cdot\sigma_y}$$ may be thought of as cosine of angle between two $n$-dimensional vectors $$ (x_1- \bar ...
0
votes
0answers
44 views

Why are dependent (predicted) variables replicating one of the explanatory variables?

In a multiplie regression problem, I have 3 different explanatory variables X1, X2, X3 that try to explain variations in the dependent variable Y. The result (Y) is always almost the same as one of ...
0
votes
0answers
50 views

What kind of optimization/regularization is this?

If I have a system: $y_{n \times 1} = A_{n \times m} x_{m \times 1}$ where $n<m$ and $rank=n$ What kind of optimization/regularization am I doing if I use: $x=\min\{\|x\|_2^2+1/2\lambda^T(y-Ax)\}...
1
vote
1answer
22 views

Choosing sample points for polynomial regression to improve accuracy

An online set of slides [ link, start on page 4] describes the following situation: we're doing polynomial regression using the model $$\texttt{prediction}(x) := w_0 + w_1x + \dotsb + w_kx^k + \...
0
votes
0answers
23 views

Consider a regression model

Consider the regression model: rcon, the (excess) rate of return on a portfolio with construction industry assets, rmrf, the (excess) rate of return on a market-wide well-diversified portfolio, $$ \...
4
votes
1answer
228 views

Basic exponential regression

Background: I've been struggling with an exponential regression problem for about 8 months now (on and off): Vertically translated depreciation curve: Update the exponential regression coefficient ...