Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

Nonlinear regression

Let's consider the model $$y_1=\cos^2(\theta)+\epsilon_1,\\ y_2=\sin^2(\theta)+\epsilon_2,$$ where $\epsilon=(\epsilon_1, \epsilon_2)^\top$, $\epsilon\sim N_{2}(0,\sigma^2I)$, $\theta\in\left[0, \...
0
votes
0answers
12 views

In linear regression what does $E( Y | X) = 0$ mean?

I have this statistics problem that I have trouble understanding the question. The regression model $Y = \beta x + e$ ($e \sim N(o, \sigma^2) $) is a model that passes by the origin meaning that $E(...
0
votes
0answers
4 views

Are stationarity and low autocorrelation the prerequisite of regression model?

As said in the title, are stationarity and low autocorrelation the prerequisite of general / linear regression model ? That is, if a time series is non-stationary or large autocorrelation, would it be ...
0
votes
0answers
10 views

How can I calculate the derivative of J(Θ) which contains sigmoid function in linear regression?

I am taking a machine learning course and since my math is extrementy rusty I am not sure how to go about solving this exercise. Given the derivative of the sigmoid function with respect to Θj I need ...
1
vote
0answers
17 views

prove that $(\hat{\beta}-\beta)'(X'X)^{-1}(\hat{\beta}-\beta)$ and SSE are independent

We have $$(\hat{\beta}-\beta)'(X'X)^{-1}(\hat{\beta}-\beta)$$ and SSE $=\sum_{i=1}^{n}(\hat{y_{i}}-y_{i})^{2}=\sum_{i=1}^{n}y_{i}^{2}-\hat{\beta'}X'y$ and I have to prove that these 2 quantities are ...
0
votes
1answer
27 views

How to prove sum of errors follow a chi square with $n-2$ degree of freedom in simple linear regression

In simple linear regression, the model is \begin{equation} Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i \end{equation} where $\varepsilon_i$ are i.i.d., and \begin{equation} \varepsilon_i \sim N(0, \...
0
votes
1answer
809 views

When fitting a polynomial to data points, how to determine the reasonable degree to use?

I have wondered the following: Suppose that there is a set of data points $(x_i,y_i)$. Then I would like to know if it is more reasonable to assume if there is a polynomial relation of degree $m$ ...
2
votes
1answer
48 views

Why do two different approaches using matrix algebra yield different results?

I am solving a problem involving linear regression and I am finding that two different approaches to the matrix algebra is yielding different results. In the second approach, I am merely simplifying ...
0
votes
0answers
22 views

Conditional probability with geometric distribution

$X$ and $Y$ are linearly independent random variables with geometric distribution. They have the same $p$ parameter. How can I calculate the following expressions: a) $P(X=k|X<Y)$=? b) $P(X=k|X=Y)...
0
votes
0answers
13 views

Apply least squares fitting when dealing with several matrices in the objective function

Let's say we have a regular photo and three low-light photos illuminated in different colors. Each pixel is a three-component vector $q=(R,G,B)$. Then $q_k^{A}$ is the $k$-th pixel of the regular ...
0
votes
1answer
46 views

Values of F and T (distribution tables) in Anova

I have to find the $p$-value but I'm having difficulty with the F values. I have this table: Now after some computation from an exercice, i calculate the F test statistic like so: $F = \text{MSS}/...
0
votes
0answers
6 views

Dynamically weighted distance function for vectors in vector space with weighted correlations between vector components

In my problem I am generating a vector space in which every vector represents a concept (representing a specific situation). The components of each vector represent different characteristics of the ...
1
vote
1answer
27 views

Understanding interaction terms in a regression

To my understanding interaction terms help if you believe the relationship between some terms undergoes a fundamental change. I am working on this exercise in my book: And I computed a linear ...
0
votes
0answers
17 views

Effects of the order of averaging and regressing

Suppose we have information (dependent variable: income, independent variables: age, gender, treatment (Boolean), etc) on individuals from different cities and we want to see the effect of treatment ...
0
votes
0answers
28 views

Good polynomials to use that can fit closed complex 2D shapes

Are there any good polynomials that can be used to fit closed complex 2D shapes? For example, the silhouette of everyday objects or people. I am aware I can use a spline for example, however, they ...
0
votes
0answers
12 views

How to solve multiple regression using canonical correlation analysis?

I've read that the results from multiple regression and canonical correlation analysis are the same, aside from scaling. No one online has shown how to prove this by hand, since most examples for ...
0
votes
0answers
23 views

linear regression-positive answers

Assume we have an over-determined linear system as $Mx=y$. We know that this system has infinitely many solutions. In case the variables($x$) are free in sign, we can find a solution to this system ...
0
votes
2answers
34 views

Is there a general way to determine the best combination of parameters to fit points?

If the number of the given points is greater than or equal to the number of the parameters in the model, is it always possible to determine those parameters? See my previous problem, Claude Leibovici ...
-1
votes
0answers
14 views

why is E(x|u)=cov(x'u)?

when I learn how to use the instrumental variable to solve the endogenous problem, I learned the mechanism that E(x|u)=cov(x'u)≠0, in other words, the x is related to the error term. However, why E(x|...
0
votes
1answer
16 views

Finding equation of best fit line in simple linear regression

To find the best fit line for a set of data $(x_i,y_i)$ by minimising the sum of least squares, one requires to minimize $\frac{\partial S}{\partial \theta{_0}}$ and $\frac{\partial S}{\partial \theta{...
0
votes
2answers
41 views

Regression of the form $y=a-be^{-x}$

Say we have the points $(x_1,y_1),(x_2,y_2),(x_3,y_3),\dots,(x_n,y_n)$ and we need to find the best fit equation that is of the form $y=a-be^{-x}$ (the red curve). See the illustration plot: Based on ...
1
vote
0answers
16 views

Is it possible to solve for the convolution kernels defining the contributions of multiple input signals to an output signal?

Suppose that I have a function, $h$, which is the sum of several convolutions $$ h = \sum_{i=1}^{n} f_{i}*g_{i} $$ where the asterisk represents convolution. By the convolution theorem we know that ...
1
vote
1answer
31 views

What is the estimate coefficient when design matrix is singular, when using the Moore-Penrose generalized inverse matrix?

The OLS estimators is But if $Y$ is a $50 \times 1$ matrix, and the design matrix $X$ is $50 \times 16$. The design matrix $X^T X$ is singular, then what will the estimate coefficient function be, ...
0
votes
1answer
46 views

Linear regression (quadratic form)

I'm doing practice problems and i'm having a bit of difficulty with this one. I'm not used to deal with quadratic forms in regression. I tried my possible to answer it but i'm not sure it's right? ...
0
votes
2answers
31 views

How to perform multivariate linear regression on this example?

So I have a table with points in time $t_i$ and measurements $s_i$ give by \begin{bmatrix}t_i&0&1&2&3\\s_i&4.30&1.48&0.56&0.24\end{bmatrix} As a regression function I'...
0
votes
0answers
19 views

Probability limit of the estimate

So I have two equations one being demand and the other supply $q_i^d = \alpha_0 + \alpha_1p_i + u_i$ and $q_i^s = \beta_0 + \beta_1p_i + v_i$ and I know that $q_i^d = q_i^s = q_i$. Additionally $E(...
0
votes
0answers
30 views

Fitting a simple polynomial to different possibilites

I have a few sets of numbers. For example, let's say my sets are - 1,5,8,12,25 2,14,36,45,47 5,9,17,55,80 29,37,64,93,102 48,69,77,86,167 Given these sets, I want to fit a ...
0
votes
0answers
33 views

How do I prove the diagonal element of the hat matrix, $h_{ii}$ (leverage) [duplicate]

Consider the simple linear regression model $y_i = \beta_0 + \beta_1x_1 + \text{error}, i = 1,2,\ldots,n$ $h_{ij}$ denotes the $[i,j]$ element of the hat matrix. How can I show that $h_{ii} = 1/n + (...
3
votes
4answers
2k views

Curve fitting with derivatives

Is there any tool to do curve fitting with derivative values? I.e. I have a bunch of values of the function at certain points, a bunch of values of the function's derivative at certain points, a bunch ...
1
vote
0answers
28 views

Please verify whether the calculating process of the WLS estimator and the variance is correct or not.

There are two Heteroscedasticity regression models 1. $$ y_i = \beta x_i + \epsilon_i, \quad i=1, \ldots, n $$ where $\epsilon_i$'s are independent and distributed as $\epsilon_i \sim N(0, \...
0
votes
0answers
15 views

What is the unit increase in coefficient analysis for logistic regression?

What does the unit increase mean in coefficient analysis for logistic regression? I have normalized my data to a mean of 0 and standard deviation of 1. How does this change the unit increase? Is this ...
0
votes
0answers
30 views

Identifying the distribution of the least squares estimate with linear regression

The model is set up as follows: Random variables $Y_1,Y_2,...,Y_n$ are given by $Y_i=\alpha+\beta x_i+\epsilon_i$ where $\alpha,\beta$ and $x_1,...,x_n$ are constants with $\sum_{i=1}^n x_i=0$ and $\...
-1
votes
0answers
6 views

T statistic for intercept and coefficient

So, assuming that W0 is the intercept and W1 is the coefficient in a simple linear regression model, the way to calculate a t statistic for W1 is (W1-0) /std error of w1 Now, my question is how do I ...
1
vote
1answer
33 views

Prove that $\frac{(n-2)s^2}{\sigma^2} \sim \chi^{2}_{n-2}$

Consider the following simple linear regression model involving the $\epsilon_i$ error term, $$y_i = \alpha + \beta x_i + \epsilon_i$$ such that, $$\epsilon_i \sim \mathcal N(0,\sigma^2)$$ we know ...
0
votes
1answer
21 views

Residual analysis in Python

when doing residual analysis do we first fit our model on our entire training set and calculate residuals between fitted values and actual values? Or do we first fit our model on the training+testing ...
1
vote
1answer
878 views

how to prove that $\hat \sigma^2$ is a consistent for $\sigma^2$

Consider a regression model $Y_n=X_n\beta +\varepsilon$, where $X_n$ is a $n \times p_n$ matrix, and $\varepsilon=(\varepsilon_1,...,\varepsilon_n)'$ consists of independent and identically ...
4
votes
1answer
266 views

Area enclosed by a function containing two power towers: $f(x)=(−\ln(x↑↑(2k)))↑↑(2k+1)$

I've been considering functions involving power towers lately and come across the following function: $$f(x)=(−\ln(x↑↑(2k)))↑↑(2k+1)$$ $$\text{Where }k∈\mathbb{Z} ^+$$ In the image below we can see ...
0
votes
0answers
9 views

Gaussian process regression for large dimensional input space

I am working for Gaussian process (GP) regression $y=f(\mathbf{x})+\epsilon$ with $\mathbf{x}\in \mathbb{R}^D$ and $D$ can be as large as few 100. I have attempted with the GPML toolbox and the exact ...
2
votes
1answer
112 views

Linear regression: equivalence of forms of the minimum variance affine unbiased estimator

Background Consider the linear regression model: $$y = X\beta + e\\E[e] = 0 \quad E[ee^T] = V$$ It is well known that the minimum variance affine unbiased estimator (MVAUE) of $\beta$ exists if and ...
0
votes
0answers
9 views

Summation Proof For $ Y_{ij} - \bar Y_j $

Currently trying to prove that the sum of squares error can be partitioned into SSPE and SSLF. $ SSE = \sum_{j=1}^c \sum_{i=1}^{nj} (Y_{ij}- \hat Y_{ij})^2 = \sum_{j=1}^c \sum_{i=1}^{nj} (Y_{ij}- \...
0
votes
1answer
40 views

Show that $X^T X = \sum_{i=1}^n \mathbf{x}_i \mathbf{x}_i^T$

Referring to this post: https://stats.stackexchange.com/questions/164223/proof-of-loocv-formula The line which says $\sum_{t=1}^TX_t'X_t=X'X$ is the result I'm trying to interpret. Or in perhaps ...
0
votes
0answers
10 views

exponential regression model in accelerate life design

Suppose time to failure $T$ follows exponential distribution and its p.d.f is $$f_T(t)=\lambda e^{-\lambda t}$$ Let $Y=lnT$, then the p.d.f for $Y$ is $$f_Y(y)=\lambda exp(-\lambda e^{y}+y)$$ $$=...
0
votes
0answers
8 views

Cross validation multivariate kernel regression in R

This question is general- I have a data set of n observations, consisting of a single response variable y and p regressor variables ( here, n ~50, p~3 or 4). I am planning to implement Nadaraya-Watson ...
-1
votes
0answers
10 views

Optimal decision rule - ROC Curve & Logistic regression

Table here - 10 entries There are 10 inputs in the above table. A binary yes or no to indicate if they voted and a model score. I plotted the ROC curve for the model and calculated the optimal ...
6
votes
1answer
5k views

Analytic solution for matrix factorization using alternating least squares

The standard form for ridge regression aims to minimize the following cost function. $$ \min\ \ \sum_i(y_i-x_i^T\beta)^2 + \lambda\sum_j\beta^2_j $$ As described here, it's possible to differentiate ...
0
votes
1answer
33 views

Consistency of OLS estimation

My question is about consistency of OLS parameter estimate for a linear regression model $y_t = x_t' b + \epsilon_t$ ($t = 1,2, \ldots, T)$. Here $x_t$ and $b$ are $1\times k$ vectors (there are $k$ ...
0
votes
1answer
74 views

Multivariate Quadratic Regression, Surface Fitting and the Hessian

I need to fit a quadratic surface to multidimensional data, one of the methods mentioned is using a polynomial basis function, $$ \phi(x) = [ 1, x_1, ..., x_n x_1^2, ..., x_n^2 x_1 x_2, ..., x_{n-1}...
1
vote
0answers
49 views

Finding Conditional Expected Value and Conditional Variance

Very stuck with this problem. Not meant to make any more assumptions than laid outin the question. I have a regression model: $\ln(𝑠𝑎𝑣_𝑖)$ = $𝛽_0 + 𝛽_1\cdot \ln (𝑖𝑛𝑐_i) + 𝑢_i$ (where sav ...
0
votes
0answers
10 views

struggling to understand R output for linear regression with categorical variables

If i where to fit some random linear regression model like fit<-lm(Height~factor(Location)+Daily.calcium.intake+Fathers.daily.prtintake,data=some data set ) as a random example where Location ...
1
vote
1answer
795 views

what is the difference between 'estimate of residual standard error' and 'residual standard error'?

What is the difference between 'estimate of residual standard error' and 'residual standard error'? Can someone please provide the formulas? Thanks!