Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Nonlinear regression

Let's consider the model $$y_1=\cos^2(\theta)+\epsilon_1,\\ y_2=\sin^2(\theta)+\epsilon_2,$$ where $\epsilon=(\epsilon_1, \epsilon_2)^\top$, $\epsilon\sim N_{2}(0,\sigma^2I)$, $\theta\in\left[0, \...
0
votes
0answers
13 views

In linear regression what does $E( Y | X) = 0$ mean?

I have this statistics problem that I have trouble understanding the question. The regression model $Y = \beta x + e$ ($e \sim N(o, \sigma^2) $) is a model that passes by the origin meaning that $E(...
0
votes
0answers
4 views

Are stationarity and low autocorrelation the prerequisite of regression model?

As said in the title, are stationarity and low autocorrelation the prerequisite of general / linear regression model ? That is, if a time series is non-stationary or large autocorrelation, would it be ...
0
votes
0answers
10 views

How can I calculate the derivative of J(Θ) which contains sigmoid function in linear regression?

I am taking a machine learning course and since my math is extrementy rusty I am not sure how to go about solving this exercise. Given the derivative of the sigmoid function with respect to Θj I need ...
1
vote
0answers
17 views

prove that $(\hat{\beta}-\beta)'(X'X)^{-1}(\hat{\beta}-\beta)$ and SSE are independent

We have $$(\hat{\beta}-\beta)'(X'X)^{-1}(\hat{\beta}-\beta)$$ and SSE $=\sum_{i=1}^{n}(\hat{y_{i}}-y_{i})^{2}=\sum_{i=1}^{n}y_{i}^{2}-\hat{\beta'}X'y$ and I have to prove that these 2 quantities are ...
2
votes
1answer
48 views

Why do two different approaches using matrix algebra yield different results?

I am solving a problem involving linear regression and I am finding that two different approaches to the matrix algebra is yielding different results. In the second approach, I am merely simplifying ...
0
votes
0answers
22 views

Conditional probability with geometric distribution

$X$ and $Y$ are linearly independent random variables with geometric distribution. They have the same $p$ parameter. How can I calculate the following expressions: a) $P(X=k|X<Y)$=? b) $P(X=k|X=Y)...
0
votes
0answers
13 views

Apply least squares fitting when dealing with several matrices in the objective function

Let's say we have a regular photo and three low-light photos illuminated in different colors. Each pixel is a three-component vector $q=(R,G,B)$. Then $q_k^{A}$ is the $k$-th pixel of the regular ...
0
votes
0answers
6 views

Dynamically weighted distance function for vectors in vector space with weighted correlations between vector components

In my problem I am generating a vector space in which every vector represents a concept (representing a specific situation). The components of each vector represent different characteristics of the ...
0
votes
0answers
17 views

Effects of the order of averaging and regressing

Suppose we have information (dependent variable: income, independent variables: age, gender, treatment (Boolean), etc) on individuals from different cities and we want to see the effect of treatment ...
0
votes
0answers
29 views

Good polynomials to use that can fit closed complex 2D shapes

Are there any good polynomials that can be used to fit closed complex 2D shapes? For example, the silhouette of everyday objects or people. I am aware I can use a spline for example, however, they ...
1
vote
1answer
27 views

Understanding interaction terms in a regression

To my understanding interaction terms help if you believe the relationship between some terms undergoes a fundamental change. I am working on this exercise in my book: And I computed a linear ...
0
votes
0answers
12 views

How to solve multiple regression using canonical correlation analysis?

I've read that the results from multiple regression and canonical correlation analysis are the same, aside from scaling. No one online has shown how to prove this by hand, since most examples for ...
0
votes
2answers
34 views

Is there a general way to determine the best combination of parameters to fit points?

If the number of the given points is greater than or equal to the number of the parameters in the model, is it always possible to determine those parameters? See my previous problem, Claude Leibovici ...
-1
votes
0answers
14 views

why is E(x|u)=cov(x'u)?

when I learn how to use the instrumental variable to solve the endogenous problem, I learned the mechanism that E(x|u)=cov(x'u)≠0, in other words, the x is related to the error term. However, why E(x|...
0
votes
2answers
41 views

Regression of the form $y=a-be^{-x}$

Say we have the points $(x_1,y_1),(x_2,y_2),(x_3,y_3),\dots,(x_n,y_n)$ and we need to find the best fit equation that is of the form $y=a-be^{-x}$ (the red curve). See the illustration plot: Based on ...
1
vote
0answers
16 views

Is it possible to solve for the convolution kernels defining the contributions of multiple input signals to an output signal?

Suppose that I have a function, $h$, which is the sum of several convolutions $$ h = \sum_{i=1}^{n} f_{i}*g_{i} $$ where the asterisk represents convolution. By the convolution theorem we know that ...
0
votes
1answer
16 views

Finding equation of best fit line in simple linear regression

To find the best fit line for a set of data $(x_i,y_i)$ by minimising the sum of least squares, one requires to minimize $\frac{\partial S}{\partial \theta{_0}}$ and $\frac{\partial S}{\partial \theta{...
0
votes
1answer
46 views

Values of F and T (distribution tables) in Anova

I have to find the $p$-value but I'm having difficulty with the F values. I have this table: Now after some computation from an exercice, i calculate the F test statistic like so: $F = \text{MSS}/...
0
votes
1answer
47 views

Linear regression (quadratic form)

I'm doing practice problems and i'm having a bit of difficulty with this one. I'm not used to deal with quadratic forms in regression. I tried my possible to answer it but i'm not sure it's right? ...
0
votes
0answers
19 views

Probability limit of the estimate

So I have two equations one being demand and the other supply $q_i^d = \alpha_0 + \alpha_1p_i + u_i$ and $q_i^s = \beta_0 + \beta_1p_i + v_i$ and I know that $q_i^d = q_i^s = q_i$. Additionally $E(...
1
vote
1answer
31 views

What is the estimate coefficient when design matrix is singular, when using the Moore-Penrose generalized inverse matrix?

The OLS estimators is But if $Y$ is a $50 \times 1$ matrix, and the design matrix $X$ is $50 \times 16$. The design matrix $X^T X$ is singular, then what will the estimate coefficient function be, ...
0
votes
0answers
30 views

Fitting a simple polynomial to different possibilites

I have a few sets of numbers. For example, let's say my sets are - 1,5,8,12,25 2,14,36,45,47 5,9,17,55,80 29,37,64,93,102 48,69,77,86,167 Given these sets, I want to fit a ...
0
votes
1answer
27 views

How to prove sum of errors follow a chi square with $n-2$ degree of freedom in simple linear regression

In simple linear regression, the model is \begin{equation} Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i \end{equation} where $\varepsilon_i$ are i.i.d., and \begin{equation} \varepsilon_i \sim N(0, \...
0
votes
0answers
33 views

How do I prove the diagonal element of the hat matrix, $h_{ii}$ (leverage) [duplicate]

Consider the simple linear regression model $y_i = \beta_0 + \beta_1x_1 + \text{error}, i = 1,2,\ldots,n$ $h_{ij}$ denotes the $[i,j]$ element of the hat matrix. How can I show that $h_{ii} = 1/n + (...
0
votes
2answers
31 views

How to perform multivariate linear regression on this example?

So I have a table with points in time $t_i$ and measurements $s_i$ give by \begin{bmatrix}t_i&0&1&2&3\\s_i&4.30&1.48&0.56&0.24\end{bmatrix} As a regression function I'...
0
votes
0answers
15 views

What is the unit increase in coefficient analysis for logistic regression?

What does the unit increase mean in coefficient analysis for logistic regression? I have normalized my data to a mean of 0 and standard deviation of 1. How does this change the unit increase? Is this ...
1
vote
0answers
28 views

Please verify whether the calculating process of the WLS estimator and the variance is correct or not.

There are two Heteroscedasticity regression models 1. $$ y_i = \beta x_i + \epsilon_i, \quad i=1, \ldots, n $$ where $\epsilon_i$'s are independent and distributed as $\epsilon_i \sim N(0, \...
0
votes
0answers
30 views

Identifying the distribution of the least squares estimate with linear regression

The model is set up as follows: Random variables $Y_1,Y_2,...,Y_n$ are given by $Y_i=\alpha+\beta x_i+\epsilon_i$ where $\alpha,\beta$ and $x_1,...,x_n$ are constants with $\sum_{i=1}^n x_i=0$ and $\...
0
votes
1answer
7 views

T statistic for intercept and coefficient

So, assuming that W0 is the intercept and W1 is the coefficient in a simple linear regression model, the way to calculate a t statistic for W1 is (W1-0) /std error of w1 Now, my question is how do I ...
1
vote
1answer
33 views

Prove that $\frac{(n-2)s^2}{\sigma^2} \sim \chi^{2}_{n-2}$

Consider the following simple linear regression model involving the $\epsilon_i$ error term, $$y_i = \alpha + \beta x_i + \epsilon_i$$ such that, $$\epsilon_i \sim \mathcal N(0,\sigma^2)$$ we know ...
0
votes
0answers
23 views

linear regression-positive answers

Assume we have an over-determined linear system as $Mx=y$. We know that this system has infinitely many solutions. In case the variables($x$) are free in sign, we can find a solution to this system ...
0
votes
1answer
21 views

Residual analysis in Python

when doing residual analysis do we first fit our model on our entire training set and calculate residuals between fitted values and actual values? Or do we first fit our model on the training+testing ...
0
votes
0answers
9 views

Gaussian process regression for large dimensional input space

I am working for Gaussian process (GP) regression $y=f(\mathbf{x})+\epsilon$ with $\mathbf{x}\in \mathbb{R}^D$ and $D$ can be as large as few 100. I have attempted with the GPML toolbox and the exact ...
0
votes
0answers
9 views

Summation Proof For $ Y_{ij} - \bar Y_j $

Currently trying to prove that the sum of squares error can be partitioned into SSPE and SSLF. $ SSE = \sum_{j=1}^c \sum_{i=1}^{nj} (Y_{ij}- \hat Y_{ij})^2 = \sum_{j=1}^c \sum_{i=1}^{nj} (Y_{ij}- \...
0
votes
1answer
40 views

Show that $X^T X = \sum_{i=1}^n \mathbf{x}_i \mathbf{x}_i^T$

Referring to this post: https://stats.stackexchange.com/questions/164223/proof-of-loocv-formula The line which says $\sum_{t=1}^TX_t'X_t=X'X$ is the result I'm trying to interpret. Or in perhaps ...
0
votes
0answers
10 views

exponential regression model in accelerate life design

Suppose time to failure $T$ follows exponential distribution and its p.d.f is $$f_T(t)=\lambda e^{-\lambda t}$$ Let $Y=lnT$, then the p.d.f for $Y$ is $$f_Y(y)=\lambda exp(-\lambda e^{y}+y)$$ $$=...
0
votes
0answers
8 views

Cross validation multivariate kernel regression in R

This question is general- I have a data set of n observations, consisting of a single response variable y and p regressor variables ( here, n ~50, p~3 or 4). I am planning to implement Nadaraya-Watson ...
-1
votes
0answers
10 views

Optimal decision rule - ROC Curve & Logistic regression

Table here - 10 entries There are 10 inputs in the above table. A binary yes or no to indicate if they voted and a model score. I plotted the ROC curve for the model and calculated the optimal ...
4
votes
1answer
267 views

Area enclosed by a function containing two power towers: $f(x)=(−\ln(x↑↑(2k)))↑↑(2k+1)$

I've been considering functions involving power towers lately and come across the following function: $$f(x)=(−\ln(x↑↑(2k)))↑↑(2k+1)$$ $$\text{Where }k∈\mathbb{Z} ^+$$ In the image below we can see ...
0
votes
1answer
74 views

Multivariate Quadratic Regression, Surface Fitting and the Hessian

I need to fit a quadratic surface to multidimensional data, one of the methods mentioned is using a polynomial basis function, $$ \phi(x) = [ 1, x_1, ..., x_n x_1^2, ..., x_n^2 x_1 x_2, ..., x_{n-1}...
0
votes
1answer
33 views

Consistency of OLS estimation

My question is about consistency of OLS parameter estimate for a linear regression model $y_t = x_t' b + \epsilon_t$ ($t = 1,2, \ldots, T)$. Here $x_t$ and $b$ are $1\times k$ vectors (there are $k$ ...
1
vote
0answers
49 views

Finding Conditional Expected Value and Conditional Variance

Very stuck with this problem. Not meant to make any more assumptions than laid outin the question. I have a regression model: $\ln(𝑠𝑎𝑣_𝑖)$ = $𝛽_0 + 𝛽_1\cdot \ln (𝑖𝑛𝑐_i) + 𝑢_i$ (where sav ...
0
votes
0answers
10 views

struggling to understand R output for linear regression with categorical variables

If i where to fit some random linear regression model like fit<-lm(Height~factor(Location)+Daily.calcium.intake+Fathers.daily.prtintake,data=some data set ) as a random example where Location ...
0
votes
0answers
12 views

Probit vs Linear Probability Model for binary independent variable

I am new to binary regression models. From my understanding, the probit is better than the LPR because it allows for varying marginal effects and constraint Y_hat to be within 0 & 1. However, if ...
0
votes
0answers
18 views

How would a simple linear regression model look like?

for example i would have 4 groups with 6 results in each of them. if i was to do a hypothesis test to see if there was a relationship between the explanatory variable and response variable i was ...
0
votes
0answers
18 views

Combining Curve Fitting Confidence Intervals and Standard Deviations for Scaling Arguments

I have done some curve fitting in MatLab so that I can generate a trend for different terms taken from my CFD simulations. In my simulations, I have three wing lengths measured by the aspect ratio $AR ...
0
votes
0answers
9 views

Logical alternative to random forest for one variable?

I have a large dataset I am analysing with a random forest regression. Part of my analysis aims to show that combining all features is better than subgroups. To highlight improvement, I want to run a ...
0
votes
0answers
16 views

Correllation between a difference and subtrahend

I have measured a total concentration of a substance (phosphorus), called totP, and one specific form of the substance (phosphate), called P, and need to understand how the diffence between these, ...
0
votes
1answer
24 views

How to calculate fitted values and residuals from a set of data

Given a set of data with 11 observations of two variables (response and predictor), I've been asked to "calculate the fitted values $\hat y_i = \hatα + \hatβx'_i $ and residuals $e_i = y_i − \hat y_i$ ...