Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

2,012 questions
Filter by
Sorted by
Tagged with
16 views

4 views

Are stationarity and low autocorrelation the prerequisite of regression model?

As said in the title, are stationarity and low autocorrelation the prerequisite of general / linear regression model ? That is, if a time series is non-stationary or large autocorrelation, would it be ...
10 views

How can I calculate the derivative of J(Θ) which contains sigmoid function in linear regression?

I am taking a machine learning course and since my math is extrementy rusty I am not sure how to go about solving this exercise. Given the derivative of the sigmoid function with respect to Θj I need ...
17 views

prove that $(\hat{\beta}-\beta)'(X'X)^{-1}(\hat{\beta}-\beta)$ and SSE are independent

We have $$(\hat{\beta}-\beta)'(X'X)^{-1}(\hat{\beta}-\beta)$$ and SSE $=\sum_{i=1}^{n}(\hat{y_{i}}-y_{i})^{2}=\sum_{i=1}^{n}y_{i}^{2}-\hat{\beta'}X'y$ and I have to prove that these 2 quantities are ...
27 views

How to prove sum of errors follow a chi square with $n-2$ degree of freedom in simple linear regression

In simple linear regression, the model is $$Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i$$ where $\varepsilon_i$ are i.i.d., and \varepsilon_i \sim N(0, \...
809 views

When fitting a polynomial to data points, how to determine the reasonable degree to use?

I have wondered the following: Suppose that there is a set of data points $(x_i,y_i)$. Then I would like to know if it is more reasonable to assume if there is a polynomial relation of degree $m$ ...
48 views

Why do two different approaches using matrix algebra yield different results?

I am solving a problem involving linear regression and I am finding that two different approaches to the matrix algebra is yielding different results. In the second approach, I am merely simplifying ...
22 views

6 views

Dynamically weighted distance function for vectors in vector space with weighted correlations between vector components

In my problem I am generating a vector space in which every vector represents a concept (representing a specific situation). The components of each vector represent different characteristics of the ...
27 views

Understanding interaction terms in a regression

To my understanding interaction terms help if you believe the relationship between some terms undergoes a fundamental change. I am working on this exercise in my book: And I computed a linear ...
17 views

Effects of the order of averaging and regressing

Suppose we have information (dependent variable: income, independent variables: age, gender, treatment (Boolean), etc) on individuals from different cities and we want to see the effect of treatment ...
28 views

Good polynomials to use that can fit closed complex 2D shapes

Are there any good polynomials that can be used to fit closed complex 2D shapes? For example, the silhouette of everyday objects or people. I am aware I can use a spline for example, however, they ...
12 views

How to solve multiple regression using canonical correlation analysis?

I've read that the results from multiple regression and canonical correlation analysis are the same, aside from scaling. No one online has shown how to prove this by hand, since most examples for ...
23 views

Assume we have an over-determined linear system as $Mx=y$. We know that this system has infinitely many solutions. In case the variables($x$) are free in sign, we can find a solution to this system ...
34 views

Is there a general way to determine the best combination of parameters to fit points?

If the number of the given points is greater than or equal to the number of the parameters in the model, is it always possible to determine those parameters? See my previous problem, Claude Leibovici ...
14 views

why is E(x|u)=cov(x'u)?

when I learn how to use the instrumental variable to solve the endogenous problem, I learned the mechanism that E(x|u)=cov(x'u)≠0, in other words, the x is related to the error term. However, why E(x|...
16 views

30 views

Fitting a simple polynomial to different possibilites

I have a few sets of numbers. For example, let's say my sets are - 1,5,8,12,25 2,14,36,45,47 5,9,17,55,80 29,37,64,93,102 48,69,77,86,167 Given these sets, I want to fit a ...
33 views

15 views

What is the unit increase in coefficient analysis for logistic regression?

What does the unit increase mean in coefficient analysis for logistic regression? I have normalized my data to a mean of 0 and standard deviation of 1. How does this change the unit increase? Is this ...
30 views

40 views

Show that $X^T X = \sum_{i=1}^n \mathbf{x}_i \mathbf{x}_i^T$

Referring to this post: https://stats.stackexchange.com/questions/164223/proof-of-loocv-formula The line which says $\sum_{t=1}^TX_t'X_t=X'X$ is the result I'm trying to interpret. Or in perhaps ...
10 views

49 views

Finding Conditional Expected Value and Conditional Variance

Very stuck with this problem. Not meant to make any more assumptions than laid outin the question. I have a regression model: $\ln(𝑠𝑎𝑣_𝑖)$ = $𝛽_0 + 𝛽_1\cdot \ln (𝑖𝑛𝑐_i) + 𝑢_i$ (where sav ...