# Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

2,716 questions
Filter by
Sorted by
Tagged with
17 views

### Technique for finding expected value Weighted Ridge Regression Coefficients

Context: We would like to approximate a linear function $f(\mathbf{x})$ at the point $\boldsymbol{\xi} \in \mathbb{R}^D$ using samples of size $N$ around $\boldsymbol{\xi}$. Assume that the input data ...
74 views

### What is the collection of functions that a given finite neural network can approximate with ease?

To my understanding, one version of the universal approximation theorem runs as follows: Let $\Phi$ be the family of (trained) feedforward neural networks of bounded width, arbitrary depth, and mild ...
• 269
59 views

### What exactly does the constants imply/mean in a regression analysis?

I have 2 regression equations $y=11.18e^{-0.972x}$ and $y=16.391e^{-2.246x}$. From my understanding of regression, for $y=Ae^{Bx}$, $A$ is the constant that represents the initial amount changing/...
98 views

43 views

### Appropriateness of one observation per unique combination of dummy variables

I am wondering what conclusions you can draw regarding the coefficients of an OLS model when you only have one observation per combination of unique dummy variables. I have seen someone else do this ...
1 vote
28 views

### Min-max optimization and prediction of a parameter in a mathematical model

Context Hello, everyone; let me preface this by saying that my background is in CS and not mathematics, but I do have a background in calculus, statistics, and discrete mathematics. The issue at hand ...
• 13
37 views

### What is the expectation of $R^2$ for for iid $y, x_1, ... x_k \sim N(0,1)$

$Y, X_1, ...., X_k$ are all iid $N(0,1)$ with $n$ samples. I can't make any progress on this problem... I don't even know an approximation but from simulation it seems to be $k/n$. What is the ...
• 249
1 vote
152 views

• 573
1 vote
93 views

45 views

### Population OLS coefficient in simple regression?

The population OLS coefficient for some $X_i \in \mathbb{R}^d, Y \in \mathbb{R}$ for the model $Y = \beta’X + e$ is defined as $$\beta =\mathbb{E}[X_iX_i']^{-1}\mathbb{E}[X_iY_i]$$ and if $X$ is a ...
• 196
12 views

### In sparse ridge regression, why we have this property

In ridge regression, we can estimate $\hat y$=$X(X^TX+\lambda I)^{-1}y$,where $X$ is covariate matrix with n rows and p column. And my teacher says that we can use SVD to rewrite this formula as:\$\hat ...
18 views

### Creating a regression model of scaling data from five points for ATAR

I am looking to calculate the scaled results for the ATAR (Australian Tertiary Admission Rank) subjects that someone inputs from the recently released 2023 data. For example if someone got a 59.00 in ...
1 vote
33 views

### What considerations should I have into account when linearizing a non-linear model for linear regressions?

I'm looking for some bibliography about what I should/must have into account when I have a model and experimental data that can be expressed in a way such that I can use a linear regression method to ...
95 views

### Using sum of squares regression for fitting a sinusoid

I am trying to use the least squares regression to fit a curve to a table of values representing a sine wave. Similar to the question: Least squares regression of sine wave Except I want to fit a more ...
21 views

### Convergence of coefficients in multivariate regression

In this thread, the convergence of coefficient for univariate dependent variable is proven. I wonder, assuming the same setup, how can the convergence be extended to multivariate as: $$Y=XW+\epsilon$$ ...
• 253