Skip to main content

Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

Filter by
Sorted by
Tagged with
0 votes
0 answers
17 views

Technique for finding expected value Weighted Ridge Regression Coefficients

Context: We would like to approximate a linear function $f(\mathbf{x})$ at the point $\boldsymbol{\xi} \in \mathbb{R}^D$ using samples of size $N$ around $\boldsymbol{\xi}$. Assume that the input data ...
user9781778's user avatar
7 votes
1 answer
74 views

What is the collection of functions that a given finite neural network can approximate with ease?

To my understanding, one version of the universal approximation theorem runs as follows: Let $\Phi$ be the family of (trained) feedforward neural networks of bounded width, arbitrary depth, and mild ...
SapereAude's user avatar
0 votes
1 answer
59 views

What exactly does the constants imply/mean in a regression analysis?

I have 2 regression equations $y=11.18e^{-0.972x}$ and $y=16.391e^{-2.246x}$. From my understanding of regression, for $y=Ae^{Bx}$, $A$ is the constant that represents the initial amount changing/...
Frankie139's user avatar
0 votes
1 answer
98 views

Exponential decay curve fitting

I have an initial set of data (Column A-C), and I noticed the % Gain/Lost follows some sort of exponential decay. I've plotted it (data from A2-A13, C2-C13) and got $y=16.391e^{-2.246x}$ and $R^2=0....
Frankie139's user avatar
-1 votes
0 answers
39 views

How to fit a sum of two weighted real-valued powers of x

Is there a method for fitting an arbitrary expression: $a x^\alpha + b x^\beta$ where $a,b,\alpha,\beta \in \mathbb{R}$ (not $\mathbb{N}$). For example, assuming that I have a bunch of $(x,y)$ data ...
mitchus's user avatar
  • 538
0 votes
0 answers
34 views

Trigamma-free Negative Binomial regression: doubts on Hessian and Fisher Information Matrix in the dispersion parameter

I have been looking at alternative versions of the Hessian and Fisher (expected) Information Matrix for the Negative Binomial regression specification, which are given by widely-cited academic sources ...
DrEti's user avatar
  • 63
1 vote
0 answers
20 views

Regrssion notation in the context of projection

I have been conditioned to the typical regression notation $y=X\beta +u$ where $X$ is a matrix and the rest are Nx1 vectors. In a book I was reading, they used the notation $y=b'x+u$ without ...
r-learning-machine's user avatar
0 votes
1 answer
38 views

the sum of $O_p$ --$ O_p(s^2\frac{\log d}{n}+s\sqrt{\frac{\log d}{n}}) $

I read papers in the area of inference for high-dimensional graphical models and these papers always state the convergence rate of the estimator. Using $O_p$ is a good choice. Maybe I made some ...
mathhahaha's user avatar
0 votes
1 answer
12 views

Derivative of Ridge regression w.r.t. to its input X

Given the Ridge Regression original formulation: $Y^* = X \theta$ $\theta = \left ( \lambda I+ X^TX\right )^{-1} X^T Y$ where $X$ and $Y$ are the input and target output matrices, respectively. What ...
Henrique Mendonça's user avatar
0 votes
0 answers
34 views

How to find best fitting trend that fits this periodic data?

I have this data only for this month and I need to find a good trend function. I used a polynomial of degree 6 but with that you can't really extrapolate. I heard you should use Fourier series? It is ...
Chase Bluehorn's user avatar
1 vote
0 answers
21 views

How to interpret t-statistics in a regression with constraints on coefficients?

Suppose I have an OLS regression, the most generic form. $z = a + b*x + c*y$ And I have one additional constraint that sum of the coefficients is 1. i.e. $b+c =1$. Here I have a couple of question on ...
qqzj's user avatar
  • 21
3 votes
1 answer
107 views

Efficient algorithm to solve a sparse recovery problem

I come across with a problem of the form $y=Hx + z \in \mathbb{R}^m$, where $z\in \mathbb{R}^m$ is the noise vector, and $x \in \mathbb{R}^N$ is partially known. $H\in \mathbb{R}^{m \times N}$ can be ...
南洋小學生's user avatar
0 votes
0 answers
32 views

Under what conditions are the remaining coefficients of a regression with a variable omitted "scaled" relative to those in the complete regression?

I was reading [this][1] article that discusses the impact of ommitting variable from a regression equation, and analyzes the impact it will have. Here is some notation they use: given an $m \times n$ ...
Paul's user avatar
  • 557
0 votes
0 answers
18 views

minimization involved $l_2$ norm

I am trying to find the minimum of the following problem $$\frac{\theta}{2}\lVert\beta-x\rVert_2^2+\lambda\lVert x\rVert_2-\frac{1}{2\tau}\lVert x\rVert_2^2+\alpha^Tx$$ by taking the derivative with ...
Simple's user avatar
  • 3,633
1 vote
0 answers
22 views

Proving OLS estimator from two linear models

In a paper I'm reading, the authors try to recover an uncontaminated signal $x$ from two time series of imaging data, $y_1$ and $y_2$, which follow the relationships below: $y_1$ = $x$ + $z$ + $\eta$ $...
malortncachaca's user avatar
0 votes
0 answers
43 views

Appropriateness of one observation per unique combination of dummy variables

I am wondering what conclusions you can draw regarding the coefficients of an OLS model when you only have one observation per combination of unique dummy variables. I have seen someone else do this ...
user25435163's user avatar
1 vote
1 answer
28 views

Min-max optimization and prediction of a parameter in a mathematical model

Context Hello, everyone; let me preface this by saying that my background is in CS and not mathematics, but I do have a background in calculus, statistics, and discrete mathematics. The issue at hand ...
A.Sal's user avatar
  • 13
0 votes
0 answers
37 views

What is the expectation of $R^2$ for for iid $y, x_1, ... x_k \sim N(0,1)$

$Y, X_1, ...., X_k$ are all iid $N(0,1)$ with $n$ samples. I can't make any progress on this problem... I don't even know an approximation but from simulation it seems to be $k/n$. What is the ...
user2330624's user avatar
1 vote
1 answer
152 views

Can anyone see a way to linearize this function for linear regression?

I have the following function: $$f(x) = \dfrac{a_1}{(x+b_1)^2+c_1} + \dfrac{a_2}{(x+b_2)^2+c_2}.$$ From multiple measurements of $f$ at known $x$ values I would like find the values of $a_1,a_2,b_1,...
HazCam's user avatar
  • 23
0 votes
0 answers
66 views

Structural Vector Autoregression (SVAR) in Python - Problem with formulating and understanding the code

I have a research question where the aim in to see how Housing Index is affected by a Central Banks purchases of covered bonds. My data consists of, Housing Index, Policy rate, Mortgage rates, Covered ...
Mateusz's user avatar
3 votes
0 answers
118 views

Kernel Regression, Similarity-Based Modeling & Weights Normalization

Let be $D=[x_j(t_i)]_{i,j} \ M_{n,p}(\mathbb{R})$ a state matrix of $n$ $x(t_i) \in M_{1,p}(\mathbb{R})$ observations of $p$ sensors $X_j$ representing the normal conditions of a system for some ...
Mistapopo's user avatar
  • 101
1 vote
2 answers
53 views

Weighted Least Squares versus ordinary least squares wiki page

If I have a $(X, Y)$ dataset and want to model $y = f(x, \beta)$. In that case for OLS, I would have $$e(x_i, \beta) = f(x_i, \beta) - y_i$$ Then obviously I would have $$SSE = \sum_{i}e(x_i, \beta)^2$...
Ghosal_C's user avatar
  • 549
1 vote
0 answers
14 views

Collaborative Planning, Forecasting, and Replenishment (CPFR) model

I'm trying to understand better the CPFR model but I can't find anywhere a numerical example of this. I'm looking for a numerical example with solution for Collaborative Planning, Forecasting, and ...
1Mathsss's user avatar
3 votes
1 answer
55 views

Treat the sum of residuals as scalar function for Gauss-Newton least squares regression

I have recently learned about Gauss-Newton method and puzzled about something. So given the model function $f(\vec{x}, \vec{\beta})$, and data points $(\vec{x_i}, y_i)$, the residuals are $S = \sum_{i}...
baronett's user avatar
1 vote
0 answers
62 views

Finding best convex approximation to piecewise linear fucntion

I am given some function $g: [0,1] -> R$ which is piecewise linear (and continuous) and my goal is to find the best convex approximation $f$ to $g$ in $L_2$. My first question is whether it somehow ...
KP4's user avatar
  • 11
1 vote
0 answers
13 views

Challenges with a Multi-label Regression Problem in a Real Estate Dataset

I am currently conducting a research where I aim to predict the selling price and deal execution time of properties in NY. To do this, I have a dataset of various properties sold in NY, containing ...
Marco Di Giacomo's user avatar
0 votes
2 answers
121 views

Converting a multivariate linear correlation into a univariate one (predicting real estate prices in NY)

I am building a prediction model of real estate properties in New York based on a few inputs: Area (size) of the property Year it was built Number of bedrooms Number of parking spots The data is ...
bru1987's user avatar
  • 1,927
1 vote
0 answers
34 views

How to measure the error between modeled and observed data?

Consider a scenario where observed data is represented in grey and modelled data in red, as below Here, the x-axis is a position, and the y-axis is an expected time, so that the slope defines, in a ...
sam wolfe's user avatar
  • 3,435
1 vote
0 answers
68 views

Matrix Calculus, finding the weights of a 2 layered non-linear neural network, with sigmoid activation functions

I'm working on a method to calculate weights of a non-linear 2 layer neural network in 1 step, instead of working with the propagation algorithm. I have chosen to make the non-linearity a sigmoid ...
mailerbot mailerbot's user avatar
1 vote
0 answers
27 views

How to use Multiple Regression Analysis to get to linear equation

I am working on a paper about MQ gas sensors and found this other study (https://www.researchgate.net/publication/...
kml's user avatar
  • 11
0 votes
0 answers
20 views

Why is maximizing the Cox partial likelihood meaningful

So the likelihood of seeing our data in survival analysis is (assuming simple case of no censorship and no ties): ...
user23773995's user avatar
1 vote
0 answers
27 views

Logistic regression with gradient descent derivation

Our maximum likelihood is: $$l(\beta)=\sum_{i=1}^{n}y\log(p(x))+(1-y)\log(1-p(x))$$ $$l(\beta) = \sum_{i=1}^{n} y^{(i)} \log(\sigma(\beta^\intercal x^{(i)})) + (1 - y^{(i)}) \log(1 - \sigma(\beta^\...
samsamradas's user avatar
0 votes
0 answers
32 views

Linear regression: $\hat{u}$ vs. $u$

I am studying statistics as a major and got some confusion watching $\hat{u}$ and $u$. My professor said $\hat{u}$ is called a residual and $u$ is an unobservable error term. Therefore this is what I ...
squid__'s user avatar
  • 67
1 vote
0 answers
41 views

Using Gradient Descent for coefficient estimation in ARMA model

I'm trying to implement an ARMA model from scratch using gradient descent with adam optimizer to estimate its coefficients . I know it might not be the ideal solution. But the thing that I'm mostly ...
LNTR's user avatar
  • 11
1 vote
0 answers
58 views

Is the expectation of the error of any projection of $\mathbb{E}[Y\mid X]$ onto subspace zero?

If we consider the following linear predictor of $Y$ based on $X$: $$ Y_{\mathbf{b}}=\boldsymbol{\Sigma}_{Y, \mathbf{X}} \boldsymbol{\Sigma}_{\mathbf{X}}^{-1}\left(\mathbf{X}-\boldsymbol{\mu}_{\mathbf{...
maskeran's user avatar
  • 573
0 votes
0 answers
19 views

closed form expression for training error in ridge regression

I'm reading the paper a random matrix approach to neural network but i'm stucked at page 5. They start from $\Sigma \in \mathbb{R}^{n\times T}$ where $T$ is the number of data points and $n$ is the ...
fabianod's user avatar
  • 155
1 vote
1 answer
112 views

MLE's for ANOVA Model

Given the ANOVA model $Y_{ij} = \mu_i + \varepsilon_{ij}, \varepsilon_{ij}\sim N(0, \sigma^2)$, $i = 1, 2, \ldots , I, \space j = 1,2, \ldots, n_i$, I am trying to find the MLE's $\hat\mu_1, \hat\mu_2,...
idk31909310's user avatar
0 votes
1 answer
166 views

Can a (bounded) linear least-squares problem include a scale factor in its solution?

I have a system of equations. $$ \small \begin{aligned} (1 - p_0)u_0 + (q_0 - 1)v_0 + u_1 - v_1 = r_0 - c t_0 \\\\ (1 - p_1)u_1 + (q_1 - 1)v_1 + u_0 - v_0 = r_1 - c t_1 \\\\ (1 - p_2)u_2 + (q_2 - 1)...
Isco's user avatar
  • 103
1 vote
2 answers
18 views

Finding the ideal phase shift when manually fitting a trigonometric curve

So essentially for a school project I've been given a scatterplot with an array of points shaped in a rough sinusodial wave, and I need to plot a line of best fit manually. So far I have all the ...
amateur's user avatar
  • 11
0 votes
0 answers
20 views

How can one compute rounding preservant integrable functions?

Background & Context : The background of the question is an engineering problem. I want to efficiently represent a set of integers as rounded real valued functions and quickly be able to calculate ...
mathreadler's user avatar
  • 26.1k
0 votes
0 answers
54 views

Is conditional expectation of the error of best linear predictor given $X$ is $0$ (Is it true that $y = a^*+b^*x + \eta$, where $E[\eta|x]=0$)?

For simplicity, assume we are working with simple regression where the predictor $x\in\mathbb{R}$. First write $y=E[y \mid x]+u$, where the variance of $u$ is a constant, and $E[u|x]=0$. I understand $...
maskeran's user avatar
  • 573
1 vote
2 answers
93 views

Why doesn't the linear regression preserve the standard deviation?

If we model $Y = \beta X$, we can estimate $\beta$ to minimize $$\sum (Y_i - \beta X_i)^2$$ Taking derivatives and solving for 0, we get $\sum 2\beta X_i^2 - 2Y_1X_i = 0 \implies \beta = \frac{\sum ...
ammo 45's user avatar
  • 11
0 votes
0 answers
20 views

Endogeneity Analysis without the access of raw data?

I currently have the correlation/covariance matrix for a set of variables and the results of regression analysis but lack access to the raw dataset. Under these constraints, is it feasible to conduct ...
Harshavardhana Srinivasan's user avatar
0 votes
0 answers
29 views

2SLS and Instrumental Variables

I don't understanding the estimation of IV thoroughoutly. The following is my puzzle. In the first stage, we have $$D_i = \alpha + \beta Z_i + u_i$$ then we get predicotrs $\hat{D_i}$. In my mind, $\...
HSINSHUO's user avatar
0 votes
1 answer
45 views

Population OLS coefficient in simple regression?

The population OLS coefficient for some $X_i \in \mathbb{R}^d, Y \in \mathbb{R}$ for the model $Y = \beta’X + e$ is defined as $$ \beta =\mathbb{E}[X_iX_i']^{-1}\mathbb{E}[X_iY_i] $$ and if $X$ is a ...
tmako's user avatar
  • 196
0 votes
0 answers
12 views

In sparse ridge regression, why we have this property

In ridge regression, we can estimate $\hat y$=$X(X^TX+\lambda I)^{-1}y$,where $X$ is covariate matrix with n rows and p column. And my teacher says that we can use SVD to rewrite this formula as:$\hat ...
MengXing Chen's user avatar
0 votes
0 answers
18 views

Creating a regression model of scaling data from five points for ATAR

I am looking to calculate the scaled results for the ATAR (Australian Tertiary Admission Rank) subjects that someone inputs from the recently released 2023 data. For example if someone got a 59.00 in ...
Wosca's user avatar
  • 3
1 vote
1 answer
33 views

What considerations should I have into account when linearizing a non-linear model for linear regressions?

I'm looking for some bibliography about what I should/must have into account when I have a model and experimental data that can be expressed in a way such that I can use a linear regression method to ...
CosmeticMichu's user avatar
0 votes
1 answer
95 views

Using sum of squares regression for fitting a sinusoid

I am trying to use the least squares regression to fit a curve to a table of values representing a sine wave. Similar to the question: Least squares regression of sine wave Except I want to fit a more ...
Adam Labuš's user avatar
0 votes
0 answers
21 views

Convergence of coefficients in multivariate regression

In this thread, the convergence of coefficient for univariate dependent variable is proven. I wonder, assuming the same setup, how can the convergence be extended to multivariate as: $$Y=XW+\epsilon$$ ...
statwoman's user avatar
  • 253

1
2 3 4 5
55