Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Is converting stationary regression output back to non-stationary format valid?

Say I want to do some regression. The thing I am interested in modelling is non-stationary. To have a well behaved noise term I differentiate my dependent and independent variables to something ...
0
votes
0answers
11 views

$\lambda$ not change in Lasso regression, when data values of some features increase

Data set has $50$ features, two of them are salary and years_of_experiences, as well as a response variable y which remain constant overtime. Apply a Lasso $(L_1)$ linear regression model with $\...
0
votes
0answers
6 views

How to add a random slope to linear mixed model with random intercept

I'm reading an introduction to LMM here https://stats.idre.ucla.edu/other/mult-pkg/introduction-to-linear-mixed-models/. I have a question related to the example of doctors and patients in the link. ...
0
votes
1answer
17 views

Are $\beta_0$ and $\beta_1$ unbiased estimators of $\hat\beta_0$ and $\hat\beta_1$?

When we are discussing simple linear regression with: $$Y_i = \beta_0 + X_i\beta_1 +u_i$$ $\hat\beta_0$ and $\hat\beta_1$ are estimates of this model using OLS. With a simple proof we get $E(\hat\...
0
votes
0answers
18 views

Solving an overdetermined system of linear equations, $Ax = b$, using least squares regression.

I'm trying to solve a particularly challenging linear algebra problem. However, unfortunately I have little linear algebra background. The problem is as follows: These linear system of equation [*...
3
votes
1answer
28 views

Does quadratic risk of MLE for multivariate linear regression go to zero with more and more data?

For the simple multivariate linear regression with Gaussian noise: $\mathbf{Y} = \mathbf{X} \boldsymbol{\beta} + \boldsymbol{\epsilon}$, where $\mathbf{Y} \in \mathbb{R}^n$: the vector of dependent ...
0
votes
0answers
13 views

Logistic Log-Loss Function with Shrinkage

I have a loss function for optimizing the beta parameters in a logistic regression model. The professor doesn't label $s$ (constant shrinkage factor?) and doesn't explain why we're transposing the ...
0
votes
2answers
29 views

Regression/forecast with an added linear constraint

I am not sure if I am asking on the right place. But given a set of independent variables $X_i$ and the dependent variable $Y_i = f(X_i, b) +c$, how can I estimate the regression equation given a set ...
0
votes
0answers
21 views

Understanding and implementing a multilateration localization algorithm.

I’m trying to implement a Time Delay of Arrival Multilateration algorithm. However, I’m struggling to understand the concept on the level necessary to produce an actual implementation. For reference, ...
2
votes
0answers
50 views

A log-regression in an Economic problem

I have this problem: I have the regression model (that satisfies MLR.1–5): $$log(taxrev_m)=\delta_0+\delta_1 taxrate_m+\epsilon_m $$ for $m=1...98$. What is the interpretation of $\delta_1$ in ...
0
votes
0answers
10 views

Can we find sequences of functions generated by iterated linear least-squares fitting?

Let us consider the problem $${\bf c_o} = \min_{\bf c}\{\|{\bf W}({\bf \Phi c - d)}\|_2^2\}$$ Where $\Phi = \begin{bmatrix}\Phi_1(x_1)& \Phi_2(x_1)&\cdots&\Phi_n(x_1)\\\Phi_1(x_2)& \...
0
votes
2answers
43 views

What happens when we minimize the sum of errors instead of the sum of errors squared?

Basically title says it all. Say we run a regression where instead of OLS we try to minimize the errors alone. Is this even possible. When I try to differentiate with respect to $\beta_0$ for the FOC ...
0
votes
0answers
35 views

Fastest runtime – calculating a matrices inverse: Row Reduction v. Gram-Schmid?

This question as irked me since finishing Linear Algebra. Question I: With regards to computational runtime – given some large matrix A, which is the fastest way to calculate the inverse: I. ...
1
vote
0answers
16 views

Hyper-parameter optimization for regression to avoid overfiting/underfiting

I am using Penalized spline to smooth noisy data. Those splines are non parametric regression models which only rely on a smoothing parameter $\lambda \geq 0$ (which has to be chosen). I would like to ...
0
votes
1answer
28 views

Probability distribution modeling of discrete differential estimates using exponential family?

For one reason or another I started to estimate histograms of gradients of natural images. Let us assume that I want to try to fit a function of the exponential family to this data. For example $$f(t) ...
0
votes
0answers
25 views

Weighted L1 Regression

I want to do weighted L1 regression that I'm trying to solve using iterative reweighted least squares and I'm running into problems with the weights of my datapoints and the weights used in the ...
1
vote
1answer
32 views

Iteratively Reweighted Least Squares: termination criterion

I'm implementing Iteratively Reweighte Least Squares with the algorithm described on https://en.wikipedia.org/wiki/Iteratively_reweighted_least_squares What I wonder now is how many iterations I ...
0
votes
1answer
25 views

Iteratively Reweighted Least Squares

I'm trying to implement iteratively reweighted least squares. Looking at the wikipedia article, I don't understand the following line $\boldsymbol\beta^{(t+1)} = \underset{\boldsymbol\beta}{ \...
-1
votes
0answers
16 views

Standard error in Data for regression

I am working on my thesis and there I need to approximate a function in the sense of sum of the square differences, i.e. $$ \min \sum_{i=1}^{n} (y_i-\eta_i)^2$$ for the function $y$ and Data $\eta$....
0
votes
0answers
17 views

Calculate commission percentage based on input amount

I want to create a function that returns a commission percentage (y) bases on an input amount (x). My math is a bit rusty (to say the least), but I recall that this should be a logarithmic regression ...
0
votes
0answers
12 views

Using Ordinal (Star Rating) variables to predict outcomes in Log lin regressions + Taking Median significant coefficients of multiple regressions

Framing the regression I am attempting to analyze the effects of several variables on clicks for Google My Business listings. Currently I'm using a Log-Lin regression model to predict the % increase ...
2
votes
0answers
46 views

Solving constraint regression problem using Lagrangian

As part of an exam some weeks ago, I had the following problem: Find a minimizer for: $ \min_{x \in \mathbb{R}^D} || Ax - y||_2 \text{ subject to } || x ||_2 \leq t, $ where $ t > 0, A \in \mathbb{...
0
votes
0answers
11 views

What is a basis function in spline regression?

I'm learning about spline regression right now and there's this notation of basis function. $$f(X)=\sum^M_{m=1}B_mh_m(X)$$ This is the notation using is The Elements of Statistical Learning - Data ...
0
votes
1answer
23 views

Why does $\frac{\delta}{\delta\beta}y^TX\beta=\frac{\delta}{\delta\beta}B^TX^Ty?$

Why does $\frac{\delta}{\delta\beta}y^TX\beta=\frac{\delta}{\delta\beta}B^TX^Ty?$ In linear regression the parameters to the function $y=X\beta + \epsilon$ can be found by calculating the derivative ...
0
votes
1answer
51 views

Find a quadratic function that respect these 3 conditions [closed]

I am developing a mobile app and I don't know how to solve this math problem. p : money d : time in days Find a quadratic $f(x)=ax^2+bx+c$ such that $f(0)=s$ (where s is for "initial y") $f(...
1
vote
0answers
16 views

How to Explain the Basis Functions in Regression Splines

I am going through the Elements of Statistical Learning and am currently working through a chapter on using splines in regression. I have a question about deriving the basis functions for the cubic ...
1
vote
1answer
61 views

Best Model ignoring Outliers

I have not taken any deeper courses in this topic, so bear with me: Often in regression analysis, we encounter outliers. These can heavily influence our model and it is clear to me that in the ...
0
votes
0answers
17 views

Show that the model with highest $AIC$ is the model with the lowest Mallows $C_p$ statistic.

$AIC = l_S - |S|, C_p = \hat{R_{tr}(S)} + 2|S|\hat{\sigma}^2$, $|S|$-is the number of the columns in design matrix, $\hat{R_{tr}(S)} = \sum_{i=1}^{n}(y-\hat{y}(S))^2$. Assume a linear regression model ...
2
votes
2answers
67 views

Does there always exist a function $ f $ for which $ Y - f ( X ) $ and $ X $ are independent?

Let $ X $ and $ Y $ be real random variables. Does there always exist a function $ f $ for which $ Y - f ( X ) $ and $ X $ are independent? I tried to prove the statement, but I couldn't do it. If the ...
0
votes
0answers
20 views

NFL Stadium Hedonic Pricing Model

Long time lurker, so first time posting on here, but I think I have something a lot of you may find interesting. I am very interested in the real estate field and thought a fun project to take on ...
1
vote
1answer
33 views

Transformation of function confusion

I am a bit confused about a concept that was recently covered in my stats class. My professor said that there's never any real need to do nonlinear regression, because a function can always be ...
0
votes
0answers
14 views

Finding a Regression Coefficient from an Orthogonal Projection

Say I have the following regression equation where $\mathbf{y} = \begin{pmatrix}1\\ 2\\ 3\end{pmatrix}$ and $\mathbf{x} = \begin{pmatrix}1.1\\ 1.5\\ 0.6\end{pmatrix}$: $$\mathbf{y} = b_0 + b_1\mathbf{...
1
vote
2answers
34 views

Ridge regression estimator in high-dimensions: is $(X^TX + \epsilon I_p)^{-1}X^Ty$ finite as $\epsilon \rightarrow 0$?

Consider the ridge regression estimator $$\hat{\beta}_{\epsilon} := (X^TX + \epsilon I_p)^{-1}X^Ty$$ where $X$ is an $n$ by $p$ matrix with $n < p$. Let $\| \hat{\beta}_{\epsilon} \|_{1} := \sum_{j=...
0
votes
1answer
21 views

Can we estimate $ y = \beta_1\exp(x) + \beta_2\exp(-x) + \beta_3$ using Linear Regression

Can we estimate the following relationship using linear regression. Here, $\beta_1, \beta_2 $ and $\beta_3$ are parameters. $$ y = \beta_1\exp(x) + \beta_2\exp(-x) + \beta_3$$
0
votes
1answer
31 views

How to account for increasing probability in odds of winning a competition when each round there is a random single decrease in the non-winner pool?

My Example: Players 1 through n are playing a game and in each round there is one winner and after the winner is determined, one of the non-winners gets randomly taken out of the total players pool (i....
1
vote
0answers
20 views

Nearest Neighbor regression for non iid data (for instance for stationary ergodic Markov)

For part of my work I am using k-NN regression for $\{X,Y\}$ be jointly stationary and ergodic Markov. I am looking for some results on the consistency of NN regression for estimating $E[Y|X=x]$, i.e.,...
0
votes
0answers
15 views

Least square with a logarithm of determinant regularization

I am trying to fit a quadratic function to 2D data. Namely, I want to find the solution to a problem similar to: $$ \arg \min_A \sum_{i=1}^N (\mathbf{x}_i A \mathbf{x}_i - b_i)^2,$$ with $A$ a ...
-2
votes
1answer
85 views

Coefficient of Determination and Standard Error of the Model

Background explaining standard concepts and standard terminology used in linear regression and analysis of variance: It will be supposed that one has data points $(X_i,Y_i),\, i = 1,\ldots,n.$ The ...
0
votes
0answers
24 views

asymptotic equivalence for nonparametric Regression

I would like to find some good literatur for asymptotic equivalence between the nonparametric multivariate regression model (with equidistant design points and non gausian errors) and the ...
0
votes
0answers
14 views

Closed form solution for Restricted Weighted Least Squares

From Greene, we know that the closed-form solution of a restricted least squares is: $\beta_{Constrained} = \beta_{Uncon} - (X'X)^{-1}R'[R(X'X)^{-1}R']^{-1}(R\beta_{Uncon}-r)$. Is there any similar ...
1
vote
0answers
45 views

Is there a way to “estimate” an indefinite integral?

I'm working with empirical data and have estimated the non-linear marginal effects of my independent variable in my dependent variable. Let's call these marginal effects $\frac{dy}{dx}(x)$. Is there ...
0
votes
0answers
43 views

Fit curve to points

I have set of (approx. 1500) points from the measurement that begins with peak and than slowly decreases to zero (blue). I would like to define the formula of the curve that fits the trend of the ...
1
vote
1answer
28 views

Should $X$ be full column rank in normal Gauss Markov model to make $(\mathbf{y'y},\mathbf{X'y})$ be a complete statistic?

Suppose normal Gauss-Markov model $\mathbf{y=Xb+e}$ where $y\sim N(\mathbf{Xb},\sigma^2 \mathbf{I})$, the pdf of y in exponential family: Set $\theta=(\mathbf{b},\sigma^2)$: $$ \begin{aligned} f _ { \...
1
vote
0answers
7 views

Estimating a data generating process (small dataset)

I wanted to get some ideas where would one look to approach the following problem: Gaussian linear models are often insufficient in practical applications, where noise can be heavy-tailed. In this ...
1
vote
0answers
11 views

What are evaluation metrics for regression-type neural net?

When neural networks are used to solve classification tasks, different evaluation metrics can be used such as Precision, Recall, classification error and F1-score, which are calculated based on ...
0
votes
0answers
17 views

How to impose constraint on quadratic equation fit based on their parameters?

I am using python's curve_fit to fit the quadratic equation to my monthly decay data. My decay lies between [0,1] . When I try ...
0
votes
0answers
15 views

Bayesian Credibility Applied to Regression

I have a set of points $(x,y)$ with the goal of predicting $y$. I am fitting a regression model of the form $\ln(y-1) = A + B\ln(x)$. I am reading an article that gives me the following information: &...
0
votes
2answers
31 views

area being finite in $\int_{0}^\infty \alpha\frac{x}{\beta+x}dx<\infty.$

For $x>0$, for what values of $\alpha$ and $\beta$, do we have: $$\int_{0}^\infty \alpha\frac{x}{\beta+x}dx<\infty.$$ This is known as the saturaion-growth model specification in nonlinear ...
1
vote
1answer
20 views

How can we interpret residual plot in case we have many variables?

In Residual plots, we try to visualize & interpret whether linearity is valid or not in the linear regression model. One way to do this is to plot error term wrt to the independent variable(say x)....
0
votes
0answers
30 views

What do the Eigenvalues and Eigenvectors Of A Coefficient Matrix Tell You?

I see eigenvalues and eigenvectors talked about a lot in relation to a coefficient matrix (in regression and time series models). What is the significance of eigenvalues and eigenvectors in this ...

1
2 3 4 5
45