Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression]

Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

0
votes
1answer
15 views

Calculating the var(β) in a least square regression model

The linear model that I'm working with is: $$y_t =α +βx_t + ε_t$$ Based on my Lecture I have: $$Var(\hatβ) = Var(Σw_tε_t)$$ where ε is the error term and $$w_t = \frac{x_t-\overline x}{Σ(x_t-\...
0
votes
2answers
19 views

How to show that these two alternative formulas for slope are equivalent

First Formula $$b_1=\dfrac{\displaystyle\sum_{i=1}^n(Y_iX_i-\bar Y \bar X)}{\displaystyle\sum_{i=1}^n(X_i^2-\bar X^2)}$$ Second Formula $$b_1=\dfrac{\displaystyle\sum_{i=1}^nY_i(X_i- \bar X)}{\...
1
vote
1answer
24 views

Proving that a given function $f^*$ is the best least square approximation

In De Boor (1972) it is stated that Let be $\$ $ a finite dimensional linear space of functions defined on the interval $[a,b]$. We are searching for the best approximation from $\$$ to $g$. ...
1
vote
0answers
19 views

On a nonlinear regression problem

Consider the function $f\colon \mathbb{R}^2\to \mathbb{R}$, $f(x_1,x_2)=x_1^2 +x_2$. Assume that I don't know the form of $f$ and I only have a set of $N$ independent "input-output" data $\{(x_1^{(i)},...
1
vote
0answers
12 views

Identifying changes in new data using previously trained regression model

I would like some ideas on the following problem. I have a data $x$: $$x(y,z) = [x_1(y,z), x_2(y,z), x_3(y,z)]$$ such that they are function of $(y,z)$, but neither of $(y,z)$ is available. Therefore ...
1
vote
2answers
39 views

Game design equation (Two variables(multiple two variables) and one Result (for every case)) how to make the Equation?

I tried to solve this using multiple methods but i couldn't figure it out. i don't know what to use in order to solve this basically i have an input x and x2 Heading when x = 1 and x2 = 1 Y = 45 ...
0
votes
0answers
38 views

Predicting maximum yield using R linear regression output

I have a question in a paper that involves calculating the maximum yield based on a linear model in which yield is explained by the amount of something being transported plus the number of days it is ...
0
votes
1answer
18 views

Linear Regression Computation as $y = ax$

I got a process which can be modelised as a Linear Regression matching an $y = ax$ equation. I can find on the internet computations to match an $y = ax +b$ equation like this $$ b = \frac{\sum y\sum ...
0
votes
0answers
35 views

Applying Regression to Optimization (Rather than Applying Optimization to Regression)?

My original question didn't generate the response I was looking for, so I'm going to ask a new but more specific question. Let's say that we compute parameters for a multiple linear regression model ...
0
votes
0answers
5 views

Bayesian approach to function fitting

Assume we are given a dataset $(x_i, y_i)_{i=1}^n$ which consists of samples of univariate normally distributed random variables following the law $y_i \sim \mathcal{N}(f(x_i), \sigma^2)$ with unknown ...
4
votes
0answers
50 views
+50

Estimating Spline curve by OLS. Is a good idea to fix the knots at Chebyshev sites?

I am writing my master's degree thesis on a novel method for fixing knots in an adaptive way and while reading the literature I've found many references to the so-called Chebyshev sites. This sites or ...
-1
votes
0answers
10 views

Conditional Multi Dimensional Data Fitting [closed]

I have data that is dependent on two variable. I need to fit the data using some analytical function. The data, say function of x and y has quite non-linearity about it. But it can be broadly ...
0
votes
0answers
6 views

If $x$ is statistically significant using $2$-sigma rule?

I have the covariance matrix (columns $y, x_1, x_2, x_3$): $$ \left[ \begin{matrix} 750 & 240 & 10 & 280 \\ 240 & 110 & 0 & 100 \\ 10 & 0 & 7 & 0 \\ 280 & 100 &...
0
votes
2answers
47 views

How to choose degree for polynomial regression?

I know how to perform polynomial regression. But is there any method to use for estimating the degree of the polynomial that is best suited? Some kind of meta-regression. With best suited I mean the ...
1
vote
0answers
46 views

Higher order lp regularization and lp least squares like regression with p>2

Is $l_p$ norm with $2<p<\infty$ used for regularization in any practical applications? Also least squares is usually used with $l_2$ norm squared. But are there any applications where $l_p$ norm ...
1
vote
1answer
20 views

Conditional Expectation Decomposition in Regression Analysis

I am currently working on my understanding of regression fundamentals and I checked this source (one can find the (even exact) same statement in multiple sources). In Theorem 3.1.1, the author claims ...
0
votes
0answers
36 views

Gradient Descent for Exponential Functions

I am trying to develop a non-linear regression for several functions (power, log and exponential). the idea was to use a log transformation to get an initial set of points, close enough to the real ...
1
vote
0answers
29 views

Linear regression of basis functions for multivariate inputs

Background - My current understanding of linear regression of basis functions: Given an input domain $\mathcal{X}$, target domain $\mathcal{Y}$, and a data set $S=\left\{ \left(x_{i},y_{i}\right)\...
2
votes
1answer
21 views

Zero conditional mean, and is regression estimating population regression function?

I am relearning econometrics to get a better understanding of it, and to clear the confusions when I had in college. Using the simple regression model, we have a population model equation as: $$ y = ...
0
votes
1answer
27 views

Finding the appropriate polynomial fit for set of data

Is there a function or library in Python to automatically compute the best polynomial fit for a set of data points? I am not really interested in the ML use case of generalizing to a set of new data, ...
0
votes
0answers
19 views

Estimating parameters of “unknown” non-linear function

I've got data of dice rolls. Every "experiment" consists of $n$ (independent) rolls with a three-sided dice (I'll call the results $A, B, C$ from now on). The chance to roll an $A$ in a single roll is ...
0
votes
1answer
25 views

What's the Most Appropriate Type of Regression for this Problem?

I have a data set from two groups: firms that use AI and their costs and firms that don't use AI and their costs. Within both groups I have data about their specific costs, e.g. fixed costs, variable ...
0
votes
1answer
26 views

Solving intercept from an equation

I am confused to solve (a) in this equation. $$y=x^2/(a+bx)^2$$ What I got is: $$a=(x-bx)/(sqrt(y)).$$ Is that right or not because when I use this equation by substituting numbers of the ...
3
votes
1answer
161 views

Deriving Bayesian logistic regression

I'm attempting to understand Bayesian logistic regression clearly, and I'm uncertain about (among other things) what is the most clear or most correct notation to use. Here is my current attempt at ...
0
votes
0answers
18 views

Can these two regression models be compared based on the coefficient of determination?

As part of a statistical project I'm trying to deal with, I've ran into the question with the following models: (1) $y=\beta_0+\beta_1x_1+\beta_2x_2+u$ (2) $y=\beta_0+\beta_1x_1+(1-\beta_1)x_2+u$ ...
0
votes
0answers
20 views

Algorithm to compute the elastic net estimator

i'm interested in finding: \begin{align*} \beta_{*} \in \underset{\beta \in \mathbb{R}^d}{\arg\min}\big\{ \| Y-X\beta \|_{2}^2 + \lambda \| \beta \|_{2}^2 + \mu \| \beta \|_{1} \big\} \end{align*} i ...
4
votes
2answers
58 views

Conditions for two optimization problems to yield the same solution

Problem: Consider the optimization problems $$\min_\beta \|y-X\beta\|^2+\alpha\|\beta\|^2 \tag 1$$ and $$\min_\beta \|\beta\|^2 \text{ subject to } \|y-X\beta\|^2 \le c \tag 2$$ where $\|x\|$ ...
0
votes
0answers
17 views

Extrapolation error of linear regression lines for a two-cluster data set

I am studying on some linear regression problems using the least-squares method and stumbled upon a problem regarding the error when extrapolating far-away datapoints. About the problem: For a point $...
1
vote
2answers
72 views

Line Of Best Fit With Perpendicular Error

The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the ...
1
vote
1answer
20 views

When will a road reach a certain condition?

I have an exponential regression equation that is designed to predict the future condition of a road at a certain age: condition = 21-EXP(0.06*age) Note: Road ...
0
votes
0answers
17 views

Finding the regression function

We have $$y(t) = B_0 +B_1(t) + ε(t)$$ $$z(t) = y^2(t) - y^2(t-1),$$ $$u(t) = x(t) - x(t-1),$$ $$v(t)= x(t)+x(t-1).$$ Find the regression function $\mathbb {E}(z(t) \vert u(t), v(t)).$ Use: $\mathbb ...
1
vote
1answer
31 views

Trouble with Gauss-Markov Theorem and with finding a Best “Non-Linear” Unbiased Estimator

Let us consider a simple model. $y_i = \beta + \epsilon_i$ If we assume that $\epsilon_i$ has 0 mean, constant variance and is uncorrelated. Then via Gauss-Markov theorem we know that $\hat{\beta} ...
1
vote
1answer
20 views

How to approximate prediction interval in linear regression

Suppose we have a linear regression model of the following format : $$ y(x) = \beta_0 + \beta_1 x_1+ \beta_2x_2+\beta_3x_3+\epsilon$$ We know that the prediction interval associated with a level $\...
0
votes
0answers
23 views

Pros and cons of multivariate interpolation techniques for scattered data?

I have a numerical simulation $f$ that takes 6 input parameters $\mathbf x = x_1, x_2, \ldots x_6$. I have randomly selected $25,000$ random combinations of these inputs and calculated $f(\mathbf x)$. ...
1
vote
1answer
19 views

Linear regression model with 2 categorical variables

Let's consider the following problem : We want to predict a variable $y$ and we have two categorical variables : $A$ that can take 3 different values and $B$ than can take 2 different values. A ...
0
votes
0answers
12 views

Model Selection (k-piece-constant function)

A $k$-piece-constant function is define by $k-1$ thresholds $-100<t_1<t_2<......<t_{k-1}<100$ and $k$ values as $a_1,a_2,......,a_k$ The function is defined as follows- If $x<t_1$ ...
0
votes
0answers
10 views

cross validation method in data set ,i want theoretical concept for selection of sample value by cross validation,

let say i have large sample size i divide it in two parts ,one is for learning and second is for validation purpose . in first step i estimate parameters from learning sample and then in second step i ...
0
votes
1answer
15 views

R-squared and variance relation

According to Wiki: https://en.wikipedia.org/wiki/Fraction_of_variance_unexplained $1 - R^2 = VAR_{err}/VAR_{tot}$ Where $VAR_{err} = \sum_{i = 1}^N (y_i - \hat{y}_i)^2$ is the variance of the ...
1
vote
1answer
14 views

What is J in while calculating SST in multiple regression?

I am little confused what actually is the J in the formula of the SST and SSR for multiple regression SST= $Y^T\left[ 1-\frac{1}{n}J\right]Y$ SSR=$Y^T\left[ H-\frac{1}{n}J\right]Y$
0
votes
1answer
24 views

Is it true that if $b\ \bot\ im(A)$ then $b\ \bot\ im(A^T)$

Is it true that: If $b\ \bot\ im(A)$ then $b\ \bot\ im(A^T)$ ? I think vaguely remembering LA class I would say it is not true. But this seems like this is what is being implied by scribe notes for ...
1
vote
1answer
32 views

Pythagorean theorem in linear regression using matrix notation

I am reading online scribe notes on linear regression and I am being quite confused (since did not do linear algebra for a while) on how Pythagorean theorem is being applied here: First of all I am ...
0
votes
0answers
17 views

Standard Error Estimate for Beta Coefficients

Suppose I have a linear regression model consisting of $\beta$ estimates, relative to a reference term. Each of these $\beta$s has a $\bar{x}$, a $s_x$, and then $n$, with a calculated $\hat{\sigma}$ ...
0
votes
0answers
9 views

Find $V(\tilde\beta|X)$

In the linear regression $Y=X\beta+\epsilon$, with $E(\epsilon_i|x_i)=0$, it is known that the true $\beta$ satisfies the restriction $M\beta=0$, where $M$ is a $q \times k$ matrix with $q<k$. $\...
0
votes
1answer
21 views

Please correct my thinking about Ridge Regression

If ridge regression biases ALL beta coefficients of a regression model towards zero, wouldn't the model massively mispredict the y-variable? I know my logic must be wrong here, but I'd appreciate if ...
2
votes
0answers
37 views

Equation for the Human Spine

I have data that I believe might fit some kind of semi-sinusoidal trend line - I'm trying to derive an equation for the length of individual vertebrae in various species (I am not a mathematician). I'...
2
votes
0answers
23 views

How to solve linear regression with an uncommon error function?

For common linear regression problems, the error terms are $l2$ norm. In other words, the error between measurement (independent values) $y$, and estimate $\hat{y}=X\beta$ is measured as $||y-\hat{y}||...
1
vote
0answers
27 views

Smoothly merging two parametric curves

Let's imagine that an object follows a path described by the known parametric curve $t(s)$ for $s \geq 0$. Now, another object follows another curve $c(s)$, that goes through a known point $c_0$. I ...
0
votes
1answer
32 views

What is the use of coefficient in in Regression

What is the meaning of coefficient values in Machine Learning. After I print model.print_summary() It shows, coefficient values of for each column. But I really ...
2
votes
0answers
19 views

Fitting a spline: find coefficients using Fourier Transform?

I came up with a idea to estimate the coefficients of a B-spline fit by using the Fourier Transform but I don't know if it makes any sense to estimate them in this way. Given that $$s(x)=\sum_kc(k)\...
0
votes
1answer
27 views

fractional curve fitting of the function $y=a+bx^{\alpha}$

Assume I have a set of data $(x_{i},y_{i})$, $i=1,...,m$. How can we find the best values of the parameters $a$ and $b$ and $\alpha$ such that the curve $y=a+bx^{\alpha}$ best fits the data. This ...