# Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

106 questions
Filter by
Sorted by
Tagged with
4k views

19k views

### Derivation of the formula for Ordinary Least Squares Linear Regression

How was the formula for Ordinary Least Squares Linear Regression arrived at? Note I am not only looking for the proof, but also the derivation. Where did the formula come from?
11k views

### easy to implement method to fit a power function (regression)

I want to fit to a dataset a power function ($y=Ax^B$). What is the best and easiest method to do this. I need the $A$ and $B$ parameters too. I'm using in general financial data in my project, which ...
197 views

38k views

### Given a data set, how do you do a sinusoidal regression on paper? What are the equations, algorithms?

Most regressions are easy. Trivial once you know how to do it. Most of them involve substitutions which transform the data into a linear regression. But I have yet to figure out how to do a ...
7k views

### Polynomial fitting where polynomial must be monotonically increasing

Given a set of monotonically increasing data points (in 2D), I want to fit a polynomial to the data which is monotonically increasing over the domain of the data. If the highest x value is 100, I don'...
1k views

### why small L1 norm means sparsity?

I'm trying to understand regularization in machine learning. one way of regularization is adding a l1 norm to the error function. This is said to produce sparsity. But I can't understand. sparsity is ...
5k views

### What is $\operatorname{Cov}(\widehat{Y},Y)$?

If $\hat{Y}$ is the OLS linear regression model for $Y$, what can I say about $\operatorname{Cov}(\hat{Y},Y)$? Is this value $0$?
9k views

Does anyone know the specific equations for the three parameters in a least-squares quadratic regression? I'm looking for something like $\beta_1=,\beta_2=,\beta_3=$ for each of $y=\beta_1+\beta_2x+\... 4answers 4k views ### Fit exponential with constant I have data whic would fit to an exponential function with a constant. $$y=a\cdot\exp(b\cdot t) + c.$$ Now I can solve an exponential without a constant using least square by taking log of y and ... 1answer 1k views ### Polynomial fitting - how to fit and what is _polynomial fitting_ I don't understand what is polynomial fitting. Can anyone explain to me how to fit a curve to given points? 3answers 88 views ### Determine which of$N$points is not on$\sin(ax + b)$, where$a$and$b$are unknown. Suppose$N$points ($(x_1,y_1), (x_2,y_2), ... (x_N,y_N)$) are given from a curve$y=\sin(ax+b)$where$a, b$values are unknown. Before giving these$N$points to you,$y$coordinate of one point is ... 2answers 1k views ### Linear trend has to pass through a point I need to interpolate a linear trend surface through a number of points but with the condition that the surface has to pass exactly through one of them. Can somebody give me any advice? 1answer 374 views ### For the simple linear regression model, show that the elements of the hat matrix$H$are… Need some help with this one. For the simple linear regression model, show that the elements of the hat matrix$H$are:$h_{ij}=1/n + (x_i -\bar x)(x_j -\bar x)/S_{xx}$and$h_{ii}=1/n + (x_i -\bar x)^...
528 views

I would like to make a polynomial regression, but for multivariate input data. In the univariate case, one can write polynomial regression as a multivariate linear regression problem and can come up ...
85 views

### Iterative Power Regression

If I have a set of data points that would fit inside a power equation of the form y = a*x^b, what is the best ITERATIVE method to find the values of 'a' and 'b'. I thought I could compute the error ...
535 views

### Proving that the estimate of a mean is a least squares estimator?

I think this is a really simple question so please bear with me - I just had my first class in regression and I'm a little confused about nomenclature/labeling. Does anyone recommend some good ...
11k views

### Why divide by $2m$

I'm taking a machine learning course. The professor has a model for linear regression. Where $h_\theta$ is the hypothesis (proposed model. linear regression, in this case), $J(\theta_1)$ is the cost ...
7k views

### Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t$$ Regression with ...
30k views

### Correlation Coefficient and Determination Coefficient

I'm really new to linear regression and am trying to teach myself. In my textbook there's a problem that asks why $R^{2}$ in the regression of $Y$ on $X =$ the sample correlation between X and Y the ...
13k views

### Why get the sum of squares instead of the sum of absolute values?

I'm self-studying machine learning and getting into the basics of linear regression models. From what I understand so far, a good regression model minimizes the sum of the squared differences between ...
5k views

### Analytic solution for matrix factorization using alternating least squares

The standard form for ridge regression aims to minimize the following cost function. $$\min\ \ \sum_i(y_i-x_i^T\beta)^2 + \lambda\sum_j\beta^2_j$$ As described here, it's possible to differentiate ...
197 views

5k views

### Mean response in linear regression

What does mean response in linear regession mean? I don't understand the definition given in wikipedia. This is the definition: Mean response is an estimate of the mean of the $y$ population ...
3k views

### derivation of simple linear regression parameters

I know there are some proof in the internet, but I attempted to proove the formulas for the intercept and the slope in simple linear regression using Least squares, some algebra, and partial ...
I'm currently in a course learning about neural networks and machine learning, and I came across these two formulas in this textbook page on linear regression: 1) $y(x) = a + bx$ and 2) \$y(x) = w^{...