# Tagged Questions

Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

54k views

### Regression vs Classification

This is more machine learning questions, but perhaps someone will be able to help. I would like to know what is the difference between regression and classification when we try to generate output for ...
11k views

### Why do we use a Least Squares fit?

I've been wondering for a while now if there's any deep mathematical or statistical significance to finding the line that minimizes the square of the errors between the line and the data points. If ...
9k views

317 views

### Minimize $||Ax-b||$ but for $A$, not $x$

I have a machine learning regression problem. I need to minimize $$\sum_i||Ax_i-b_i||_2^2$$ However I am trying to find matrix $A$, not the usual $x$, and I have lots of example data for $x_i$ and ...
2k views

### Derivation of the formula for Ordinary Least Squares Linear Regression

How was the formula for Ordinary Least Squares Linear Regression arrived at? Note I am not only looking for the proof, but also the derivation. Where did the formula come from?
15k views

### Given a data set, how do you do a sinusoidal regression on paper? What are the equations, algorithms?

Most regressions are easy. Trivial once you know how to do it. Most of them involve substitutions which transform the data into a linear regression. But I have yet to figure out how to do a ...
705 views

### Linear regression is wrong?

I tried asking this in electronics as a question related to oscillators, but I wasn't able to get a satisfactory answer. I think the more math-y types here may shed some additional light on the ...
801 views

### Why do people fit polynomials?

Could someone explain the justification and limits of fitting polynomials to arbitrary data points? I mean what about square roots or fractional or inverse powers? Most of the time some wants to ...
386 views

### Are there variations on least-squares approximations?

In least-squares approximations the normal equations act to project a vector existing in N-dimensional space onto a lower dimensional space, where our problem actually lies, thus providing the "best" ...
294 views

### How to find line that has least distance to all points?

I need to find the line having minimal distance to all points. I found linear regression and linear interpolation algorithms. But their minimal distance is only in y-axis: $D = y - f(x)$. But I need ...
11k views

### How to fit a curve to a sinusoidal wave

I am wondering how to fit a sinusoidal wave (approximation). I would like to fit it in the form: $y = A\sin(Bx + C) + D$ where $A,\,B,\,C$ and $D$ are constants. The only constants I really care about ...
372 views

### Theoretical basis for overfitting

There are many examples in which making more "precise" predictions gives worse performance (e.g. Runge's phenomenon). My professor implied that there was a sound basis for choosing "simple" functions ...
36k views

### Linear regression: degrees of freedom of SST, SSR, and RSS

I'm trying to understand the concept of degrees of freedom in the specific case of the three quantities involved in a linear regression solution, i.e. $SST=SSR+SSE,$ i.e. Total sum of squares = ...
5k views

### Correlation Coefficient and Determination Coefficient

I'm really new to linear regression and am trying to teach myself. In my textbook there's a problem that asks why $R^{2}$ in the regression of $Y$ on $X =$ the sample correlation between X and Y the ...
278 views

### Can I build a program that will tell me if a real world data set looks linear, logarithmic, exponential etc?

I have a bunch of real world data sets and from manually plotting some of the data in graphs, I've discovered some data sets look pretty much logarithmic and some look linear, or exponential (and some ...
108 views

### Formula for straight part of a slightly bumpy line

Given a straight line that deviates from the horizontal by at most 15 degrees. On this straight line there are bumps on top at random places on the line. The combined width of the bumps is at most ...
270 views

### Recommended Reading on Regression Analysis?

For a university project, I am implementing an automated regression analysis tool. However, I have very little background in statistics. So what books / articles / material would you suggest I could ...
6k views

38 views

### single variable is significant but overall test is not

I do a multiple regression with 3 independent variables $X_1$, $X_2$ and $X_3$. The correlation between $Y$ and $X_1$, $Y$ and $X_2$, and $Y$ and $X_3$, are each large and statistically significant. ...