# Tag Info

4

A direct method of fitting (no guessed initial values required, no iterative process) is shown below. For the theory, see the paper (pp.16-17) : https://fr.scribd.com/doc/14674814/Regressions-et-equations-integrales

2

Let $n$ be the sample size and $q$ be the number of parameters. The multiple regression equation in matrix form is $$Y=X\beta+\epsilon$$ where $Y$ and $\epsilon$ are $n\times 1$ vactors; $X$ is a $n\times q$ matrix; $\beta$ is a $q\times 1$ vector of parameters. The model is usually written in vector form as $$Y_i=X_i'\beta+\epsilon_i$$

1

The problem is intrinsically nonlinear since you want to minimize $$SSQ(a,b,c)=\sum_{i=1}^N\big(ae^{bt_i}+c-y_i\big)^2$$ and the nonlinear regression will require good (or at least reasonable and consistent) estimates for the three parameters. But, suppose that you assign a value to $b$; then defining $z_i=e^{bt_i}$ the problem turns to be linear $(y=az+c)$ ...

1

As said in previous answers, the problem depends on what is measured. Suppose that $x$ is with no error and $y$ is measured with some error. So, the vertical distance is the good choice. For sure the problem can be different : suppose that for a know value of $x$ you measure $y$ (with some error) and also $z$ (with some error too) and, for any reason, you ...

1

The $x$-variable may be in kilograms and the $y$-variable in dollars. If you change kilograms to grams or to metric tons, or change dollars to cents, then the lines that were perpendicular in the plane before, no longer are. No such problem afflicts conventional least-squares. And suppose you classify people according to occupation (any of six types of ...

1

The reason for using vertical distance is that often you have a knob you turn that controls the important parameter of the experiment and then you measure the output. We believe you can set this parameter exactly (or so close that any error of the point is all in the measurement of the $y$ value, not the $x$ value). This is appropriate as long as the error ...

1

To use the "nearest point on the line" involves drawing a perpendicular from the point to the line. It is more complicated than the expression in the OP. There is a good reason apart from simplicity to use the vertical distance. In many circumstances, the data points are not just arbitrary scattered points in the plane. The x-coordinate represents known ...

1

As Occupy commented, using weighted least squares is the solution. Let me take something similar from data validation and data reconciliation. In this area, what we want to minimize is $$\Phi=\sum_{i=1}^n \Big(\frac{y_i^*-y_i}{\sigma_i}\Big)^2$$ where $y_i^*$, $y_i$ and $\sigma_i$ are respectively the reconciled value, the measured value and the standard ...

1

A non-standard approach: Take all points in turn and estimate $\alpha=-\frac{2\tanh^{-1}(y)}x$. Then take the median $\alpha$. In case of outliers, you can even try all $\alpha$ and choose the one that minimizes some error criterion such as the Sum of Absolute Differences. (Unfortunately, this is an $O(n^2)$ procedure.)

1

The rule that you are talking about is known and it is called 3-$\sigma$ rejection rule. This is the simplest way of robustifying the regression model. You can find anything you are searching for here.

1

Here's my guess. Well suppose you have N data points. If you fit a polynomial of order M, then you essentially have N equations with M unknowns. If $M>N$ then you have more unknowns than equations. This is definitely not advisable (the regression requirements are not satisfied). If $M=N$, you have M equations and N unknowns and would get a perfect fit. ...

1

For something like a $t$-test, it is not always all that crucial that the data distributions are normal because the true assumption is that the estimators for the two sample means are normal, which will be approximately true if you have enough data points and the true data distributions aren't too crazy (this is the central limit theorem). Similarly, for a ...

1

The regression line is by definition the line $\hat y = ax + b$ that minimizes the SSE. On the other hand, the SST is the SSE resulting from the line $y = \bar y$. By this definition, we see that we must always have SSE $\leq$ SST.

1

for what it is worth, the least squares solution to $$H \hat{x} = z$$ is considered by multiplying on the left by $H^t,$ giving $$H^t H \hat{x} = H^t z.$$ The matrix $H^t H$ is square and symmetric, indeed positive semidefinite. It is called the Gramian of the system in applications. If the Gramian is nonsingular, therefore invertible, there is a ...

1

Suppose that $\exists y \in Ker(A) / y \neq 0$. Now, let $\hat{x}$ be a least squares solution. That is: $\hat{x}$ is such that $||A\hat{x} - b||_2 = min_{x \in V} ||Ax - b||_2$. Notice that $A(\hat{x} + y) = A\hat{x} + Ay = A \hat{x}$, hence $||A\hat{x} - b||_2 = ||A(\hat{x} + y) - b||_2$. This means that you have just created another solution that also ...

1

Your least squares solution is minimizing $\hat x^T A \hat x$ If $A$ does not have full rank, there is some vector $y$ such that $Ay=0$. Then $(\hat x+y)^TA(\hat x+y)=\hat x^T A \hat x$ so you can add any multiple of $y$ to your solution and get the same product.

Only top voted, non community-wiki answers of a minimum length are eligible