# Questions tagged [least-squares]

Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere.

1,063 questions
22 views

### Using the least squares method for problems with two independent variables

This may be quite a specific question and I apologise however I have struggled to find any information regarding a method. I have 6 given $P_i$ values and 6 given $E_i$ and $F_i$ values and I want to ...
39 views

### In this constrained minimization problem, should the Lagrange multipliers be positive?

Consider the following (real, block ?) matrix $Z_{n\times k+1}=[1_{n\times 1},X_{n\times k}]$. Note how $z\equiv v^TZZ^Tv$ can be written as: $v^T11^Tv+v^TXX^Tv=v^TJ_nv+v^TWv$, where $J_n$ is a unit ...
25 views

### OLS estimator of AR($1$) is biased

Suppose that we have a sample $X_0,X_1,\ldots,X_n$ from the AR($1$) model given by $$X_t=\phi X_{t-1}+\varepsilon_t$$ for $t\in\mathbb Z$, where $|\phi|<1$ and $\{\varepsilon_t\}_{t\in\mathbb Z}$ ...
10 views

### Househbolder transformation identity matrix dimensions

When performing a householder transformation and generating an elementary reflector matrix of the form: $$H = I - 2\dfrac{vv^T}{v^Tv}$$ How do we know the dimensions of the identity matrix?
23 views

19 views

26 views

53 views

### Why the average of a set of value has the least square error?

Now we have the equation $$\sum_{i}(x_i-\hat x_i)^2,$$ where $x_i$ is the observed value of a data sample $S$. Here is the question: Why does this expression get its minimum value when $\hat x_i$ ...
39 views

### solving large scale and ill-posed least square problem

I want to estimate unknown value using least square. my matrix is very large, dense(full numerical) and ill_conditioned. I have a pc with 128-gigabyte memory, In this system, only the calculations of ...
16 views

### 2d basis for approximating curvature of spherical surface

I have a surface $h(x,y)=z$ which is point-wise defined and approximately hemispheral. I am using moving least squares to find best-fit function of that surface, and then use known approximation of ...
33 views

### Tips for optimisation problem

I have an optimization (minimize) problem which can be written down as: $f(\vec{x})=\sum_1^m{(max(\vec{a_1}*x_1,\vec{a_2}*x_2,\vec{a_3}*x_3,...,\vec{a_n}*x_n)-\vec{a_0})^2}$ Where $m$ is the size of ...
34 views

26 views

### Derivation of the ordinary least squares estimator β1 and the sampling distribution?

I am trying to derive the ordinary least squares and its sampling distribution for the model: $$y = \beta_0 + \beta_1 x + \epsilon$$ How can I obtain the estimator for $\beta_1$
27 views

### Least-squares regularization matrix is not positive definite?

I am fitting data $(x_i,y_i)$ to the following model: $$f(x) = \sum_j a_j g_j(x) = a^T g(x)$$ where $g_j(x)$ are well-conditioned basis functions (in my application they are B-splines, but I don't ...