# Questions tagged [least-squares]

Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere.

1,337 questions
Filter by
Sorted by
Tagged with
7 views

### Condition inequality in perturbed LS

I have two matrices $A \in \mathcal{M}_{n,d}(\mathbb{R})$ and $B \in \mathcal{M}_{d,d}(\mathbb{R})$ with $B$ being symmetric definite-positive. I am trying to find a condition on $A$ for which I have ...
31 views

### Making a Negative Number Possible to Square Root

We are able to solve x^2+4=0 by square rooting both sides, but if we have x^2=-4 we can't solve. Firstly, why? Aren't they equal expressions? Secondly, if we have x^2=-4, why can't we bring the four ...
18 views

### Solving nonlinear least-squares with first order Taylor expansion

I'm referencing this Wikipedia article. I understand that a Taylor expansion of a function $f(x)$ around $x = a$ can be given by $f(x) = \sum_{n = 0}^{\infty} \frac{f^{(n)}(a)}{n!}(x - a)^n$ ...
17 views

### Deriving OLS estimator dependent on X being orthogonal

Let $X \in \mathbb{R}^{n \times d}$ be a predictor matrix with orthonormal columns and $y \in \mathbb{R}^n$ as output vector and vector (OLS estimator) $\beta_{LS} \in \mathbb{R}^d$ for estimating a ...
20 views

13 views

### Covariance generated from best-fit chi error function

I came across definition of covariance matrix that is defined from best fit error equation. I would like to clarify correctnes of procedure. We define equation for error: \begin{equation}\label{eq:...
19 views

### Compare the least square estimators and the residuals of two Linear Regression with an alternative regressor

So I have been given this as an assignment for my econometrics course and I seriously can't understand where to begin here: Consider the least square regression $y ∈ R^n$ on $X ∈ R^{n*k}$, and the ...
56 views

### Comparison of two least-squares optimization problems

I have come across two least square minimization problems. The first one is: $$\min_{\beta\in \mathbb{R}} \lvert y_j-x_j\beta\rvert, \quad \text{where}\ j = 1, \dots, n.$$ Here $y$ is the dependent ...
70 views

### Least Squares: Derivation of Normal Equations with Chain Rule (Revisited)

My question pertains to someone else's answered question that has made me curious. The OP wanted to differentiate the following using the chain rule: $$J(\theta)=\frac12(X\theta-y)^T(X\theta - y)$$ ...
27 views

### Calculating coefficients for least squares [cross-posted from CrossValidated]

In this blog, this author says to calculate the coefficients for the equation $$Flat(x, y) = A + Bx + Cy + Dx^2 + Ey^2 + Fxy$$ using least squares. I found this PDF that shows how to do the ...
37 views

### Proof that least squares estimators are unbiased under gauss-markov assumptions

I saw two different derivations of $E[\hat{\beta}] = \beta$, and they don't appear to be equivalent to me. Method 1 (from https://www.youtube.com/watch?v=T5kjKqkCvHc). Method 2 (from https://www....
10 views

### How to increase speed by skipping calculations when fitting a curve with conjugate gradient?

Let us assume we have a least squares fitting problem. $${\bf v_o} = \min_{\bf v}\{\|{\bf \Phi v-d}\|_2^2\}$$ Where $${\bf \Phi} \in \mathbb R^{N\times k}\\{\bf d} \in \mathbb R^{N\times1}$$ ...
13 views

### least squares: solve for b hat without knowing y

I am trying to solve for $\hat{B}$ without knowing the $y$ vector. I know $\hat{B}=((X'X)^{-1})X'y$ and i am given the norm of $y$, the projection of $y (X\hat{B})$, and the $X$ matrix (non-invertible)...
58 views

### Minimizing $\| x A - B\|_F^2$ With a Constraint

I have previously asked an optimization question Here. I will reiterate the question and simply add a constraint to it: I have 2 known grayscale images (256×256 matrices) $A$ and $B$ and want to find ...
40 views

### Find all solutions of least squares problem

I have the following exercise (this is exercise 4.39 of Fundamentals of Matrix Comuptations - Watkins) : I am not sure about how to find all the solutions(item e). I think I must use itens c) and d) ...
39 views

### How do I proof that $A=\sum\limits_{i=1}^{m}x_{i}x_{i}^{T}$ is invertible if and only if $X$ has full rank?

Show that $A=\sum\limits_{i=1}^mx_ix_i^T$ is invertible if and only if $x_1,\cdots,x_m$ span $\mathbb R^d$ for $x_i\in\mathbb R^d$. Here are my thoughts: If $A$ is invertible $Aw=0$ only has the ...
### Fundamentals of Matrix Computations, Watkins, exercise $4.3.9(e)$, SVD.
Given that $$A=\begin{bmatrix} 1 & 2 \\ 2 & 4 \\ 3 & 6\end{bmatrix}, \qquad b=\begin{bmatrix} 1 \\ 1\\ 1\end{bmatrix},$$ what is the method to find all solutions of the least-squares ...