# Questions tagged [least-squares]

Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere.

1,364 questions
Filter by
Sorted by
Tagged with
37 views

### Proof that least squares estimators are unbiased under gauss-markov assumptions

I saw two different derivations of $E[\hat{\beta}] = \beta$, and they don't appear to be equivalent to me. Method 1 (from https://www.youtube.com/watch?v=T5kjKqkCvHc). Method 2 (from https://www....
10 views

### How to increase speed by skipping calculations when fitting a curve with conjugate gradient?

Let us assume we have a least squares fitting problem. $${\bf v_o} = \min_{\bf v}\{\|{\bf \Phi v-d}\|_2^2\}$$ Where $${\bf \Phi} \in \mathbb R^{N\times k}\\{\bf d} \in \mathbb R^{N\times1}$$ ...
59 views

### Minimizing $\| x A - B\|_F^2$ With a Constraint

I have previously asked an optimization question Here. I will reiterate the question and simply add a constraint to it: I have 2 known grayscale images (256×256 matrices) $A$ and $B$ and want to find ...
41 views

### Find all solutions of least squares problem

I have the following exercise (this is exercise 4.39 of Fundamentals of Matrix Comuptations - Watkins) : I am not sure about how to find all the solutions(item e). I think I must use itens c) and d) ...
39 views

### How do I proof that $A=\sum\limits_{i=1}^{m}x_{i}x_{i}^{T}$ is invertible if and only if $X$ has full rank?

Show that $A=\sum\limits_{i=1}^mx_ix_i^T$ is invertible if and only if $x_1,\cdots,x_m$ span $\mathbb R^d$ for $x_i\in\mathbb R^d$. Here are my thoughts: If $A$ is invertible $Aw=0$ only has the ...
42 views

### Fundamentals of Matrix Computations, Watkins, exercise $4.3.9(e)$, SVD.

Given that $$A=\begin{bmatrix} 1 & 2 \\ 2 & 4 \\ 3 & 6\end{bmatrix}, \qquad b=\begin{bmatrix} 1 \\ 1\\ 1\end{bmatrix},$$ what is the method to find all solutions of the least-squares ...
44 views

22 views

### Prove that there are unique values $w,b$ such that $L_2(w,b) = \sum_{i=1}^{n}(wx_i + b - y_i)^2$ is minimized.

I would like to prove following result: Suppose we have pair of points $D = \{(x_1,x_1),\cdots,(x_n,y_n)\}$. At least two of these points do not overlap (meaning that there is at least one pair of ...
55 views

### Curve fitting exponential function

in my schoolwork mathematical exploration, I am struggling to find ways in which I can model a function after a set of data. From a simulation software, I have gathered data that resembles an ...
36 views

### How to write $(AX-B)^{T}(AX-B)$ into the form $(X-K)^{T}\Sigma(X-K)$?

I have a problem in the derivation of matrix. Suppose $A$ is some $m \times n$ matrix, with $m>n$. $B$ is a $m \times 1$ vector. $X$ is a $n \times 1$ vector. If we define $M=(A^{T}A)^{-1}A^{T}$,...
28 views

50 views

### Plotting a parabola based on data points

I am trying to draw a parabola inside a chart which I am developing using D3.Js library and using SVG paths to draw the curve. I have a set of 5 points for drawing the parabola: ...
I have the following system of 3 equations and 3 unknowns: $$c_{0} = \frac{x_0}{x_0 + x_1},\ \ c_{1} = \frac{x_1}{x_1 + x_2},\ \ \ c_{2} = \frac{x_2}{x_2 + x_0},$$ where $c_i\!\in\!(0,1)$ are known ...