Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere. See https://en.wikipedia.org/wiki/Least_squares

learn more… | top users | synonyms

0
votes
1answer
49 views

Ordinary Least Squares (OLS) + Matlab

Please help me to solve the equation with Ordinary Least Squares (OLS) method. Given two same length vectors: $x=(x_{1} + ... + x_{n})$ $y=(y_{1} + ... + y_{n})$ 1) Find coefficients a,b,c for: ...
0
votes
1answer
16 views

Summatory problem | Ordinary least square estimator

How I can transform the first expression in the second? \begin{align} \hat{\beta}_{1} & =\frac{n\sum X_{i}Y_{i}-\sum X_{i}\sum Y_{i}}{n\sum X_{i}^{2}-\left(\sum X_{i}\right)^{2}} \\ & = ...
0
votes
0answers
14 views

Clarification of matrix equation needed in recursive least squares example.

I was looking at the answer to the post entitled "simple example of recursive least squares" and I would like to post a question concerning the matrix equation that is presented in the answer. First ...
1
vote
0answers
35 views

Least squares solutions of the linear system

I'm doing problems from old exams, and my solutions don't add up with the professor's solution. The problem is as followed: Find all least squares solutions of the linear system. I checked my ...
0
votes
1answer
25 views

Least squares w.r.t. different basis

I am looking to solve the following equation, where $A$ is a diagonal matrix: $$\min_x\ (Lx - f)^T A (Lx - f)$$ which I recognize to be similar to least squares, but then with respect to a scaling ...
0
votes
0answers
12 views

How to Solve Using Recursive Least Squares Approach

We start with the initial point $\hat{P}_0\!=\left(x_0,y_0\right)$ and the function $f\!\left(x,y\right)=K$ where $K$ is a constant real number and where $f\!\left(x_0,y_0\right)\!{\ne}K$. We are ...
0
votes
1answer
28 views

B-spline weighted least squares fit

Can someone please point to an easy to read source for Bspline curve fitting with weighted least squares. Basically I want to fit a function, and I have some points which are more important then ...
0
votes
0answers
21 views

Solve $\min d(\mathbf{x},\theta) = \min \|\mathbf{y} - \mathbf{S}(\theta)\mathbf{A}\mathbf{x}\|$

I am trying to solve the following least squares problem: $$\min d(\mathbf{x},\theta) = \min \|\mathbf{y} - \mathbf{S}(\theta)\mathbf{A}\mathbf{x}\|$$ where $\mathbf{S}(\theta)$ is a complex-valued ...
2
votes
0answers
23 views

For fat, full-rank matrix $A$, why does $I - A^T(AA^T)^{-1}A$ give projection onto $\mathcal{N}(A)$?

Suppose we are trying to solve the least-norm problem for underdetermined equations, i.e. we want to minimize $\|x\|$ given that $Ax = y$ with $A \in \mathbf{R}^{m \times n}$ and $m < n$. I know ...
0
votes
2answers
29 views

Minimizing this complex expression

I am working through Ahlfors' Complex Analysis book. I have come to the section in Chapter 1 on inequalities. Among the exercises in this section is this: Given complex numbers $a$ and $b$, choose ...
1
vote
0answers
35 views

Least squares have null determinant

I want fitting my data using bicubic interpolation: $$f(x,y)=\sum_{i=0}^{3}\sum_{j=0}^{3}a_{ij}x^iy^j$$ Let known $$f(0, 0)=1; f(2, 0)=1;f(1, 1)=0;f(0, 2) = 1; f(2, 2)=1$$ I used least squares method, ...
0
votes
3answers
23 views

The minimum of $\sum_{i=1}^m(c_ix-b_i)^2$

We know the minimum of $(c_ix-b_i)^2$ is $$x_i^*=\frac{b_i}{c_i}$$ How to show that the minimum of $\sum_{i=1}^m(c_ix-b_i)^2$ with $c_i \neq0$ is ...
-1
votes
0answers
24 views

analytical solution of non-linear least square problem

I am implementing a trust region optimization algorithm and I would like to compare it against already done similar work, where authors measures performance on this problem. $$ \min_{u,\gamma}\Bigg\{ ...
2
votes
0answers
30 views

linear regression - least squares fit error behaviour

I'm trying to be as clear as possible but please be patient as I am very new to the subject of curve fitting. I come from a specific type of problem. I have an input/output relationship I get from a ...
0
votes
0answers
12 views

Differentiate function returning vector

can I differentiate a function which is returning a vector? I'm trying to implement Least Squares method on sets of points, but I'm stuck at defining Jacobian, which is numeric, but then, I have no ...
1
vote
0answers
18 views

Determine mutual location of two coordinate systems, given two sets of points

My problem is: we've got tracking device and a robot. Tracking device provides set of $n$ points in cartesian coordinates(taken from marker on robot arm) and robot driver returns position of TCP(tool ...
1
vote
2answers
42 views

How to find a “least squares” line with a known slope?

I have gps trackings that I know they fall into a certain pattern - a line with a known angle. How do I find the line that minimizes the distances of the points from it but is in the correct angle? ...
0
votes
0answers
37 views

How to find best plane passes through the center of $n$ points

Consider $n$ points $x_1,\ldots,x_n$ in $\mathbb{R}^n$. How to find the best plane passes through the center of these points with following approach: If we assume that the such plane has form ...
0
votes
0answers
18 views

finite differences nonlinear least squares

I am facing to following nonlinear least-squares problem: $$\min_{u,\gamma} \frac{1}{1000} \int_{ \gamma(x,y)^2} + \int_{ [u(x,y)−u (x,y)]^2} + \int_{ [∆u(x,y)−\gamma(x,y)u(x,y)]^2}$$ where the ...
0
votes
0answers
15 views

Is total least square solution only valid for isotropic error

Let $\mathbf{y} = \mathbf{Ax}$ represent a system of equation where, $\mathbf{y}\in\mathbb{R}^n, \mathbf{A}\in\mathbb{R}^{n\times m}$. However due to error in sensor, what we observe is the following ...
0
votes
0answers
14 views

Iterative Shrinkage Thresholding Algorithm (ISTA) vs Iterative Reweighted Least Algorithm (IRLS)

I am new in the field of compressed sensing. I am confused between when to use Iterative Shrinkage Thresholding Algorithm and Iterative Reweighted Least Algorithm. What i could find is that both are ...
0
votes
0answers
23 views

Matrix Factorization with Arbitrary Dimensions

Continuation of a previous question here. Suppose I have a $n\times m$ matrix $A$. I choose some $k$, and want to find a factorization $A=XY$ where $X$ is $n\times k$ and $Y$ is $k\times m$. In ...
2
votes
2answers
40 views

Using Least Squares to calculate a matrix in an equation.

I have two sets of vectors $v_i$ and $w_i$, in some $d$ dimensional space. I know that $v_i \approx M \cdot w_i$ for all i. I.e., I know that the $v$ vectors are a linear transformation of the $w$ ...
2
votes
1answer
77 views

Least-squares solution to a matrix equation?

Suppose I have $n$ observations of $m$ dependent variables $y_1,\dots,y_m$, and I believe they follow some model wherein they can all be written as linear combinations of some underlying variables ...
0
votes
0answers
30 views

Identification of non-linear functions:polynomial+exponential

Is there a way to perform a non linear least square to identify the following function: $$\alpha_2\cdot x^2 + \alpha_1\cdot x + \alpha_0 + \beta e^{\frac{\gamma}{x}}=Y$$ I aim at identifying the ...
0
votes
0answers
15 views

Non-linear least squares and Bundle Adjustment

In METHODS FOR NON-LINEAR LEAST SQUARES PROBLEMS, 2nd Edition, April 2004 by K. Madsen, H.B. Nielsen, O. Tingleff on page 17 it states: Given a $f: R^n \mapsto R^m$ with $m \geq n$ We want ...
1
vote
3answers
54 views

How come least square can have many solutions?

I know there always exists a least-square solution $\hat{x}$, regardless of the properties of the matrix $A$. However, I keep finding online that least-square can have infinitely many solutions, if ...
0
votes
0answers
31 views

The equivalent of least squares, but for vectors

Given a set of poins, one can use a fitting method such as least squares to find the straight (or the parabola, or the 3rd grade equivalent) that's closest to all points at the same time (via ...
1
vote
2answers
63 views

Proving that $\hat{x} = x$ in least square

I am trying to show that if $Ax=b$ has a unique solution $x$, then the least square solution $\hat{x}$ is the exact one (i.e., $\hat{x} = x$). My attempt: We know that for $A x = b$ to have an exact ...
0
votes
1answer
29 views

Polynomial least squares fit — restrictions on order?

If we're finding an interpolating polynomial for 10 data pairs, the order of the polynomial has to be 9. In class, my professor said that when doing a polynomial least squares fit, if you have 10 ...
0
votes
1answer
32 views

Rain forecast model and least squares

Good evening everyone, I'm trying to create a rain forecast model, I have about 720 data, which correspond to monthly rainfall in 60 years during the 12 months of the year. I have a matrix ...
0
votes
0answers
29 views

Calculation of error limits on linear least squares coefficients

I am developing software to find a 'good' solution for the over-constrained problem $Ax=b$, where $A$ is a known matrix $A_{i,j}$, $i=1,\ldots, M$, $j = 1,\ldots,N$, $M > N$, $b$ is a known ...
0
votes
0answers
27 views

Estimate Beam and Ball Problem System Parameters

I'm trying to estimate the parameters of beam and ball problem model. In the problem we have output as ball position and input as gear rotation angle. The issue that i want to ask is that our ...
0
votes
1answer
35 views

Least-squares solution to a linear matrix equation

Let $\\A$ be a matrix of size $\\(m, n)$, $\\b$ a column vector of size $\\m$, $\\x$ a column vector of size $\\n$ and $\\a$ a real number. If $\begin{bmatrix} x \\ a \end{bmatrix}$ is the ...
0
votes
3answers
39 views

Least squares solution when $Ax=B$ actually has a solution

I'm searching for an easy proof for this theorem: (Given $A$ and $b$) If $Ax=b$ has a solution for $x$, then this solution = the least squares solution. This is how I did it , but I'm not sure ...
1
vote
1answer
54 views

Least squares in matrix form demonstration

I want to know why $$\min ||AX-B||^2 <=> A^tAX = A^tB$$ and I'm having a hard time finding a demonstration that I can understand. I'm pretty sure I have to start by doing $Y=AX$ and $B = B_1 + ...
1
vote
0answers
31 views

Technique to solve 2 x 2 block Toeplitz system

I want to know how to solve this system of equations: $$ \begin{bmatrix} R_{N} &-Q_{NM} \\ -Q_{NM}^T & P_M \end{bmatrix} \begin{bmatrix} a_N \\ b_M \end{bmatrix} = \begin{bmatrix} ...
4
votes
5answers
47 views

calculating least squares fit

I read this thread talking about 'why we use least squares' for curve fitting Why do we use a Least Squares fit? One answer by Chris Taylor begins with the assumption that we should look for $$ ...
3
votes
1answer
44 views

Projection of $z$ onto $\{x\mid Ax = b\}$

Suppose $A$ is fat(number of columns > number of rows) and full row rank. The projection of $z$ onto $\{x\mid Ax = b\}$ is (affine) $$P(z) = z - A^T(AA^T)^{-1}(Az-b)$$ How to show this? ...
1
vote
0answers
26 views

Quadratic fit with least square : any simple analytical expression?

We consider the least square problem in the case where we got only one independant variable $x_i$ and only one dependant variable $y_i$. The number of observations is $n$. In the case of the linear ...
1
vote
0answers
22 views

Overdetermined system with discrete data.

The setup I have a set of experimental data (subscript 1) which calculates two variables $u_1(x,y,z)$ $v_1(x,y,z)$ I can calculate the three spatial gradients for my two variables ($u_1$ and ...
0
votes
0answers
25 views

Derivation of least squares for a line $y=a+bx$

I was trying to obtain the formula for the least squares regression for a line: I'm not able to compute the formula that gives the errors on the two parameter. For the "true value" I obtained for ...
0
votes
0answers
21 views

Find least-square equation of ellipsoidal cylinder from a set of point

I work in mechanical engineering , and I made a 3D- measure of a drilled surface. SO now I have a set of cartesian coordinates(x,y,z) of the surface and I know the surface has the shape of a cyclinder ...
0
votes
2answers
54 views

“Averaging” transformation matrices?

I have a question on how best to "average" transformation matrices. Say that I have n number of 4x4 transformation matrices, and I wanted to find a matrix that approximated each one of the n 4x4 ...
0
votes
1answer
41 views

Difference between orthogonal projection and least squares solution

When you find the least squares solution you solve $$A^TA = A\vec b$$ but to find the orthogonal projection into the "subspace" A, you multiply this result (the least squares solution) with the ...
2
votes
0answers
34 views

[Levenberg-Marquardt]What is the link between positive-definiteness and well-conditioning?

Working on optimization problems through neural networks, I use the Levenberg-Marquardt algorithm. I have read this assertion that I do not understand : A positive definite diagonal matrix is ...
0
votes
1answer
52 views

Least squares and pseudo-inverse

Let $b\in \mathbb{R}^m$,$A\in M_{m\times n}(\mathbb{R})$ with $m>n$ and $rank(A)=n$, and the element $x^*\in \mathbb{R}^m$ solution of least squares of $Ax=b$. i) Show that $r^*=b-Ax^*\in N(A^T)$ ...
2
votes
3answers
67 views

Parameters estimation for gaussian function with offset

I've read the paper Least square fitting of a Gaussian function to a histogram by Leo Zhou on how to perform a Least Square Fitting of a gaussian function to a histogram. The Gaussian function used ...
2
votes
3answers
60 views

Least Squares method and Octave/Matlab [closed]

I'll try to be as clear as possible so that you understand what I'm trying to do and can help me I have twelve pairs of data $(x_1,y_1),....,(x_{12},y_{12})$ and from this data we established a model ...
0
votes
0answers
13 views

General Equation for rotating, expanding and shifting a sample track

Dear Mathematics Community, The problem is that I have two functions which are shifted, rotated and distorted to each other and I'd like to adjust the two redish looking tracks so that the distance ...