Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere. See https://en.wikipedia.org/wiki/Least_squares

learn more… | top users | synonyms

0
votes
0answers
12 views

Non-linear least squares and Bundle Adjustment

In METHODS FOR NON-LINEAR LEAST SQUARES PROBLEMS, 2nd Edition, April 2004 by K. Madsen, H.B. Nielsen, O. Tingleff on page 17 it states: Given a $f: R^n \mapsto R^m$ with $m \geq n$ We want ...
1
vote
3answers
47 views

How come least square can have many solutions?

I know there always exists a least-square solution $\hat{x}$, regardless of the properties of the matrix $A$. However, I keep finding online that least-square can have infinitely many solutions, if ...
0
votes
0answers
29 views

The equivalent of least squares, but for vectors

Given a set of poins, one can use a fitting method such as least squares to find the straight (or the parabola, or the 3rd grade equivalent) that's closest to all points at the same time (via ...
1
vote
2answers
62 views

Proving that $\hat{x} = x$ in least square

I am trying to show that if $Ax=b$ has a unique solution $x$, then the least square solution $\hat{x}$ is the exact one (i.e., $\hat{x} = x$). My attempt: We know that for $A x = b$ to have an exact ...
0
votes
1answer
25 views

Polynomial least squares fit — restrictions on order?

If we're finding an interpolating polynomial for 10 data pairs, the order of the polynomial has to be 9. In class, my professor said that when doing a polynomial least squares fit, if you have 10 ...
0
votes
1answer
32 views

Rain forecast model and least squares

Good evening everyone, I'm trying to create a rain forecast model, I have about 720 data, which correspond to monthly rainfall in 60 years during the 12 months of the year. I have a matrix ...
0
votes
0answers
25 views

Calculation of error limits on linear least squares coefficients

I am developing software to find a 'good' solution for the over-constrained problem $Ax=b$, where $A$ is a known matrix $A_{i,j}$, $i=1,\ldots, M$, $j = 1,\ldots,N$, $M > N$, $b$ is a known ...
0
votes
0answers
19 views

Estimate Beam and Ball Problem System Parameters

I'm trying to estimate the parameters of beam and ball problem model. In the problem we have output as ball position and input as gear rotation angle. The issue that i want to ask is that our ...
0
votes
1answer
25 views

Least-squares solution to a linear matrix equation

Let $\\A$ be a matrix of size $\\(m, n)$, $\\b$ a column vector of size $\\m$, $\\x$ a column vector of size $\\n$ and $\\a$ a real number. If $\begin{bmatrix} x \\ a \end{bmatrix}$ is the ...
0
votes
3answers
35 views

Least squares solution when $Ax=B$ actually has a solution

I'm searching for an easy proof for this theorem: (Given $A$ and $b$) If $Ax=b$ has a solution for $x$, then this solution = the least squares solution. This is how I did it , but I'm not sure ...
1
vote
1answer
50 views

Least squares in matrix form demonstration

I want to know why $$\min ||AX-B||^2 <=> A^tAX = A^tB$$ and I'm having a hard time finding a demonstration that I can understand. I'm pretty sure I have to start by doing $Y=AX$ and $B = B_1 + ...
1
vote
0answers
29 views

Technique to solve 2 x 2 block Toeplitz system

I want to know how to solve this system of equations: $$ \begin{bmatrix} R_{N} &-Q_{NM} \\ -Q_{NM}^T & P_M \end{bmatrix} \begin{bmatrix} a_N \\ b_M \end{bmatrix} = \begin{bmatrix} ...
4
votes
5answers
42 views

calculating least squares fit

I read this thread talking about 'why we use least squares' for curve fitting Why do we use a Least Squares fit? One answer by Chris Taylor begins with the assumption that we should look for $$ ...
3
votes
1answer
37 views

Projection of $z$ onto $\{x\mid Ax = b\}$

Suppose $A$ is fat(number of columns > number of rows) and full row rank. The projection of $z$ onto $\{x\mid Ax = b\}$ is (affine) $$P(z) = z - A^T(AA^T)^{-1}(Az-b)$$ How to show this? ...
1
vote
0answers
22 views

Quadratic fit with least square : any simple analytical expression?

We consider the least square problem in the case where we got only one independant variable $x_i$ and only one dependant variable $y_i$. The number of observations is $n$. In the case of the linear ...
1
vote
0answers
20 views

Overdetermined system with discrete data.

The setup I have a set of experimental data (subscript 1) which calculates two variables $u_1(x,y,z)$ $v_1(x,y,z)$ I can calculate the three spatial gradients for my two variables ($u_1$ and ...
0
votes
0answers
21 views

Derivation of least squares for a line $y=a+bx$

I was trying to obtain the formula for the least squares regression for a line: I'm not able to compute the formula that gives the errors on the two parameter. For the "true value" I obtained for ...
0
votes
0answers
16 views

Find least-square equation of ellipsoidal cylinder from a set of point

I work in mechanical engineering , and I made a 3D- measure of a drilled surface. SO now I have a set of cartesian coordinates(x,y,z) of the surface and I know the surface has the shape of a cyclinder ...
0
votes
2answers
24 views

“Averaging” transformation matrices?

I have a question on how best to "average" transformation matrices. Say that I have n number of 4x4 transformation matrices, and I wanted to find a matrix that approximated each one of the n 4x4 ...
0
votes
1answer
23 views

Difference between orthogonal projection and least squares solution

When you find the least squares solution you solve $$A^TA = A\vec b$$ but to find the orthogonal projection into the "subspace" A, you multiply this result (the least squares solution) with the ...
2
votes
0answers
22 views

[Levenberg-Marquardt]What is the link between positive-definiteness and well-conditioning?

Working on optimization problems through neural networks, I use the Levenberg-Marquardt algorithm. I have read this assertion that I do not understand : A positive definite diagonal matrix is ...
0
votes
1answer
48 views

Least squares and pseudo-inverse

Let $b\in \mathbb{R}^m$,$A\in M_{m\times n}(\mathbb{R})$ with $m>n$ and $rank(A)=n$, and the element $x^*\in \mathbb{R}^m$ solution of least squares of $Ax=b$. i) Show that $r^*=b-Ax^*\in N(A^T)$ ...
2
votes
3answers
58 views

Parameters estimation for gaussian function with offset

I've read the paper Least square fitting of a Gaussian function to a histogram by Leo Zhou on how to perform a Least Square Fitting of a gaussian function to a histogram. The Gaussian function used ...
2
votes
3answers
57 views

Least Squares method and Octave/Matlab [closed]

I'll try to be as clear as possible so that you understand what I'm trying to do and can help me I have twelve pairs of data $(x_1,y_1),....,(x_{12},y_{12})$ and from this data we established a model ...
0
votes
0answers
13 views

General Equation for rotating, expanding and shifting a sample track

Dear Mathematics Community, The problem is that I have two functions which are shifted, rotated and distorted to each other and I'd like to adjust the two redish looking tracks so that the distance ...
0
votes
1answer
20 views

Excluding wrong points in least square method

I am using least square method to find linear equation but getting some problems below. I would like to exclude wrong points before calculating linear line. Is there any idea for this?
0
votes
1answer
28 views

fitting by linear combination of exponential functions

Suppose that we have a set of points $(x_1,y_1), \ldots (x_n,y_n)$, and we want to fit a function of the form $f(x) = ae^{2x} + be^x + c$ to those points. If we make $z=e^x$, then our function becomes ...
3
votes
1answer
28 views

Least square approximation where X vector is given.

I want to find a curve of the form $y = a + b \sqrt{x}$ that best fits the points: $(3, 1.5)$, $(7, 2.5)$ and $(10, 3)$ by substituting the $x$ vector $= \sqrt{x}$ My understanding of the process to ...
0
votes
0answers
18 views

What is 'bursting' in least squares estimation, and what causes it?

I know as much that 'bursting' is some sort of unstable behavior of the least squares calculation, but more precisely what can one expect to see in the estimates in a bursting situation, what causes ...
0
votes
0answers
19 views

Least square matrix form will fail, if the inverse property not satisfied?

In the matrix form of least squares , the inverse of ( X transpose X ) we are calculating . So, what if that matrix does not posses inverse properties. I mean what if it is not invertible ? Sorry if ...
0
votes
0answers
23 views

How accurate the solution of over-determined linear system of equation could be using least square method?

I have read the theory of least square method. It is used to minimize the Frobenius norm of equation residual vector. but I searched the internet and I did not find how to determine the actual value ...
0
votes
1answer
14 views

Demeaned fixed effects invariant to base category

Consider the following regression equation: $\gamma_{ib}=\beta_{b}+\alpha_{i}$. Where $\gamma_{ib}$ is matched bank-firm loan growth between $t$ and $t-1$. $\beta_{b}$ is a set of $B$ dummies (one for ...
0
votes
0answers
22 views

advantage and disadvantage of using SVD to solve least square problems

I usually just use $AA^T$ or QR decomposition of A to solve least square problems. But SVD seems to be the popular way to solve the problem. what is the advantage and disadvantage of SVD? thanks!
1
vote
1answer
37 views

(half) hyperboloid least squares problem

I have five equations as follows, , where i = 1, 2, 3, 4, 5 and only (x, y, z) are unknown. The five equations above are half-side hyperboloids. It could be seen as . I want to find the solution (x, ...
1
vote
2answers
59 views

least squares using exponential model

I'm trying to fit values from this model $$y(x)=ae^{−bx}+c$$ where a, b and c are 3 different parameters that I want to find with least squares. So using least squares I want to find the value of a, b ...
0
votes
0answers
26 views

Even least squares approximation

Can anyone help me with this problem or give me a tip on where to start. Let's consider $\theta_n$ a class of approximations with the following properties: all functions $\varphi \in ...
-1
votes
1answer
25 views

What is the best fit (in the sense of least-squares) to the data?

A) Find the best fit (in the sense of least-squares) to the data $x_1$ $(1,-1,-1,1)$ $x_2$ $(1,1,-1,-1)$ $y$ $(5,1,1,1)$ by a linear function of the form $y$=$a$+$bx_1$+$cx_2$ B) Find ...
0
votes
1answer
26 views

Fit exponential distribution with noise

I'm trying to fit an exponential with noise (which in this case is a constant $c$) like this one $$y(x)=αe^{−αx}+c,$$ having $(x_i, y_i)$ values (So $α$ and $c$ are unknown and are the ones that I ...
1
vote
2answers
34 views

Least Square method, find vector x that minimises $ ||Ax-b||_2^2$

Given Matrix A = | 1 0 1 | | 1 1 2 | | 0 -1 -1| and b = $[1\ \ 4\ -2]^T$ find x such that $||Ax - b||_2^2$ is minimised. I know I have to do something along the line $A^TAx = A^Tb$ got the ...
0
votes
1answer
36 views

Finding the least squares solution for the system of equations $y=Ax^2+B$

Find the least squares solution for the system of equations $y = Ax^2 + B$ where $(x, y)$ belongs to the set {$(0, 1),(1, 5),(−1, 3)$}. What is the geometric (graphical) interpretation of the ...
0
votes
1answer
32 views

approximation of $x^2$ in hilbert spaces

use the least squares to find the best linear approximation to $f(x)=x^2$ on [-1,1]. that is find the line $y=a_0+a_1x$ that minimizes $\int_{-1}^1|f(x)-y(x)|^2$ solution I used the theory of ...
0
votes
0answers
11 views

General formula for $\hat{b}$ (least squares) using SVD and pseudoinverse

In a situation with an SVD for A given by $A=U\Sigma V^T$ I know about the relation $ x=(A'A)^{-1}A'b=A^+b $ Given b and matrix A, which general formula can one use to find $\hat{b}$?
2
votes
1answer
33 views

Least square problem with constraint on the unit sphere

It is easy to answer the minimum of $\|Ax-b\|_2$, when $A$ has full column rank. But how is the case when we add an constraint $\|x\|_2=1$? Or to be explicit, $$\min_{\|x\|_2=1}\|Ax-b\|_2=?.$$ My idea ...
1
vote
1answer
30 views

$_Linear regression for polynomial fitting

I am doing some curve fitting. The theoretical curve is hyperbolic and have the form $(x-x_0)(y-y_0)=c$. This is not linear, so the normal linear least square regression is not apply immediately. ...
0
votes
1answer
24 views

Solving Constrained Least-Squares

I need to solve a constrained least-squares (LS) problem as follows $min_X \text{ } ||Y-AX||_F^2$ $s.t. \text{ } {X\in \chi}$ where $A\in R^{n\times m}$, $(n\ge m)$ , $X\in R^{m\times k}$ and ...
0
votes
0answers
19 views

SVD and least squares solution: orthogonal projection $ \vec{\widehat b} $ of $ \vec{b} $ onto $Col(A^T)$

Given the following: $ A = \left(\begin{array}{rrr} -2 & 3 & 2 \\ 2 & 2 & 3 \end{array}\right).$ A has the SVD: $A = USV^T$ $ b = \left(\begin{array}{rrr} -6\\ 1 \\ 4 ...
0
votes
1answer
20 views

Are approximate least square intersections unique?

I seem to be getting a different approximate intersections for the same three lines by multiplying one of the line equations (so that the equation still defines the same line but has different numbers ...
0
votes
0answers
12 views

Solutions to a laplacian preservation

I'm trying to write an impementation to that paper over here http://www.cs.jhu.edu/~misha/Fall07/Papers/Sorkine04.pdf The main idea is that i have a series of points, and i displace some of them. ...
1
vote
0answers
30 views

Least squares with errors in input, errors also a function

this is my first post here, so I hope I'll word everything correctly. I am an amateur mathematician, who does his problems for fun. I am tackling a system of non linear equations, with errors in the ...
0
votes
0answers
55 views

Equivalence of two linear least squares problem

Here I want to build a subspace representation $Uq$ to approximate $x$, where $q$ is the reduced coordinates. We know that the best approximation to $x$ is the linear least squares solution $q_1 = ...