Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere. See https://en.wikipedia.org/wiki/Least_squares

learn more… | top users | synonyms

0
votes
0answers
9 views

Differentiate function returning vector

can I differentiate a function which is returning a vector? I'm trying to implement Least Squares method on sets of points, but I'm stuck at defining Jacobian, which is numeric, but then, I have no ...
1
vote
0answers
14 views

Determine mutual location of two coordinate systems, given two sets of points

My problem is: we've got tracking device and a robot. Tracking device provides set of $n$ points in cartesian coordinates(taken from marker on robot arm) and robot driver returns position of TCP(tool ...
1
vote
2answers
31 views

How to find a “least squares” line with a known slope?

I have gps trackings that I know they fall into a certain pattern - a line with a known angle. How do I find the line that minimizes the distances of the points from it but is in the correct angle? ...
0
votes
0answers
33 views

How to find best plane passes through the center of $n$ points

Consider $n$ points $x_1,\ldots,x_n$ in $\mathbb{R}^n$. How to find the best plane passes through the center of these points with following approach: If we assume that the such plane has form ...
0
votes
0answers
16 views

finite differences nonlinear least squares

I am facing to following nonlinear least-squares problem: $$\min_{u,\gamma} \frac{1}{1000} \int_{ \gamma(x,y)^2} + \int_{ [u(x,y)−u (x,y)]^2} + \int_{ [∆u(x,y)−\gamma(x,y)u(x,y)]^2}$$ where the ...
0
votes
0answers
15 views

Is total least square solution only valid for isotropic error

Let $\mathbf{y} = \mathbf{Ax}$ represent a system of equation where, $\mathbf{y}\in\mathbb{R}^n, \mathbf{A}\in\mathbb{R}^{n\times m}$. However due to error in sensor, what we observe is the following ...
0
votes
0answers
8 views

Iterative Shrinkage Thresholding Algorithm (ISTA) vs Iterative Reweighted Least Algorithm (IRLS)

I am new in the field of compressed sensing. I am confused between when to use Iterative Shrinkage Thresholding Algorithm and Iterative Reweighted Least Algorithm. What i could find is that both are ...
0
votes
0answers
22 views

Matrix Factorization with Arbitrary Dimensions

Continuation of a previous question here. Suppose I have a $n\times m$ matrix $A$. I choose some $k$, and want to find a factorization $A=XY$ where $X$ is $n\times k$ and $Y$ is $k\times m$. In ...
2
votes
2answers
39 views

Using Least Squares to calculate a matrix in an equation.

I have two sets of vectors $v_i$ and $w_i$, in some $d$ dimensional space. I know that $v_i \approx M \cdot w_i$ for all i. I.e., I know that the $v$ vectors are a linear transformation of the $w$ ...
2
votes
1answer
75 views

Least-squares solution to a matrix equation?

Suppose I have $n$ observations of $m$ dependent variables $y_1,\dots,y_m$, and I believe they follow some model wherein they can all be written as linear combinations of some underlying variables ...
0
votes
0answers
29 views

Identification of non-linear functions:polynomial+exponential

Is there a way to perform a non linear least square to identify the following function: $$\alpha_2\cdot x^2 + \alpha_1\cdot x + \alpha_0 + \beta e^{\frac{\gamma}{x}}=Y$$ I aim at identifying the ...
0
votes
0answers
14 views

Non-linear least squares and Bundle Adjustment

In METHODS FOR NON-LINEAR LEAST SQUARES PROBLEMS, 2nd Edition, April 2004 by K. Madsen, H.B. Nielsen, O. Tingleff on page 17 it states: Given a $f: R^n \mapsto R^m$ with $m \geq n$ We want ...
1
vote
3answers
51 views

How come least square can have many solutions?

I know there always exists a least-square solution $\hat{x}$, regardless of the properties of the matrix $A$. However, I keep finding online that least-square can have infinitely many solutions, if ...
0
votes
0answers
31 views

The equivalent of least squares, but for vectors

Given a set of poins, one can use a fitting method such as least squares to find the straight (or the parabola, or the 3rd grade equivalent) that's closest to all points at the same time (via ...
1
vote
2answers
63 views

Proving that $\hat{x} = x$ in least square

I am trying to show that if $Ax=b$ has a unique solution $x$, then the least square solution $\hat{x}$ is the exact one (i.e., $\hat{x} = x$). My attempt: We know that for $A x = b$ to have an exact ...
0
votes
1answer
27 views

Polynomial least squares fit — restrictions on order?

If we're finding an interpolating polynomial for 10 data pairs, the order of the polynomial has to be 9. In class, my professor said that when doing a polynomial least squares fit, if you have 10 ...
0
votes
1answer
32 views

Rain forecast model and least squares

Good evening everyone, I'm trying to create a rain forecast model, I have about 720 data, which correspond to monthly rainfall in 60 years during the 12 months of the year. I have a matrix ...
0
votes
0answers
27 views

Calculation of error limits on linear least squares coefficients

I am developing software to find a 'good' solution for the over-constrained problem $Ax=b$, where $A$ is a known matrix $A_{i,j}$, $i=1,\ldots, M$, $j = 1,\ldots,N$, $M > N$, $b$ is a known ...
0
votes
0answers
26 views

Estimate Beam and Ball Problem System Parameters

I'm trying to estimate the parameters of beam and ball problem model. In the problem we have output as ball position and input as gear rotation angle. The issue that i want to ask is that our ...
0
votes
1answer
29 views

Least-squares solution to a linear matrix equation

Let $\\A$ be a matrix of size $\\(m, n)$, $\\b$ a column vector of size $\\m$, $\\x$ a column vector of size $\\n$ and $\\a$ a real number. If $\begin{bmatrix} x \\ a \end{bmatrix}$ is the ...
0
votes
3answers
37 views

Least squares solution when $Ax=B$ actually has a solution

I'm searching for an easy proof for this theorem: (Given $A$ and $b$) If $Ax=b$ has a solution for $x$, then this solution = the least squares solution. This is how I did it , but I'm not sure ...
1
vote
1answer
53 views

Least squares in matrix form demonstration

I want to know why $$\min ||AX-B||^2 <=> A^tAX = A^tB$$ and I'm having a hard time finding a demonstration that I can understand. I'm pretty sure I have to start by doing $Y=AX$ and $B = B_1 + ...
1
vote
0answers
30 views

Technique to solve 2 x 2 block Toeplitz system

I want to know how to solve this system of equations: $$ \begin{bmatrix} R_{N} &-Q_{NM} \\ -Q_{NM}^T & P_M \end{bmatrix} \begin{bmatrix} a_N \\ b_M \end{bmatrix} = \begin{bmatrix} ...
4
votes
5answers
43 views

calculating least squares fit

I read this thread talking about 'why we use least squares' for curve fitting Why do we use a Least Squares fit? One answer by Chris Taylor begins with the assumption that we should look for $$ ...
3
votes
1answer
44 views

Projection of $z$ onto $\{x\mid Ax = b\}$

Suppose $A$ is fat(number of columns > number of rows) and full row rank. The projection of $z$ onto $\{x\mid Ax = b\}$ is (affine) $$P(z) = z - A^T(AA^T)^{-1}(Az-b)$$ How to show this? ...
1
vote
0answers
23 views

Quadratic fit with least square : any simple analytical expression?

We consider the least square problem in the case where we got only one independant variable $x_i$ and only one dependant variable $y_i$. The number of observations is $n$. In the case of the linear ...
1
vote
0answers
22 views

Overdetermined system with discrete data.

The setup I have a set of experimental data (subscript 1) which calculates two variables $u_1(x,y,z)$ $v_1(x,y,z)$ I can calculate the three spatial gradients for my two variables ($u_1$ and ...
0
votes
0answers
23 views

Derivation of least squares for a line $y=a+bx$

I was trying to obtain the formula for the least squares regression for a line: I'm not able to compute the formula that gives the errors on the two parameter. For the "true value" I obtained for ...
0
votes
0answers
21 views

Find least-square equation of ellipsoidal cylinder from a set of point

I work in mechanical engineering , and I made a 3D- measure of a drilled surface. SO now I have a set of cartesian coordinates(x,y,z) of the surface and I know the surface has the shape of a cyclinder ...
0
votes
2answers
36 views

“Averaging” transformation matrices?

I have a question on how best to "average" transformation matrices. Say that I have n number of 4x4 transformation matrices, and I wanted to find a matrix that approximated each one of the n 4x4 ...
0
votes
1answer
28 views

Difference between orthogonal projection and least squares solution

When you find the least squares solution you solve $$A^TA = A\vec b$$ but to find the orthogonal projection into the "subspace" A, you multiply this result (the least squares solution) with the ...
2
votes
0answers
24 views

[Levenberg-Marquardt]What is the link between positive-definiteness and well-conditioning?

Working on optimization problems through neural networks, I use the Levenberg-Marquardt algorithm. I have read this assertion that I do not understand : A positive definite diagonal matrix is ...
0
votes
1answer
50 views

Least squares and pseudo-inverse

Let $b\in \mathbb{R}^m$,$A\in M_{m\times n}(\mathbb{R})$ with $m>n$ and $rank(A)=n$, and the element $x^*\in \mathbb{R}^m$ solution of least squares of $Ax=b$. i) Show that $r^*=b-Ax^*\in N(A^T)$ ...
2
votes
3answers
59 views

Parameters estimation for gaussian function with offset

I've read the paper Least square fitting of a Gaussian function to a histogram by Leo Zhou on how to perform a Least Square Fitting of a gaussian function to a histogram. The Gaussian function used ...
2
votes
3answers
59 views

Least Squares method and Octave/Matlab [closed]

I'll try to be as clear as possible so that you understand what I'm trying to do and can help me I have twelve pairs of data $(x_1,y_1),....,(x_{12},y_{12})$ and from this data we established a model ...
0
votes
0answers
13 views

General Equation for rotating, expanding and shifting a sample track

Dear Mathematics Community, The problem is that I have two functions which are shifted, rotated and distorted to each other and I'd like to adjust the two redish looking tracks so that the distance ...
0
votes
1answer
21 views

Excluding wrong points in least square method

I am using least square method to find linear equation but getting some problems below. I would like to exclude wrong points before calculating linear line. Is there any idea for this?
0
votes
1answer
36 views

fitting by linear combination of exponential functions

Suppose that we have a set of points $(x_1,y_1), \ldots (x_n,y_n)$, and we want to fit a function of the form $f(x) = ae^{2x} + be^x + c$ to those points. If we make $z=e^x$, then our function becomes ...
3
votes
1answer
29 views

Least square approximation where X vector is given.

I want to find a curve of the form $y = a + b \sqrt{x}$ that best fits the points: $(3, 1.5)$, $(7, 2.5)$ and $(10, 3)$ by substituting the $x$ vector $= \sqrt{x}$ My understanding of the process to ...
0
votes
0answers
18 views

What is 'bursting' in least squares estimation, and what causes it?

I know as much that 'bursting' is some sort of unstable behavior of the least squares calculation, but more precisely what can one expect to see in the estimates in a bursting situation, what causes ...
0
votes
0answers
21 views

Least square matrix form will fail, if the inverse property not satisfied?

In the matrix form of least squares , the inverse of ( X transpose X ) we are calculating . So, what if that matrix does not posses inverse properties. I mean what if it is not invertible ? Sorry if ...
0
votes
0answers
23 views

How accurate the solution of over-determined linear system of equation could be using least square method?

I have read the theory of least square method. It is used to minimize the Frobenius norm of equation residual vector. but I searched the internet and I did not find how to determine the actual value ...
0
votes
1answer
16 views

Demeaned fixed effects invariant to base category

Consider the following regression equation: $\gamma_{ib}=\beta_{b}+\alpha_{i}$. Where $\gamma_{ib}$ is matched bank-firm loan growth between $t$ and $t-1$. $\beta_{b}$ is a set of $B$ dummies (one for ...
0
votes
0answers
24 views

advantage and disadvantage of using SVD to solve least square problems

I usually just use $AA^T$ or QR decomposition of A to solve least square problems. But SVD seems to be the popular way to solve the problem. what is the advantage and disadvantage of SVD? thanks!
1
vote
1answer
37 views

(half) hyperboloid least squares problem

I have five equations as follows, , where i = 1, 2, 3, 4, 5 and only (x, y, z) are unknown. The five equations above are half-side hyperboloids. It could be seen as . I want to find the solution (x, ...
1
vote
2answers
59 views

least squares using exponential model

I'm trying to fit values from this model $$y(x)=ae^{−bx}+c$$ where a, b and c are 3 different parameters that I want to find with least squares. So using least squares I want to find the value of a, b ...
0
votes
0answers
26 views

Even least squares approximation

Can anyone help me with this problem or give me a tip on where to start. Let's consider $\theta_n$ a class of approximations with the following properties: all functions $\varphi \in ...
-1
votes
1answer
26 views

What is the best fit (in the sense of least-squares) to the data?

A) Find the best fit (in the sense of least-squares) to the data $x_1$ $(1,-1,-1,1)$ $x_2$ $(1,1,-1,-1)$ $y$ $(5,1,1,1)$ by a linear function of the form $y$=$a$+$bx_1$+$cx_2$ B) Find ...
0
votes
1answer
27 views

Fit exponential distribution with noise

I'm trying to fit an exponential with noise (which in this case is a constant $c$) like this one $$y(x)=αe^{−αx}+c,$$ having $(x_i, y_i)$ values (So $α$ and $c$ are unknown and are the ones that I ...
1
vote
2answers
35 views

Least Square method, find vector x that minimises $ ||Ax-b||_2^2$

Given Matrix A = | 1 0 1 | | 1 1 2 | | 0 -1 -1| and b = $[1\ \ 4\ -2]^T$ find x such that $||Ax - b||_2^2$ is minimised. I know I have to do something along the line $A^TAx = A^Tb$ got the ...