Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere. See https://en.wikipedia.org/wiki/Least_squares

learn more… | top users | synonyms

2
votes
2answers
23 views

Parameters estimation for gaussian function with offset

I've read the paper Least square fitting of a Gaussian function to a histogram by Leo Zhou on how to perform a Least Square Fitting of a gaussian function to a histogram. The Gaussian function used ...
2
votes
3answers
42 views

Least Squares method and Octave/Matlab [on hold]

I'll try to be as clear as possible so that you understand what I'm trying to do and can help me I have twelve pairs of data $(x_1,y_1),....,(x_{12},y_{12})$ and from this data we established a model ...
0
votes
0answers
11 views

General Equation for rotating, expanding and shifting a sample track

Dear Mathematics Community, The problem is that I have two functions which are shifted, rotated and distorted to each other and I'd like to adjust the two redish looking tracks so that the distance ...
0
votes
1answer
17 views

Excluding wrong points in least square method

I am using least square method to find linear equation but getting some problems below. I would like to exclude wrong points before calculating linear line. Is there any idea for this?
0
votes
1answer
20 views

fitting by linear combination of exponential functions

Suppose that we have a set of points $(x_1,y_1), \ldots (x_n,y_n)$, and we want to fit a function of the form $f(x) = ae^{2x} + be^x + c$ to those points. If we make $z=e^x$, then our function becomes ...
3
votes
1answer
24 views

Least square approximation where X vector is given.

I want to find a curve of the form $y = a + b \sqrt{x}$ that best fits the points: $(3, 1.5)$, $(7, 2.5)$ and $(10, 3)$ by substituting the $x$ vector $= \sqrt{x}$ My understanding of the process to ...
0
votes
0answers
16 views

What is 'bursting' in least squares estimation, and what causes it?

I know as much that 'bursting' is some sort of unstable behavior of the least squares calculation, but more precisely what can one expect to see in the estimates in a bursting situation, what causes ...
0
votes
0answers
16 views

Least square matrix form will fail, if the inverse property not satisfied?

In the matrix form of least squares , the inverse of ( X transpose X ) we are calculating . So, what if that matrix does not posses inverse properties. I mean what if it is not invertible ? Sorry if ...
0
votes
0answers
23 views

How accurate the solution of over-determined linear system of equation could be using least square method?

I have read the theory of least square method. It is used to minimize the Frobenius norm of equation residual vector. but I searched the internet and I did not find how to determine the actual value ...
0
votes
1answer
13 views

Demeaned fixed effects invariant to base category

Consider the following regression equation: $\gamma_{ib}=\beta_{b}+\alpha_{i}$. Where $\gamma_{ib}$ is matched bank-firm loan growth between $t$ and $t-1$. $\beta_{b}$ is a set of $B$ dummies (one for ...
0
votes
0answers
13 views

advantage and disadvantage of using SVD to solve least square problems

I usually just use $AA^T$ or QR decomposition of A to solve least square problems. But SVD seems to be the popular way to solve the problem. what is the advantage and disadvantage of SVD? thanks!
1
vote
1answer
33 views

(half) hyperboloid least squares problem

I have five equations as follows, , where i = 1, 2, 3, 4, 5 and only (x, y, z) are unknown. The five equations above are half-side hyperboloids. It could be seen as . I want to find the solution (x, ...
1
vote
2answers
55 views

least squares using exponential model

I'm trying to fit values from this model $$y(x)=ae^{−bx}+c$$ where a, b and c are 3 different parameters that I want to find with least squares. So using least squares I want to find the value of a, b ...
0
votes
0answers
24 views

Even least squares approximation

Can anyone help me with this problem or give me a tip on where to start. Let's consider $\theta_n$ a class of approximations with the following properties: all functions $\varphi \in ...
-1
votes
1answer
23 views

What is the best fit (in the sense of least-squares) to the data?

A) Find the best fit (in the sense of least-squares) to the data $x_1$ $(1,-1,-1,1)$ $x_2$ $(1,1,-1,-1)$ $y$ $(5,1,1,1)$ by a linear function of the form $y$=$a$+$bx_1$+$cx_2$ B) Find ...
0
votes
1answer
23 views

Fit exponential distribution with noise

I'm trying to fit an exponential with noise (which in this case is a constant $c$) like this one $$y(x)=αe^{−αx}+c,$$ having $(x_i, y_i)$ values (So $α$ and $c$ are unknown and are the ones that I ...
1
vote
2answers
30 views

Least Square method, find vector x that minimises $ ||Ax-b||_2^2$

Given Matrix A = | 1 0 1 | | 1 1 2 | | 0 -1 -1| and b = $[1\ \ 4\ -2]^T$ find x such that $||Ax - b||_2^2$ is minimised. I know I have to do something along the line $A^TAx = A^Tb$ got the ...
0
votes
1answer
32 views

Finding the least squares solution for the system of equations $y=Ax^2+B$

Find the least squares solution for the system of equations $y = Ax^2 + B$ where $(x, y)$ belongs to the set {$(0, 1),(1, 5),(−1, 3)$}. What is the geometric (graphical) interpretation of the ...
0
votes
1answer
29 views

approximation of $x^2$ in hilbert spaces

use the least squares to find the best linear approximation to $f(x)=x^2$ on [-1,1]. that is find the line $y=a_0+a_1x$ that minimizes $\int_{-1}^1|f(x)-y(x)|^2$ solution I used the theory of ...
0
votes
0answers
10 views

General formula for $\hat{b}$ (least squares) using SVD and pseudoinverse

In a situation with an SVD for A given by $A=U\Sigma V^T$ I know about the relation $ x=(A'A)^{-1}A'b=A^+b $ Given b and matrix A, which general formula can one use to find $\hat{b}$?
2
votes
1answer
33 views

Least square problem with constraint on the unit sphere

It is easy to answer the minimum of $\|Ax-b\|_2$, when $A$ has full column rank. But how is the case when we add an constraint $\|x\|_2=1$? Or to be explicit, $$\min_{\|x\|_2=1}\|Ax-b\|_2=?.$$ My idea ...
1
vote
1answer
29 views

$_Linear regression for polynomial fitting

I am doing some curve fitting. The theoretical curve is hyperbolic and have the form $(x-x_0)(y-y_0)=c$. This is not linear, so the normal linear least square regression is not apply immediately. ...
0
votes
1answer
22 views

Solving Constrained Least-Squares

I need to solve a constrained least-squares (LS) problem as follows $min_X \text{ } ||Y-AX||_F^2$ $s.t. \text{ } {X\in \chi}$ where $A\in R^{n\times m}$, $(n\ge m)$ , $X\in R^{m\times k}$ and ...
0
votes
0answers
17 views

SVD and least squares solution: orthogonal projection $ \vec{\widehat b} $ of $ \vec{b} $ onto $Col(A^T)$

Given the following: $ A = \left(\begin{array}{rrr} -2 & 3 & 2 \\ 2 & 2 & 3 \end{array}\right).$ A has the SVD: $A = USV^T$ $ b = \left(\begin{array}{rrr} -6\\ 1 \\ 4 ...
0
votes
1answer
20 views

Are approximate least square intersections unique?

I seem to be getting a different approximate intersections for the same three lines by multiplying one of the line equations (so that the equation still defines the same line but has different numbers ...
0
votes
0answers
10 views

Solutions to a laplacian preservation

I'm trying to write an impementation to that paper over here http://www.cs.jhu.edu/~misha/Fall07/Papers/Sorkine04.pdf The main idea is that i have a series of points, and i displace some of them. ...
1
vote
0answers
28 views

Least squares with errors in input, errors also a function

this is my first post here, so I hope I'll word everything correctly. I am an amateur mathematician, who does his problems for fun. I am tackling a system of non linear equations, with errors in the ...
0
votes
0answers
51 views

Equivalence of two linear least squares problem

Here I want to build a subspace representation $Uq$ to approximate $x$, where $q$ is the reduced coordinates. We know that the best approximation to $x$ is the linear least squares solution $q_1 = ...
0
votes
0answers
28 views

What is my error in this matrix / least squares derivation?

I'm doing a simple problem in linear algebra. It is clear that I have done something wrong, but I honestly can't see what it is. let, $y = Ax$, $y_{ls} = Ax_{ls}$ where A is skinny and full rank, ...
1
vote
0answers
37 views

Minimizing Sum of Least Squares in Matlab

I am working on this minimization problem for image warping that I want to solve in Matlab: Each feature $p$ can be presented by a 2D bilinear interpolation of the four vertices $V_p = [v_p^1, ...
0
votes
0answers
52 views

Pseudo-inverse gives the minimum norm for x

So the pseudo-inverse gives the solution $x = A^+ b$ which minimizes $||Ax - b||_2$. How do I prove that $x$ also has the smallest 2-norm for all $x_i$ where $||Ax - b||_2 = ||Ax_i - b||_2$?
0
votes
1answer
18 views

Regarding least squares, value of n in a scatter plot

I am currently in a college algebra class wherein I am required to do a rather lengthy project regarding least squares. One particular exercise posits the following (keep in mind that in this project ...
0
votes
0answers
29 views

How to solve the least square with $L_2$ norm constraint directly?

I answered the question Why are additional constraint and penalty term equivalent in ridge regression? earlier, but I myself still have some questions on it. To solve \begin{align} \min_{\beta} ...
1
vote
0answers
28 views

Increase the probability of correct prediction using multiple regression

First off let me begin by saying that I'm brand new to statistics and I would appreciate it if you could dumb down any answers for my problem. I am trying to create a general prediction of how much a ...
0
votes
1answer
24 views

How to approach this least square projection question?

A simple linear regression model as follows, \begin{align} Y=\beta_0+\beta_1 X+\epsilon \end{align} Now I would like to replace $X$ with another variable $Z$. I only know $X$ and $Z$ are correlated ...
0
votes
0answers
27 views

Motivation for gradient descent method over OLS/MLE for simple linear regression?

I am beginner in machine learning and I am currently trying to find the motivation for gradient descent method. I am confused why we want to employ gradient descent method for linear regression? I see ...
0
votes
1answer
22 views

Does $\vec{b}$ have to be in $\text{im }(A)$ if $|| \vec{b}-A\vec{x}^*||=0$?

I'm going through a least squares computation where $A=\begin{bmatrix}3&2\\5&3\\4&5\end{bmatrix}$ and $\vec{b}=\begin{bmatrix}5\\9\\2\end{bmatrix}$. From ...
-1
votes
2answers
85 views

What are some real life applications of least squares problem?

I'm looking for some applications that require solving the least square problem. I know polynomial fitting is one of them, but sure there are many others. Thanks
1
vote
4answers
42 views

Finding initial values with the help of least square method

$D: \begin{bmatrix}1&4.19\\2&3.40\\3&2.80\\4&2.30\\5&1.99\\6&1.70\\7&1.51\\8&1.34\\9&1.21\\10&1.09\end{bmatrix}$ In this table of data the first column is ...
0
votes
1answer
33 views

Least squares fit to a an exponential equation with one unknown

I have this equation $$y = s - cx^{1.85}$$ where s is a known integer and c is unknown. I want to use the least squares method to find the best value of c that fits a set of points. I've used ...
1
vote
1answer
27 views

Least squares approximation to a subspace.

Consider the inner product space $C[0,1]$ with inner product $$\langle f,g\rangle =\int_0^1f(x)g(x)\,dx$$ Let $S$ be the subspace spanned by $1$ and $2x-1$ Find the best least squares approximation ...
0
votes
0answers
12 views

A least squares subject to orthogonal constraint

A least square minimization subject to orthogonal constraint: Assume we are given two matrices $B$ and $A$, we aim to find the following $X$, $\min_A \|B - XA \|_F^2, \mbox{subject to} ~XX^T=I$, ...
0
votes
0answers
13 views

marginal return as a function of full return and inverse diagonal

I met this formula somewhere in our system, and even remember that I proved something like that a lot time ago. Now I became older and probably my mind is not so flexible as before,so I am confused. ...
0
votes
0answers
15 views

algorithms for constrained linear least-squared problems

Given $x_\text{opt}, b\in \mathbb{R}^n$, $A\in\mathbb{R}^{n\times n}$, I'm looking for $x\in \mathbb{R}^n$ such that $$ \min_x \|x - x_\text{opt}\|_2^2,\\ A x \le b,\\ x \ge 0. $$ Apparently, this ...
0
votes
0answers
15 views

curve fittin with non-gaussian noise

Fitting with the least squares method results in the ML fit assuming the given points have a gaussian distributed noise. What methods are there for non-gaussian noise distributions, especially ...
1
vote
0answers
13 views

3D topographic progress compensation by the least squares method.

I'm looking for an explanation of the least squares method used in the case of a correction of 3D point network. We have reference points with known coordinates XYZ, we calculate intermediate points ...
0
votes
0answers
19 views

Express the estimation of the difference of means between two grooups under equal variance using a linear model.

Using a linear model, I've constructed two variables $y$ and $z$: $y = \alpha + \beta x + \epsilon $ $z = \alpha' +\beta'x + \epsilon' $ and I am assuming the difference of the two variables can be ...
0
votes
3answers
102 views

Solving $Ax=b$ when $x$ and $b$ are given.

I am trying to study least square and linear regression and I understand the solution for $Ax = b$ when x is the unknown and the LS solution is given by $(A^TA)^{-1}A^TA$. Now, I was wondering if ...
2
votes
1answer
29 views

why conventional approximation method is true?

why the text book method for finding the fitting curve is right ? we have n data we want to approximate with a polynomial of degree m $P_m(x)$ (m < n-1). and of course $E = \sum_{i=1}^m ...
0
votes
0answers
17 views

Measure best fitting major and minor axis length given 3 points on an ellipse

I am trying to measure the parameters of an ellipse in an image. I have the center, the rotation of the ellipse. I am trying to find the best fitting major and minor axis length based on 3 given ...