The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
2answers
42 views

Least Squares: Derivation of Normal Equations with Chain Rule

I'm new to Stackexchange so please bear with me. I'm struggling with the least squares formula. Now Wikipedia does show ways to derive the "normal equations". But I'd like to get the same result ...
0
votes
0answers
20 views

numerical (script) fit of function with 2 arguments

I would like to find the least-square fit for a 1D-function that takes two arguments. m(x,y) = d * (x-x0)^2 / (y-y0)^2 I would like to write a c++ routine to ...
0
votes
1answer
17 views

How can missing data be organised or classified (Interpolation vs Approximation)?

I'm looking for a way to distinguish between the various types of missing data techniques? Can someone help to clarify or organize these categories in sub-sections or indicate similarities or ...
1
vote
2answers
29 views

orthogonal matrices vs. orthogonal columns

I'm just reading a book on econometrics and now I'm stuck with a problem: There is a Theorem on "Orthogonal Partitioned Regression" which says: "In the multiple linear least squares regression of ...
0
votes
1answer
43 views

Unique least square solutions

There is a theorem in my book that states: If $A$ is $m\times n$, then the equation $Ax = b$ has a unique least square solution for each $b$ in $\mathbb{R}^m$. But can we find a counter-example to ...
0
votes
0answers
22 views

Optimizing choice of data points with known model

my question is fairly simple to explain but I'm not quite sure how to solve it. Basically lets say I am measuring some value at 8 time points. I get to choose these 8 time points. I also know the ...
2
votes
0answers
23 views

Levenberg's original article “A method for the solution of certain problems in least squares”

Does there exist any digital copy of the original article (or a transcript) K. Levenberg, A method for the solution of certain problems in least-squares, Quart. Appl. Math. 2 (1944): 164-168? It is ...
1
vote
0answers
21 views

Non linear least square ellipse fitting

I am trying to find a Non linear leasts squares ellipse fit for a set of 100 data points data points $(x,y)$. Now i have found the values of $A,B,C,D,E,F$ according to the conical equation of the ...
4
votes
0answers
43 views

How to solve this equation (may be with least squares)?

I have a system of linear equations in the following form. How can I solve it? $$\operatorname*{argmin}_{a,b} \sum_{i,j} \left( \left| X(i,j)-aY(i,j)\right|-b \right)^2$$ Where $X$ and $Y$ are ...
0
votes
1answer
25 views

How to find parameters that minimize the sum of squares, using Matlab?

I have a system of linear equations in the following form. How can I solve it in Matlab? $$\operatorname*{argmin}_{a,b} \sum_{i,j} [X(i,j)-a\times Y(i,j)-b]^2$$ Where X and Y are known. I need to ...
0
votes
1answer
47 views

minimizing sum of different least squares?

Can we write the minimization problem: $$\operatorname{min}\limits_{x\in\mathbb{R}^n}\sum_{i=1}^{n}\|C_i x-b_i\|_2^2$$ as a least square problem?
1
vote
1answer
35 views

Least squares fitting issue

I deal with MRI image processing and while reading one of the articles in this field I faced with the next mathematical formula: $$ \widetilde{R_2}(t) = K_1*\overline{R_2}(t) + K_2 * \int_0^t ...
0
votes
0answers
30 views

Solver for least squares

I'm looking for a numerical solution to the constrained least squares problem below: $$ \min_\mathbf{x}\|\mathbf{a+Bx}\|^2 ~~\text{s.t}~~\|\mathbf{x}\|^2 \leq \alpha^2$$ where $\mathbf{a} \in ...
0
votes
0answers
16 views

Linearizing non-linear least squares: Problem with derivatives

We want to approximate $$y_i \approx a b^{x_i}$$ and thus have $$S=\sum_{i=1}^m (ab^{x_i}-y_i)^2$$ as least squares error term. This term is not linear in b, so it is not easy to calculate its ...
1
vote
0answers
22 views

efficient least squares A = BX+CXD (solving for matrix X)

I am interested in solving a least-squares solution of the form $$ \operatorname{argmin}_X \| A - BX - CXD \|_F^2 $$ for large (rank in hundreds to thousands) matrices $A,B,C,D,X$ I know this is ...
1
vote
1answer
71 views

Non-linear least squares with two dependent variables

I have data in the form $(t_i,x_i,y_i)$, i.e. position in 2D as a function of time. I have non-linear equations which I want to fit to the data. They give me a position $(X,Y)$ as a function of time ...
-1
votes
0answers
28 views

how to calculate least square root of a nonlinear equation?

I have $z= x*y$, and I want to calculate the least square root of this equation. So it becomes: $(z+dz)^2=(x+dx)(y+dy)$ then I can continue with replacing $z$ with $x\cdot y$ and $dz$ with $dxdy$ but ...
1
vote
1answer
60 views

Least squares with a quadratic inequality constraint

Is there a closed form solution for the following least squares problem: $$ \min_\mathbf{x}\|\mathbf{a+Bx}\|^2 ~~\text{s.t}~~\|\mathbf{x}\|^2 \leq \alpha^2$$ where $\mathbf{a} \in \mathbb{C^{M\times ...
0
votes
1answer
17 views

Regression when the variance of the residuals depends on the independent variable

When the residuals follow a normal distribution, the most likely function that fits the data is found using least squares. In that case: $y = f(x_i) + r_i, \quad r\sim\mathcal{N}(0, \sigma^2)$ ...
0
votes
1answer
27 views

Minimal number of points to define a rotated ellipse?

What is the minimal number of points $N$ to uniquely define the semi-major axis $a$, the semi-minor axis $b$ and the rotation angle $\omega$ of an ellipse whose the center is known/fixed (this is ...
0
votes
0answers
50 views

Find scaling factor that minimizes f(x) - round(f(x))?

Let's say I have a function $f(x)$, which has a fractional component $\{ f(x) \} = f(x) - \lfloor f(x) \rfloor$. I would like to add a scaling factor $h(x)$, where $h(x)$ is a polynomial, such that ...
0
votes
1answer
23 views

Find optimal least square solution to the normal equation

What is the optimal solution for $\beta_1$ and $\beta_2$ in the following normal equation: $$\beta _{ 1 }\sum _{ i=1 }^{ n }{ { x }_{ i } } +\beta _{ 0 }=\sum _{ i=1 }^{ n }{ { y }_{ i } } $$ EDIT ...
0
votes
0answers
12 views

Derivative and linear fitting model

Let $V=v_1,v_2,\ldots,v_n$ be the measured velocities and $A=a_1,a_2,\ldots,a_n$ be the measured accelerations of a vehicle at times $T=t_1,t_2,\ldots,t_n$. Let $Y=c_1+c_2t+c_3t^2$ be the best ...
0
votes
0answers
25 views

Finding the least square fit for 3 parameters in Linear Algebra

I know how to find least square for $y = mx+b$ when we have two parameters. But this question has $3$ parameters, am trying to think of how to approach it but so far no success, I can't find any ...
0
votes
2answers
35 views

using lagrange multipliers to fit a curve through a point

So this is part math/ part statistics. I have a set of data I'm fitting a 2nd order curve through using least squares method (matrix form). However, I've been given the requirement to pass the curve ...
0
votes
1answer
32 views

Sums of positive and negative distances to the least squares plane

Let $A_{1}, A_{2}, \ldots, A_{n}$ be points in $\mathbb{R}^{3}$ and $\pi_{*}$ be the least squares plane, i. e. $$ \sum \limits_{i = 1}^{n}\rho^{2}(A_{i}, \pi_{*}) = \min_{\pi}\sum \limits_{i = ...
0
votes
0answers
35 views

An error in least square optimization problem in Matlab

I am new to MATLAB and I want to formulate the following lease square expression in Matlab. I have some codes that I am typing here. But the optimization problem solution seems not to be correct. Does ...
0
votes
2answers
35 views

Find the least squares approximation g(x) = a0 + a1x of the function f(x) = sqrt(x), 1 <= x <= 4.

HELP!! I'm floundering here.... Find the least squares approximation $g(x) = a_0 + a_1x $of the function $f(x) = \sqrt(x),$ from $1 \le x \le 4$. I'm not sure how to set up this problem. The problem ...
0
votes
1answer
21 views

Rank degenerate non negative least squares

I'm following an algorithm in the book "Solving Least Squares Problems" by Lawson and Hanson (#15 in Siam's Classics in Applied Mathematics) for solving non negative least squares. That is, minimize ...
1
vote
1answer
33 views

Efficient Algorithm for Iteratively Reweighted Least Squares Problem

I'm interested in solving a weighted least squares problem of the form $X^T W X \beta = X^T W Y$ where $W$ is a diagonal, positive definite matrix, $X \in R^{m \times n}$, $Y \in R^{m \times 1}$ and ...
1
vote
2answers
37 views

How to find a line that minimizes the average squared perpendicular distance from the given points to the line?

I have set of points scattered around the origin. How to find a vector, such that the average squared distance (perpendicular distance) from points to the vector is minimised? Added For example, ...
1
vote
1answer
19 views

How to perform a monotonic function fitting of data points?

I'm seeking suggestions for general purpose function fitting of a set of data points, where, based on physical intuition, the relationship is expected to be "monotonic", i.e. the function should be ...
1
vote
1answer
36 views

Derivative of diagonal function

I'm working on a sightly modified least-squares method which must minimize the quantity: $$ [Y-\text{diag}(\mu X^T)]^T\cdot [Y-\text{diag}(\mu X^T)] $$ where $Y$ is a $n$-dimensional vector and $\mu$ ...
0
votes
0answers
19 views

least square line fitting in 4D space

I have two sets of points in $4D$ space as: $(P, Q_1, Q_2, Q_3)$ like: Set one: $(1, 2, 3, -1)$ $(3, 1, 2, -2)$ $(2, 4, 3, -3)$ and Set two: $(4, 2, 3, -1)$ $(7, 1, 2, -2)$ $(9, 4, 3, -3)$ ...
0
votes
1answer
19 views

Unicity of solution of a constrained least square problem

I am interested in the linear least square problem with the solution constrained to the closed standard simplex: $$ \min_x \|Ax-b\|^2$$ subject to $x_i \ge 0$ and $\| x \|_1 = 1$. More precisely, I ...
1
vote
3answers
52 views

Simplify function with polynomial via least-squares

I want to "adjust" (simplify) $f(x)$, a function, by $g(x)$, a polynomial, via least-squares. I want to write code for that. Apperently my code is issuing wrong results, so I was wondering if my ...
0
votes
1answer
29 views

Least squares fitting of vectors

I want to find: $argmin_{\lambda} = \sum_{i\epsilon I}\left \| \vec{P_{i}} - \lambda\vec{Q_{i}} \right \|^2$ where $P = (x^{'}, y^{'}, z^{'})$ and $Q = (x^{''}, y^{''}, z^{''})$ are representations ...
1
vote
2answers
37 views

Autocorrelation and var-cov matrix

$$Y_t=\beta_1+\beta_2 X_{t2}+\dots +\beta_k X_{tk}+\epsilon_t \qquad (t=1,\dots,T)$$ $$\epsilon_t=\rho \epsilon_{t-1}+v_t, \qquad v_t \sim \mathrm{i.i.d.}(0,\sigma^2_v)$$ GLS estimation under AR(1) ...
1
vote
2answers
35 views

$\operatorname{Var}(\hat{\beta_1})=\frac{\sigma^2}{\sum(X_i-\bar{X})^2}$ : how to derive this?

$\hat{\beta_1}$ is an OLS estimator for parameter $\beta_1$: $Y_i=\beta_0+\beta_1 X_i+\epsilon_i$, So $\hat{\beta_1}=\frac{\sum(X_i-\bar{X})(Y_i-\bar{Y})}{\sum(X_i-\bar{X})^2}$ and ...
0
votes
1answer
63 views

About sphere equation $z = a+bx+cy+dx^2 +ey^2$

I'm trying to fit a sphere from points. I tried a first way to estimate the sphere but I'm not satisfied. I saw in an article a way to get a best fitting sphere from the equation : $z = a+bx+cy+dx^2 ...
2
votes
1answer
38 views

Finding the least square line passing through the origin

So, I am trying to solve a least square problem. I have the matrix $$ A= \begin{bmatrix} \ 1&-1 \\ 1 & 1 \\ 1&2 \end{bmatrix}. $$ and the matrix $$ b = \begin{bmatrix} 7 \\ 7 \\ 21 ...
2
votes
1answer
46 views

Solving programmatically a least squares problem with one constrain

I need to solve the following problem (preferably in python but any other suggestion is welcome) $$ \min_x||Ax - b||_2 $$ $$ s.t. \: Dx = Dy $$ everything except x is known. $A$ and $D$ are square ...
0
votes
0answers
37 views

solve a linear least square problem with inequality constraints

given an overdetermined system, we can solve it with following solution see Linear least squares However, if I introduce some linear inequations to the system, how can I solve the system? I've ...
0
votes
0answers
15 views

Fitting by an ellipsoid with a known center?

Consider a set of $N$ points in 3D of coordinates : $$p_{i} = \left\{x_{i}, y_{i}, z_{i} \right\}$$ The very general question I ask is : how to fit these points by the surface of an ellipsoid ...
0
votes
0answers
33 views

Linear model: Show that $\hat{\theta}$ and $\hat{e}$ are independent

Show that under the assumptions $Y\sim N(X\theta,\sigma^2I_n)$and $\text{rang}(X)=\text{rang}(\theta)$ the residual vector $\hat{e}$ and the least squares estimator $\hat{\theta}$ are ...
0
votes
0answers
20 views

Derivation of Recursive Least Squares with Forgetting Factor

I have the following set of equations: $\begin{bmatrix} y_1\\ y_2\\ \vdots\\ y_n \end{bmatrix} = \begin{bmatrix} a_1 \theta_1\\ a_2 \theta_2\\ \vdots\\ a_n \theta_3 \end{bmatrix}$ In the ...
0
votes
0answers
8 views

Constrained Time-Series OLS

I'm trying to solve an OLS problems over a several number of time-series variables with zero mean except the last one. Namely speaking, my problem is ...
1
vote
2answers
49 views

total least squares derivation with matrices

Taken from a computer vision book: "to minimize the sum of the perpendicular distances between points and lines, we need to minimize $$ \sum_i (ax_i + by_i +c)^2$$ subject to $a^2 +b^2 =1$. Now using ...
0
votes
0answers
32 views

Least squares over the integers (diophantine least squares?)

I have the following problem and I do not even know under which mathematical field I should look for an answer, so any hint is highly appreciated: Let S be the ellipsoid $$ ...
0
votes
0answers
15 views

Total Least Squares problem when some columns of data matrix have no error

I'm reading through Golub and Van Loan and they mention that to solve the total least-squares problem $(A + E)x = b + r$, where the first $s$ columns of E are zero, then we can solve the problem by ...