The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
7 views

Least squares with three quadratic constraints (Ellipse fitting based on algebraic distance)

I would like to fit an ellipse to a given set of scattered data in $\mathcal{R}^2$. The fitting problem is in form least squares, minimizing the sum of squared algebraic distances \begin{equation} ...
2
votes
0answers
22 views

Least square regression of $(x_1,x_2)$ onto $(w,2w)$? [on hold]

I don't understand why the answer of the least square regression of $(x_1,x_2)$ onto $(w,2w)$ gives $$w=\frac{1}{5}(x_1+2x_2)$$
0
votes
0answers
13 views

best fit straight line MAXIMIZING k-y values

I understand the minimization of the sum of the least squares approach to obtain a best fit straight line. This approach, however, unduly weights the "outliers" more than those points close to the ...
2
votes
1answer
40 views

Step by step LMS for learning a linear function

Disclaimer Since this is an exercise assignment I'm not looking for a complete solution but for help that enables me to solve it on my own The task Given the error function ...
0
votes
0answers
14 views

Finding a relative error measure on a data set proportional to another

I have a set of exact data points $\mathcal{X}=\{X_i\}$ and another approximate one $\mathcal{Y}=\{Y_i\}$ where there is a correspondence between $X_i$ and $Y_i$ for all $i$. If $\mathcal{Y}$ was ...
0
votes
1answer
42 views

Why does SVD provide the least squares solution to $Ax=b$?

I am studying the Singular Value Decomposition and its properties. It is widely used in order to solve equations of the form $Ax=b$. I have seen the following: When we have the equation system $Ax=b$, ...
0
votes
1answer
14 views

How to find variance of a vector?

I am given a set of measurements: $\tilde{y_1}=x+v_1$ $\tilde{y_2}=x+v_2$ Where $v$'s are random variables with $E\{v_1\}=E\{v_2\}=E\{v_1v_2\}=0, E\{v_1^2\}=a, E\{v_2^2\}=b$. A least squares ...
0
votes
1answer
27 views

Understanding how to solve a Cost Function?

I'm having trouble seeing the relationship in the following equation. Let's assume $J(0,1)$ and $m=4$. First I figure out my hypothesis function ...
1
vote
0answers
77 views

Normal equations for minimization of Frobenius norm least squares error

I'm having a hard time understanding the most efficient sequence of steps for deriving the normal equations for Frobenius norm least squares minimization. Here I want to minimize the norm of a matrix ...
1
vote
0answers
38 views

Non-linear least squares solver to solve a system of non-linear equations?

Can I use a non-linear least squares solver to find the solutions of a system of non-linear equations? From Wikipedia: "The method of least squares is a standard approach to the approximate ...
0
votes
1answer
38 views

Taking derivative with respect to a vector

From time to time, I come across with derivation operations which are executed with regard to a vector. For example, the least squares estimation method with more than one explanatory variables is ...
0
votes
0answers
22 views

Perpendicular distances versus vertical distances

Why is it better to use perpendicular distance rather than vertical distance along a particular coordinate axis when finding the best fit subspace? This is an exercise question in a chapter related to ...
0
votes
0answers
39 views

When can I solve in closed form this curve fitting problem?

I have $n$ real values $x_1,x_2,\ldots,x_n$ and $n$ real values $y_1,y_2,\ldots,y_n$; then I have a function $f(x,\boldsymbol\theta)$ from $\mathbb{R}$ to $\mathbb{R}$ and depending on $m$ parameters ...
1
vote
1answer
47 views

Prove that $E(\mathbf{u}|\mathbf{X})=\mathbf{0}$ implies $Cov(\mathbf{x},\mathbf{u})=0$

Let \begin{equation} \mathbf{y}=\mathbf{X}\mathbf{\beta}+\mathbf{u} \end{equation} where $\mathbf{y}=\begin{bmatrix}y_1 \\ \vdots \\ y_n\end{bmatrix}$, $\mathbf{X}=\begin{bmatrix}X_{11} & ...
1
vote
1answer
72 views

how to apply non-linear least square

I'm trying to implement the example of estimating an angle between a target $\textbf{x}$ and a sensor $x_{p}$. I'm using the example in this book. There are three available measurements of the angle ...
1
vote
0answers
11 views

GMM estimation of linear regression with intercept restriction

Say I have a time series regression as follows: $$y_t = a_i + \beta_i x_t + \varepsilon_t^i \ \ ; \ \ t = 1, 2, \cdots, T \ \ \text{for each } i$$ Now say I impose the following restriction on the ...
1
vote
0answers
8 views

force singular value decomposition :: multiple solutions

Well I'm writing a code to solve a positioning problem. given arrival times from multiple sources I want to invert and get the receiver position. obviously I have the xyz of each receiver. so I ...
1
vote
0answers
24 views

Least square / Linear regression over a simplex

I have to solve the following least square problem: $$\hat{x} = \arg \min_{x \in S} \|Ax - b\|^2$$ If $S = \mathbb{R}^n$, then the solution is given by $$\hat{x} = (A^TA)^{-1}A^Tb$$ having posed ...
0
votes
1answer
37 views

Why is $E(u)=0$ when an intercept is included in OLS Estimation?

I am reading Wooldridge's graduate econometrics text. There he states that when estimating the equation $y=\mathbf{x\beta}+u$ by OLS, if an intercept (constant term) is included in your $\mathbf{x}$ ...
0
votes
0answers
27 views

OLS standard error that corrects for autocorrelation but not heteroskedasticity

Question: By mapping the OLS regression into the GMM framework, write the formula for the standard error of the OLS regression coefficients that corrects for autocorrelation but not ...
0
votes
1answer
25 views

how to find measurement matrix for least square.

I know how to use least square for estimating a constant value given a bunch of measurements. It is the average assuming measurements have same weight of variance. $$ \hat{x} = (H^{T}H)^{-1} H^{T}z ...
3
votes
1answer
41 views

Generalized inverse/Pseudo Inverse

Let $A_{m. n}$ be a matrix with rank $p$ where $p\leq m$ and $p\leq n$. First Question: We need to show that $A$ can be decomposed as a product of two matrices $A=BC$ where $B$ is an $m$ by $p$ and ...
2
votes
0answers
57 views

How to stop iteration in inverse problem using nonlinear least square problem?

I am having a real trouble with stopping criterion in iteration of Generalized Nonlinear Least Square. My problem is that I do not know exactly how to stop my iteration. First, I will give a short ...
0
votes
0answers
25 views

Convex sets and minimum points

Let $X$ be the convex set formed by the convex combination of the $n$ points $\{x_1, x_2, ... x_n\}$ in $\mathbb{R}^n$. Let $X^* \subseteq X$ be the convex set of minimal points w.r.t to the convex ...
0
votes
0answers
31 views

Least squares and simplex

I am interested in the linear least square problem with the solution with the following constraints : $$ \min_x \|Ax-b\|^2$$ subject to $0 \le x_i \le 1$ and $\Sigma_{i=1}^n x_i= 1$. Because of the ...
1
vote
1answer
30 views

Least squares fitting using cosine function?

Hello I am trying to fit a harmonic of the form $$y = b + c\cos(x)$$ to four data points (0,6.1) (.5,5.4) (1,3.9) (1.5,1.6) using least squares for homework. I know that the error $= Y_i - f(x_i)$ but ...
3
votes
3answers
47 views

linear solution of curve fitting on multiple linear functions differing by a multiplier

I recently posted this question here but I thought this could be of interest also in mathematics, given I found a partially related question here I am facing the following problem. I know nonlinear ...
0
votes
0answers
4 views

Least square estimator: $N( \beta x_i, \sigma^2)$

Let $ Y_1,...,Y_n$ be i.i.d $N(\beta x_i, \sigma^2) $ with known $ x_i's$. It is asked to find the Mean Squared Estimator for $\beta.$ I didn't understandmuch about this method of pbtaining an ...
1
vote
1answer
36 views

Prediction error in least squares with a linear model

In the classical linear model with $$Y=X\beta +\epsilon,$$ where $Y \in \mathbb{R}^n$ is the observation, $X\in \mathbb{R}^{n\times p}$ is the known covariates, $\beta \in \mathbb{R}^p$ is the ...
1
vote
2answers
26 views

Simple linear regression seems off

I have the following datapoints: $$p1(52,730)$$ $$p2(53,409)$$ $$p3(52,250)$$ $$p4(52,90)$$ Now I want to find the best fitting line between these points. When I use simple linear regression I get $$y ...
1
vote
1answer
27 views

Solving least-squares: why ever use iterative descent methods over pseudoinverse?

I recall doing an assignment in machine learning where we ran regression tests on a data set, both using our own implemented gradient descent program, and then using the (right) pseudoinverse ...
1
vote
0answers
12 views

Pseudo inverse does not satisfy original problem although matrix has sufficient rank

I have a $4(\text{rows}) \times 5(\text{col})$ matrix. Lets call it $A$. I want to solve $AX = b$ where $b$ is $4 \times 1$ vector. Verified with Matlab that ...
3
votes
1answer
45 views

Weighted least squares with angular data

Suppose I have a system whose state is $\Theta=(\theta_1,\theta_2,\ldots,\theta_n)$, where $\theta_i\in[-\pi,\pi)$ (i.e., they are angles). I'd like to determine the most likely estimate of $\Theta$ ...
0
votes
2answers
102 views

Least Squares: Derivation of Normal Equations with Chain Rule

I'm new to Stackexchange so please bear with me. I'm struggling with the least squares formula. Now Wikipedia does show ways to derive the "normal equations". But I'd like to get the same result ...
0
votes
0answers
22 views

numerical (script) fit of function with 2 arguments

I would like to find the least-square fit for a 1D-function that takes two arguments. m(x,y) = d * (x-x0)^2 / (y-y0)^2 I would like to write a c++ routine to ...
0
votes
1answer
33 views

How can missing data be organised or classified (Interpolation vs Approximation)?

I'm looking for a way to distinguish between the various types of missing data techniques? Can someone help to clarify or organize these categories in sub-sections or indicate similarities or ...
1
vote
2answers
34 views

orthogonal matrices vs. orthogonal columns

I'm just reading a book on econometrics and now I'm stuck with a problem: There is a Theorem on "Orthogonal Partitioned Regression" which says: "In the multiple linear least squares regression of ...
0
votes
1answer
49 views

Unique least square solutions

There is a theorem in my book that states: If $A$ is $m\times n$, then the equation $Ax = b$ has a unique least square solution for each $b$ in $\mathbb{R}^m$. But can we find a counter-example to ...
0
votes
0answers
29 views

Optimizing choice of data points with known model

my question is fairly simple to explain but I'm not quite sure how to solve it. Basically lets say I am measuring some value at 8 time points. I get to choose these 8 time points. I also know the ...
2
votes
0answers
55 views

Levenberg's original article “A method for the solution of certain problems in least squares”

Does there exist any digital copy of the original article (or a transcript) K. Levenberg, A method for the solution of certain problems in least-squares, Quart. Appl. Math. 2 (1944): 164-168? It is ...
1
vote
0answers
32 views

Non linear least square ellipse fitting

I am trying to find a Non linear leasts squares ellipse fit for a set of 100 data points data points $(x,y)$. Now i have found the values of $A,B,C,D,E,F$ according to the conical equation of the ...
5
votes
0answers
58 views

How to solve this equation (may be with least squares)?

I have a system of linear equations in the following form. How can I solve it? $$\operatorname*{argmin}_{a,b} \sum_{i,j} \left( \left| X(i,j)-aY(i,j)\right|-b \right)^2$$ Where $X$ and $Y$ are ...
0
votes
1answer
26 views

How to find parameters that minimize the sum of squares, using Matlab?

I have a system of linear equations in the following form. How can I solve it in Matlab? $$\operatorname*{argmin}_{a,b} \sum_{i,j} [X(i,j)-a\times Y(i,j)-b]^2$$ Where X and Y are known. I need to ...
0
votes
1answer
50 views

minimizing sum of different least squares?

Can we write the minimization problem: $$\operatorname{min}\limits_{x\in\mathbb{R}^n}\sum_{i=1}^{n}\|C_i x-b_i\|_2^2$$ as a least square problem?
1
vote
1answer
42 views

Least squares fitting issue

I deal with MRI image processing and while reading one of the articles in this field I faced with the next mathematical formula: $$ \widetilde{R_2}(t) = K_1*\overline{R_2}(t) + K_2 * \int_0^t ...
0
votes
0answers
32 views

Solver for least squares

I'm looking for a numerical solution to the constrained least squares problem below: $$ \min_\mathbf{x}\|\mathbf{a+Bx}\|^2 ~~\text{s.t}~~\|\mathbf{x}\|^2 \leq \alpha^2$$ where $\mathbf{a} \in ...
0
votes
0answers
18 views

Linearizing non-linear least squares: Problem with derivatives

We want to approximate $$y_i \approx a b^{x_i}$$ and thus have $$S=\sum_{i=1}^m (ab^{x_i}-y_i)^2$$ as least squares error term. This term is not linear in b, so it is not easy to calculate its ...
1
vote
0answers
27 views

efficient least squares A = BX+CXD (solving for matrix X)

I am interested in solving a least-squares solution of the form $$ \operatorname{argmin}_X \| A - BX - CXD \|_F^2 $$ for large (rank in hundreds to thousands) matrices $A,B,C,D,X$ I know this is ...
1
vote
1answer
78 views

Non-linear least squares with two dependent variables

I have data in the form $(t_i,x_i,y_i)$, i.e. position in 2D as a function of time. I have non-linear equations which I want to fit to the data. They give me a position $(X,Y)$ as a function of time ...
1
vote
1answer
69 views

Least squares with a quadratic inequality constraint

Is there a closed form solution for the following least squares problem: $$ \min_\mathbf{x}\|\mathbf{a+Bx}\|^2 ~~\text{s.t}~~\|\mathbf{x}\|^2 \leq \alpha^2$$ where $\mathbf{a} \in \mathbb{C^{M\times ...