The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
1answer
17 views

Fitting a polynomial model to the data

I want to fit a line to the following data. But the line I have obtained is far from the data. What is wrong in the following least square problem? ...
1
vote
0answers
12 views

Trouble with horner function in MATLAB [closed]

I have the following homework question: Apply linear least squares with the two models S1(A, B, C) = Ax^2 + Bx + C and S2(A, B, C, D) = Ax^3 + Bx^2 + Cx + D to the data set (0, 4), (1, −1), (2, ...
0
votes
1answer
36 views

Please help with linear algebra least squares problem

The problem is as follows: "Please set up (but do not solve) the normal equations for the following least squares approximation problem: Find $(a, b, c, d)$ such that the plane H described by $ax + ...
1
vote
1answer
16 views

Prove that Least squares and decorrelator are equivalent

Here is the problem: $$\mathbf{y}=\mathbf{Ax}+\mathbf{b}$$ where $\mathbf{y,x,b}$ are vectors, and$\mathbf{A}$ is matrix(generally rectangular, but with full column rank). The least squre solution ...
-2
votes
0answers
39 views

Property Moore-Penrose inverse [closed]

Let $u \in \mathbb C^{m}$ and $v \in \mathbb C^n$ with $v \neq 0$. Show that $\| uv^{\dagger}\|_2 =\|u\|_2/\|v\|_2$ how I prove this? $v^{\dagger}=\frac{x^*}{\|v\|_2^2}$ so, ...
1
vote
1answer
32 views

Toeplitz equality constrained least-square optimization

What is the fastest known algorithm for least-square optimization problem with a linear equality constrain \begin{align*} &\min \|K x - y\|^2 + \mu \|x\|^2\\ \text{s. t. }& Q x = v ...
0
votes
1answer
28 views

Is k-means clustering guaranteed to converge if using Manhattan distance?

The k-means algorithm is an iterative clustering algorithm that partitions the data points into K clusters (with centroids {$\mu_1, ... , \mu_k$}, minimizing the Sum-of-Squared-Error: $$ SSE = ...
1
vote
1answer
34 views

Solving Linear Least Squares with Constraint

I would like to use linear least squares to solve for $x \in \mathbb{R}^5$ where $$ Ax = b \rightarrow x = (A^TA)^{-1}A^Tb $$ but would like to include the constraint $$ x_1^2 + x_2^2 = 1 $$ I ...
0
votes
1answer
13 views

How does minimum squared error relate to a linear system?

Given some system $U*x = b$, I've solved for $x^*$, the least squares solution. I then compute the minimum squared error by $||U*x^* - b||^2$. I know that the least squares solution minimizes the ...
0
votes
0answers
15 views

Different SVD results in Matlab

my question relates to calculating SVD in Matlab. I have been reading a lot and somehow I have jumbled up all the facts. It would be great if you experts could get me to the right track. My task is ...
3
votes
1answer
29 views

When is Block-Partitioned Matrix Invertible?

Suppose I have a block partitioned matrix \begin{equation} \begin{bmatrix} \mathbf{X}_1^{\top}\mathbf{X}_1 & \mathbf{X}_1^{\top}\mathbf{X}_2 \\ \mathbf{X}_2^{\top}\mathbf{X}_1 & ...
0
votes
1answer
25 views

Bivariate rational (quadratic over linear) model fitting by least squares

I am trying to fit a simple model over 2D data points, in the frame of an image formation model with perspective and optical distortion. My model is the ratio of a second degree polynomial over a ...
1
vote
2answers
40 views

Show $E\left(\mathbf{X}_i \otimes \mathbf{u}_i\right)=\mathbf{0}$ implies $E\left(\mathbf{X}_i^{\top}\mathbf{G}\mathbf{u}_i\right)=\mathbf{0}$

Let $\mathbf{X}_i$ be a $G \times K$ random matrix, and let $\mathbf{u}_i$ be a $G \times 1$ random vector, and suppose we have a sample of $i=1,\ldots,N$ of each. Suppose the following condition ...
1
vote
1answer
32 views

How to Change Summation Expression $\sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$ into Matrix Expression

Let $\mathbf{X}_i$ be a $G \times K$ matrix, and suppose are $i=1,...,N$ of these matrices. Note that \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{X}_i &= \begin{bmatrix} ...
1
vote
0answers
35 views

Which projection, in $L_\infty$ norm or $L_2$ norm, is non-expansion?

I am just wondering which projection is non-expansion? Basically, I am wondering if $F$ is a projection operator then which norm would satisfy the following non-expansion property, where for a given ...
1
vote
0answers
41 views

Matrix Decompositions: Difference between Cholesky Decomposition, Eigendecomposition and Jordan Normal Form Decomposition

I recently created a related topic about the square root matrix, in case you'd like to refer to that one. Here's what we want: Consider the matrix $\Omega=E(\mathbf{u}^{\top}\mathbf{u})$, where ...
0
votes
1answer
12 views

Least Squares with equality Constraint

So I have a problem in the form of Y = Ax, where A is a matrix and x and Y are vectors. A is a skinny matrix and I would like to do a least squares solution to solve for x. Lets say though that x ...
1
vote
2answers
86 views

Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

I know in general that if a matrix $A$ is positive definite, then there exists a (unique?) square root matrix $B$, which is also positive definite, such that $BB=A$. Therefore, suppose $A$ is ...
0
votes
0answers
27 views

Least squares with three quadratic constraints (Ellipse fitting based on algebraic distance)

I would like to fit an ellipse to a given set of scattered data in $\mathcal{R}^2$. The fitting problem is in form least squares, minimizing the sum of squared algebraic distances \begin{equation} ...
2
votes
0answers
41 views

Least square regression of a vector onto a space [closed]

Suppose the basis vectors for a space are [1 0 0] and [0 1 2]. Now, I would like to find the least square projection of the vector [a b b] onto the mentioned space. How do I approach this?
0
votes
0answers
19 views

best fit straight line MAXIMIZING k-y values

I understand the minimization of the sum of the least squares approach to obtain a best fit straight line. This approach, however, unduly weights the "outliers" more than those points close to the ...
0
votes
1answer
31 views

Well-defined $\xi$-weighted (Euclidean) norm

Suppose $\xi$ is a vector, that is used for $\parallel z\parallel_\xi$ calculation. Should every element of $\xi$ be positive, $\xi(i)>0$?
2
votes
1answer
78 views

Step by step LMS for learning a linear function

Disclaimer Since this is an exercise assignment I'm not looking for a complete solution but for help that enables me to solve it on my own The task Given the error function ...
0
votes
0answers
22 views

Finding a relative error measure on a data set proportional to another

I have a set of exact data points $\mathcal{X}=\{X_i\}$ and another approximate one $\mathcal{Y}=\{Y_i\}$ where there is a correspondence between $X_i$ and $Y_i$ for all $i$. If $\mathcal{Y}$ was ...
0
votes
1answer
51 views

Why does SVD provide the least squares solution to $Ax=b$?

I am studying the Singular Value Decomposition and its properties. It is widely used in order to solve equations of the form $Ax=b$. I have seen the following: When we have the equation system $Ax=b$, ...
0
votes
1answer
16 views

How to find variance of a vector?

I am given a set of measurements: $\tilde{y_1}=x+v_1$ $\tilde{y_2}=x+v_2$ Where $v$'s are random variables with $E\{v_1\}=E\{v_2\}=E\{v_1v_2\}=0, E\{v_1^2\}=a, E\{v_2^2\}=b$. A least squares ...
0
votes
1answer
32 views

Understanding how to solve a Cost Function?

I'm having trouble seeing the relationship in the following equation. Let's assume $J(0,1)$ and $m=4$. First I figure out my hypothesis function ...
1
vote
0answers
98 views

Normal equations for minimization of Frobenius norm least squares error

I'm having a hard time understanding the most efficient sequence of steps for deriving the normal equations for Frobenius norm least squares minimization. Here I want to minimize the norm of a matrix ...
1
vote
0answers
46 views

Non-linear least squares solver to solve a system of non-linear equations?

Can I use a non-linear least squares solver to find the solutions of a system of non-linear equations? From Wikipedia: "The method of least squares is a standard approach to the approximate ...
0
votes
2answers
52 views

Taking derivative with respect to a vector

From time to time, I come across with derivation operations which are executed with regard to a vector. For example, the least squares estimation method with more than one explanatory variables is ...
0
votes
0answers
27 views

Perpendicular distances versus vertical distances

Why is it better to use perpendicular distance rather than vertical distance along a particular coordinate axis when finding the best fit subspace? This is an exercise question in a chapter related to ...
0
votes
0answers
42 views

When can I solve in closed form this curve fitting problem?

I have $n$ real values $x_1,x_2,\ldots,x_n$ and $n$ real values $y_1,y_2,\ldots,y_n$; then I have a function $f(x,\boldsymbol\theta)$ from $\mathbb{R}$ to $\mathbb{R}$ and depending on $m$ parameters ...
1
vote
1answer
52 views

Prove that $E(\mathbf{u}|\mathbf{X})=\mathbf{0}$ implies $Cov(\mathbf{x},\mathbf{u})=0$

Let \begin{equation} \mathbf{y}=\mathbf{X}\mathbf{\beta}+\mathbf{u} \end{equation} where $\mathbf{y}=\begin{bmatrix}y_1 \\ \vdots \\ y_n\end{bmatrix}$, $\mathbf{X}=\begin{bmatrix}X_{11} & ...
1
vote
1answer
77 views

how to apply non-linear least square

I'm trying to implement the example of estimating an angle between a target $\textbf{x}$ and a sensor $x_{p}$. I'm using the example in this book. There are three available measurements of the angle ...
1
vote
0answers
12 views

GMM estimation of linear regression with intercept restriction

Say I have a time series regression as follows: $$y_t = a_i + \beta_i x_t + \varepsilon_t^i \ \ ; \ \ t = 1, 2, \cdots, T \ \ \text{for each } i$$ Now say I impose the following restriction on the ...
1
vote
0answers
10 views

force singular value decomposition :: multiple solutions

Well I'm writing a code to solve a positioning problem. given arrival times from multiple sources I want to invert and get the receiver position. obviously I have the xyz of each receiver. so I ...
1
vote
0answers
32 views

Least square / Linear regression over a simplex

I have to solve the following least square problem: $$\hat{x} = \arg \min_{x \in S} \|Ax - b\|^2$$ If $S = \mathbb{R}^n$, then the solution is given by $$\hat{x} = (A^TA)^{-1}A^Tb$$ having posed ...
0
votes
1answer
44 views

Why is $E(u)=0$ when an intercept is included in OLS Estimation?

I am reading Wooldridge's graduate econometrics text. There he states that when estimating the equation $y=\mathbf{x\beta}+u$ by OLS, if an intercept (constant term) is included in your $\mathbf{x}$ ...
0
votes
0answers
28 views

OLS standard error that corrects for autocorrelation but not heteroskedasticity

Question: By mapping the OLS regression into the GMM framework, write the formula for the standard error of the OLS regression coefficients that corrects for autocorrelation but not ...
0
votes
1answer
26 views

how to find measurement matrix for least square.

I know how to use least square for estimating a constant value given a bunch of measurements. It is the average assuming measurements have same weight of variance. $$ \hat{x} = (H^{T}H)^{-1} H^{T}z ...
3
votes
1answer
52 views

Generalized inverse/Pseudo Inverse

Let $A_{m. n}$ be a matrix with rank $p$ where $p\leq m$ and $p\leq n$. First Question: We need to show that $A$ can be decomposed as a product of two matrices $A=BC$ where $B$ is an $m$ by $p$ and ...
0
votes
0answers
25 views

Convex sets and minimum points

Let $X$ be the convex set formed by the convex combination of the $n$ points $\{x_1, x_2, ... x_n\}$ in $\mathbb{R}^n$. Let $X^* \subseteq X$ be the convex set of minimal points w.r.t to the convex ...
0
votes
0answers
32 views

Least squares and simplex

I am interested in the linear least square problem with the solution with the following constraints : $$ \min_x \|Ax-b\|^2$$ subject to $0 \le x_i \le 1$ and $\Sigma_{i=1}^n x_i= 1$. Because of the ...
1
vote
1answer
32 views

Least squares fitting using cosine function?

Hello I am trying to fit a harmonic of the form $$y = b + c\cos(x)$$ to four data points (0,6.1) (.5,5.4) (1,3.9) (1.5,1.6) using least squares for homework. I know that the error $= Y_i - f(x_i)$ but ...
3
votes
3answers
61 views

linear solution of curve fitting on multiple linear functions differing by a multiplier

I recently posted this question here but I thought this could be of interest also in mathematics, given I found a partially related question here I am facing the following problem. I know nonlinear ...
0
votes
0answers
4 views

Least square estimator: $N( \beta x_i, \sigma^2)$

Let $ Y_1,...,Y_n$ be i.i.d $N(\beta x_i, \sigma^2) $ with known $ x_i's$. It is asked to find the Mean Squared Estimator for $\beta.$ I didn't understandmuch about this method of pbtaining an ...
1
vote
1answer
37 views

Prediction error in least squares with a linear model

In the classical linear model with $$Y=X\beta +\epsilon,$$ where $Y \in \mathbb{R}^n$ is the observation, $X\in \mathbb{R}^{n\times p}$ is the known covariates, $\beta \in \mathbb{R}^p$ is the ...
1
vote
2answers
26 views

Simple linear regression seems off

I have the following datapoints: $$p1(52,730)$$ $$p2(53,409)$$ $$p3(52,250)$$ $$p4(52,90)$$ Now I want to find the best fitting line between these points. When I use simple linear regression I get $$y ...
1
vote
1answer
33 views

Solving least-squares: why ever use iterative descent methods over pseudoinverse?

I recall doing an assignment in machine learning where we ran regression tests on a data set, both using our own implemented gradient descent program, and then using the (right) pseudoinverse ...
1
vote
0answers
12 views

Pseudo inverse does not satisfy original problem although matrix has sufficient rank

I have a $4(\text{rows}) \times 5(\text{col})$ matrix. Lets call it $A$. I want to solve $AX = b$ where $b$ is $4 \times 1$ vector. Verified with Matlab that ...