The tag has no wiki summary.

learn more… | top users | synonyms

1
vote
0answers
9 views

Solving a sparse Least Squares problem or breaking it to separate least squares problems

I would like to solve a Least Squares (LS) problem of the form y=Fx. Let's assume for simplicity that $y \in \mathbb{R}^{2N}$ vector, $F= \left[ \begin{array}{cc} F_1 & 0 \\ 0 & F_2 ...
0
votes
0answers
12 views

Is R(A) = ker(A^t)?, where R(A) is the space generated by the columns of A

I'm looking at this deduction of the normal equations that solve the linear least squares problem. It goes like this: R(A) is the space generated by the columns of A $\hat{X}$ is the solution of the ...
2
votes
1answer
13 views

Least squares approximation problem of $t^3$ in a subspace spanned by even degree polynomials.

I am having trouble solving the following question, Let $P_9 ([-1,1])$ be the complex vector space consisting of polynomials $p:[-1,1] \rightarrow\mathbb{C}$ with degree 9 or lower. With the inner ...
0
votes
0answers
9 views

Equivalence of the partial least square regresssion's iterative algorithm and its optimization problem

I am reading The Elements of Statistical Learning. This is a page from the partial least square section: The exercise asks to prove the equivalence between Algorithm 3.3 and Eq. (3.64). Here's my ...
1
vote
0answers
116 views

Can gradient descent solve this problem $\text{argmin}_x \|Ax-[Var(Ax)]^{\frac{1}{2}}-b\|^2$?

How can I find the (approximate) solution to the following problem: $$\text{argmin}_x \|Ax-[Var(Ax)]^{\frac{1}{2}}-b\|^2,$$ where $Var(.)$ denotes the variance? $A$ is matrix and $b$ and $x$ are ...
0
votes
1answer
13 views

Determining parameters for static system

So i have to determine parameters for static system: $y=o1 + o2u$ So, my idea was to pick mesurments for witch $\det[]!=0$ (2 of them as L=1 and R=2) Parameters :$n=1 u=-2 y=9$ and $n=2 u=1 y=-2$ ...
1
vote
0answers
18 views

incremental approach to solve positive least square problem

Is there any incremental (approximate) solution for the following positive least squares problem: $$\min_x \|Ax-b\|^2\qquad \textrm{s.t.}\qquad x_i> 0,~b_1=1,~b_{i>1}=0$$
0
votes
0answers
15 views

Least squares: Calculus to find residual minimizers?

Reading a section on simple regression in "An Introduction to Statistical Learning with Applications in R" I got a question on residual sum of squares minimization. Quoting from the book: [quote] ... ...
1
vote
1answer
35 views

Least-squares in integral form

In the linear least squares problem one minimizes the norm of the residual vector: $$ \chi^2=\|\vec{b}-A\vec{x}\|_2^2=\sum_i^m{|b_i-\sum_j^nA_{ij}x_j|^2} $$ where $A \in \mathbb{R}^{m\times n}, ...
1
vote
0answers
11 views

Is there any way how logistic regression parameters can be preserved under a linear projection?

I have a logistic regression model and OLS estimation of its parameters $\hat{w}$: $$y=\sigma(x^Tw + b)$$ Now, I would like to reduce the dimensionality of the space in which $x \in X$ lies by some ...
0
votes
1answer
25 views

X values for least squares approximation

If I have a yearly quantity (eg. 2000 - 45, 2001-67, 2002 - 38.....2010 - 38) and I need to find the least squares line for this relationship, what should I use for the X values? Should I use the ...
0
votes
1answer
31 views

Least Square fit for signal data (360 points)

I would like to analyze data to get the maximum value out of 360 points. I used least square fitting because I get the data from signal strengths. I want to remove any outliers I get from my data ...
3
votes
0answers
49 views

Tikhonov regularization vs truncated SVD

To find $\mathbf{x}$ such that $$A\mathbf{x}=\mathbf{b}$$ we can use least squares when the problem is not well posed. Further, we can use Tikhonov regularization when $A$ is ill-conditioned. In ...
3
votes
1answer
46 views

$x^TAx-b^Tx\leq 0\Rightarrow\|x\|_2\leq\|A^{-1}b\|_2$?

Let $x$ be a vector, $A\succ 0$ an inverse matrix and $b$ a vector with proper dimensions. If $$x^TAx-b^Tx\leq 0,$$ do we have $$\|x\|_2\leq\|A^{-1}b\|_2?$$ I don't think it's a hard problem, but I ...
1
vote
1answer
47 views

Find cubic Bézier control points given four points

What I need is to generate an SVG file while having a series of (x,y) ready. P0(x0,y0) P1(x1,y1) P2(x2,y2) P3(x3,y3) P4(x4,y4) P5(x5,y5) ... I need to make a ...
0
votes
0answers
33 views

Wording: l2/SSE/Sum-of-Squares Objective Function

The least-square problem is a very common optimization problem, where the objective function describes the sum over squared residua $r_n$ with respect to a parameter vector $p$: $$p \mapsto ...
2
votes
1answer
32 views

Analytic solution for matrix factorization using alternating least squares

The standard form for ridge regression aims to minimize the following cost function. $$ \min\ \ \sum_i(y_i-x_i^T\beta)^2 + \lambda\sum_j\beta^2_j $$ As described here, it's possible to differentiate ...
0
votes
0answers
37 views

Linear optimization w/ linear and non-linear inequality constraints

Given dependent variables $Q_i$ and independent variables $x_i$, $y_i$, $z_i$ where $i=1,⋯, N $ which are related via the following system of N linear equations with parameters $P_1$, $P_2$ and ...
1
vote
1answer
70 views

MATLAB curve fitting - least squares method - wrong “fit” using high degrees

Anyone here that could help me with the following problem? The following code calculates the best polynomial fit to a given data-set, that is; a polynomial of a specified degree. Unfortunately, ...
1
vote
1answer
34 views

Am I doing this approximation correctly? (least squares method)

Here is the problem. Find the function $f$ of the type $f(x) = a\cos x + b\sin x$ which best approximates the function $g$ in the points : $$ \begin{array}{ c | c | c | c | c | c | c } x & ...
0
votes
1answer
32 views

Linear regression involving angles in a triangle.

In a survey experiment, three independent measurements $29.5^{\circ}$, $30.5^{\circ}$, $120.5^{\circ}$ are obtained from the three angles $\alpha,\beta,\gamma$ of a triangle. Formulate the ...
1
vote
1answer
27 views

Least Squares in Matlab

I'm stuck on part (d) I'm not sure how to code it so that it approximates that function in matlab. I'm also not sure if my (a) thru (c) are correct. But this is what I have so far. ...
0
votes
0answers
24 views

Estimate expected gain for least square

The following data indicate the gain in reading speed vs the number of weeks in the program of 10 students in a speed-reading program: weeks 2 3 8 11 4 5 9 7 5 7 Speedgain 21 42 102 130 52 57 105 85 ...
0
votes
0answers
42 views

Curve fitting and regression: reading speed

The following data indicate the gain in reading speed vs the number of weeks in the program of 10 students in a speed-reading program: weeks 2 3 8 11 4 5 9 7 ...
1
vote
2answers
38 views

Least Squares approximation for item prices

Let's say that $A$, $B$, $C$ are different items with different values. $R$ is a unit of currency, for simplicity I'll let it be $1$. Traders frequently trade these items on an open market. Price is ...
1
vote
2answers
36 views

Minimum value of an integral with least square?

I have a problem. The question is: given a parabola $$p(t) = a + bt + ct^2$$ I need to evaluate the least squares straight line $$A + Bt$$ for which this integral is minimum: $$ \int_{t_1}^{t_2} ...
0
votes
0answers
9 views

Range-Based Localization of a Point using LSE

Suppose that we a set of points $P = \{p_1, p_2, \ldots, p_n\}$ in 3D. The coordinates of these points are known. In addition, we have another point, called $p$, We have Euclidean distances ...
0
votes
0answers
13 views

How to get the minimum singular value of some points covariance matrix?

I'm having trouble understanding the context of my question. I have a set of points which correspond to some 3D coordinates. I guess i need a minimum of two for my question. So the points would be ...
0
votes
0answers
36 views

Best fitting circle to points in 3D

I have a set of n ≥ 3 points in 3D that are measurements of a possible circle. The measured points are "noisy" so best-fitting algorithms are involved. I'm programming in C# and have put together some ...
1
vote
2answers
51 views

Lattice fitting to points

I have a set of points (shown as little black circles) which ideally form a hexagonal lattice shape, each point having an equal distance to all of its neighboring points. (Sorry for my drawing, some ...
2
votes
0answers
33 views

Does Least squares solution exist for this case?

$ {\bf{Z}} = {\bf{H}} \cdot {\bf{S}} + {\bf{N}} $ Dimensions of the matrices are as follows: Z = m X m H = m X n S = n X m (matrix S is an orthogonal matrix) N = m X m. All the elements of the ...
0
votes
1answer
22 views

Fitting a polynomial model to the data

I want to fit a line to the following data. But the line I have obtained is far from the data. What is wrong in the following least square problem? ...
0
votes
1answer
39 views

Please help with linear algebra least squares problem

The problem is as follows: "Please set up (but do not solve) the normal equations for the following least squares approximation problem: Find $(a, b, c, d)$ such that the plane H described by $ax + ...
1
vote
1answer
18 views

Prove that Least squares and decorrelator are equivalent

Here is the problem: $$\mathbf{y}=\mathbf{Ax}+\mathbf{b}$$ where $\mathbf{y,x,b}$ are vectors, and$\mathbf{A}$ is matrix(generally rectangular, but with full column rank). The least squre solution ...
1
vote
1answer
36 views

Toeplitz equality constrained least-square optimization

What is the fastest known algorithm for least-square optimization problem with a linear equality constrain \begin{align*} &\min \|K x - y\|^2 + \mu \|x\|^2\\ \text{s. t. }& Q x = v ...
0
votes
1answer
51 views

Is k-means clustering guaranteed to converge if using Manhattan distance?

The k-means algorithm is an iterative clustering algorithm that partitions the data points into K clusters (with centroids {$\mu_1, ... , \mu_k$}, minimizing the Sum-of-Squared-Error: $$ SSE = ...
1
vote
1answer
39 views

Solving Linear Least Squares with Constraint

I would like to use linear least squares to solve for $x \in \mathbb{R}^5$ where $$ Ax = b \rightarrow x = (A^TA)^{-1}A^Tb $$ but would like to include the constraint $$ x_1^2 + x_2^2 = 1 $$ I ...
0
votes
1answer
16 views

How does minimum squared error relate to a linear system?

Given some system $U*x = b$, I've solved for $x^*$, the least squares solution. I then compute the minimum squared error by $||U*x^* - b||^2$. I know that the least squares solution minimizes the ...
0
votes
0answers
30 views

Different SVD results in Matlab

my question relates to calculating SVD in Matlab. I have been reading a lot and somehow I have jumbled up all the facts. It would be great if you experts could get me to the right track. My task is ...
3
votes
1answer
44 views

When is Block-Partitioned Matrix Invertible?

Suppose I have a block partitioned matrix \begin{equation} \begin{bmatrix} \mathbf{X}_1^{\top}\mathbf{X}_1 & \mathbf{X}_1^{\top}\mathbf{X}_2 \\ \mathbf{X}_2^{\top}\mathbf{X}_1 & ...
0
votes
1answer
31 views

Bivariate rational (quadratic over linear) model fitting by least squares

I am trying to fit a simple model over 2D data points, in the frame of an image formation model with perspective and optical distortion. My model is the ratio of a second degree polynomial over a ...
1
vote
2answers
45 views

Show $E\left(\mathbf{X}_i \otimes \mathbf{u}_i\right)=\mathbf{0}$ implies $E\left(\mathbf{X}_i^{\top}\mathbf{G}\mathbf{u}_i\right)=\mathbf{0}$

Let $\mathbf{X}_i$ be a $G \times K$ random matrix, and let $\mathbf{u}_i$ be a $G \times 1$ random vector, and suppose we have a sample of $i=1,\ldots,N$ of each. Suppose the following condition ...
1
vote
1answer
34 views

How to Change Summation Expression $\sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$ into Matrix Expression

Let $\mathbf{X}_i$ be a $G \times K$ matrix, and suppose are $i=1,...,N$ of these matrices. Note that \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{X}_i &= \begin{bmatrix} ...
1
vote
0answers
40 views

Which projection, in $L_\infty$ norm or $L_2$ norm, is non-expansion?

I am just wondering which projection is non-expansion? Basically, I am wondering if $F$ is a projection operator then which norm would satisfy the following non-expansion property, where for a given ...
1
vote
0answers
64 views

Matrix Decompositions: Difference between Cholesky Decomposition, Eigendecomposition and Jordan Normal Form Decomposition

I recently created a related topic about the square root matrix, in case you'd like to refer to that one. Here's what we want: Consider the matrix $\Omega=E(\mathbf{u}^{\top}\mathbf{u})$, where ...
0
votes
1answer
14 views

Least Squares with equality Constraint

So I have a problem in the form of Y = Ax, where A is a matrix and x and Y are vectors. A is a skinny matrix and I would like to do a least squares solution to solve for x. Lets say though that x ...
1
vote
2answers
98 views

Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

I know in general that if a matrix $A$ is positive definite, then there exists a (unique?) square root matrix $B$, which is also positive definite, such that $BB=A$. Therefore, suppose $A$ is ...
0
votes
0answers
39 views

Least squares with three quadratic constraints (Ellipse fitting based on algebraic distance)

I would like to fit an ellipse to a given set of scattered data in $\mathcal{R}^2$. The fitting problem is in form least squares, minimizing the sum of squared algebraic distances \begin{equation} ...
0
votes
1answer
32 views

Well-defined $\xi$-weighted (Euclidean) norm

Suppose $\xi$ is a vector, that is used for $\parallel z\parallel_\xi$ calculation. Should every element of $\xi$ be positive, $\xi(i)>0$?
1
vote
1answer
82 views

Step by step LMS for learning a linear function

Disclaimer Since this is an exercise assignment I'm not looking for a complete solution but for help that enables me to solve it on my own The task Given the error function ...