# Questions tagged [least-squares]

Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere.

1,357 questions
Filter by
Sorted by
Tagged with
9 views

### Confidence Intervals from Nonlinear Least Squares (Numerical): How do I ensure reasonable values?

Summary: In numerical (solver-based) non-linear least squares, the smaller the step size scale (for finite differences), the smaller the 95% confidence intervals are. Ergo, by using a smaller step ...
31 views

### How to solve for BLUE in linear regression?

I have been studying OLS in matrix form and I understand that when $y=X\beta +\varepsilon$, and when $E[\varepsilon|X]=0$ and $Cov[\varepsilon|X]=\sigma^{2}I$, $\hat{\beta}=(X^TX)^{-1}X^{T}y$ is the ...
17 views

### How extreme singular values affects least square for Gaussian covariates.

Consider the generative model for linear regression w.r.t. the true parameter $w^* \in S^{d-1}$ $$y=Xw^*+e$$ with i.i.d. Gaussian error $e \sim N(0, \sigma^2I_n)$. Let $X \in \mathbb{R}^{n\times d}$ ...
11 views

### How to Reconstruct a curves from coordinate points that I extracted from edge detection technique?

I have extracted the coordinate points of a diagram using edge detection technique in opencv. I want to remap only those coordinate points that contribute to the curves in the diagram and make it look ...
45 views

### Sensitivity of least squares solution to a data point change

Consider computing the least squares solution in $x$: $$\text{min}.~\|Ax - b\|^2,$$ when $A$ is an $m \times n$ matrix, and $b$ is a column vector. Let's suppose we now switch exactly one row of $A$...
35 views

### Fitting least squares to 3-D points to get an equation for a plane

Here is a thread that is inspiring this question. I don't agree with Ben's answer for the reason wcochran stated. I am trying to find the equation for a plane specifically using least squares. So say ...
42 views

### How to find plane equation from 8 3D points with least square method

I have been working on school projects to find a plane equation from 8 3D points. Normally from 3 points, we can create a plane equation but when we have a lot of points, we want to find a good ...
37 views

### Solve differential equation with a least square method

I'm trying to solve the following differential equation: $$\frac{d^2u}{dx^2}=\frac{du}{dx}*u+u^2+x$$ $$x \in \Omega=[0,1]$$ $$BCS:u|_{x=0}=1;\frac{du}{dx}|_{x=0}=1$$ Least square method is one of ...
18 views

### Invertible $X^TX$ - what happens when you clone rows of $X$?

My question is inspired by https://stats.stackexchange.com/questions/70899/what-correlation-makes-a-matrix-singular-and-what-are-implications-of-singularit, in particular ttnphns's answer where they ...
10 views

### Least Squares and producing chemistry substances - ask references

This question came to me after discuss if there are nice examples of applied linear algerba topics in chemistry. A chemical compound is often a bunch of less complex chemical compouds put together to ...
26 views

22 views

### Issue in calculating Cramer-Rao Lower Bound from Fisher information matrices

I am having trouble understanding an apparent paradox in calculating the CRLBs from a Fisher information matrix. Lets say that I have some data which is fitted to a model of $$S=Aexp(-bt^2)$$ where S ...
22 views

### Statistical inference-wise, should I just worry about the standard error of the random part of my model?

I am modelling my real world system as: $$Y = f(x) + E$$ where $f$ is my deterministic model with some parameters $\theta_{1}$, $\theta_{2}$, ... and $E$ is the random part (stuff I can't explain, ...
21 views

### Use of least-squares minimization to solve noisy systems of linear equations

This question is somewhat connected to a previous one I posted two months ago concerning solving linear, over-determined systems of equations of the form $Ax = b$, where $A$ is a matrix of ...
17k views

### Derivative of squared Frobenius norm of a matrix

In linear regression, the loss function is expressed as $$\frac1N \left\|XW-Y\right\|_{\text{F}}^2$$ where $X, W, Y$ are matrices. Taking derivative w.r.t $W$ yields $$\frac 2N \, X^T(XW-Y)$$ Why ...
58 views

### Find two numbers to minimize the sum of squares

Suppose we are given a sequence of positive numbers $0<a_1<a_2< \cdots < a_n$. Step 1. Choose an integer $m$ where $m \in \{1,2,\cdots,n\}$. After choosing $m$, we divide our numbers into ...
Suppose I have a linear system $y=Ax+\varepsilon$ where $y\in\mathbb R^n$, $x\in\mathbb R^m$ and $A\in\mathbb R^{n\times m}$. I have known that the least squares solution for $x$ is $\hat{x}=A^+y$ ...