Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [weighted-least-squares]

The tag has no usage guidance.

0
votes
0answers
12 views

Baseball: How are data sets with differing number of occurrences compared?

I have two data sets, each showing batting statistics for batters hitting against a specific pitcher. These tables show lifetime batting statistics of batters hitting against pitchers, therefore the ...
1
vote
0answers
23 views

From non-linear least squares to weighted linear least squares

Given an overdetermined linear system $A \in \mathbb{R}^{m \times n}$, $b \in \mathbb{R}^{m \times 1}$. And a non-linear function $f(x)$. Given a non-linear least squares: $$ e^* = \min_g \left\lVert ...
1
vote
0answers
24 views

Iterative least squares for TV-like regularizer

I am trying to implement a TV-like surface regularization in a least squares solver. The formulation is as follows: $E_{surface} = \sum |dA_\theta| $ Background $dA_{\theta} = \frac{\theta_1}{f_x ...
0
votes
0answers
27 views

Inverse of matrix sum, one symmetric PSD and one near-constant diagonal

Question How can split the calculation of a real matrix inverse $(S + D)^{-1}$ when I know that $S$ is symmetric and PSD and $D$ diagonal with only a handful of unique values (=diag$(a,a...a,b...b,c.....
0
votes
0answers
12 views

Iteratively Weighted Least Squares and Hessian

The question is about stationary point of a least squares being identical to the stationary point of an error function as stated in: Computer Vision: Algorithms and Applications by Szelinski et al. p....
0
votes
0answers
14 views

Weighted rank 1 approximation to matrix

I want to solve the following problem: $$\arg\min_{u,v} \|W\odot(u v^\mathsf{T}-M)\|_\mathrm{F}$$ where $u$ and $v$ are $N\times 1$ vectors, $W$ and $M$ are $N\times N$ matrices, $\odot$ represents ...
0
votes
1answer
47 views

fractional curve fitting of the function $y=a+bx^{\alpha}$

Assume I have a set of data $(x_{i},y_{i})$, $i=1,...,m$. How can we find the best values of the parameters $a$ and $b$ and $\alpha$ such that the curve $y=a+bx^{\alpha}$ best fits the data. This ...
0
votes
0answers
12 views

Statistics: Higher confidence, higher weight

So we have an experiment going on where a group of colleagues evaluate each other's skill levels, ranging from 1 to 5. We have $n$ colleagues, so in the end we should get $n^2$ evaluations. However, ...
0
votes
0answers
38 views

When can Levenberg-Marquardt fitting algorithm be used with least absolute residuals (LAR) method and not Bisquare method for residual minimization?

I am sorry for asking a trivial question but though I have found an answer in the following link, I would like to know some more insight on the situations when one residual minimizing method is used ...
1
vote
0answers
31 views

Relationship between Fundamental Lemma of Calculus of Variations and the Weighted-Residual Statement

The Weighted-Residual Method states that the integral of the Residual R(x) times the weighting function w(x) is equal to zero which means that R(x) = 0 On the other hand, Fundamental Lemma of ...
0
votes
0answers
55 views

Alternatives to least square, overestimate?

Ok so I have a very simple linear system of equations with three unknowns and eight equations. Is there anyone that can tell me if there is a method or some way to find a solution which overestimates ...
0
votes
1answer
28 views

Understanding an identity for least squares regression line gradient

In section 2.2 of this paper, Gelman and Park present the following identity for the gradient of the least squares line through a set of 2D points: ...we recall a simple algebraic identity that ...
1
vote
0answers
76 views

Nearest (with respect to weights) symmetric positive semidefinite matrix

I want to compute the nearest symmetric positive semidefinite matrix, similar as Higham did. But here also weights (given by an inverse co-variance matrix) should be taken into account. So the ...
0
votes
0answers
45 views

Weighted least squares estimator for a non-zero intercept regression

If I have the following regression model with intercept $\alpha$ $$y=X\beta + \alpha + \epsilon$$ Is the Weighted Least Squares (WLS) estimator for $\beta$ the same as in the zero-intercept ...
0
votes
0answers
45 views

How to solve conditional least square?

I'm studying least square, so I can calculate optimal solution at overdetermined system. weighted least square But when some conditions are given, I can't calculate optimal solution(sub?) For ...
0
votes
1answer
182 views

Matrix notation for weighted sum of squares

While going through page 1 of Lecture 24: Weighted and Generalized Least Squares [PDF], I got the following questions. Weighted sum of squares is defined as below: $$ \sum_{i = 0}^{n}{w_i(Y_i - X_ib)...
0
votes
0answers
51 views

Weighted Least Squares - Categorical Data vs. Numerical Data

Over on Stackoverflow, I am trying calculate the Weighted Least Squares (WLS) of a data set in a python library called Numpy as compared to using a library called Statsmodels. However, I noticed ...
0
votes
0answers
34 views

Minimizing a multivariable function with different vector length?

Assume that I want to minimize a function to a measured vector. $$V_{min} = \sum_{n = 0, k = 0}^{N, K}{|T_n - f(t_n, p_n, r_k)|}$$ Where $T_n, t_n, p_n$ has the same vector length $N$, but not $r_k$ ...
1
vote
2answers
167 views

How can I get the gradient of the normal equation for weighted linear regression?

The normal equation for weighted linear regression looks like this: $$J(\theta) = (X\theta - y)^TW(X\theta - y),$$ where $X\in\Re^{m\times n}$, $\theta\in\Re^{n\times n}$, $y\in\Re^{m\times 1}$, $W\...
0
votes
2answers
43 views

Is there a least squares estimator for correlated, non-constant variance errors?

For OLS we have $\hat{\beta} = (X^TX)^{-1}X^Ty$, For non-constant variance we have $\hat{\beta} = (X^TWX)^{-1}X^TWy$, but what if we have, for example $Y = X\beta + \epsilon $ where $\epsilon \sim N(...
1
vote
1answer
228 views

Understanding Galerkin method of weighted residuals

I have a puzzlement regarding the Galerkin method of weighted residuals. The following is taken from the book A Finite Element Primer for Beginners, from chapter 1.1. If I have a one dimensional ...
0
votes
1answer
261 views

Weighted Least Squares Without Intercept

I am studying the WLS model for $y=\beta x+\epsilon$, where $\beta$ and $x$ are vectors and $\epsilon$ is the error term. This is a multiple regression model without an intercept. How would I find the ...
1
vote
1answer
46 views

polynomial least squares derivation: normal equations

Suppose we have the problem $$ \min_{q \in \mathcal{P}_n} \|f - q\|_{L^2(w)}^2 $$ where $\mathcal{P}_n$ is the space of polynomials of degree $n$, $w$ is some weight function (measure with continuous ...
1
vote
1answer
115 views

Weighted Least Squares for Parabola Coefficients Estimation

I am in trouble to find where I am making a mistake... I have to estimate the parameters a and b of the curve modeled by: $y = a x^2 + bx$ I have to do that from K measures of the curve, each ...
2
votes
2answers
484 views

Linear Fit when Data has Uncertainty

I am attempting to find the slope and y-intercept (along with their uncertainty) from a set of data. In this case, I am graphing Gamma Energy (MeV) vs. Peak Centroid (Channel). Here is my data: Gamma ...
1
vote
1answer
26 views

what is the meaning of weighting in mathematics?

What is the mathematical meaning of weighted by a Gaussian for numbers or vectors or Weighting by bilinear and weighted vectors? Regards and thanks in advance!
0
votes
0answers
27 views

Assign automatic weightage for set of negative numbers

I have set of negative numbers e.g. [-0.189, -3.55, -19.90, -0.0001] now I have to convert this set to percentage such that largest number will have highest ...
1
vote
1answer
216 views

Stuck in concavity proof of least squares cost as a function of weights

This problem comes from Boyd & Vandenberghe Convex Optimization, example 3.9 in page 81. All derivations make sense for me except the last step which says: $$g(w)=b^TWb-b^TWA(A^TWA)^{-1}A^TWb ...
0
votes
2answers
47 views

How to apply weightings to least squares slope formula

Im using this formula to find the slope of the regression line of given $x,y$ samples using LS calculation : $a=\frac{(n\sum xy-\sum x\sum y)}{(n\sum x^2-(\sum x)^2)}$ How do i change it to apply ...
2
votes
2answers
67 views

The derivative and extremum of a matrix function

$$f(W)=(Ax-b)^TW(Ax-b)=x^TA^TWAx-2b^TWAx+b^TWb$$ where $f(W)$ is a function of $W$, $A$ is a known matrix, $x$ and $b$ are vectors ($b$ is known). How to get $\frac{\partial f}{\partial W}$?
0
votes
0answers
81 views

Solving the two-parameter generalized Eigenvalue problem

Is there a solution for the problem: $$ (A_0 + \alpha \ A_1 + \beta \ A_2 ) x = 0 $$ where: $ A_0, A_1,$ and $A_2 \in \mathbb{R}^{n\times n}$, $\alpha$ and $\beta \in \mathbb{C}$, and $x \in \mathbb{...
2
votes
0answers
77 views

Sensitivity of weighted least squares estimation method

I am trying to understand the weighted least squares estimation method, and I'd really appreciate it if you could shed some light on me. Let me explain my problem briefly: Consider a linear model in ...
2
votes
1answer
244 views

minimize a sum of weighted square distances

Let $f_1, f_2$ be given polynomials of degree $k$ and we want to find a polynomial $f$ of degree $k+1$ that solves the following minimization problem on $[0,1]$: $$f=\operatorname*{argmin}_{\hat{f}\in ...
1
vote
2answers
258 views

Weighted least squares with nuclear norm minimization

Nuclear norm minimization is very popularization and formulation is least squares term with nuclear norm term as following, $$\min\limits_{X} \frac{1}{2}\|X_{3\times3}-Y_{3\times3}\|_F^2+\lambda\|X_{...