Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [weighted-least-squares]

The tag has no usage guidance.

2
votes
2answers
67 views

The derivative and extremum of a matrix function

$$f(W)=(Ax-b)^TW(Ax-b)=x^TA^TWAx-2b^TWAx+b^TWb$$ where $f(W)$ is a function of $W$, $A$ is a known matrix, $x$ and $b$ are vectors ($b$ is known). How to get $\frac{\partial f}{\partial W}$?
2
votes
1answer
247 views

minimize a sum of weighted square distances

Let $f_1, f_2$ be given polynomials of degree $k$ and we want to find a polynomial $f$ of degree $k+1$ that solves the following minimization problem on $[0,1]$: $$f=\operatorname*{argmin}_{\hat{f}\in ...
2
votes
2answers
552 views

Linear Fit when Data has Uncertainty

I am attempting to find the slope and y-intercept (along with their uncertainty) from a set of data. In this case, I am graphing Gamma Energy (MeV) vs. Peak Centroid (Channel). Here is my data: Gamma ...
2
votes
0answers
79 views

Sensitivity of weighted least squares estimation method

I am trying to understand the weighted least squares estimation method, and I'd really appreciate it if you could shed some light on me. Let me explain my problem briefly: Consider a linear model in ...
1
vote
2answers
177 views

How can I get the gradient of the normal equation for weighted linear regression?

The normal equation for weighted linear regression looks like this: $$J(\theta) = (X\theta - y)^TW(X\theta - y),$$ where $X\in\Re^{m\times n}$, $\theta\in\Re^{n\times n}$, $y\in\Re^{m\times 1}$, $W\...
1
vote
1answer
244 views

Understanding Galerkin method of weighted residuals

I have a puzzlement regarding the Galerkin method of weighted residuals. The following is taken from the book A Finite Element Primer for Beginners, from chapter 1.1. If I have a one dimensional ...
1
vote
1answer
46 views

polynomial least squares derivation: normal equations

Suppose we have the problem $$ \min_{q \in \mathcal{P}_n} \|f - q\|_{L^2(w)}^2 $$ where $\mathcal{P}_n$ is the space of polynomials of degree $n$, $w$ is some weight function (measure with continuous ...
1
vote
1answer
27 views

what is the meaning of weighting in mathematics?

What is the mathematical meaning of weighted by a Gaussian for numbers or vectors or Weighting by bilinear and weighted vectors? Regards and thanks in advance!
1
vote
0answers
25 views

From non-linear least squares to weighted linear least squares

Given an overdetermined linear system $A \in \mathbb{R}^{m \times n}$, $b \in \mathbb{R}^{m \times 1}$. And a non-linear function $f(x)$. Given a non-linear least squares: $$ e^* = \min_g \left\lVert ...
1
vote
0answers
24 views

Iterative least squares for TV-like regularizer

I am trying to implement a TV-like surface regularization in a least squares solver. The formulation is as follows: $E_{surface} = \sum |dA_\theta| $ Background $dA_{\theta} = \frac{\theta_1}{f_x ...
1
vote
0answers
31 views

Relationship between Fundamental Lemma of Calculus of Variations and the Weighted-Residual Statement

The Weighted-Residual Method states that the integral of the Residual R(x) times the weighting function w(x) is equal to zero which means that R(x) = 0 On the other hand, Fundamental Lemma of ...
1
vote
0answers
77 views

Nearest (with respect to weights) symmetric positive semidefinite matrix

I want to compute the nearest symmetric positive semidefinite matrix, similar as Higham did. But here also weights (given by an inverse co-variance matrix) should be taken into account. So the ...
1
vote
1answer
118 views

Weighted Least Squares for Parabola Coefficients Estimation

I am in trouble to find where I am making a mistake... I have to estimate the parameters a and b of the curve modeled by: $y = a x^2 + bx$ I have to do that from K measures of the curve, each ...
1
vote
1answer
218 views

Stuck in concavity proof of least squares cost as a function of weights

This problem comes from Boyd & Vandenberghe Convex Optimization, example 3.9 in page 81. All derivations make sense for me except the last step which says: $$g(w)=b^TWb-b^TWA(A^TWA)^{-1}A^TWb ...
1
vote
2answers
264 views

Weighted least squares with nuclear norm minimization

Nuclear norm minimization is very popularization and formulation is least squares term with nuclear norm term as following, $$\min\limits_{X} \frac{1}{2}\|X_{3\times3}-Y_{3\times3}\|_F^2+\lambda\|X_{...
0
votes
1answer
48 views

fractional curve fitting of the function $y=a+bx^{\alpha}$

Assume I have a set of data $(x_{i},y_{i})$, $i=1,...,m$. How can we find the best values of the parameters $a$ and $b$ and $\alpha$ such that the curve $y=a+bx^{\alpha}$ best fits the data. This ...
0
votes
1answer
29 views

Understanding an identity for least squares regression line gradient

In section 2.2 of this paper, Gelman and Park present the following identity for the gradient of the least squares line through a set of 2D points: ...we recall a simple algebraic identity that ...
0
votes
1answer
271 views

Weighted Least Squares Without Intercept

I am studying the WLS model for $y=\beta x+\epsilon$, where $\beta$ and $x$ are vectors and $\epsilon$ is the error term. This is a multiple regression model without an intercept. How would I find the ...
0
votes
2answers
48 views

How to apply weightings to least squares slope formula

Im using this formula to find the slope of the regression line of given $x,y$ samples using LS calculation : $a=\frac{(n\sum xy-\sum x\sum y)}{(n\sum x^2-(\sum x)^2)}$ How do i change it to apply ...
0
votes
0answers
12 views

Weighted Least Squares and the unreal effect on the a posteriori MSE estimator

I expose the initial situacion and the questions are derived afterwards. Thanks a lot in advance, I hope the question is not too simple for this community. When solving normal or weighted least ...
0
votes
0answers
11 views

Restricted Weighted Linear Regression in R

I have to follwing issue. I would like to run a linear regression imposing a constraint on the weighted coefficients. Let me construct an example: Consider the following cross-sectional regression $...
0
votes
0answers
12 views

How to scale a weighted variable for linear regression?

I would like to scale a variable such that after weighting it is scaled to mean zero and standard deviation one: $\sum_{i} w_{i} x_{i} = 0$ and $\sum_{i} w_{i} x_{i}^2 = 1$ where $w_{i}$ are ...
0
votes
0answers
17 views

Restricted weighted OLS

I would like to run a restricted weighted OLS regression in R. Let me first state the problem mathematically: $\arg\min_{\beta} \sum_{i=1}^{n} w_{i} \lvert y_{i} - \sum_{k=1}^{m} x_{i,k} \beta_{k}\...
0
votes
0answers
12 views

Baseball: How are data sets with differing number of occurrences compared?

I have two data sets, each showing batting statistics for batters hitting against a specific pitcher. These tables show lifetime batting statistics of batters hitting against pitchers, therefore the ...
0
votes
0answers
28 views

Inverse of matrix sum, one symmetric PSD and one near-constant diagonal

Question How can split the calculation of a real matrix inverse $(S + D)^{-1}$ when I know that $S$ is symmetric and PSD and $D$ diagonal with only a handful of unique values (=diag$(a,a...a,b...b,c.....
0
votes
0answers
14 views

Iteratively Weighted Least Squares and Hessian

The question is about stationary point of a least squares being identical to the stationary point of an error function as stated in: Computer Vision: Algorithms and Applications by Szelinski et al. p....
0
votes
0answers
16 views

Weighted rank 1 approximation to matrix

I want to solve the following problem: $$\arg\min_{u,v} \|W\odot(u v^\mathsf{T}-M)\|_\mathrm{F}$$ where $u$ and $v$ are $N\times 1$ vectors, $W$ and $M$ are $N\times N$ matrices, $\odot$ represents ...
0
votes
0answers
12 views

Statistics: Higher confidence, higher weight

So we have an experiment going on where a group of colleagues evaluate each other's skill levels, ranging from 1 to 5. We have $n$ colleagues, so in the end we should get $n^2$ evaluations. However, ...
0
votes
0answers
42 views

When can Levenberg-Marquardt fitting algorithm be used with least absolute residuals (LAR) method and not Bisquare method for residual minimization?

I am sorry for asking a trivial question but though I have found an answer in the following link, I would like to know some more insight on the situations when one residual minimizing method is used ...
0
votes
0answers
56 views

Alternatives to least square, overestimate?

Ok so I have a very simple linear system of equations with three unknowns and eight equations. Is there anyone that can tell me if there is a method or some way to find a solution which overestimates ...
0
votes
0answers
45 views

Weighted least squares estimator for a non-zero intercept regression

If I have the following regression model with intercept $\alpha$ $$y=X\beta + \alpha + \epsilon$$ Is the Weighted Least Squares (WLS) estimator for $\beta$ the same as in the zero-intercept ...
0
votes
0answers
45 views

How to solve conditional least square?

I'm studying least square, so I can calculate optimal solution at overdetermined system. weighted least square But when some conditions are given, I can't calculate optimal solution(sub?) For ...
0
votes
1answer
199 views

Matrix notation for weighted sum of squares

While going through page 1 of Lecture 24: Weighted and Generalized Least Squares [PDF], I got the following questions. Weighted sum of squares is defined as below: $$ \sum_{i = 0}^{n}{w_i(Y_i - X_ib)...
0
votes
0answers
54 views

Weighted Least Squares - Categorical Data vs. Numerical Data

Over on Stackoverflow, I am trying calculate the Weighted Least Squares (WLS) of a data set in a python library called Numpy as compared to using a library called Statsmodels. However, I noticed ...
0
votes
0answers
36 views

Minimizing a multivariable function with different vector length?

Assume that I want to minimize a function to a measured vector. $$V_{min} = \sum_{n = 0, k = 0}^{N, K}{|T_n - f(t_n, p_n, r_k)|}$$ Where $T_n, t_n, p_n$ has the same vector length $N$, but not $r_k$ ...
0
votes
2answers
45 views

Is there a least squares estimator for correlated, non-constant variance errors?

For OLS we have $\hat{\beta} = (X^TX)^{-1}X^Ty$, For non-constant variance we have $\hat{\beta} = (X^TWX)^{-1}X^TWy$, but what if we have, for example $Y = X\beta + \epsilon $ where $\epsilon \sim N(...
0
votes
0answers
28 views

Assign automatic weightage for set of negative numbers

I have set of negative numbers e.g. [-0.189, -3.55, -19.90, -0.0001] now I have to convert this set to percentage such that largest number will have highest ...
0
votes
0answers
81 views

Solving the two-parameter generalized Eigenvalue problem

Is there a solution for the problem: $$ (A_0 + \alpha \ A_1 + \beta \ A_2 ) x = 0 $$ where: $ A_0, A_1,$ and $A_2 \in \mathbb{R}^{n\times n}$, $\alpha$ and $\beta \in \mathbb{C}$, and $x \in \mathbb{...