Questions tagged [weighted-least-squares]

This tag is for questions relating to weighted least squares, a generalization of ordinary least squares and linear regression in which the errors covariance matrix is allowed to be different from an identity matrix.

Filter by
Sorted by
Tagged with
0
votes
1answer
13 views

How is iteratively reweighted least squares used for $L^p$ norm linear regression?

The iterative scheme that I see everywhere in this context is $$\theta _{k+1}=\left(X^{\:t}W_k\:X\right)^{-1}\left(X^{\:t}W_k\:Y\right)$$ With the weight $W_k$ being a diagonal matrix of $$w_i=\left(...
0
votes
0answers
7 views

weighted nonlinear least regression fitting

This question relates to fit real life data by using exponential. I understand that if the variance are not constant at each sampling point I can use weight to correct that. But what if the sampling ...
0
votes
0answers
22 views

weighted least squares with negative weights

so, if I have an overconstrained set of equations yi = ai * x0 + bi * x1 + ci * x3... * xn with i>n then I know I can solve it by writing it as y = A * x and then solving x = (A^T * A)^-1 * A^T * y ...
0
votes
0answers
22 views

Preconditioned least-squares problem

Given the least squares problem $\min_x \|Ax-b\|$, show that the preconditioned version $\min_x \|MAx-Mb\|$ has the same unique solution as the original version given that $A$ is full-rank and $M$ is ...
0
votes
0answers
11 views

Can we find sequences of functions generated by iterated linear least-squares fitting?

Let us consider the problem $${\bf c_o} = \min_{\bf c}\{\|{\bf W}({\bf \Phi c - d)}\|_2^2\}$$ Where $\Phi = \begin{bmatrix}\Phi_1(x_1)& \Phi_2(x_1)&\cdots&\Phi_n(x_1)\\\Phi_1(x_2)& \...
0
votes
0answers
64 views

Solving a coupled system of linear ODEs one second order, the other first order

i want to find Aerodynamic center (y,z) and its moment(M ac) for vehicle , so i searched a lot , and i found these information . Aerodynamic center position doesn't change wrt angle , drag ,lift ...
0
votes
0answers
16 views

What are diagonal weighted matrices in Total Least-Square Approach

I am going through Total least square and came across two diagonal matrices $D$ and $T$ $$ D= \operatorname{diag}(d_1, ...,d_m)$$ $$ T = \operatorname{diag}(t_1,...,t_{n+k})$$ Total Least Square $$(A+ ...
0
votes
0answers
12 views

Weighted Hotelling T2

Suppose I have a covariance matrix defined by: $\hat{\mathbf{\Sigma}}=\frac{1}{n-1} \sum_{i=1}^{n}\left(\mathbf{x}_{i}-\overline{\mathbf{x}}\right)^{T}\left(\mathbf{x}_{i}-\overline{\mathbf{x}}\right)$...
0
votes
0answers
20 views

Closed form solution for Restricted Weighted Least Squares

From Greene, we know that the closed-form solution of a restricted least squares is: $\beta_{Constrained} = \beta_{Uncon} - (X'X)^{-1}R'[R(X'X)^{-1}R']^{-1}(R\beta_{Uncon}-r)$. Is there any similar ...
0
votes
1answer
21 views

Convexity conditions for an Infimum operation

I came across these 2 seemingly contradictory statements in the book 'Convex Optimization' by Boyd. and So the top image is said to be concave. For the simplest case where $n=1$ the function is the ...
0
votes
0answers
9 views

Typo in Wasserman explanation of Reweighted Least Squares?

My test (All of Statistics, second edition, Wasserman) contains the following: Reweighted Least Squares Algorithm (for Logistic Regression) Choose starting values $\hat{\beta}^0 = (\hat{\...
0
votes
0answers
7 views

How to do a weighted extrapolation using a Burg (or similar)?

I had extrapolated data using the Burg Method (octave arburg) but in the period with no existing data the extrapolated value kept on asymptotically tending to an arbitrary value even when the data ...
0
votes
1answer
73 views

SVD decomposition in Weighted Least Square

Consider the Linear Regression below: c_hat=argmin ‖b-Xc‖ (1) Where The Least Square solution is as follows: c_hat=inv(X'X)X'b (2) It is possible to decompose data matrix as ...
0
votes
1answer
29 views

Estimate weight/impact of different variables on time

I am trying to estimate digestion time for food given their nutritional content. So the idea is a formula that takes nutritional content of a food as input and outputs time in hours: ...
0
votes
0answers
7 views

How to calculate weighted average cost for a business units when there is more than one weight parameters to account?

Trying to solve how to allocate a cost (based on weight) to different units to justify cost distribution: Here is the problem. There are certain cost known to a live stock firm and only know at grand ...
1
vote
0answers
60 views

Weighted Average Loss, does my approach make sense?

nevermind, no one replies anyway.
1
vote
0answers
35 views

Please verify whether the calculating process of the WLS estimator and the variance is correct or not.

There are two Heteroscedasticity regression models 1. $$ y_i = \beta x_i + \epsilon_i, \quad i=1, \ldots, n $$ where $\epsilon_i$'s are independent and distributed as $\epsilon_i \sim N(0, \sigma^2 ...
0
votes
0answers
16 views

Please help me compare the variances of linear regression parameter estimates obtained by OLS and WLS

If the errors from the error vector $\varepsilon$ are independent, but have distinct variances, so that $Var(\varepsilon|X)=\Sigma=diag(\sigma _1^2,...,\sigma _N^2)$. Variance-covariance matrix of ...
1
vote
0answers
14 views

How to calculate weight for least squares if both values error is known?

Usually minimizing $$\chi^2 = \sum_i w_i (y_i - f(a_0, a_1, \dots, a_n, x_i))^2$$ where $a_k$ are parameters is done by taking $w_i = 1/\sigma_i^2$ where $\sigma_i$ is a observation error of $y_i$ ...
0
votes
0answers
13 views

Deriving the weighting matrix for the efficient GLS estimator

$u_{it} = \nu _{it} - \theta \nu _{i\left ( t-1 \right )}$ for $t>1$ $u_{i1} = \nu _{i1}$ and the $\nu _{it}$ are white noise with variance equal to $\sigma^{2}$. This is for a system of $T$ ...
0
votes
1answer
22 views

Restricted Weighted Linear Regression in R

I have the following issue. I would like to run a linear regression imposing a constraint on the weighted coefficients. Let me construct an example: Consider the following cross-sectional regression ...
0
votes
0answers
23 views

Restricted weighted OLS

I would like to run a restricted weighted OLS regression in R. Let me first state the problem mathematically: $\arg\min_{\beta} \sum_{i=1}^{n} w_{i} \lvert y_{i} - \sum_{k=1}^{m} x_{i,k} \beta_{k}\...
1
vote
0answers
39 views

From non-linear least squares to weighted linear least squares

Given an overdetermined linear system $A \in \mathbb{R}^{m \times n}$, $b \in \mathbb{R}^{m \times 1}$. And a non-linear function $f(x)$. Given a non-linear least squares: $$ e^* = \min_g \left\lVert ...
1
vote
0answers
28 views

Iterative least squares for TV-like regularizer

I am trying to implement a TV-like surface regularization in a least squares solver. The formulation is as follows: $E_{surface} = \sum |dA_\theta| $ Background $dA_{\theta} = \frac{\theta_1}{f_x ...
0
votes
1answer
74 views

fractional curve fitting of the function $y=a+bx^{\alpha}$

Assume I have a set of data $(x_{i},y_{i})$, $i=1,...,m$. How can we find the best values of the parameters $a$ and $b$ and $\alpha$ such that the curve $y=a+bx^{\alpha}$ best fits the data. This ...
0
votes
0answers
131 views

Alternatives to least square, overestimate?

Ok so I have a very simple linear system of equations with three unknowns and eight equations. Is there anyone that can tell me if there is a method or some way to find a solution which overestimates ...
0
votes
1answer
57 views

Understanding an identity for least squares regression line gradient

In section 2.2 of this paper, Gelman and Park present the following identity for the gradient of the least squares line through a set of 2D points: ...we recall a simple algebraic identity that ...
1
vote
0answers
133 views

Nearest (with respect to weights) symmetric positive semidefinite matrix

I want to compute the nearest symmetric positive semidefinite matrix, similar as Higham did. But here also weights (given by an inverse co-variance matrix) should be taken into account. So the ...
0
votes
0answers
56 views

How to solve conditional least square?

I'm studying least square, so I can calculate optimal solution at overdetermined system. weighted least square But when some conditions are given, I can't calculate optimal solution(sub?) For ...
1
vote
1answer
524 views

Matrix notation for weighted sum of squares

While going through page 1 of Lecture 24: Weighted and Generalized Least Squares [PDF], I got the following questions. Weighted sum of squares is defined as below: $$ \sum_{i = 0}^{n}{w_i(Y_i - X_ib)...
0
votes
0answers
134 views

Weighted Least Squares - Categorical Data vs. Numerical Data

Over on Stackoverflow, I am trying calculate the Weighted Least Squares (WLS) of a data set in a python library called Numpy as compared to using a library called Statsmodels. However, I noticed ...
0
votes
0answers
63 views

Minimizing a multivariable function with different vector length?

Assume that I want to minimize a function to a measured vector. $$V_{min} = \sum_{n = 0, k = 0}^{N, K}{|T_n - f(t_n, p_n, r_k)|}$$ Where $T_n, t_n, p_n$ has the same vector length $N$, but not $r_k$ ...
3
votes
2answers
387 views

How can I get the gradient of the normal equation for weighted linear regression?

The normal equation for weighted linear regression looks like this: $$J(\theta) = (X\theta - y)^TW(X\theta - y),$$ where $X\in\Re^{m\times n}$, $\theta\in\Re^{n\times n}$, $y\in\Re^{m\times 1}$, $W\...
0
votes
2answers
49 views

Is there a least squares estimator for correlated, non-constant variance errors?

For OLS we have $\hat{\beta} = (X^TX)^{-1}X^Ty$, For non-constant variance we have $\hat{\beta} = (X^TWX)^{-1}X^TWy$, but what if we have, for example $Y = X\beta + \epsilon $ where $\epsilon \sim N(...
1
vote
1answer
666 views

Understanding Galerkin method of weighted residuals

I have a puzzlement regarding the Galerkin method of weighted residuals. The following is taken from the book A Finite Element Primer for Beginners, from chapter 1.1. If I have a one dimensional ...
0
votes
1answer
474 views

Weighted Least Squares Without Intercept

I am studying the WLS model for $y=\beta x+\epsilon$, where $\beta$ and $x$ are vectors and $\epsilon$ is the error term. This is a multiple regression model without an intercept. How would I find the ...
1
vote
1answer
81 views

polynomial least squares derivation: normal equations

Suppose we have the problem $$ \min_{q \in \mathcal{P}_n} \|f - q\|_{L^2(w)}^2 $$ where $\mathcal{P}_n$ is the space of polynomials of degree $n$, $w$ is some weight function (measure with continuous ...
1
vote
1answer
167 views

Weighted Least Squares for Parabola Coefficients Estimation

I am in trouble to find where I am making a mistake... I have to estimate the parameters a and b of the curve modeled by: $y = a x^2 + bx$ I have to do that from K measures of the curve, each ...
2
votes
2answers
2k views

Linear Fit when Data has Uncertainty

I am attempting to find the slope and y-intercept (along with their uncertainty) from a set of data. In this case, I am graphing Gamma Energy (MeV) vs. Peak Centroid (Channel). Here is my data: Gamma ...
1
vote
1answer
29 views

what is the meaning of weighting in mathematics?

What is the mathematical meaning of weighted by a Gaussian for numbers or vectors or Weighting by bilinear and weighted vectors? Regards and thanks in advance!
0
votes
0answers
58 views

Assign automatic weightage for set of negative numbers

I have set of negative numbers e.g. [-0.189, -3.55, -19.90, -0.0001] now I have to convert this set to percentage such that largest number will have highest ...
1
vote
2answers
336 views

Stuck in concavity proof of least squares cost as a function of weights

This problem comes from Boyd & Vandenberghe Convex Optimization, example 3.9 in page 81. All derivations make sense for me except the last step which says: $$g(w)=b^TWb-b^TWA(A^TWA)^{-1}A^TWb ...
0
votes
2answers
101 views

How to apply weightings to least squares slope formula

Im using this formula to find the slope of the regression line of given $x,y$ samples using LS calculation : $a=\frac{(n\sum xy-\sum x\sum y)}{(n\sum x^2-(\sum x)^2)}$ How do i change it to apply ...
2
votes
2answers
74 views

The derivative and extremum of a matrix function

$$f(W)=(Ax-b)^TW(Ax-b)=x^TA^TWAx-2b^TWAx+b^TWb$$ where $f(W)$ is a function of $W$, $A$ is a known matrix, $x$ and $b$ are vectors ($b$ is known). How to get $\frac{\partial f}{\partial W}$?
0
votes
0answers
88 views

Solving the two-parameter generalized Eigenvalue problem

Is there a solution for the problem: $$ (A_0 + \alpha \ A_1 + \beta \ A_2 ) x = 0 $$ where: $ A_0, A_1,$ and $A_2 \in \mathbb{R}^{n\times n}$, $\alpha$ and $\beta \in \mathbb{C}$, and $x \in \mathbb{...
1
vote
0answers
100 views

Sensitivity of weighted least squares estimation method

I am trying to understand the weighted least squares estimation method, and I'd really appreciate it if you could shed some light on me. Let me explain my problem briefly: Consider a linear model in ...
2
votes
1answer
293 views

minimize a sum of weighted square distances

Let $f_1, f_2$ be given polynomials of degree $k$ and we want to find a polynomial $f$ of degree $k+1$ that solves the following minimization problem on $[0,1]$: $$f=\operatorname*{argmin}_{\hat{f}\in ...
1
vote
2answers
312 views

Matrix Weighted Least Squares with Nuclear Norm Minimization

Nuclear norm minimization is very popularization and formulation is least squares term with nuclear norm term as following, $$\min\limits_{X} \frac{1}{2}\|X_{3\times3}-Y_{3\times3}\|_F^2+\lambda\|X_{...