Questions tagged [weighted-least-squares]
The weighted-least-squares tag has no usage guidance.
0
votes
0answers
13 views
Weighted Least Squares and the unreal effect on the a posteriori MSE estimator
I expose the initial situacion and the questions are derived afterwards. Thanks a lot in advance, I hope the question is not too simple for this community.
When solving normal or weighted least ...
0
votes
0answers
11 views
Restricted Weighted Linear Regression in R
I have to follwing issue. I would like to run a linear regression imposing a constraint on the weighted coefficients. Let me construct an example:
Consider the following cross-sectional regression
$...
0
votes
0answers
12 views
How to scale a weighted variable for linear regression?
I would like to scale a variable such that after weighting it is scaled to mean zero and standard deviation one:
$\sum_{i} w_{i} x_{i} = 0$ and
$\sum_{i} w_{i} x_{i}^2 = 1$
where $w_{i}$ are ...
0
votes
0answers
17 views
Restricted weighted OLS
I would like to run a restricted weighted OLS regression in R. Let me first state the problem mathematically:
$\arg\min_{\beta} \sum_{i=1}^{n} w_{i} \lvert y_{i} - \sum_{k=1}^{m} x_{i,k} \beta_{k}\...
0
votes
0answers
12 views
Baseball: How are data sets with differing number of occurrences compared?
I have two data sets, each showing batting statistics for batters hitting against a specific pitcher. These tables show lifetime batting statistics of batters hitting against pitchers, therefore the ...
1
vote
0answers
25 views
From non-linear least squares to weighted linear least squares
Given an overdetermined linear system $A \in \mathbb{R}^{m \times n}$, $b \in \mathbb{R}^{m \times 1}$. And a non-linear function $f(x)$. Given a non-linear least squares:
$$ e^* = \min_g \left\lVert ...
1
vote
0answers
24 views
Iterative least squares for TV-like regularizer
I am trying to implement a TV-like surface regularization in a least squares solver. The formulation is as follows:
$E_{surface} = \sum |dA_\theta| $
Background
$dA_{\theta} = \frac{\theta_1}{f_x ...
0
votes
0answers
29 views
Inverse of matrix sum, one symmetric PSD and one near-constant diagonal
Question
How can split the calculation of a real matrix inverse $(S + D)^{-1}$ when I know that $S$ is symmetric and PSD and $D$ diagonal with only a handful of unique values (=diag$(a,a...a,b...b,c.....
0
votes
0answers
16 views
Iteratively Weighted Least Squares and Hessian
The question is about stationary point of a least squares being identical to the stationary point of an error function as stated in: Computer Vision: Algorithms and Applications by Szelinski et al. p....
0
votes
0answers
16 views
Weighted rank 1 approximation to matrix
I want to solve the following problem:
$$\arg\min_{u,v} \|W\odot(u v^\mathsf{T}-M)\|_\mathrm{F}$$
where $u$ and $v$ are $N\times 1$ vectors, $W$ and $M$ are $N\times N$ matrices, $\odot$ represents ...
0
votes
1answer
48 views
fractional curve fitting of the function $y=a+bx^{\alpha}$
Assume I have a set of data $(x_{i},y_{i})$, $i=1,...,m$. How can we find the best values of the parameters $a$ and $b$ and $\alpha$ such that the curve $y=a+bx^{\alpha}$ best fits the data.
This ...
0
votes
0answers
12 views
Statistics: Higher confidence, higher weight
So we have an experiment going on where a group of colleagues evaluate each other's skill levels, ranging from 1 to 5. We have $n$ colleagues, so in the end we should get $n^2$ evaluations. However, ...
0
votes
0answers
42 views
When can Levenberg-Marquardt fitting algorithm be used with least absolute residuals (LAR) method and not Bisquare method for residual minimization?
I am sorry for asking a trivial question but though I have found an answer in the following link, I would like to know some more insight on the situations when one residual minimizing method is used ...
1
vote
0answers
31 views
Relationship between Fundamental Lemma of Calculus of Variations and the Weighted-Residual Statement
The Weighted-Residual Method states that the integral of the Residual R(x) times the weighting function w(x) is equal to zero which means that R(x) = 0
On the other hand, Fundamental Lemma of ...
0
votes
0answers
56 views
Alternatives to least square, overestimate?
Ok so I have a very simple linear system of equations with three unknowns and eight equations. Is there anyone that can tell me if there is a method or some way to find a solution which overestimates ...
0
votes
1answer
29 views
Understanding an identity for least squares regression line gradient
In section 2.2 of this paper, Gelman and Park present the following identity for the gradient of the least squares line through a set of 2D points:
...we recall a simple algebraic identity that ...
1
vote
0answers
77 views
Nearest (with respect to weights) symmetric positive semidefinite matrix
I want to compute the nearest symmetric positive semidefinite matrix, similar as Higham did. But here also weights (given by an inverse co-variance matrix) should be taken into account.
So the ...
0
votes
0answers
45 views
Weighted least squares estimator for a non-zero intercept regression
If I have the following regression model with intercept $\alpha$
$$y=X\beta + \alpha + \epsilon$$
Is the Weighted Least Squares (WLS) estimator for $\beta$ the same as in the zero-intercept ...
0
votes
0answers
45 views
How to solve conditional least square?
I'm studying least square, so I can calculate optimal solution at overdetermined system. weighted least square
But when some conditions are given, I can't calculate optimal solution(sub?)
For ...
0
votes
1answer
200 views
Matrix notation for weighted sum of squares
While going through page 1 of Lecture 24: Weighted and Generalized Least Squares [PDF], I got the following questions.
Weighted sum of squares is defined as below:
$$ \sum_{i = 0}^{n}{w_i(Y_i - X_ib)...
0
votes
0answers
54 views
Weighted Least Squares - Categorical Data vs. Numerical Data
Over on Stackoverflow, I am trying calculate the Weighted Least Squares (WLS) of a data set in a python library called Numpy as compared to using a library called Statsmodels. However, I noticed ...
0
votes
0answers
36 views
Minimizing a multivariable function with different vector length?
Assume that I want to minimize a function to a measured vector.
$$V_{min} = \sum_{n = 0, k = 0}^{N, K}{|T_n - f(t_n, p_n, r_k)|}$$
Where $T_n, t_n, p_n$ has the same vector length $N$, but not $r_k$ ...
1
vote
2answers
177 views
How can I get the gradient of the normal equation for weighted linear regression?
The normal equation for weighted linear regression looks like this:
$$J(\theta) = (X\theta - y)^TW(X\theta - y),$$ where $X\in\Re^{m\times n}$, $\theta\in\Re^{n\times n}$, $y\in\Re^{m\times 1}$, $W\...
0
votes
2answers
45 views
Is there a least squares estimator for correlated, non-constant variance errors?
For OLS we have $\hat{\beta} = (X^TX)^{-1}X^Ty$,
For non-constant variance we have $\hat{\beta} = (X^TWX)^{-1}X^TWy$,
but what if we have, for example $Y = X\beta + \epsilon $ where $\epsilon \sim N(...
1
vote
1answer
244 views
Understanding Galerkin method of weighted residuals
I have a puzzlement regarding the Galerkin method of weighted residuals. The following is taken from the book A Finite Element Primer for Beginners, from chapter 1.1.
If I have a one dimensional ...
0
votes
1answer
272 views
Weighted Least Squares Without Intercept
I am studying the WLS model for $y=\beta x+\epsilon$, where $\beta$ and $x$ are vectors and $\epsilon$ is the error term. This is a multiple regression model without an intercept. How would I find the ...
1
vote
1answer
46 views
polynomial least squares derivation: normal equations
Suppose we have the problem
$$
\min_{q \in \mathcal{P}_n} \|f - q\|_{L^2(w)}^2
$$
where $\mathcal{P}_n$ is the space of polynomials of degree $n$, $w$ is some weight function (measure with continuous ...
1
vote
1answer
118 views
Weighted Least Squares for Parabola Coefficients Estimation
I am in trouble to find where I am making a mistake...
I have to estimate the parameters a and b of the curve modeled by:
$y = a x^2 + bx$
I have to do that from K measures of the curve, each ...
2
votes
2answers
559 views
Linear Fit when Data has Uncertainty
I am attempting to find the slope and y-intercept (along with their uncertainty) from a set of data. In this case, I am graphing Gamma Energy (MeV) vs. Peak Centroid (Channel). Here is my data:
Gamma ...
1
vote
1answer
27 views
what is the meaning of weighting in mathematics?
What is the mathematical meaning of weighted by a Gaussian for numbers or vectors or Weighting by bilinear and weighted vectors?
Regards and thanks in advance!
0
votes
0answers
28 views
Assign automatic weightage for set of negative numbers
I have set of negative numbers
e.g. [-0.189, -3.55, -19.90, -0.0001]
now I have to convert this set to percentage such that largest number will have highest ...
1
vote
1answer
218 views
Stuck in concavity proof of least squares cost as a function of weights
This problem comes from Boyd & Vandenberghe Convex Optimization, example 3.9 in page 81.
All derivations make sense for me except the last step which says:
$$g(w)=b^TWb-b^TWA(A^TWA)^{-1}A^TWb
...
0
votes
2answers
48 views
How to apply weightings to least squares slope formula
Im using this formula to find the slope of the regression line of given $x,y$ samples using LS calculation :
$a=\frac{(n\sum xy-\sum x\sum y)}{(n\sum x^2-(\sum x)^2)}$
How do i change it to apply ...
2
votes
2answers
67 views
The derivative and extremum of a matrix function
$$f(W)=(Ax-b)^TW(Ax-b)=x^TA^TWAx-2b^TWAx+b^TWb$$
where $f(W)$ is a function of $W$, $A$ is a known matrix, $x$ and $b$ are vectors ($b$ is known). How to get $\frac{\partial f}{\partial W}$?
0
votes
0answers
81 views
Solving the two-parameter generalized Eigenvalue problem
Is there a solution for the problem:
$$ (A_0 + \alpha \ A_1 + \beta \ A_2 ) x = 0 $$
where:
$ A_0, A_1,$ and $A_2 \in \mathbb{R}^{n\times n}$, $\alpha$ and $\beta \in \mathbb{C}$, and $x \in \mathbb{...
2
votes
0answers
80 views
Sensitivity of weighted least squares estimation method
I am trying to understand the weighted least squares estimation method, and I'd really appreciate it if you could shed some light on me. Let me explain my problem briefly:
Consider a linear model in ...
2
votes
1answer
247 views
minimize a sum of weighted square distances
Let $f_1, f_2$ be given polynomials of degree $k$ and we want to find a polynomial $f$ of degree $k+1$ that solves the following minimization problem on $[0,1]$:
$$f=\operatorname*{argmin}_{\hat{f}\in ...
1
vote
2answers
264 views
Weighted least squares with nuclear norm minimization
Nuclear norm minimization is very popularization and formulation is least squares term with nuclear norm term as following,
$$\min\limits_{X} \frac{1}{2}\|X_{3\times3}-Y_{3\times3}\|_F^2+\lambda\|X_{...
