# Tag Info

## New answers tagged regression

0

$$||A w+eb||^2=(Aw+eb)^T(Aw+eb)=mb^2+2be^TAw+w^TA^TAw\\b=-e^TAw/m$$ The effect is that you can remove $b$ by subtracting the average vector from each row of $A$. In matrix language, let $B=A-e(e^TA/m)$ where $e$ is a column vector of $m$ ones. Now, using Lagrange Multipliers to make sure that $w\cdot w=1$: $$F(w)=w^TB^TBw+\lambda(1-w^Tw)\\ ... 0 Hint: In 3-D coordinate system, the distance between a given point (x_0, y_0, z_0) & the plane: ax+by+cz+d=0$$=\frac{|ax_0+by_0+cz_0+d|}{\sqrt{a^2+b^2+c^2}}$$0 It really matters in what sense you want the piecewise constant function to be "close" to the continuous function. The "regression tree" you mentioned looks like it uses the same concept I am going to suggest: L^2 closeness, aka, Least Squares. As you mentioned it is designed for points but we can make sense of minimizing the squared difference for ... 0 take the Cross-Correlation function: https://en.wikipedia.org/wiki/Cross-correlation 1 I found where it was in my notes. We went over it very briefly, so it was a small section. Here's what you need to do.$$ \frac{\partial{f}}{\partial{m}} = \sum_{i=1}^{5}2(y_i-mx_i-b)*(-x_i)\\ = -2(\sum x_iy_i-m\sum x_i^2-b \sum 1) = 0\\ \frac{\partial{f}}{\partial{b}} = -2(\sum y_i -mx_i-b)\\ -2(\sum y_i -m\sum x_i-b\sum i) = 0\\ \text{now we do some ...

1

Let our best fit line be described as $mx + b$ We would hope that we could solve for $m$ and $b$ without any contradictions using the system of equations: $\begin{cases} 2m+b = 1\\ 3m + b = 2\\ 3m + b = 3\\ 4m + b = 6\\ 5m + b = 5\end{cases}$ Which can be expressed as the following matrix equation: $\begin{bmatrix} 2 & ... 1 Denote$x=(x_1,\dots,x_n)$. Assuming that$\mathbb{E}[u_i|x]=0\mathbb{E}[\hat\beta_1|x]=\mathbb{E}\left[\frac{\sum x_i y_i}{\sum x_i^2}\mid x\right]=\mathbb{E}\left[\frac{\sum x_i (\beta_0 + \beta_1 x_i + u_i)}{\sum x_i^2}\mid x\right]=\frac{\beta_0\sum x_i+\beta_1\sum x_i^2+\sum x_i\mathbb{E}[u_i|x]}{\sum x_i^2}=\beta_1+\beta_0\frac{\sum x_i}{\sum ... 1 \begin{align} \text{E} \left ( \frac{\sum_{i=1}^{n} x_i Y_i}{\sum_{j=1}^{n} x_j^2} \right ) &= \left ( \sum_{j=1}^{n} x_j^2 \right )^{-1} \sum_{i=1}^{n} x_i \text{E}(Y_i) \\ &= \left ( \sum_{j=1}^{n} x_j^2 \right )^{-1} \sum_{i=1}^{n} x_i (\beta_0 + \beta_1 x_i) \\ &= \beta_0 n\bar{x} \left ( \sum_{j=1}^{n} x_j^2 \right )^{-1} + \beta_1 . ... 1 There are many interpolation and approximation methods available, depending on your requirements (in terms of smoothness, robustness to noise, computational cost...). Most of them are base on polynomials or piecewise polynomial functions. Interpolation makes sure that f(x_i)=z_i, while approximation realizes f(x_i)\approx z_i. ... 0 Your question is too vague. There exists an infinite number of functions that fit the requirements you listed so far. For example, one such function is:f(x) = \begin{cases}z_1; & x=x_1\\ z_2; & x=x_2\\ \vdots\\ z_n; & x=x_n\\ 0; &\text{else} \end{cases}$$This function perfectly maps all values x_i to ther targets. It is probably not ... 0 If n is large enough, linear regression should do the trick. In other words, you can jut consider the residuals of the linear regression. 1 This results from applying Leibniz's Rule \begin{eqnarray*} \frac{d}{d\hat x} E_{ϱ_τ}(X-\hat x) &=& (τ -1)\frac{d}{d\hat x}\int_{-∞}^{\hat x}(x-\hat x)\,dF(x)+τ\frac{d}{d\hat x}\int_{\hat x}^{∞}(x-\hat x)\,dF(x) \\ &=& (τ -1)\left((x - \hat x)|_{\hat x} + \int_{-∞}^{\hat x} \frac{d}{d\hat x}(x-\hat x)\,dF(x)\right)+τ\left((x - \hat x)|_{\hat ... 1 In general \textbf Y and \textbf X are known because you have a sample. This sample have a dataset of m points: (x_{11},x_{12},\ldots,x_{1m},y_1), (x_{21},x_{22},\ldots,x_{2m},y_2), (x_{31},x_{32},\ldots,x_{3m},y_3), \ldots, (x_{n1},x_{n2},\ldots,x_{nm},y_m). The values of x_{ij} are represented by \textbf X and the values of y_i are represented ... 1 Let f:b\in\mathbb{R}^n\rightarrow 2y^TXb\in\mathbb{R} where X\in M_{p,n},y\in\mathbb{R}^p. f is linear, then the derivative is Df_b:h\in\mathbb{R}^n\rightarrow 2y^TXh\in \mathbb{R}; note that the matrix associated to Df_b is a row. We consider the scalar product over \mathbb{R}^n: (u,v)=u^Tv. Then the gradient \nabla(f) is defined by: for ... 1 It's because they don't only look for the derivative of the map f:B\mapsto AB\in \mathbb R, which is indeed just A; but rather for the gradient vector \nabla f=\tfrac{df}{dB} such that \forall H,$$ \langle\nabla f,H\rangle=df_B(H)=A(H)=AH=\langle A^T,H\rangle .$$0 Most likely you are being confused because he wrote every variable as a capital letter instead of the usual convention: matrices are capital, and vectors are lowercase. In your situation, y is a column vector, X is a matrix, b is a column vector, and e is a column vector. Therefore you have:$$e'e=y'y-2y'Xb+b'X'Xb$$Taking the gradient with ... 0 Your first interpretation is correct, the expected value of the dependent value conditional on the third quarter (and on the remaining regressor variables) is defined as you did. Given a significance level \alpha=0.05, all the variables except the second quarter are individually significant since the p-values related to the t-test statistics are lower ... 0 As a complement to the answer given by @MichaelHardy, substituting Y = M\beta + \varepsilon (i.e., the regression model) in the expression of the least squares estimator may be helpful to see why the OLS estimator is normally distributed.$$ \begin{eqnarray} \begin{array}{l} \hat\beta &=& (M^\top M)^{-1}M^\top \underbrace{Y}_{Y = M\beta + ... 0 With dummy variables, everything is relative to the omitted group. So lets say in the first regression, you included all categories of 1-24, if none are significant, it suggests that you cannot tell the difference between those values and the value for the 25th category. This could, for example, be a function of the implied coefficient for the 25th category ... 0 The Gamma distribution is commonly used. Your plot does look like it might have a sharper peak than that, in which case you might use an inverse Gaussian distribution. Both these distributions have two parameters so you'd need to fit them to your data using Maximum Likelihood Estimation. The equations for these are given in the Wikipedia pages. 0 Regression problems for one explained variable of one independent variable are generally of the form $$f(x,y;a,b,c\cdots)\approx0$$where you want to find the "best" values of the unknown parametersa,b,c\cdots$for a given data set$(x_i,y_i)$. The common procedure is to minimize the global quadratic error, computed as ... 0 Hint If you turn the problem into an optimization problem, Occupy Gezi gave the solution and this could be the simplest. Another way would be to set$x=y z + \lambda^2$and to run the regression using$\lambda,y,z$as tuning parameters. 0 They run OLS regressions; it would be odd if the axis intercept ($c$) would happen to be$0$for all variables. But as they are focussing on to autogregressions of the variable vector, I suppose, they simply do not consider the intercept interesting enough to put it in the estimation results. I agree that it is somewhat unconventional that they give ... 1 As Occupy commented, using weighted least squares is the solution. Let me take something similar from data validation and data reconciliation. In this area, what we want to minimize is $$\Phi=\sum_{i=1}^n \Big(\frac{y_i^*-y_i}{\sigma_i}\Big)^2$$ where$y_i^*$,$y_i$and$\sigma_i$are respectively the reconciled value, the measured value and the standard ... 0 EDIT: Assuming$s > 0$and$B$is positive definite, take any nonzero vector$v$, let$t = v' B^{-1} v$and let$A = (s/t)^{1/2} v$. 1 for what it is worth, the least squares solution to $$H \hat{x} = z$$ is considered by multiplying on the left by$H^t,$giving $$H^t H \hat{x} = H^t z.$$ The matrix$H^t H$is square and symmetric, indeed positive semidefinite. It is called the Gramian of the system in applications. If the Gramian is nonsingular, therefore invertible, there is a ... 1 Suppose that$\exists y \in Ker(A) / y \neq 0$. Now, let$\hat{x}$be a least squares solution. That is:$\hat{x}$is such that$||A\hat{x} - b||_2 = min_{x \in V} ||Ax - b||_2$. Notice that$A(\hat{x} + y) = A\hat{x} + Ay = A \hat{x}$, hence$||A\hat{x} - b||_2 = ||A(\hat{x} + y) - b||_2$. This means that you have just created another solution that also ... 2 Your least squares solution is minimizing$\hat x^T A \hat x$If$A$does not have full rank, there is some vector$y$such that$Ay=0$. Then$(\hat x+y)^TA(\hat x+y)=\hat x^T A \hat x$so you can add any multiple of$y\$ to your solution and get the same product.

Top 50 recent answers are included