Hot answers tagged

2

You have two degrees of freedom (the slope $m$ and $y$-intercept $b$ of the regression line, say), but also two constraints: The partial derivatives of the squared total error $E$ with respect to $m$ and with respect to $b$ must vanish. That means you expect only finitely many (local) minima. (As A.E. says, $E$ is strictly convex, so there is at most one ...


2

As @hardmath mentioned in the comment, the results are perfectly logical. If $Y$ and $X$ are independent and each one is $\mathcal{N}(0,1)$, so clearly (from independence) $cov(X,Y)=0$ and the real intercept is $0$, because $(0,0)=(\mathbb{E}X, \mathbb{E}Y)$. Hence, the real regression line is simply $y=0+0 x+\epsilon=\epsilon$, where $\epsilon \sim \mathcal{...


2

(since I can't add a comment I will repost a answer I posted here) There is a simple proof that requires only linear algebra. First notice that if you take $\beta=(\beta_0,\beta_1,\ldots,\beta_{n−1})^T$ and the matrix $X=[\mathbb 1,x_1,x_2,\ldots,x_{n−1}]$, where $\mathbb 1=(1,1,\ldots,1)^T$ and $x_i=(x_{i1},\ldots,x_{i(n−1)})^T$, you can write the model ...


2

You can find a least squares estimate of $x$ and $y$. You want to "solve" the overdetermined system $Az = c$, where \begin{equation} A = \begin{bmatrix} a_1 & b_1 \\ a_2 & b_2 \\ a_3 & b_3 \\ a_4 & b_4 \end{bmatrix} \end{equation} and \begin{equation} c = \begin{bmatrix} c_1 \\ c_2 \\ c_3 \\ c_4 \end{bmatrix}. \end{equation} In a least ...


2

From a Bayesian point of view, this is equivalent to assuming that your data is generated by a line plus Gaussian noise, and finding the maximum likelihood line based on that assumption. Using the absolute values means assuming that your noise has pdf proportional to $e^{-|x|}$ which is substantially less natural than assuming Gaussian noise (e.g. Gaussian ...


1

If the regression data are symmetric with respect to changing the sign of $y$, the least-squares approximation is the line $y=0$. The error is a sum of pairs $((y+a)^2 + (y-a)^2)$ all of which are minimized at $y=0$. If the data are samples from a symmetric distribution then $y=0$ is the expected regression line and the actual line will be a small random ...


1

As Jyrki Lahtonen commented, the problem reduces to finding the best value of $c$ for the model $$y=c(x-x_k)^2+y_k$$ based on $n$ data points $(x_i,y_i)$. If you define $z_i=(y_i-y_k)$ and $t_i=(x_i-x_k)^2$ then, the model reduces to $$z=c t$$ which corresponds to a linear regression without intercept. If you want to minimize the sum of squared errors, you ...


1

1) For the Alcohol retention, you don't even need a linear regression, as the relationship is perfectly linear (straight line), thus your model is: $$ y(t) = 0.19 -0.015t, $$ where $y$ denotes the Alcohol amount (g/dl) and $t$ time. 2) For the Caffeine, assuming that exponential decay model is reasonable fit, you should first perform a linearization, ...


1

Basically, as you said, you are minimizing sum of squares, i.e., $$ \underset\beta{\arg\min} \sum_{i=1}^n(y_i-\beta_0 - \sum_{j=1}^p\beta_jx_j)^2 = \underset\beta{\arg\min}||y-X\beta|| $$ which has to be strictly convex function over the parametric space $\mathcal{B}$, otherwise the Hessian matrix of this quadratic form, $X'X$, won't be positive definite, ...


1

Assuming that you have $n$ data points $(x_i,y_i)$, you want to minimize $$SSQ=\sum_{i=1}^n\left( c_0+c_1x_i+\frac 12 c_2x_i^2+\frac 13 c_3x_i^3-y_i\right)^2$$ subject to constraints $a_i\leq c_i\leq b_i$. This is clearly an optimization problem with only bound constraints. Have a look here for doing it using Matlab. Edit By the way, if you have access to ...



Only top voted, non community-wiki answers of a minimum length are eligible