Tag Info

You have two degrees of freedom (the slope $m$ and $y$-intercept $b$ of the regression line, say), but also two constraints: The partial derivatives of the squared total error $E$ with respect to $m$ and with respect to $b$ must vanish. That means you expect only finitely many (local) minima. (As A.E. says, $E$ is strictly convex, so there is at most one ...
As @hardmath mentioned in the comment, the results are perfectly logical. If $Y$ and $X$ are independent and each one is $\mathcal{N}(0,1)$, so clearly (from independence) $cov(X,Y)=0$ and the real intercept is $0$, because $(0,0)=(\mathbb{E}X, \mathbb{E}Y)$. Hence, the real regression line is simply $y=0+0 x+\epsilon=\epsilon$, where $\epsilon \sim \mathcal{... 2 (since I can't add a comment I will repost a answer I posted here) There is a simple proof that requires only linear algebra. First notice that if you take$\beta=(\beta_0,\beta_1,\ldots,\beta_{n−1})^T$and the matrix$X=[\mathbb 1,x_1,x_2,\ldots,x_{n−1}]$, where$\mathbb 1=(1,1,\ldots,1)^T$and$x_i=(x_{i1},\ldots,x_{i(n−1)})^T$, you can write the model ... 2 You can find a least squares estimate of$x$and$y$. You want to "solve" the overdetermined system$Az = c$, where $$A = \begin{bmatrix} a_1 & b_1 \\ a_2 & b_2 \\ a_3 & b_3 \\ a_4 & b_4 \end{bmatrix}$$ and $$c = \begin{bmatrix} c_1 \\ c_2 \\ c_3 \\ c_4 \end{bmatrix}.$$ In a least ... 2 From a Bayesian point of view, this is equivalent to assuming that your data is generated by a line plus Gaussian noise, and finding the maximum likelihood line based on that assumption. Using the absolute values means assuming that your noise has pdf proportional to$e^{-|x|}$which is substantially less natural than assuming Gaussian noise (e.g. Gaussian ... 1 If the regression data are symmetric with respect to changing the sign of$y$, the least-squares approximation is the line$y=0$. The error is a sum of pairs$((y+a)^2 + (y-a)^2)$all of which are minimized at$y=0$. If the data are samples from a symmetric distribution then$y=0$is the expected regression line and the actual line will be a small random ... 1 As Jyrki Lahtonen commented, the problem reduces to finding the best value of$c$for the model $$y=c(x-x_k)^2+y_k$$ based on$n$data points$(x_i,y_i)$. If you define$z_i=(y_i-y_k)$and$t_i=(x_i-x_k)^2$then, the model reduces to $$z=c t$$ which corresponds to a linear regression without intercept. If you want to minimize the sum of squared errors, you ... 1 1) For the Alcohol retention, you don't even need a linear regression, as the relationship is perfectly linear (straight line), thus your model is: $$y(t) = 0.19 -0.015t,$$ where$y$denotes the Alcohol amount (g/dl) and$t$time. 2) For the Caffeine, assuming that exponential decay model is reasonable fit, you should first perform a linearization, ... 1 Basically, as you said, you are minimizing sum of squares, i.e., $$\underset\beta{\arg\min} \sum_{i=1}^n(y_i-\beta_0 - \sum_{j=1}^p\beta_jx_j)^2 = \underset\beta{\arg\min}||y-X\beta||$$ which has to be strictly convex function over the parametric space$\mathcal{B}$, otherwise the Hessian matrix of this quadratic form,$X'X$, won't be positive definite, ... 1 Assuming that you have$n$data points$(x_i,y_i)$, you want to minimize $$SSQ=\sum_{i=1}^n\left( c_0+c_1x_i+\frac 12 c_2x_i^2+\frac 13 c_3x_i^3-y_i\right)^2$$ subject to constraints$a_i\leq c_i\leq b_i\$. This is clearly an optimization problem with only bound constraints. Have a look here for doing it using Matlab. Edit By the way, if you have access to ...