# Yet another Least Squares matrix derivation

I understand the solution to the well known Least squares as explained in the following post

Least-squares solution to a matrix equation?

We solve for β so that below expression has minimal value.

$$\mathbf(Y−Xβ)′ × (Y−Xβ)$$

In above, lets assume that we have N samples each with D features

• Y is N*1
• X is N*D
• β is D*1

I am wondering how the derivation steps would change had we assumed Y output shape is 1 * N

• Y is N*1

So the equation for Y would be

$$\mathbf Y = β'X'$$

and not

$$\mathbf Y = Xβ$$

as in the original derivation.

Again, while I completely understand the steps in the original derivation, I could not solve if I had assumed Y = β′X′

$$\mathbf (Y−β′X′)′×(Y−β′X′)$$

Expanding the above for derivative wrt β yields below - but I couldn't proceed further

$$\mathbf Y′Y − Y′β′X′ - XβY - Xββ′X′$$

For academic interest, I like to understand if it at all possible to solve for β this route and the steps

• Please use MathJax to format. May 6, 2020 at 6:45
• You can use the linked solution by replacing $Y$ by its transpose, i.e. the same substitution you've made in the problem statement.
– greg
May 6, 2020 at 14:55
• Thanks. Specifically, what are the derivative of terms Y'β'X' and XβY wrt β (I can't get a dimensionally compatible value for these terms. I may be missing something obvious. May 6, 2020 at 15:16

In brief, the steps to solve a least squares problem are \eqalign{ y &= X\beta \quad\implies \min_\beta\,\|X\beta-y\|^2 \quad\implies \beta = (X^TX)^{-1}X^Ty \\ } The decision to use $$Y=y^T$$ instead of $$y$$ has no effect on these steps, i.e. \eqalign{ Y^T &= X\beta \quad\implies \min_\beta\,\|X\beta-Y^T\|^2 \quad\implies \beta = (X^TX)^{-1}X^TY^T \\ } The approach can be summarized as:
$$\quad$$Substitute the new $$Y^T$$ variable where ever the original $$y$$ variable appears.