# Transform the squared error loss function to matrix form

I have n data, $$x_i$$ are the inputs, and $$y_i$$ are the labels. $$x_i$$ is $$d$$-dimension, and $$y_i$$ is $$p$$-dimension. We have a prediction function that is $$f(x) = W^Tx$$. $$W$$ is a matrix of dimension $$d \times p$$.

The squared error loss function of the prediction function is: $$J(W) = \sum_{i=1}^n||y_i - W^Tx_i||_2^2$$

I would like to write $$J(W)$$ in matrix form. How should I convert it to an equation involving $$Y$$ ($$n \times p$$ dimension), $$W$$ ($$d \times p$$ dimension) and $$X$$ ($$n \times d$$ dimension). Hence the loss function will be in matrix form instead of a sum?

Given the set of data $$D_{m}:=\left\{\left(x_{i}, y_{i}\right) \mid i=1,2, \cdots, m\right\}$$ such that $$x_{i} \neq x_{j}$$ for $$i \neq j .$$ we must find the line $$y=\alpha x+\beta$$ that best fits the data, i.e. find $$(\alpha, \beta)$$ such that for all $$(a, b) \in \mathbb{R}^{2}$$ we have $$$$0 \leq \Phi(\alpha, \beta)=\sum_{i=1}^{m}\left(y_{i}-\alpha x_{i}-\beta\right)^{2} \leq \sum_{i=1}^{m}\left(y_{i}-a x_{i}-b\right)^{2}=\Phi(a, b)$$$$ in matrix form, $$\Phi(\alpha, \beta)=\Phi(z)=\|y-A z\|_{2}^{2}$$, thus inequality $$(1)$$ is equivalent to the least square problem: $$\Phi(z)=\|y-A z\|_{2}^{2}=\min_{w \in \mathbb{R}^{n}}\|y-A w\|_{2}^{2}=\min_{w \in \mathbb{R}^{n}} \Phi(w)$$ Since $$y_{i}-a x_{i}-b=(y-A w)_{i}$$ where : $$w=[a ; b], z=[\alpha ; \beta], \mathbf{y}=\begin{bmatrix}y_{1} \\ y_{2} \\ y_{3} \\ \vdots \\ y_{m}\end{bmatrix}, \mathbf{A}=\begin{bmatrix}x_{1} & 1 \\ x_{2} & 1 \\ x_{3} & 1 \\ \vdots & \vdots \\ x_{m} & 1\end{bmatrix}, \mathbf{w}=\begin{bmatrix}a \\ b\end{bmatrix},$$ and $$\mathbf{z}=\begin{bmatrix}\alpha \\ \beta\end{bmatrix}$$.
• So you are saying that $J(W) = \sum_{i=1}^n||y_i - W^Tx_i||_2^2$ = $||Y - WX||_2^2$ ? Mar 26 '21 at 2:19
With $$X = \begin{bmatrix}x_1 & \dots & x_n\end{bmatrix}$$ and $$Y = \begin{bmatrix}y_1 & \dots & y_n\end{bmatrix}$$, you simply obtain: $$j(W) = ||Y-W^TX||_{2,1},$$ where the $$L_{2,1}$$ norm is the sum of the Euclidean norms of the columns of the matrix $$Y-W^TX$$.
Note however the dimensions: $$X \in \mathbb{R}^{d \times n}$$ and $$Y \in \mathbb{R}^{p \times n}$$.