For context, we are looking at the perceptron algorithm, where we want to find $(\beta,\beta_0)$ such that: $$\beta,\beta_0 = \underset{\beta,\beta_0}{ \text{argmin } } D(\beta,\beta_0) = \underset{\beta,\beta_0}{ \text{argmin } } -\sum_{i\in\mathcal{M}} y_i\left( x_i^T\beta + \beta_0 \right)$$ where $\mathcal{M}$ is the index set of misclassified points. (Note that $x_i^T\beta + \beta_0<0$ for $i\in\mathcal{M}$ ). So we can use stochastic gradient descent: $$(\beta,\beta_0) \leftarrow (\beta,\beta_0) + \gamma(y_ix_i,y_i)$$ where $(y_ix_i,y_i)=\nabla D$ with learning rate $\gamma$.
Notice it is not obvious how to calculate $\tilde{\beta}=(\beta,\beta_0)$ in closed form (i.e. not iteratively). That's because there are many $\tilde{\beta}$ that can satisfy this criterion. Any separating plane gives $D=0$, but there can be many ways to achieve a separating plane.