A critical point is a point at which the derivative vanishes. So definitely, $1$ and $4$ are not critical points.
Now those points are at the boundary of the domain of $f$ and are extremas.
However, consider a point $x$ which is a minimum or a maximum of a differentiable function $f$ and which belongs to the interior of the the domain of $f$. Then $f^\prime(x)=0$.
An extrema belonging to the interior of the domain of a differentiable map is a critical point.
An extrema may not be a critical point, if it belongs to the frontier of the domain. Example: the function of the question.
And obviously the derivative of a function can vanish at a point belonging to the frontier. In that case the point is a critical point.
Lastly a critical point may not be an extrema. Example $f: x \mapsto x^3$ at $x=0$.