This really isn't a question specific to arguments and complex numbers.
Essentially the arctan function horizontally asymptotes to $pi/2$ and $-pi/2$, as seen by the graph:
So there actually isn't a value of $x$ (infinity is not a number!) that gives an output of $-pi/2$.
Rather, the limit as $x$ approaches negative infinity for $arctan(x$) yields the $-pi/2$ you are seeking.
In turn $x$ can be made to approach infinity by minimizing the denominator for the opposite/adjacent triangle that defines the tangent. This means a complex number with a minimized real part and a negative imaginary part will yield $-pi/2$. So what is the limit of $x$ (the real part), when it approaches $0$? Well it is $0$ itself.
So the complex number with an argument of $-pi/2$ can be concluded to be of the form $0-ki$ (where $k$ is a positive real number). And do note, we have taken limits to find this.
In order to really make this whole system of complex arguments be less tedious, one can simply define such problematic cases to make them conform to our desires. So always simply state that a complex number only with imaginary parts either has an argument of $3pi/2$ or $pi/2$ - this, we can say is the "definition" and so questions regarding it cease (although is good to have used limits to rigourously justify our definition ;) )!