Solving $\sin[\cot^{-1}(x + 1)]=\cos[\tan^{-1}x]$ in two ways gives contradictory results I came across this problem to solve for $x$:
$$\sin[\cot^{-1}(x + 1)]=\cos[\tan^{-1}x]$$
I tried to do it initially by converting the cosine term on the right to sine by using $\sin(\frac{\pi}{2}-x)=\cos x$, but for some reason this doesn't work:
$$\sin[\cot^{-1}(x + 1)]=\sin[\frac{\pi}{2}-\tan^{-1}x]$$
$$\implies\cot^{-1}(x + 1)=\frac{\pi}{2}-\tan^{-1}x$$
$$\implies\cot^{-1}(x + 1)=\cot^{-1}x$$
$$\implies x + 1=x$$
$$\implies 1=0$$
Yet, if instead I use $\sin(\frac{\pi}{2}+x)=\cos x$:
$$\sin[\cot^{-1}(x + 1)]=\sin[\frac{\pi}{2}+\tan^{-1}x]$$
$$\implies\cot^{-1}(x + 1)=\frac{\pi}{2}+\tan^{-1}x$$
$$\implies\frac{\pi}{2}-\tan^{-1}(x + 1)=\frac{\pi}{2}+\tan^{-1}x$$
$$\implies\tan^{-1}(-x - 1)=\tan^{-1}x$$
$$\implies-x-1=x$$
$$\implies x=-\frac{1}{2}$$
And this is indeed the correct answer according to the book. What I don't get is why the first method failed. The identity $\tan^{-1}x + \cot^{-1}x = \frac{\pi}{2}$ should hold for all real numbers, so I don't think that is the problem. Moreover, if I plug $\frac 1 2$ and $-\frac 1 2$ into $\cot^{-1}x$, I get different answers, so this isn't a problem with the cancellation either. What's going wrong here?
EDIT: It appears that the problem is that since the value of $x$ is negative, the term $\frac \pi 2 - \tan^{-1}x$ becomes positive and lands outside the range where $\sin x$ is injective, and so the two inputs are different despite the fact that the output of their sine is equivalent. I guess a follow up question here is how one would figure this out without knowing that the answer is negative. It seems impossible to know which conversion to use beforehand without that knowledge.
 A: If $\sin[\cot^{-1}(x + 1)]=\cos[\tan^{-1}x]$, then
$$\sin[\cot^{-1}(x + 1)]=\sin\left[\frac{\pi}{2}-\tan^{-1}x\right]$$
If $\sin(\alpha) = \sin(\beta)$ then, for some integer, $n$
$$\alpha = \beta + 2n\pi \quad \text{or} \quad \alpha = (\pi - \beta) + 2n\pi$$
So
\begin{align}
   \cot^{-1}(x + 1) &= \left(\frac{\pi}{2}-\tan^{-1}x\right) + 2n\pi \\
   &\text{or} \\
   \cot^{-1}(x + 1) &= \left(\frac{\pi}{2}+\tan^{-1}x\right) + 2n\pi \\
\end{align}
You can work out the rest.
A: The reason the first method refuses to work is that it gives us $$\arctan x+\text{arccot} (x+1)=\frac π2.$$ But this is wrong, since the arguments of the involved functions are not equal for any value of $x.$
The correct identity is $$\arctan x+\text{arccot}x=\frac π2.$$
A: To start with,
$$
\cos(\tan^{-1}x) = \frac1{\sqrt{x^2 + 1}} > 0.  \tag1
$$
The range of the inverse cotangent is
$\left(-\frac\pi2,0\right) \cup \left(0,\frac\pi2\right],$
but from $(1)$ we know that we must have $\sin(\cot^{-1}(x + 1)) > 0$
and this eliminates $-\frac\pi2<\cot^{-1}(x + 1) < 0.$
Therefore $\cot^{-1}(x + 1) > 0$ and $x > -1.$
Under those constraints,
$$
\sin(\cot^{-1}(x + 1)) = \frac1{\sqrt{(x + 1)^2 + 1}}. \tag2
$$
Alternatively, if you didn't first observe that $\cot^{-1}(x + 1) > 0$
you could write $(2)$ with a $\pm$ sign in front of the right-hand side,
and you should then realize that when equating $(1)$ and $(2)$ there is no possible solution for the $-$ sign so it can be eliminated.
It's then easy to work out the rest of the problem.
This way you avoid all the complications of figuring out which of the infinitely many possible solutions to $\sin(\theta) = y$ are relevant when removing the $\sin$ function from both sides of an equation.
