Gram-Schmidt process on complex space

Let $\mathbb{C}^3$ be equipped with the standard complex inner product. Apply the Gram-Schmidt process to the basis: $v_1=(1,0,i)^t$, $v_2=(-1,i,1)^t$, $v_3=(0,-1,i+1)^t$ to find an orthonormal basis $\{u_1,u_2,u_3\}$.

I have found $u_1 = \dfrac{1}{\sqrt{2}} (1,0,i)^t$ and $u_2 = \dfrac{1}{\sqrt{2}}\left(\dfrac{i-1}{2},0,\dfrac{i+1}{2}\right)^t$.

I then try to use the following formulae to work out $u_3$ but I keep going wrong and can't figure out why: $w_3 = v_3 - \langle v_3, u_1\rangle u_1 - \langle v_2, u_2\rangle u_2$ and $u_3 = \dfrac{w_3}{\|w_3\|}$.

Your $u_1$ is correct, but your $u_2$ is incorrect; as noted in the comments, $u_1\cdot u_2 \neq 0$.

Recall that the standard inner product on $\mathbb{C}^n$ is given by $\langle u, v\rangle = u\cdot\bar{v}$. With this in mind, let's calculate $u_2$. First we have

$$\langle v_2, u_1\rangle = (-1, i, 1)\cdot\overline{\frac{1}{\sqrt{2}}(1, 0, i)} = \frac{1}{\sqrt{2}}(-1, i, 1)\cdot(1, 0, -i) = \frac{-1-i}{\sqrt{2}},$$

so

$$w_2 = v_2 - \langle v_2, u_1\rangle u_1 = \left[\begin{array}{c} -1\\ i\\ 1\end{array}\right] + \frac{1+i}{\sqrt{2}}\frac{1}{\sqrt{2}}\left[\begin{array}{c} 1\\ 0\\ i\end{array}\right] = \left[\begin{array}{c} \frac{-1+i}{2}\\ i\\ \frac{1+i}{2}\end{array}\right].$$

As

\begin{align*} \|w_2\|^2 &= \sqrt{\left|\frac{-1+i}{2}\right|^2 + |i|^2 + \left|\frac{1+i}{2}\right|^2}\\ &= \sqrt{\left(\frac{-1}{2}\right)^2 + \left(\frac{1}{2}\right)^2 + 0^2 + 1^2 + \left(\frac{1}{2}\right)^2 + \left(\frac{1}{2}\right)^2}\\ &= \sqrt{2} \end{align*}

we have

$$u_2 = \frac{1}{\|w_2\|}w_2 = \frac{1}{\sqrt{2}}\left[\begin{array}{c} \frac{-1+i}{2}\\ i\\ \frac{1+i}{2}\end{array}\right].$$

Let's check to see if $u_2$ is orthogonal to $u_1$:

\begin{align*} \langle u_1, u_2\rangle &= \frac{1}{\sqrt{2}}(1, 0, i)\cdot\overline{\frac{1}{\sqrt{2}}\left(\frac{-1+i}{2}, i, \frac{1+i}{2}\right)}\\ &= \frac{1}{2}(1, 0, i)\cdot\left(\frac{-1-i}{2}, -i, \frac{1-i}{2}\right)\\ &= \frac{1}{2}\left(\frac{-1-i}{2} + 0 + \frac{1+i}{2}\right)\\ &= 0. \end{align*}

Note, we didn't have to normalise before we checked orthogonality; i.e. we could have checked $\langle u_1, w_2\rangle = 0$ instead.

I won't do the calculation of $u_3$ now. It is similar, but there are more computations. Note however that you have a typo in your formula for $w_3$; it should be

$$w_3 = v_3 - \langle v_3, u_1\rangle u_1 - \langle v_3, u_2\rangle u_2.$$

Now that you have the correct $u_2$ and the correct formula for $w_3$, the computation for $u_3$ should work out and produce $u_3 = \frac{1}{2}(i, -1-i, 1)$.