Orthogonal complement of complex subspace

I have a dot product defined by a matrix $$A= \begin{pmatrix} 2 & 0 & 1+i \\ 0 & 3 & -1 \\ 1-i & -1 & 3 \\ \end{pmatrix}$$ and I want to find an orthogonal complement of a subspace $$W=span\{(-i,1,1+i),(-1-i,1,3)\}$$

I think I need to find the null space of $$W.A=0$$ where W is a matrix with row vectors $$(-i,1,1+i)(-1-i,1,3)$$.

The thing is, I am not sure whether to conjugate the vectors of subspace first or not, as I tried and my complement ended up being $$(-3-3i,i,1)$$ but it doesn't seem right.

Can someone give me a general idea how to solve otrthogonal complement of complex subspace when the dot product is defined by a orthogonal matrix?

• The starting point is: what does it means that a dot product is defined by a matrix? What is your answer? – Emilio Novati Feb 27 '18 at 19:59
• Matrix is a mapping, so the matrix that defines a dot vector is a mapping of two vectors that give us a number. So the standard dot product is defined by identity matrix. I'm sorry I am not a native english speaker so using the verb define is maybe not used with matrixes. – Zuzana Mitterová Feb 27 '18 at 20:03

Given an hermitian positive definite matrix , as your $A$, the corresponding inner product on $\mathbb{C}^3$ is the Hermitian form defined as $$\langle x,y\rangle=y^\dagger A x$$ where $y^\dagger$ is the conjugate transpose of $y$.
So in your case you have to found a vector $y=(y_1,y_2,y_3)^T$ such that:
$$\begin{pmatrix} \overline{y_1}&\overline{y_2}&\overline{y_3} \end{pmatrix} \begin{pmatrix} 2 & 0 & 1+i \\ 0 & 3 & -1 \\ 1-i & -1 & 3 \\ \end{pmatrix} \begin{pmatrix} -i\\1\\1+i \end{pmatrix}=0$$ and $$\begin{pmatrix} \overline{y_1}&\overline{y_2}&\overline{y_3} \end{pmatrix} \begin{pmatrix} 2 & 0 & 1+i \\ 0 & 3 & -1 \\ 1-i & -1 & 3 \\ \end{pmatrix} \begin{pmatrix} -1-i\\1\\3 \end{pmatrix} =0$$
The searched orthogonal complement subspace is the span of this vector $y$.