I wrote out Xor operation as $(\lnot x \land y) \lor (x \land \lnot y)$ but don't know how to simplify from here. I don't know if De Morgan's laws will just make things more complicated, and I don't know how to factor anything out when there are NOTS on everything.
You can't simplify this expression with boolean algebra as xor is it's own logical connective often denoted with $\oplus$. Thus what you wrote down could be written $x\oplus y$
What would a simplification look like to you? There aren't much simpler ways to write the exclusive disjunction in terms of $\neg, \land, \lor$. You could write $\neg(x \leftrightarrow y)$, but it's not much cleaner, as usually $x \leftrightarrow y$ is just an abbreviation for $(x \to y) \land (y \to x)$.
(Personally, I find $(x \lor y) \land \neg (x \land y)$ the clearest -- that's how I would describe the exclusive disjunction in words.)