An empty set has no elements, so what I would guess is that X would
have to be the same set as A, since removing all real elements of X
would leave you with an empty set. Does that make sense?
Yes, it makes sense. In removing the elements of $A$ you end up removing all the elements of $X$, so all the elements of $X$ must be elements of $A$ and removed.
But that just tells you the elements of $X$ are in $A$. It doesn't tell you that anything about the elements of $A$ that may or may not be in $X$.
So that means $X \subset A$. But it doesn't mean $A = X$. It is possible that $X \subsetneq A$.
As for symmetric difference, if I understand it correctly, is $(X\setminus A) \cup (A\setminus X) = (X\cup A)\setminus (X\cap A) $ is the set of all elements in $X$ or in $A$ but not both, right?
In that case ....$X \Delta Y=\emptyset$ means There are no elements in $X$ that aren't in $A$ and there aren't any elements in $A$ that aren't in $X$ so.... $X$ and $A$ have the same elements.
So $X = A$.