Pieter21 has provided you with a valid and efficient solution.
Your attempt was incorrect since you subtracted the sum of the number of ways of selecting five men and one woman and the number of ways of selecting five women and one man. However, it is possible to select a committee of six people containing at most one of A and B that contains more than one person of each sex.
Here is an alternate solution:
A committee that contains at most one of A and B either contains neither A nor B or it contains exactly one of them.
If the committee of six people contains neither A nor B, we must select six of the other eleven available people, which can be done in $$\binom{11}{6}$$ ways.
If the committee contains exactly one of A and B, we must select one of A and B, which can be done in $\binom{2}{1}$ ways, and select five of the other eleven people, which can be done in $\binom{11}{5}$ ways. Hence, the number of committees of six people that contain exactly one of A and B is
$$\binom{2}{1}\binom{11}{5}$$
Therefore, the number of committees that can be formed with at most one of A and B is
$$\binom{11}{6} + \binom{2}{1}\binom{11}{5}$$