Solving System of Equations using transformation rotation I've never had to post the same question twice, but my last post is getting filled out with work and I'm going about it a different way so I figured i'd try a whole different question
So This is the problem I've been working with: $$ \text{Let} \ a,b \ \text {be a positive real number with} \ a\neq b \ \text{Solve the system:}$$ $$a^2x^2-2abxy+b^2y^2-2a^2bx-2ab^2y+a^2b^2=0$$$$\text {and}$$$$abx^2+(a^2-b^2)xy-aby^2+ab^2x-a^2by=0$$ 
I have solved the top for $x$ plugged into the bottom and was working on solving the quartic for $y$. When someone gave me a new Idea to go about this.
They said to try and substitue $$x=\frac {u-v}{\sqrt 2} \ \text {and} \ y=\frac {u+v}{\sqrt 2}$$
How did they pick those two equations for $x$ and $y$ and how does that even help solving the two systems for x and y? Because If I use those substitutions to rotate the system, won't I have to somehow rotate them back?
Thanks for any and every idea!
FIRST EDIT:
So now I have used the substitutions $$x = \hat{x}\cos\theta - \hat{y}\sin\theta$$ and $$y = \hat{x}\sin\theta + \hat{y}\cos\theta$$ 
I also used the exact value of $$\cos(\frac{\pi}8)=\frac{\sqrt{2+\sqrt2}}2 \qquad \text {and also} \ \qquad \cos^2(\frac{\pi}8)=\frac{2+\sqrt2}4 $$ and $$\sin(\frac{\pi}8)=\frac{\sqrt{2-\sqrt2}}2 \qquad \text {and also} \ \qquad \sin^2(\frac{\pi}8)=\frac{2-\sqrt2}4$$
Which I was able to simplify my equation down some, but how does that help the overall process of solving the systems?
SECOND EDIT:
So I substituted and rearranged the top equations and got: $$a^2\hat{x}^2\frac {2+\sqrt{2}}4-a^2\hat{x}\hat{y}a^2\hat{y}^2\frac{2-\sqrt2}4-ab\hat{x}^2-2ab\hat{x}\hat{y}+ab\hat{y}^2+b^2\hat{x}^2\frac{2-\sqrt2}4+b^2\hat{x}\hat{y}+b^2\hat{y}2\frac{2+\sqrt2}4-2ab\hat{x}\frac{\sqrt{2+\sqrt2}}2+2ab\hat{y}\frac{\sqrt{2-\sqrt2}}2-2ab^2\hat{x}\frac{\sqrt{2-\sqrt2}}2+2ab^2\hat{y}\frac{\sqrt{2+\sqrt2}}2+a^2b^2=0$$
Then I did the same thing with the bottom equation and got: $$\frac{a^2\hat{x}^2}2+a^2\hat{x}\hat{y}\frac{\sqrt2}2-\frac{a^2y^2}2+ab\hat{x}^2\frac{2+\sqrt2}4-ab\hat{x}\hat{y}+ab\hat{y}^2\frac{2-\sqrt2}4-\frac{b^2\hat{x}^2}2+\frac{b^2\hat{y}^2}2b^2\hat{x}\hat{y}\frac{\sqrt2}2-ab\hat{x}\frac{\sqrt{2-\sqrt2}}2-ab\hat{y}\frac{\sqrt{2+\sqrt2}}2+ab^2\hat{x}\frac{\sqrt{2+\sqrt2}}2-ab^2\hat{y}\frac{\sqrt{2-\sqrt2}}2-a^2b\hat{x}\frac{\sqrt{2-\sqrt2}}2+a^2b\hat{y}\frac{\sqrt{2+\sqrt2}}2=0$$
Now once I have it simplified what can I do now?
 A: The idea is that in rotating, the solution (in transformed coordinates) becomes much easier to find. The general approach is as follows:
Given any conic section $Ax^2 + Bxy + Cy^2 + Dx + Ey + F = 0$, we can eliminate the $xy$ component by rotating the equation by an angle $\theta$ that satisfies $\text{cot}2\theta = \frac{A - C}{B}$.
EDIT: Good catch on that minus sign.
For your case we have that $\text{cot}2\theta = -\frac {a^2 - b^2}{2ab} = \frac {2ab}{a^2-b^2} = undefined $ (no number is equal to the negative of its reciprocal), or $\theta = \mathbf{\frac{\pi}{4}}$. 
In order to rotate equations by an angle $\theta$, we use the substitution $x = \hat{x}\cos\theta - \hat{y}\sin\theta$ and $y = \hat{x}\sin\theta + \hat{y}\cos\theta$.
The substitution you were given will rotate your axes by the amount we previously found.
Finally, you are correct that your solution will be in terms of the rotated coordinates, so you must remember to rotate them back the same amount (using the same equations, and a negative $\theta$ should do the trick).

Sorry for sending you down a rabbit hole on that $\pi/8$ bit. The point is the substitution will eliminate the $xy$ components in your equations, which makes them easier to work with.
A: My approach is to solve both equations for $x$ and then set them equal. The I go through stages of expansion and collection of terms. As I write, I'm stopping for the night. The next stage will be separation of radicals to one side and of non-radicals to the other side. Then both sides will be squared, the remaining radical will be separated, and both sides will be squared again. At that point a solution for $y$ should be possible even if it takes WolframAlpha to find it.
$$a^2x^2-2abxy+b^2y^2-2a^2bx-2ab^2y+a^2b^2=0\\ \implies
x = \frac{(2 a^2 b + 2 a b y) ± \sqrt{(-2 a^2 b - 2 a b y)^2 
- 4 a^2 (a^2 b^2 - 2 a b^2 y + b^2 y^2)}}{2 a^2} \\$$
$$abx^2+(a^2-b^2)xy-aby^2+ab^2x-a^2by=0\\ \implies
x = \frac{(a^2 (-y) - a b^2 + b^2 y) ± \sqrt{(a^2 y + a b^2 - b^2 y)^2 
- 4 a b (a^2 (-b) y - a b y^2)}}{2 a b}\\ $$
$$b\bigg((2 a^2 b + 2 a b y) ± \sqrt{(-2 a^2 b - 2 a b y)^2 
- 4 a^2 (a^2 b^2 - 2 a b^2 y + b^2 y^2)}\bigg) \\
=a\bigg( (a^2 (-y) - a b^2 + b^2 y) ± \sqrt{(a^2 y + a b^2 - b^2 y)^2 
- 4 a b (a^2 (-b) y - a b y^2)}\bigg)\\
\implies\\
b (\bigg(2 a^2 b + 2 a b y) ± 4 \sqrt{a^3 b^2 y}\bigg)=\\
a \bigg((a^2 (-y) - a b^2 + b^2 y) ± 
\sqrt{4 a^2 b^2 y (a + y) + (a^2 y + a b^2 - b^2 y)^2}\bigg) \implies \\
4 b \sqrt{a^3 b^2 y} + a^3 y + 3 a^2 b^2 - a \sqrt{a^4 y^2 + 6 a^3 b^2 y + a^2 b^4 + 2 a^2 b^2 y^2 - 2 a b^4 y + b^4 y^2} + a b^2 y = 0$$
