1967 HSC 4 unit Mathematics Question 2 Screenshot from the examination paper
[...asking about partial fraction decomposition of $$\frac{1 - abx^2}{(1-ax)(1-bx)} $$ and related formulas...]
This question is taken from the New South Wales Higher School Certificate 4 unit (highest level possible) paper of 1967. I wasn't sure how to approach this problem. In the end, I decided to let x = 1 and this gave me u$_r$ = -1/n and R$_n$(x) = (1 - ab)x when I compared terms on left and right but I'm not sure if this is the correct approach. Any help appreciated on this one.
 A: Let us do things formally without worrying about convergent issues. As mentioned by Jack Lam,  the partial fraction decomposition of the function is given by
\begin{align}
\frac{1-abx^2}{(1-ax)(1-b)} = \frac{1}{1-bx}+\frac{1}{1-ax}-1.
\end{align}
Formally, we will us the geometric series representation for the two fractions on the right-hand side, i.e. we have
\begin{align}
\frac{1}{1-bx} + \frac{1}{1-ax} =&\ \sum^\infty_{r=0} b^rx^r + \sum^\infty_{r=0}a^rx^r = \sum^\infty_{r=0}(b^r+a^r)x^r \\
=&\ \sum^n_{r=0}(b^r+a^r)x^r + \sum^\infty_{r=n+1} b^rx^r+\sum^\infty_{r=n+1}a^rx^r\\
=&\ \sum^n_{r=0}(b^r+a^r)x^r + b^{n+1}x^{n+1}\sum^\infty_{r=0}b^rx^r+ a^{n+1}x^{n+1}\sum^\infty_{r=0}a^rx^r\\
=&\ \sum^n_{r=0}(b^r+a^r)x^r +  \frac{b^{n+1}x^{n+1}}{1-bx}+ \frac{a^{n+1}x^{n+1}}{1-ax}.
\end{align}
Now, going back to the original function, we have
\begin{align}
\frac{1-abx^2}{(1-ax)(1-bx)} =&\  \sum^n_{r=0}(b^r+a^r)x^r +  \frac{b^{n+1}x^{n+1}}{1-bx}+ \frac{a^{n+1}x^{n+1}}{1-ax}- 1\\
=&\ \sum^n_{r=1}(b^r+a^r)x^r + \frac{b^{n+1}x^{n+1}}{1-bx}+ \frac{a^{n+1}x^{n+1}}{1-ax}+1.
\end{align}
Multiply both sides of the above equation by $(1-ax)(1-bx)$ yields
\begin{align}
1-abx^2 =&\ (1-ax)(1-bx)\left[\sum^n_{r=1}(b^r+a^r)x^r+1 \right] + [(1-ax)b^{n+1} +(1-bx)a^{n+1}]\ x^{n+1}\\
=&\ (1-ax)(1-bx)\sum^n_{r=0}u_rx^r + x^{n+1} R_n(x)
\end{align}
where 
\begin{align}
R_n(x) = (1-ax)b^{n+1} +(1-bx)a^{n+1}
\end{align}
and
\begin{align}
u_r =
\begin{cases}
1 & \text{ if } r =0,\\
a^r+b^r & \text{ if } r\geq 1
\end{cases}.
\end{align}
