Let $z = [x,y]$, observe $z' = [x',y'] = [z(2)-z(1)^3,-z(1)-z(2)^3]$.
Your initial condition is $z_0 = [x_0,y_0] = [1,0]$
The trick is now we have a vector $z$ and we know its derivative $z'$ and initial values, we can certainly solve it (numerically).
Matlab solvers like ode45 or ode23 can easily tackle this, or you can write your own solver using Euler's method and so on.
f = @(t,z)[z(2)-z(1)^3,-z(1)-z(2)^3];
[t z] = ode45(f,[1 100],[1 0]);
Remember now your $z$ vector contains both $x$ and $y$.
$z(:,1)$ will give you $x$ vector in correspondence to $t$ vector and $z(:,2)$ will give the $y$ vector
You can for example
to see how x evolve along t or
to see how y evolve along t. Or
to see the phase plot.
Regarding plotting against $A$ and $B$, I think you might have misunderstood the problem. We have x(t) and y(t) and we solved them, this is different from solving z(x,y,t). Don't get them confused.
Your A(x(t),y(t)) is basically a parametrized curve in 2 dimension (z against t). You can plot by
A = @(x,y) x.^2+y.^2;
And your B has nothing to do x(t) and y(t), it only depends on t, so again a 2 dimension plot.
B = @(t) 1./t;
To plot them on the same graph you can do either
plot(t,A(z(:,1),z(:,2)); hold on; plot(t,B(t)); hold off;