Ratio of balls in a box A box contains some identical tennis balls. The ratio of the total volume of the tennis
balls to the volume of empty space surrounding them in the box is $1:k$, where $k$ is an
integer greater than one.
A prime number of balls is removed from the box. The ratio of the total volume of the
remaining tennis balls to the volume of empty space surrounding them in the box is 1:$k^2$.
Find the number of tennis balls that were originally in the box.
A few questions regarding this problem:
Does the shape of the box matter? I just let the volume of the box be a constant $V$. Also I noted that the ratio $\frac{1}{k^2} = \left( \frac{1}{k} \right)^2$. ie. New ratio is the old ratio squared. I also let the amount of balls in the box be $n$ and the amount of balls taken out be $p$ where $p$ is a prime, so the new amount of balls in the box is $n-p$.
This is about all I could do in this problem but I would like to be guided towards solving the problem (and I'm also interested in your thought processes and what ideas you initially think of so I can get a better idea of what to think of when doing problem solving) than just being given the solution.
Your help would be much appreciated.
 A: Let $n$ be the number of balls, $p$ be the number of balls taken away and $V$ be the volume of the box. 
When I first saw the problem, what stands out is the condition "$p$ is a prime". This suggests us setup some equations between $n$, $k$ and $p$ and then uses this condition to pose some constraint on $k$. If we do that, we have
$$n ( 1 + k ) = V = (n - p)(1 + k^2)\quad\implies\quad p(k^2+1) = nk(k-1)$$ 
Now $p$ is on LHS together with the factor $k^2+1$. To use the condition "$p$ is a prime",
we probably need to pick a factor $?$ from RHS so that $\gcd(?,k^2+1) = 1$. $k$ seems
to do the job,
$$k |p(k^2+1)\quad\stackrel{\gcd(k,k^2+1)=1}{\implies}\quad k | p \quad\stackrel{p \text{ is prime}}{\implies}\quad k = p$$
Once we get this, the remaining step is obvious. We express $n$ in terms of $k = p$
and see what can we do with that.
$$n = \frac{k^2+1}{k-1} = k + 1 + \frac{2}{k-1}\quad\stackrel{n\text{ is integer}}{\implies}\quad k = 2 \text{ or }3 \implies n = \frac{k^2+1}{k-1} = 5
$$
The result just follows.
$$(n,p,V) = (5,2,15) \text{ or } (5,3,20)$$
A: You've done what you can to move in the right direction. As you observe, nothing's said about the shape of the box, so the only relevant item seems to be the volume, $V$. There's also the volume of an individual ball. We can (by changing our unit of measurement if necessary) assume that this volume is 1. That means that we have 
initial state: n balls; box volume $V = nk$. 
final state: n-p balls, box volume $V = (n-p)(k^2)$
So you know that $nk = (n-p) k^2$. We can divide both sides by $k$ (I'm just following my nose here!) to get
$$ 
n = (n-p) k
$$
At this point, I'd figure you could just try winging it. You'd say "$n$ has to be a composite, because the right hand side is a factorization of it, unless $k = 1$. But that's not possible, because it'd mean that $p =0$, which isn't prime. OK, so $n$ is composite. Let's try a small prime, like $p = 2$. I need to write $n = (n-2) k$. Well, $4 = (4-2) 2$ seems to work. Hunh." 
So you say "There were 4 balls originally, in a box large enough to hold 8 of them. The volume ratio is 2. Then you took away 2 balls (a prime number!) and you have 2 balls in a box that's got the volume of 8,. That's a ratio of 4, which is $2^2$. Looks like a solution."
But are there other solutions? Seems likely.  For instance
n = 6, p = 3, k = 2
seems to work as well. In other works, your problem doesn't seem to have a unique solution. So there's no "process" by which to arrive at "the solution." 
A: The ratio $1:k$ means in particular that if $\alpha$ is the total volume of the balls originally in the box, then $k\alpha$ is the total volume of the empty space surrounding them in the box. In general, if $V_t$ is the total volume of the tennis balls and $V_b$ is the total volume of the box, then the total volume of empty space surrounding the tennis balls in the box is $V_b-V_t.$ (This has nothing to do with the shape of the box. We simply need the box to be large enough that it can hold all the tennis balls originally in it, and still be closed.) What this means is that the total volume of the box (which will not be changing) is $\alpha+k\alpha=(1+k)\alpha.$
Now, let's suppose that there are $n$ balls in the box originally, and since they are all tennis balls, then we can assume that they all have the same volume. (In fact, if they had different volumes, there would be no way to do the problem, so we need to assume it.) Say that the volume of a single tennis ball is $\beta.$ Then $\alpha=n\beta,$ so the volume of the box is $(1+k)n\beta.$
Next, we remove a prime number of balls from the box, say $p,$ so that there are now $n-p$ balls in the box, so that the total volume of the tennis balls remaining is $(n-p)\beta.$ The total volume of the box is $(1+k)n\beta,$ so the total volume of the empty space around the remaining tennis balls in the box is $$(1+k)n\beta-(n-p)\beta=n\beta+kn\beta-n\beta+p\beta=(kn+p)\beta.$$ By assumption, then, the ratio $1:k^2$ is the same as the ratio $(n-p)\beta:(kn+p)\beta,$ which is clearly the same as $n-p:kn+p.$ This means that $$kn+p=k^2(n-p)\\kn+p=k^2n-k^2p\\k^2p+p=k^2n-kn\\(k^2+1)p=(k^2-k)n.$$ Now, note that since we removed a prime (so non-zero) number of balls from the box, then our new ratio cannot be the same as our old one--that is, $k^2\ne k$--so we may divide by $k^2-k$ to get $$n=\frac{k^2+1}{k^2-k}p.$$ At this point, I don't think there's much more we can say, unless there's some additional information about $k,p,$ or $n$ that you haven't shared. We can conclude readily that $\frac{k^2+1}{k^2-k}$ is a rational number greater than $1,$ but that doesn't really help to determine what $n$ has to be (at least, not as far as I can see).

Oops! I missed the condition that $k$ was an integer! Ignore "Now, note that since we removed...as far as I can see)."
Now, since $k$ is an integer, then $k^2+1$ and $k^2-k$ are integers. In particular, since $$(k^2+1)p=k(k-1)n,$$ then $k\mid(k^2+1)p,$ so since $k$ and $k^2+1$ are relatively prime, then we must have that $k\mid p.$ Since $p$ is prime and $k$ is an integer greater than $1,$ it then follows that $k=p,$ so we have $$(k^2+1)p=(k-1)np\\k^2+1=(k-1)n\\k^2+1=kn-n\\k^2-kn+n+1=0.$$ This gives us a quadratic in $k,$ whose solutions are $$\begin{align}k &= \frac{n\pm\sqrt{n^2-4(n+1)}}2\\ &= \frac{n\pm\sqrt{n^2-4n-4}}2.\end{align}$$ Since $k$ is an integer, then we require $\sqrt{n^2-4n-4}$ to be rational, which means it must be a positive integer, say $m.$ Hence, $$n^2-4n-4=m^2\\n^2-4n+4-8=m^2\\(n-2)^2-8=m^2\\(n-2)^2-m^2=8\\\bigl((n-2)+m\bigr)\bigl((n-2)-m\bigr)=8\\(n-2+m)(n-2-m)=8.$$ Since $m$ is a positive integer, then we can conclude that $m=1,$ whence $n=5.$ (I leave it to you to show this.)
A: $V$ is the volume of the box and $s$ is the volume of each ball. Then we can write:
$\frac{ns}{V-ns}=\frac{1}{k} \to \frac{ns}{V}=\frac{1}{k+1}$
$\frac{(n-p)s}{V-(n-p)s}=\frac{1}{k^2} \to \frac{(n-p)s}{V}=\frac{1}{k^2+1}$
$\frac{n}{n-p}=\frac{k^2+1}{k+1} \to 1+\frac{p}{n-p}=k+1-\frac{2k}{k+1}$
$\frac{p}{n-p}=\frac{k^2-k}{k+1}$
$\frac{p}{n}=\frac{k^2-k}{k^2+1}$
if $gcd(n,p)=1$ then $k(k-1)n=p(k^2+1)$ so $k=ap$ or $k=ap+1$.
if $k=ap$ then $n=\frac{(a^2p^2+1)}{a^2p-a}=p+\frac{pa+1}{a^2p-a}$ so $\frac{pa+1}{a^2p-a} \geq 1$ so $a+1 \geq a^2p-pa$ so either $a=1$ or $\frac{a+1}{a^2-a}\geq p$ that is impossible for $a>1$ so $a=1$ and $n=\frac{p^2+1}{p-1}=p+1\frac{2}{p-1}$ and it is only natural when $p = 2$ so $n=5$ and $k=2$. or $p=3,n=5,k=3$
if $k=ap+1$ then $n=\frac{a^2p^2+2ap+2}{a^2 p+a}=p+\frac{ap+2}{a^2p+a}$ so $ap+2 \geq a^2p+a$ which is impossible for $a>1$, but for $a=1$,$p+\frac{p+2}{p+1}=p+1+\frac{1}{p+1}$ will not be a natural number so we should dismiss this case. 
if $gcd(n,p)=p$ then $\frac{1}{t}=\frac{k^2-k}{k^2+1} \to t=\frac{k^2+1}{k^2-k}$ that cannot be a natural number for $k>1$, be cause it is strictly decreasing for $k>1$ and for $k=3$ it is less than 2 and for $t=2$ it is not integer.
Note that in the second case $n$ cannot be equal to $p$ because for no natural $k$, $k^2+1=k^2-k$. 
So the final result is: $n=5,k=2,p=2$ or $n=5,k=3,p=3$
