Is there any formula to calculate how many circles of radius r fit in a single bigger circle of radius R?
It is a very complicated and investigated problem, usually formulated in the form: given a natural number $n$, which is the smallest radius $R=R(n)$ of the disk containing $n$ non-overlapping unit disks (that is, disks of radius 1). You can see results for small $n$ ($\le 20$) at Packing Center by Erich Friedman. For large $n$ ($\gg 10$) it is practically impossible to obtain tight lower bounds for $R(n)$ and upper bounds are found by computers, trying to find tighter and more tighter packings. I can share with you some articles and a program Pack 1.0 related with this problem. I upload them when my file sharing website will work.