# Calculating the Optimal Number of Server Instances

At time $t_{0}$ I have $N$ server instances running. Each instance is at $100\%$ CPU utilization. Collectively, the instances are attempting to respond to $R_{0}$ web requests. Some unknown percent of $R_{0}$ requests are being ignored because all the instances are under too much load.

At time $t_{1}$, the load lessens, and all $N$ instances are now at only $50\%$ CPU utilization. Collectively, the instances are now successfully responding to $100\%$ of $R_{1}$ web requests. Note $R_{1}$ < $R_{0}$.

How many additional server instances ($N_{\text{additional}}$ so that $N_{\text{new}} = N + N_{\text{additional}}$) should you create so that should $R_{0}$ traffic occur again, CPU utilization on each instance will be no more than $75\%$?

My approach is to use the performance under $50\%$ load to calculate the predicted maximum number of requests per instance, and then divide the $R_{0}$ by this amount to get the estimated number of instances needed, multiplying by some amount to over-estimate to get the $75\%$ max CPU utilization.

e.g.

Maximum $R$ PerInstance $= 100 \times \frac{\frac{R_1}{N}}{50}$

Predicted Number of Instances $= \frac{\text{Predicted R}}{\text{Max } R \text{ Per Instance}} \times 1.25$

Does that seem like the right approach?

-

Your idea/approach is correct but from where do you get the factor $1.25$? You need to multiply by a factor $\frac{4}{3}$ since you now want them to be working at $75\%$ utilization. ($75\%$ of $\frac{4}{3}$ is $1$).

Assume that each web request has the same CPU utilization.

$R_1$ web requests corresponds to $0.5 \times N$ instances.

$1$ web request corresponds to $\frac{N}{2R_1}$ instances.

$R_0$ web requests corresponds to $\frac{NR_0}{2R_1}$ instances.

And this should be $0.75 \times N_{\text{new}}$ instances.

Hence, $N_{\text{new}} = \frac{4}{3} \frac{NR_0}{2R_1} = \frac{2}{3} \frac{NR_0}{R_1}$.

So the number of additional server instances you need is $\frac{2}{3} \frac{NR_0}{R_1} - N$.

(Note that you have $R_0 > 2R_1$ and hence $\frac{2}{3} \frac{NR_0}{R_1} - N > \frac{4}{3} N - N = \frac{N}{3}$. Hence the additional server instance you need is greater than $\frac{N}{3}$).

-