Why Response time goes to infinite when Utilization approaches 1? I've started to learn queueing theory & model.
When we take a look at the graph (Utilization vs. Response time), it shows Response time goes to infinite when Utilization approaches 1, which is not always true in reality.
Can anybody explain me what assumptions make this distortion intuitively?
 A: We generally assume that the interarrival and service times are random, and the average delay time increases with the variation of these times. Intuitively this makes sense - in a system with deterministic interarrival and service times, there is no waiting time as long as $\lambda\leqslant\mu$, and in a system where the service times have high variability, a long service time can result in many customers being delayed.
To be more precise, Kingman's formula approximates the mean delay time in a one-server queue: $$\mathbb E[W_q]\approx\left(\frac\rho{1-\rho}\right)\left(\frac{c_A^2+c_S^2}2\right)\mathbb E[S]. $$ Here $\rho=\frac\lambda\mu$ is the server utilization, $c_A^2$ and $c_S^2$ are the coefficients of variation of the interarrival and service times, respectively, and $\mathbb E[S]$ is the mean service time. This approximation is generally very accurate, regardless of the probability distributions of the interarrival and service times, especially when $\rho$ is close to $1$.
A: Here's a good article that describes the variables in response vs. utilization: http://queue.acm.org/detail.cfm?id=1854041
I'd also like to offer the social observation that little efficiencies can speed things up, for instance if the second customer says, "I'll have the same as the previous order."  That might speed up order input time due to "caching." And might speed up other parts of the order-processing and delivery sequence depending on implementation.
