From Patterson et al' Computer Organization and Design:
Throughput and Response Time
Do the following changes to a computer system increase throughput, decrease response time, or both?
- Replacing the processor in a computer with a faster version
- Adding additional processors to a system that uses multiple processors for separate tasks—for example, searching the web
Decreasing response time almost always improves throughput. Hence, in case 1, both response time and throughput are improved. In case 2, no one task gets work done faster, so only throughput increases.
Although the book doesn't mention, am I right that the above is based on Little's Law for the relation between occupancy, latency and throughput for a single queue? In the book, the queue would be for queuing tasks being processed.
If, however, the demand for processing in the second case was almost as large as the throughput, the system might force requests to queue up. In this case, increasing the throughput could also improve response time, since it would reduce the waiting time in the queue. Thus, in many real computer systems, changing either execution time or throughput often affects the other.
Is the model now a network of two queues: one for processing, and the other for waiting to be processed?
Is there a result between throughput, latency and occupancy for this network, similar to the little's law for a single queue?
What are some keywords to look up for this network model and its "Little's Law"? And some references?