A server that is busy almost all the time looks efficient and positive for the owners, but from the customer’s side, the same system can feel painfully slow. Queueing theory explains why a small increase in demand can cause a much larger increase in delay, even before average demand reaches the system’s capacity.
Consider the M/M/1 queue. Jobs arrive independently according to a Poisson process with a constant average rate. Service times are independent of each other and of arrivals, and exponentially distributed. One server handles jobs in arrival order. The model assumes an unlimited waiting area and an arrival rate below the average service capacity. In steady state, its average total time in the system is:
W = 1/(μ – λ)
W is the average time from arrival to completion. The symbol λ, ‘lambda’, is the arrival rate, while μ, ‘mu’, is the service rate while the server is busy. Both rates use the same time unit. W includes waiting and service, so it measures the full delay experienced by a job. [1]
Suppose the server completes an average of ten jobs per minute when busy. With five arrivals per minute, the model predicts an average total time of twelve seconds. At 9 arrivals per minute, that becomes 60 seconds. At 9.5 arrivals per minute, it reaches two minutes. Moving from 90 to 95 per cent utilisation doubles the predicted time.
Randomness explains the sharp increase. Several jobs sometimes arrive close together, or a job takes unusually long to finish. Spare capacity lets the server clear the backlog. Close to full utilisation, recovery is slow, and another burst may arrive before the queue has emptied.
The same relationship makes additional capacity especially valuable near the limit. At 9 arrivals per minute, increasing the service rate from 10 to 11 jobs per minute reduces the predicted total time from 60 seconds to 30. A 10% increase in capacity halves this measure of delay.
Real queues may behave differently. Arrivals can occur in scheduled batches, service times may be nearly fixed, and several servers may share the work. These differences change the appropriate model and its numerical predictions. Two systems with identical average rates can have very different waiting times when their variability differs. [2]
Capacity planning therefore needs more than a comparison of average demand and average output. It also needs information about fluctuations and the delays users can tolerate. An idle server during a quiet period may be providing the spare capacity that clears the next burst. Removing every quiet moment can leave the system with little ability to recover.
Leave a comment