In my last blog entry I discussed the prevalence of 'microbursts' in market data feeds – short, occasional bursts of activity in the feed during which data rates reach levels much higher than average. We saw that during busy periods the LatencyStats.com feeds can send many high-rate microbursts in succession, each burst lasting much less than one second. If the data rate during a microburst exceeds the capacity of a network link or a feed handler, the excess data will be forced to queue. This adds latency to the feed messages, and in the worst case messages may be discarded if a system runs out of buffer space to hold waiting data.
How should network and computing systems be engineered to handle microbursts? A simple but rather conservative approach is to ensure that the available capacity always exceeds the highest microburst data rate measured at some short timescale. For example, if the speed of a network link carrying a market data feed exceeds the feed’s highest 1-millisecond data rate, then the link will never be continuously busy for more than 1 millisecond at a time. Therefore no data will be delayed on the link for more than 1 millisecond.
In practice the capacity needed to keep latency below 1 millisecond is normally much less than the peak 1-millisecond data rate. This is because the microbursts in the feed tend not to be sustained over time. A 1-millisecond microburst exceeding link capacity will build up a queue of data waiting to be processed. But provided the system can buffer the queue and clear it quickly when the burst ends, it can still prevent any data from being delayed more than a millisecond. For example, on LatencyStats.com we display both the peak 1-millisecond rate and the network bandwidth required to avoid 1-millisecond latency for each feed – the latter value is computed using an algorithm based on queuing theory. At the time of writing (28 September 2010) the values displayed for ArcaBook for the last seven days are: