Using Latency to Evaluate Computer System Performance

Abstract

Building high performance computer systems requires an understanding of the behaviour of systems and what makes them fast or slow In addition to our file system performance analysis we have a number of projects in measuring evaluating and understanding system performances The conventional methodology for system performance measurement which relies primarily on throughput-sensitive benchmarks and throughput metrics has major limitations when analyzing the behaviour and performance of interactive workloads The increasingly interactive character of personal computing demands new ways of measuring and analyzing system performance In this paper we present a combination of measurement techniques and benchmark methodologies that address these problems We use some simple methods for making direct and precise measurements of event handling latency in the context of a realistic interactive application We analyze how results from such measurements can be used to understand the detailed behaviour of latency-critical events We demonstrate our techniques in an analysis of the performance of two releases of Windows 9x and Windows XP Professional Our experience indicates that latency can be measured for a class of interactive workloads providing a substantial improvement in the accuracy and detail of performance information over measurements based strictly on throughput