Case Study: Measuring Process Performance

Service time is defined by the Agile and Lean Glossary as the wall clock time necessary to provide a service from start to finish.  It is measured by noting the start time and the end time of a unit of customer value as it flows through a process.  Service time can be averaged over a period of time to determine how long customers must wait to receive a product or service.  But averages can be misleading, especially if there is a wide variation in delivered service times.

Figure #1 is a histogram of the service times of an organization’s customer service requests over a four month period prior to implementing a Lean process improvement project.  The histogram was created by counting the number of service requests that completed for each time bucket across the bottom axis.  Those that completed from 0 to 10 days were counted in the “10” bar on the chart.  Those that were completed from 11 to 20 days were counted in the “20” bar, etc..  The total service requests for each month are shown using a different color segment on each bar.

Figure #1

The distribution of service requests shows a peak around 40 days with a long tail out beyond 300 days.  Building to an initial peak followed by a long tail is a common distribution curve of service times.  In this case, the tail was unusually long, with almost 20% of the requests taking over a year to complete.

Averages can be Misleading

Prior to development of these service time histograms, the organization simply calculated the average service time of their service requests.  Although many improvements made as part of a Lean process improvement project, the average service time remained stubbornly flat at 150 days.  Simply calculating the average service time did not reflect the impacts from the improvements being made.

Figure #2 is a histogram of service times for the same organization two years later.

Figure #2

Comparing the histograms of Figure #1 and Figure #2, the latter shows a tighter distribution of fulfilled requests at a peak of around 30 days.  Then a rapid fall off at 60 days to a long flat tail.  The histogram of Figure #2 shows progress over Figure #1 by reducing the customer service time for the majority of customers.  Even though the average service time each month was still around 150 days.

Average service time was never an accurate measure of performance because of the length of the distribution tail.  The average service time only rose or fell in a given month based on the number of 300+ day service requests that were delivered in that month.  It did not take many 300+ day service requests to increase the average service time calculation; even with service times in the 30’s. 

One of the efforts of the Lean initiative was to reduce the work-in-process (WIP) and close out old service requests.  During those months the average service time was worse than 150 days.

Why the long tail?
In this case study root causes were:

1.   Customers learned to put projects in the queue long before they were needed just to insure they would be completed on time.

2.   There was a mix of customer requests; some fully ready for execution and others that first required complex analysis. There was only one process to handle both alternatives.  The second peak at 130 days in Figure #2 represented complex projects.

3.   There was a lot of WIP with no priority to which work should be completed first; causing some work to flow through quickly and other work to literally get lost on someone’s desk

Process Predictability
The lesson learned from this case study is that average service time is not a sufficient to measure process performance.  Average service time is easy to calculate.  Little’s Law defines the average service time (total cycle time) as ST = WIP / Throughput.  But this equation does not consider which element of WIP is throughput next.  

Charting the distribution of service time provides insight into the predictability of a process.  Providing a customer a service ready date based on an average calculation can cause dissatisfaction if their actual service ready date can vary greatly from the target.  A tight service time distribution with a Lean Ratio of 2 or less is an indicator of a high performing Lean process that delivers maximum value for minimum cost.