Tracking Level of Service Goals

As discussed in the Developing Level of Service Goals section, goals should be written such that they are measurable so that progress towards meeting these goals can be tracked over time. Tracking goals should be manageable and kept at a level the system can handle. For example, progress towards goals can be tracked monthly for some goals, quarterly for others, or annually for goals that are time consuming to track. The most important aspect is to track the goals on some routine basis and report the results of this tracking to both the elected officials/upper management and the customers. Tracking can be done with something as high-tech as a Computerized Maintenance Management System or as simple as a spreadsheet. When determining how often to track a particular goal, system staff should consider: 

• How often do we need to report this type of information to elected officials, the board, or decision makers?
• How often do we need to communicate with our customers on meeting this goal?
• How frequently will the data we need for these communications be available?
• How often will it be possible to make adjustments if we find we’re not meeting the goal?
• How much time will it take to get the data for tracking?
• How will any changes impact our resources?

The frequency with which a system tracks an individual goal should match the frequency of data availability. For example, if a system wants to track water pressure throughout the distribution system on a monthly basis, but operators only measure the pressure once a quarter, something needs to change. Either the system needs to change the way it operates by taking monthly readings, or the goal should be evaluated quarterly as data become available.

Similarly, the frequency of communication to customers can be used to set the time frame for tracking a goal. If a system has an annual meeting with its members, then the goal can be communicated annually so maybe it does not need to be tracked more often than that. If a system has a quarterly newsletter, then the goal may be tracked on a quarterly basis to provide information to the newsletter. Lastly, the system should consider how often it is possible to make adjustments to the goal in determining how often to measure it. If adjustments can only be made on a quarterly basis, then measuring the goal on a monthly basis is too frequent. However, an annual basis is not frequent enough. An example of this type of situation is the following. Assume the system has set a goal of responding to breaks within 6 hours during normal business hours. The system has a contract with an operator to respond to breaks, and the system can only change the contract on an annual basis. In this scenario, even if the system finds that one contract operator cannot meet the goal of 6 hours after the first quarter, it won’t be able to make an adjustment until the end of the year when it comes time to renew the contract. In this case, measuring annually, or semi-annually makes more sense than measuring monthly. 

The main point regarding frequency of measuring progress towards meeting goals is that there should be a balance between the resources it takes to gather the data in a usable format (time and money) in order to measure progress, and the importance of having the data. If a goal will take a significant amount of time to collect and analyze but it is not a significant goal, it is probably worth a closer look and possibly rewriting or eliminating the goal. An example of goals and tracking to see if goals are met, is shown below.

Table 2: Level of Service Goals Tracking

Once this data is examined on a monthly, quarterly, semi-annually, or annual basis, the system can decide how well they are doing in meeting the goals. There are several possibilities for each goal:

1. The system met and exceeded the goal.
2. The system just met the goal.
3. The system missed the goal, but not by a lot.
4. The system was not close to meeting the goal.
5. Some condition outside the utility’s control caused the goal to be missed (e.g., natural disaster, pandemic.)

In the case of 1) and 2), where the goal was met, the system can decide if any further action is required. If the system is consistently much better than the goal, there are a few possibilities. The system can:

• Leave the goal as is and continue activities related to the goal as currently practiced.
• Leave the goal the same but reduce the level of activities in this area so that the system is closer to the actual goal level. The reduced activities can free up resources to be spent in another activity.
• Adjust the target level for the goal to be in closer in line with the actual performance.

In the case of 3) and 4) where the goal was missed, the system can determine if any actions should be taken to allow the system to meet this goal. The system can:

• Leave the goal as is and determine what needs to change in order to meet the goal in the future.
• Leave the goal as is, leave activities the same and determine if the goal can be met in the future with no changes.
• Revise the target level for the goal to make it possible to reach the goal with the same level of effort used in the past. In this case, there should be a good reason to revise the target level; the target level should not be changed just because the goal was not met. Rather, the target level could be revised because the cost of meeting the goal may not be worth the benefit, and the customers will accept a change in the goal level without significant concern.
• Leave the goal as is for now, but plan to have customer listening sessions or input methods to determine if the goal should be revised or if resources should be spent to meet the goal.

In the case of 5) where the reason the goal was not met is related to an incident outside of the system’s control, it is worthwhile to determine how important it is to continue to meet a particular goal in this type of event. For example, if the goal is reliability (e.g., “No customer will be without water for more than 4 hours”) and a massive power outage caused customers to be without water for a few days, the system may want to investigate emergency generators as an option to prevent future disruptions. If, on the other hand, the goal was “the system will respond in person to all customer complaints related to water quality within 24 hours” and the response required an in-person follow-up, this type of goal is unlikely to be met during a pandemic, such as COVID. It might be determined that the goal can be changed during the pandemic to say that follow-up will now be virtual instead of in-person. The goal may also have to be revised in terms of the time for response as the pandemic may affect the availability of staff to respond.

Any adjustments to target levels or goals themselves should be made through careful consideration of data related to the goal and target levels as well as conversations with appropriate stakeholders. The stakeholders for external goals are customers and elected leaders. The stakeholders for internal goals include management and operations personnel. At a minimum, a review of all goals should be undertaken on an annual basis. If the system tracks information regarding how well it is meeting the level of service criteria on a regular basis, it can use this information to prepare an annual report regarding how well the system met these criteria over the course of a year. This information can be presented to the customers at an annual meeting or newsletter so that customers are aware of how well the system met the overall goals. A meeting would also be an opportunity to discuss any changes needed, based on the data tracked. Decisions on changes to level of service goals will directly impact customers, so it is important to use the opportunity of the annual meeting to discuss the potential options with them. If there is no annual meeting, customer feedback can be gathered through online polls or other similar means.

Using qualitative and quantitative benchmarking methods – Frank Roth, Sr. Policy Manager, Chair of AM Steering Committee, Albuquerque Bernalillo County Water Utility Authority, Albuquerque, NM