Creating Probability of Failure Ratings

To calculate the criticality score of an asset, the probability of failure of an asset needs to be quantified. A probability of failure rating structure that includes a numeric rating as well as a description of each rating ensures that all staff are scoring assets the same way. One of the best ways to develop these standardized criteria is to engage a cross-section of system personnel who have different viewpoints and different experiences with the assets. Staff should choose a rating scale, such as 1 to 5 or 1 to 10, and keep the descriptions broad enough so the ratings can apply to any assets (gray and green) in the system. Creating these ratings does not have to be a long, time-intensive activity. A small system should be able to complete the process of developing the rating structure by meeting a few times for a few hours each. A larger system may take longer and may wish to have a more expansive rating system. 

There are no industry standards for probability of failure for given assets, so they are best defined by utility staff. Consider how many levels of ratings will work best for your utility.  

• • • An even number of ratings can deter staff from the natural temptation to pick a middle value (e.g., 3 on a scale of 1-5).  
    • Avoid too many levels in a rating structure. The difference between levels could be so minute that it is challenging to determine what its probability of failure is. A 1-4, 1-5, or 1-6 scale are often the highest recommended. 

Once developed, staff will use the structure to assign a rating to each asset based on how likely the asset is to fail. See the following examples of possible rating structures below. In these examples, 1 is the lowest probability of failure.  

Table 2. In-depth probability of failure ratings with descriptions.

This is a more detailed rating structure. This structure is often suited to utilities who are further along in their asset management journey. A more descriptive rating structure helps to ensure that ratings are as objective as possible. 

Once a rating structure is created, it should be tested in the field using a variety of asset types to make sure it is understandable to those who will be applying it and to make sure it achieves the correct results. Following a successful test of the structure, the utility should create standard operating procedures (SOPs) for the application of probability of failure ratings to each asset to ensure the process is performed consistently among staff members. The SOPs can also be used to train new staff members on the process and ensure that the ratings stay consistent when current staff members leave the utility, change roles, or retire. 

Consider all four failure modes when assigning a rating to make informed judgements about the probability of failure across all types of assets. If the utility can collect and store the information, it is also valuable to identify which failure mode(s) are likely. The more relevant empirical data is collected on an asset, the more accurate and objective the probability of failure estimate will be and the more consistent they will be with industry best practice. The following examples use a rating scale of 1-5 with 1 being least likely to fail. 

Example: Asset A’s probability of failure rating is a four and is likely to fail via level of service failure.  

Example: Asset B’s probability of failure rating is a 2 and is likely to fail via mortality failure.  

Example: Asset C’s probability of failure rating is a 5 and is likely to fail via mortality and level of service failure. 

Collecting data to help determine the probability of failure for green assets may be more challenging initially because their inclusion in this type of analysis is a relatively new practice compared to gray assets. Additionally, collecting the appropriate data may require expertise that is outside the typical knowledge of a water or wastewater system (e.g., fire, forestry, geomorphology.) 

It is not appropriate or advisable to compare asset probability of failure ratings from two unrelated facilities (e.g., a water system in Massachusetts and a water system in Arizona.) The intent is to compare assets within a system. However, if an entity is managing multiple systems in the same area (e.g., the city owns 4 treatment plants that serve its population), the same rating system can and should be used in each of these plants. The entity is likely to want to determine which plant(s) require the most investment and comparing on the same basis will be beneficial.  

Once all assets are assigned a probability of failure rating, staff should review the results to see if they make sense or if there seem to be anomalies with certain types of assets or specific assets. If necessary, adjustments can be made to the rating structure to ensure it adequately represents the situation.