Creating Consequence of Failure Ratings

As with the probability of failure of an asset, the consequence of failure of that asset also needs to be quantified. Assets should be assigned a consequence of failure rating just like they are assigned a probability of failure rating. Systems can quantify the consequence of failure by creating a rating structure that includes a numeric rating and a degree of impact description for each rating. From an implementation perspective, it may be easiest to use the same scale for both consequence and probability of failure. If a 1 to 5 scale was used for probability of failure, systems may want to use a 1 to 5 scale for the consequence of failure, but it is not necessary to do it this way. If a system wishes to weight one of the factors more heavily than the other, it is possible to have a 1 to 5 rating for probability of failure and 1 to 10 rating for consequence of failure or vice versa. If the scale is 1 to 5 for both probability and consequence, the overall risk will be between 1 and 25. A rating of “0” is never used because it would result in an overall risk of 0 and no asset is completely without any risk of failure or consequence of failure.

One of the best ways to develop standardized criteria is to engage a cross-section of system personnel who have different viewpoints and different experiences with the assets. Staff should choose a rating scale, such as 1 to 5 or 1 to 10, and keep the descriptions broad enough so the ratings can apply to any assets (gray and green) in the system. Creating these ratings does not have to be a long, time-intensive activity. A small system should be able to complete the process of developing the rating structure by meeting a few times for a few hours each. A larger system may take longer and may wish to have a more extensive rating system.

Once developed, staff will use the structure to assign a rating to each asset based on the significance of the consequences. Three examples of possible rating structures are shown below. Two use a 1 to 5 scale and the third a 1 to 4 scale with the lowest number being the lowest probability and the highest number being the highest probability.

Table 5: In-depth consequence of failure ratings with descriptions

Each consequence description should be generic enough so it can describe the impact from a green or gray asset failure. This allows the impacts of failures for each asset type to be considered in a common way. Rating impacts by this method allows systems to assess whether the impact from a specific degraded green asset renders the same degree of impact to the system as a specific degraded gray asset. The number of ratings a system creates does not matter (1 to 5 versus a 1 to 10) as long as those ratings have consistent definitions and are agreed upon by system staff. Some systems prefer to have an even number of ratings to avoid the natural temptation to pick the middle value (e.g., a rating of 3 on a scale of 1 to 5). Systems should avoid creating a rating structure with so many levels (e.g., a scale of 1 to 50), that the difference between each is minute and hard to decipher. It can also take much more time to apply this type of rating structure to the assets and the additional time may not be worth the benefit.

Once a rating structure is created, it should be tested in the field using a variety of asset types to make sure it is understandable to those who will be applying it and to make sure it achieves the correct results. Following a successful test of the structure, the system should create standard operating procedures (SOPs) for the application of consequence of failure ratings to each asset, so the process is performed consistently among staff members. The SOPsStandard Operating Procedures can also be used to train new staff members on the process and ensure that the ratings stay consistent when current staff members leave the system or retire.

System staff will assign a consequence of failure to an asset rating based on the rating structure they created. Staff must consider all three consequence categories when assigning a rating in order to make informed judgements about the consequence of failure across all types of assets. If the system has the ability to collect and store the information, it is also valuable to identify which types of consequences (financial, environmental, social) are likely to dominate the analysis. For example, Asset A has the potential for serious social consequences and the likelihood is rated a 4. Asset B has minor financial consequences, and the consequence is rated 2. There are no industry standards for consequence of failure for given assets, so they are best defined by system personnel as described previously. However, the more applicable empirical data that is collected on an asset, the more accurate and objective the consequence of failure estimate will be and the more consistent they will be with industry best practice.

Collecting data to help determine the consequence of failure for green assets may be more challenging initially because their inclusion in this type of analysis is a relatively new practice compared to gray assets. Additionally, collecting the appropriate data may require expertise that is outside the typical knowledge of a water or wastewater system (e.g., fire, forestry, geomorphology).

It is not appropriate or advisable to compare asset consequence of failure ratings from two unrelated facilities (e.g., a wastewater system in Tennessee and a wastewater system in Ohio.) The intent is to compare assets within a system. However, if an entity is managing multiple systems in the same area (e.g., the city owns 4 treatment plants that serve its population), the same rating system can and should be used in each of these plants. The entity is likely to want to determine which plant(s) require the most investment and comparing on the same basis will be beneficial. The goal is to determine which of your assets have more consequence if they fail than other assets in your system. For example, a major distribution line that has the potential to cause major failures, collateral social damage, and legal consequences, and which is expensive to repair could be rated “5” while a small valve serving a residential area that has low costs of repair and essentially little to no social or environmental consequence would be given a rating of “1.” In this way, there is a qualitative assessment of which assets have a greater consequence than others, but no specific quantitative assessment is performed. A more robust analysis can be performed that would assign costs of consequences in each category to each asset (or to at least the major assets), thus allowing the system to compare the assets with actual costs of consequences. Some of the costs would be known, such as the cost of a repair, while others would need to be estimated using the best information available, such as the cost of legal action. Past experience or experience from other systems could be used to help estimate costs. A system can start out with a simple estimate of consequence of failure and move to a more robust analysis over time.

In some cases, it may be easier to assess all the assets of a class at the same time for easy comparison during the rating process. For example, all the hydrants could be assessed, then all the pumps, and so on. However, in larger systems, this approach may be more time consuming than assessing the assets based on their geographic location.

Once all assets are assigned a consequence of failure rating, staff should review the results to see if they make sense or if there seem to be anomalies with certain types of assets or specific assets. If necessary, adjustments can be made to the rating structure to ensure it adequately represents the situation.