11.4 Priority Ranking Methods and Economic Assessment

The previous chapter discussed how to identify risk, while the earlier part of this chapter discussed the use of effective interventions to address the risks that have been identified. The next important step is to determine the priority of different treatments. In most situations, there are likely to be financial constraints, which means that not all worthy projects or programmes can be funded. Therefore, a method is needed to identify which projects/programmes should be undertaken as the highest priority. There are also likely to be several options for addressing a risk, and it is necessary to see which of these options will produce the best safety benefit for the cost. Economic appraisals or evaluations provide a comparison basis which can be used for prioritising, comparing and selecting road safety interventions. They help identify measures that yield the highest social return.

At the strategic level, it may also be necessary to establish the relative importance of proactive and reactive measures and decide on the budget proportion that will be allocated to each approach. The guidance provided in this chapter can be used at the strategic, programme or project level.

PIARC (2012) has produced the State of the practice for cost-effectiveness analysis, cost-benefit analysis and resource allocation document, which provides comprehensive advice on methods to appraise projects and allocate resources. This document defines project appraisal as an assessment of the value of a project in order to establish whether the project meets the country’s economic and social objectives. Evaluation approaches include cost-effectiveness analysis (CEA) and cost-benefit analysis (CBA) (also referred to as benefit-cost analysis (BCA)). The outcome indicators from this analysis (BCR, NPV, FYRR and IRR) are discussed in Assessment Criteria in Priority Ranking Methods and Economic Assessments. The following sections provide a summary of some of the key material on these issues.

Cost-effectiveness analysis involves comparing the cost of a proposed countermeasure with the outcome or effect it produces. Within cost-effectiveness, projects are ranked and screened according to their cost and effectiveness in improving road safety or achieving policy objectives. Effects are generally expressed in non-monetary units, e.g. the change in the number of crashes. Cost-effectiveness is mainly applied when comparing alternative projects, programmes and policies with a similar outcome. The cost-effectiveness is expressed as the cost-effectiveness ratio (CER), which is calculated by dividing the number of crashes prevented by the cost of the measure.

Cost-benefit analysis uses monetary values to compare total benefits with total costs of any given policy, programme or project. It is mainly used to determine the worth of an investment based on the total benefits and costs of the investment, and to compare a project with any alternative projects. Cost-benefit analysis is used in road safety economic appraisal to help build a business case and secure funding for different projects or programmes. It enables comparisons between alternative road safety measures, identifying both the cost and the benefits to society as a whole to determine if the project should be undertaken and to establish priorities for approved projects. This, in turn, encourages the efficient allocation of limited resources to competing policies.

Yannis et al. (2008) provide a useful summary of cost-effective infrastructure interventions in an analysis for the Conference of European Directors of Roads (CEDR). They examined 55 road infrastructure investments, including reviews of both the costs and benefits of each. Based on this analysis they identified several best practice examples that should be considered in the efficient planning of investments. The cost-effective intervention options were:

• roadside treatments (clear zones and safety barriers);
• speed limits;
• junctions layout (roundabout, realignment, staggered junctions and channelisation);
• traffic control at junctions (traffic signs and traffic signals);
• traffic calming schemes.

Data Requirements

The key data requirements or parameters for estimating countermeasure benefits and costs are as follows:

Initial cost

Initial costs refer to implementation costs (e.g. installation, material and labour costs) for each countermeasure. The costs differ by road environment type, traffic volumes, local environment, local labour costs, and availability of materials. There is greater uncertainty surrounding implementation costs in most LMICs as the information is not readily available. The Road Safety Toolkit (http://toolkit.irap.org) outlines general cost levels for different countermeasures. These values can be used as indicative measures where the treatments have not been implemented before or in cases where the cost information is not readily available.

Annual maintenance and operating costs

Annual maintenance and operating costs refer to routine and periodic maintenance costs and running costs. The level and regularity of maintenance and associated running costs depend on the countermeasure.

Terminal salvage value

Some countermeasures may have a residual value if they are removed. For example, an intersection may be temporarily equipped with traffic signals for a number of years until a by-pass is completed, and after completion, lower subsequent traffic flows may warrant the removal of the traffic signals. If the countermeasure asset can be used elsewhere, the recovery of this cost should be taken into account. However, in most cases, any residual value is likely to be negligible.

Service/treatment life

A countermeasure’s service life refers to the time period over which a treatment will deliver safety benefits before major rehabilitation or replacement is required. It varies with:

• the type and scope of the project;
• climate conditions (which may cause infrastructure to deteriorate);
• traffic volumes (higher volumes might cause infrastructure to deteriorate, while growth in volumes may result in the need for different infrastructure) ;
• local standards (more stringent construction requirements may result in slower deterioration);
• resource and materials availability (certain materials are more resistant to deterioration);
• regularity of maintenance.

For projects involving multiple treatments, e.g. network or national blackspot programmes, the service life applied is that of the longest-lived component. Table 11.2 gives example maximum treatment lives for different countermeasures. Given the issues listed above, this is likely to vary substantially for individual projects. As an example, in the US, the treatment life for line-marking is expected to be one year, especially in States that experience snow and ice conditions.

Source: Adapted from Turner & Comport (2010).

Estimate of resulting crash changes

The main benefits of road safety projects are expressed in terms of monetary savings from crash reductions or prevention of casualties (fatalities and injuries) over a given number of years.

Treatment effectiveness can be expressed as crash modification factors (CMFs). Several comprehensive resources that provide CMFs for different interventions are provided in Intervention Option and Selection, including the CMF Clearinghouse database (http://www.cmfclearinghouse.org/) and the Road Safety Toolkit (toolkit.irap.org). As also discussed previously, the effectiveness and magnitude of crash changes can vary according to their context/environment.

In cases where several treatments are applied at the same location (multiple countermeasures), estimates of overall benefits need to be made. Some approaches have only included the crash savings from the primary or main treatment, but it is preferable that total benefits be calculated. Care needs to be taken to ensure that benefits are not counted more than once for interventions that improve safety in similar ways. For example, to address crashes at a curve, interventions such as advanced warning signs, audio-tactile edgelines, and improved road surface friction may be applied. The total benefit of these treatments will not equal the sum of benefits for each treatment, as each is alike in terms of its effect on crashes. For situations where treatments are linked, an adjustment needs to be made. Although several complex approaches have been devised to calculate the total benefit from multiple treatments, the simple approach outlined by Shen et al. (2004) will usually suffice. They suggest a multiplicative formula similar in form to that shown below:

As an example, if three countermeasures are being considered in one location, with CMFs of 0.6, 0.75 and 0.8, the results would be as follows:

CMFt = 0.6 x 0.75 x 0.8

= 0.36, or 36% of crashes will remain (i.e. a 64% reduction in crashes).

A 64% reduction in casualties is obviously less than the 85% reduction that would be calculated if each reduction was added together.

Roberts and Turner (2007) were able to compare safety benefits at locations where packages of treatments were used, with locations where the same treatments were used but as single treatments. By applying the above formula, they identified that this approach tended to overestimate the true benefit of treatments. They suggested the results be multiplied by 0.66 to provide a more conservative approach (for the above example, this would produce a result of 42% reduction).

For a detailed discussion on the effectiveness of multiple treatment projects, see AASHTO (2010), iRAP (2013), and Elvik (2007).

Monetary values for different categories of road crashes

The benefits resulting over time from safety countermeasures are estimated by placing an economic value on crashes, and applying this to the expected reduction in crashes. Values should not be derived on a project-by-project basis, but should be set at the national level by transport economists and updated annually.

This economic value, referred to as the social cost of crashes, is the value of property damage caused by vehicle crashes, medical and ambulance costs, insurance and administration costs, loss of output costs, police costs, and human costs associated with the pain and suffering caused by death and injury. The different cost components are outlined in Table 11.3.

Source: PIARC (2012).

There have been many projects and considerable debate about the best way to determine crash costs (Hills & Jones-Lee, 1983; Alfaro et al., 1994; Jacobs, 1995), but it is now generally accepted that only two methods should be considered – the willingness-to-pay (WTP) approach and the human capital (HC) approach. These approaches are summarised in Table 11.4.

Source: Based on Hills and Jones-Lee (1983).

For a detailed description and discussion of the HC and WTP approaches, see PIARC (2013), HEATCO (2006), Transport Research Laboratory (1995), and Asian Development Bank (2005). Although both approaches are widely used, the willingness-to-pay (WTP) assessment method is generally recommended (DaCoTA, 2012; McMahon & Dahdah, 2008).

Costs must be determined for crashes of varying levels of severity, usually fatal, serious, slight or minor, and property damage only. These severity levels have been defined in Effective Management And Use Of Safety Data where it was indicated that a fatal crash involved a situation where at least one person died within 30 days of a crash; while a serious injury crash involved at least one person admitted to hospital, but there were no deaths. A minor crash indicates that at one person received some form of injury, but no one was killed or seriously injured.

For the purpose of prioritising actions aimed at reducing crash frequency, a single average cost for all injury crashes is generally considered sufficient, particularly since it is difficult to predict the specific severities of accidents that might be prevented.

Costs are always based on average values, and in some countries, are also determined for broad road categories (e.g. urban, rural, freeway). The social cost of crashes provides an estimate of the economic burden that different crash and injury types place on the economy. For illustrative purposes, an example of costs by road category and accident severity for the UK in 2012 is shown in Table 11.5.

It can be seen that costs increase from built-up roads to non-built-up roads to freeways, indicating the effect of greater speeds on crash severity levels. It can also be seen that there are approximately ten-fold increases in costs between severity levels. That is, the cost of a slight crash is about ten times that of a damage only crash, a serious crash is about ten times that of a slight crash, and a fatal crash is about ten times that of a serious crash.

Source: Adapted from Department for Transport (2013).

Calculation of crash costs is generally undertaken at the country level, and development of an accurate figure can be a complex process regardless of which method is used. If no figure is available at the country level, a simple method for obtaining the value of crashes, especially in the absence of the data required for the HC and WTP approaches, is the iRAP ‘rule of thumb’ (McMahon & Dahdah, 2008). This method uses the information from countries that have already carried out WTP calculations, and analyses the relationship between the value of statistical life (VSL) and gross domestic product (GDP) per capita.

The assumptions are (McMahon & Dahdah, 2008 Table 10):

• The value of statistical life is 60–80 times the GDP per capita (current price) for the country concerned.
• Ten serious injuries occur for every fatality.
• The value of serious injury is equal to 25% of the value of statistical life.

The approach was originally developed using WTP values from a limited number of LMICs. These values were recently updated using a larger dataset (Milligan et al., 2014). The update showed that the rule of thumb tends to underestimate VSL for countries with GDP per capita above $7000 (Milligan et al., 2014).

Generation of crash costs can be a significant issue in LMICs, even with the availability of crash cost estimates, or when using the ‘rule of thumb’. Due to low GDP per capita in many countries, the crash costs can be low, while the cost for installing engineering treatments can remain high. The example (Box 11.5) below illustrates this issue using an example from Papua New Guinea.

Box 11.2: Potential crash reductions and savings from improved road maintenance in Papua New Guinea

Alternatively, the value of different injury severities can be derived using quality adjusted life years (QALYs) and disability adjusted life years (DALYs). QALYs measure the value of a fatality prevented, taking into account the quantity and quality of life. They place a weight of one for a year of perfect health and zero for death. DALYs on the other hand, measure the quality of life lost or loss of life years due to illness or injury. They account for the burden of injury or illness and can also be used to measure property damage. DALYs and QALYs are widely used in health economics and very rarely in road safety. An example application of DALYs and QALYs in Colombia road safety is outlined in Box 11.6.

Box 11.3: Estimating the cost of crashes in Columbia

Discount rate used for schemes

In any economic road project assessment, it is important to identify a base year from which all future costs and benefits can be assessed. This is because the value of a dollar received in the future is less than the value of a dollar now (also referred to as the ‘time value of money’). The discount rate is used to compare benefits and costs received at different points in time over a project’s treatment life, converting future benefits and costs to present values.

The choice of discount rate can have significant effects on the desirability and selection of projects, especially where benefits and costs accrue later in the treatment’s life. A higher discount rate reduces the value of benefits and costs occurring later in the treatment’s life, favouring projects where benefits occur early in the project. The World Bank recommends present value calculations at 12% discount rates (2014 values) be included in road project business case submissions (see PIARC, 2012; AASHTO, 2010). It is important to note, however, that this value is not necessarily relevant for every country, and the discount rate actually used can be significantly different. For instance, the discount rate is close to 5% in several western European countries.

Assessment Criteria

As indicated above, the standard approach for the ranking of treatments is to carry out a cost-benefit analysis, i.e. to compare the estimated benefits of each scheme (in terms of the value of crashes that will be prevented) in relation to its costs (implementation, maintenance, etc.). The treatments are then prioritised in accordance with the best economic returns.

As previously mentioned, estimating likely crash reductions resulting from remedial work is often difficult, because it can only be based on previous experience with similar schemes (Turner & Hall, 1994; Kulmala, 1994; Mackie, 1997).

The selection of countermeasure options is based on the first year rate of return (FYRR), the internal rate of return (IRR), the benefit-cost ratio (BCR), and the incremental benefit-cost ratio (IBCR), as well as net present value (NPV). However, the two main indicators in assessing a project or treatment are the BCR and the NPV. These measures indicate whether the benefits of the proposed treatment outweigh the costs, and if the preferred treatment has the greatest net social benefit.

First year rate of return (FYRR)

This is simply the net monetary value of savings and drawbacks anticipated in the first year of the scheme, expressed as a percentage of the total capital cost.

Note that the last two elements might be considered to be small, particularly for low-cost schemes, and are often ignored.

This is not a rigorous evaluation criterion for prioritisation since it ignores any benefits or changes in maintenance costs after the first year. However, it is very simple to calculate, and given that road safety engineering schemes often produce first year rates of return in excess of 100%, more sophisticated decision criteria may not be necessary. This method usually yields high values with low-cost schemes but with relatively small crash savings, and for this reason it is less consistent with the Safe System approach.

The FYRR can also be used to assess the timing of a particular project by comparing it with the discount rate. If the FYRR is greater than the discount rate, the project can, in theory, proceed. This says nothing, however, of how it compares with other projects. If the FYRR is less than the discount rate, the project should, at the very least, be postponed.

More detailed assessments will be needed for schemes where crashes and traffic levels are expected to change substantially from year to year. For example, a scheme with an 80% FYRR may not be worthwhile if subsequent road closures due to the construction of a new road limit the benefit to just one year.

Internal rate of return (IRR)

Another important criteria used for assessing costs and benefits of highway schemes is the internal rate of return (IRR). This is the discount rate that makes the NPV equal to zero or makes the BCR equal to one. A theoretical example of how the discount rate affects the NPV of a project is shown in Figure 11.6

.

At discount rates of 8% or 10%, the project has a positive NPV, while it is negative at 12% or 14%. The NPV is zero at 11% discount rate, which is known as the internal rate of return (IRR). The IRR is preferred by multilateral aid agencies, such as the World Bank, because it avoids the use of local discount rates which, depending on their value, can significantly affect the NPV or NPV/PVC ratio. The IRR is not particularly useful for ranking projects, but is included for comprehensiveness.

Benefit-cost ratio (BCR) and incremental benefit-cost ratio (IBCR)

Benefit cost ratio (BCR) is defined as the present value of benefits (PVB) divided by the present value of costs (PVC):

When the NPV of any given project is positive, the BCR is greater than one. The greater the BCR, the higher the benefits are. The BCR is used to rank projects where there is a budget constraint, and it serves as an indicator of a project’s economic efficiency.

The IBCR involves ranking a pairwise comparison of all alternatives with a BCR greater than one in order to determine the marginal benefit obtained for a marginal increment in cost. Then, after eliminating all schemes with a BCR of less than one, the schemes are listed in order of ascending cost and the marginal BCR is determined by a pairwise comparison of alternatives, starting with the lowest and second-lowest cost alternatives. That is:

If the IBCR is greater than one, the alternative x+1 is preferred, since the marginal benefit is greater than the marginal cost. Conversely, if the IBCR is less than one, alternative x is preferred. The preferred option is then taken and the pairwise comparison is continued until only a single alternative remains, which should then be the most economically desirable of all the options considered.

However, Ogden (1996) concludes that the BCR approach is more cumbersome to use than the NPV approach and may produce more ambiguous and misleading results depending on how benefits and costs are defined. It is of particular note that low-cost measures are typically favoured when using BCR as the basis for selection. For example, installing advanced warning signs are likely to have a limited (but beneficial) effect on severe crash outcomes, but due to the low cost of installation, they are likely to produce a high BCR. In contrast, roadside barriers are likely (in the right situation) to have a significant effect on reducing fatal and serious crash outcomes. However, given the greater cost for installation and maintenance, the BCR is likely to be lower. The aim of road safety is to produce a net reduction in fatal and serious injury. Using solely the BCR approach may produce outcomes that are inconsistent with this objective. Therefore the NPV/PVC approach in association with BCR is much preferred.

Net present value (NPV)

This type of evaluation expresses the difference between discounted costs and benefits of a scheme, which may extend over a number of years. As stated earlier, future benefits must be adjusted or discounted before being summed to obtain a present value. Changes may also take place over the life of the scheme which will affect benefits in future years.

Let us assume (for ease of calculation) that the current rate used by the government for highway schemes is 10%, which in the prevailing economic climate might be considered as somewhat high in most countries. This means that $100 in benefits accruing this year will be worth 10% less if it accrues next year. A further year's delay will reduce the benefit again, and so on. These figures can be summed over the life of the scheme to obtain the present value of benefits (PVB).

Net present value is defined as the difference between the discounted monetary value of all the benefits and costs of a particular project or measure. The NPV is expressed as the PVB minus the PVC. A positive NPV indicates an improvement in economic efficiency compared with the base case.

With respect to implementation priorities, the economic criteria for scheme assessment using the NPV approach are:

• all schemes with a positive NPV are worthwhile in economic terms
• for a particular site, the most worthwhile option is the one with the highest NPV.

Care needs to be taken in using NPV as the only investment criterion, since it tends to indicate projects with higher costs.

Recommended approach to prioritisation

The choice of assessment criteria depends primarily on available data, as well as the scope of the treatment. The different assessment criteria provide information on the project. The NPV provides information on the total welfare gain over a project’s treatment life; the BCR highlights the relationship between the present value benefits and implementation costs of a project; while the IRR shows the rates at which benefits are realised after investing in a countermeasure (PIARC, 2012).

The NPV is the preferred criterion as it provides an estimate of the absolute size of the treatment’s net social benefit. The BCR on the other hand provides the relative size of the costs and benefits of a treatment and depends on the classification of the project’s impacts. Table 11.6 provides guidance on when to use the different criteria.

Source: Austroads (2005).

For a comprehensive step-by-step approach on economic appraisals or evaluation, as well as a summarised discussion of the assessment criteria, see PIARC (2012), EU (2006), and HEATCO (2006). Box 11.7 provides an example economic evaluation from Belize.

Box 11.4: Belize  – economic evaluation

Economic Appraisal Tools

A variety of tools assist with the economic appraisal process in road safety. Some examples are provided below.

SafetyAnalyst

SafetyAnalyst (also discussion in Management Tools) includes an economic appraisal tool developed by the American Association of State Highway and Transportation Officials (AASHTO; see Harwood et al., 2010). It assesses whether countermeasures for a specific highway site are economically efficient. The tool allows the user to specify the costs, traffic volumes, and all other data inputs. It also provides default values for the specified treatment, which can be used in the estimation. The user can specify the economic appraisal to be performed, with options for cost-effectiveness analysis, benefit-cost ratio or net present value. The effectiveness measures are obtained from observed, expected and predicted crash patterns at the specific site. The countermeasures and sites are ranked using the Priority Ranking Tool component of SafetyAnalyst, which uses the same measures obtained by the economic appraisal tool and also provides the optimal option given budget constraints. The choice of prioritisation criteria lies with the user.

COBALT (cost and benefit to accidents – light touch)

COBALT is an economic appraisal tool developed in 2012 by the UK Department for Transport. It was derived from the broader transport appraisal tool, COBA (Cost Benefit Analysis tool). COBALT focuses solely on road safety appraisals using the same CBA approach as COBA as outlined in DfT (2011).

Economic evaluation manual, New Zealand

While New Zealand does not have a dedicated road safety economic appraisal tool, the Economic evaluation manual (EEM) (New Zealand Transport Agency 2013) provides clear guidance and templates that can be used in the evaluation process. The EEM is a guidance tool outlining procedures for economic evaluations of transport investment proposals. It provides descriptions of basic concepts in economic evaluations and simple and detailed procedures for evaluations. The simple procedures are aimed at low cost activities while the detailed procedures are for large scale evaluations. Step by step methodologies for evaluating benefits and costs are also provided through downloadable spreadsheets.

There are different spreadsheets for different evaluations. The road safety promotion spreadsheet contains six procedural worksheets and four other worksheets for working notes, cost estimates and sensitivity analysis. Worksheet 1 is a summary of general project information and data used for the evaluation. Worksheet 2 is used for calculating the present value of project costs, worksheet 3 is used for calculating the social cost of crashes per person and worksheet 4 is used for calculating the present value of project benefits. Worksheet 5 is used to calculate the BCR per head. The cover worksheet is a summary of all the information and calculations in the spreadsheet. For each of the steps, there is guidance offered on the necessary information and input data.

European guidance on economic appraisals and prioritisation of road safety countermeasures is also available to practitioners. Examples of this include European Road Safety Observatory (ERSO- http://www.erso.eu/), Developing Harmonised European Approaches for Transport Costing and Project Assessment (HEATCO- http://heatco.ier.uni-stuttgart.de/), and Road Safety and Environmental Benefit-Cost and Cost-Effectiveness Analysis for Use in Decision-Making (ROSEBUD, http://partnet.vtt.fi/rosebud/).

The process applied by iRAP (Section 10.4.4) not only identifies problems and effective interventions, it also produces detailed business plans, including the cost-effectiveness of the interventions identified. An example from the Ukraine of one such investment plan is provided in Figure 11.7

Pathway to Effective Intervention Selection and Prioritisation