“To reduce user costs on the priority sections of trunk and regional road networks, to improve the institutional framework for road safety and to introduce innovation to road maintenance practices in Bosnia and Herzegovina" (Financing Agreement Schedule 1, page 6)

Apart from explicitly referring to the types of road network targeted, the objective formulation in the project appraisal document (PAD) was the same: "To reduce user costs on the priority sections of the magistral and regional road networks, to improve the institutional framework for road safety and to modernize road maintenance practices "(PAD, p. 11)

In accordance with current practice, this review assesses the objectives as formulated in the Financing Agreement for this project.

1.Rehabilitation of Selected Roads (estimated cost US$24 million, actual cost US$30.36 million). This component was to finance civil works to rehabilitate and repair 240km of the trunk and regional road network in the two Entities, the Federation and the Republika Srpska. 2. Road Safety (estimated cost US$5 million, actual cost US$ 2.5 million), This component had two subcomponents, one focused on the road safety, which included implementation of a road safety strategy, building capacity and establishment of a monitoring framework for road safety activities; the other subcomponent was to test pilot measures in high-risks sections. Typical interventions included procurement of police equipment, installation of signs and marking, implementation of small civil works, local promotion campaigns and multi-sector investigations following accidents. 3. Technical Assistance and Capacity Building (estimated cost US$1 million, actual cost US$32,000 This component was to finance consulting services to provide technical guidance on introducing output and performance-based maintenance contracts for routine and winter road maintenance in the two Entities and to finance the capacity building of the Road Directorates based on the recommendations of the 2007 Transport Sector Review. Civil works of the maintenance contract was not financed by the project. The consultant services for this specific project were actually financed by the project preparation facility at project preparation stage rather than by the project itself.

The ICR reported three changes to the component activities during project implementation:

1. Component 1: some roads and bridges activities were funded from outside of the project due to the project cost increase;2. Component 2: Pilot road safety activities were scaled down as a result of the reallocation of funds from Component 2 to Component 1;3. Component 3: The pilot of the Output Performance-Based Maintenance Contract was under its first year of implementation in the Federation at project completion, when the equivalent contract for the Republika Srpska had only just been signed. The format of these two contracts was also changed into a hybrid consisting of traditional contract method (inputs type) for routine maintenance and performance-based contract for winter road maintenance;

Related to the above listed changes, the project was restructured twice, on September 16, 2010 and on July 6, 2011. The first restructuring was to reallocate funds from component 2 and 3 to component 1 to allow the completion of the road works under component 1 and the second restructuring was to reallocate funds from component 3 to 2 in the Republika Srpska and to extend project closing date by 6 months from December 31, 2011 to June 30, 2012.

Project cost: The actual cost information acquired by the project team from the implementation agency showed that an actual cost of US$33.16 million, about US$3 million more than the appraised cost. The cost increase was due to higher actual cost of component 1 road rehabilitation activities as a result of higher design standards the works and more funds than envisaged for the major repairs of one bridge in the Federation.

Financing: US$0.15 million was cancelled at project completion. The actual disbursed amount at project completion was US$0.5 million equivalent more than the original Bank financing due to exchange rate fluctuations.

Borrower Contribution:The Borrower contribution committed at project appraisal was US$5 million for the Road Safety component only. The actual contribution was US$ 2.48 million, the change being mainly due to the scaling down of project road safety activities.

Project closing date: The project closing date was extended for six months from December 31, 2011 to June 30, 2012 to allow the completion of some project activities which could not be finished on time as planned due to initial project implementation delays.

3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Rated: Substantial.The project development objectives were relevant to Country and Bank priorities and strategies at the project preparation stage. As recorded in the Project Appraisal Document (PAD p. 9), the Country's Medium Term Development Strategy (2004-2007) emphasized that effective transport infrastructure was essential to expand domestic and foreign trade, increase income, attract foreign investment and create employment. One of the priorities set out in that Strategy was to provide conditions for effective and safe road transport. The project development objectives were also relevant to the Bank's Country Assistance Strategy to Bosnia and Herzegovina (FY05-07), which called for promoting private-sector-led growth and investing in key social and economic infrastructure including the roads infrastructure. The project’s financing prioritized road rehabilitation and introducing innovation in road maintenance would contribute to reducing road user costs and transport time, which in turn would help the private sector become more competitive.

The project objectives are still relevant today. The Country Partnership Strategy (FY12-15) sets within the competitiveness pillar the continuation of investment in regional transport and the promotion of Public Private Partnership (PPP)in infrastructure services. The project objective was relevant in terms of increasing the competitiveness of the country by reducing road user cost, involving private sector in road maintenance and promoting road safety to help private sector become more competitive.

b. Relevance of Design:

Rated: Substantial.The designed project activities would directly contribute to the project development objectives of reducing user costs on the priority sections of trunk and regional road networks, improving the institutional framework for road safety and introducing innovation to road maintenance practices in Bosnia and Herzegovina. The links between implementing the project activities and achieving the project development objectives are listed below : i) The roads and bridges rehabilitation would improve road conditions and consequently lower road user costs on those improved sections; ii) The support for the establishment of a road safety agency, financing the preparation of road strategies and testing of pilot road safety measures in high-risks sections would contribute to the second project objective of improving the institutional framework for road safety;iii) The introduction of output performance based road maintenance contract is an innovation to road maintenance practice for Bosnia and Herzegovina. The benefits of this type of road maintenance contract include lower maintenance costs and higher road service standards overall. Its effectiveness has been demonstrated by the positive experiences of such contracts in many countries worldwide.

Therefore, the result chain from project inputs to outputs then to the achievement of project development objectives was clear and logical.

4. Achievement of Objectives (Efficacy) :

At project completion, the ICR reported the following project outputs:

i) Component 1, 293km of roads were rehabilitated under the project as against the original target of 240km of roads. ii) Component 2, a road safety agency was established in Republika Srpska (the Federation already had one). Road strategies have been prepared, but yet to be adopted by both Entities; pilot road safety measures in high-risk sections were completed yet with delay and reduced scope due to budget and time constraints; iii) Component 3: one hybrid Output Performance Based Road Maintenance contract was introduced in the Federation on a184.66 km road section and was at its first year of implementation; In Republika Srpska, a hybrid contract like this was only signed shortly before project closure.

Outcomes in achieving project development objectives are assessed here separately under each of the three aspects of the project objective:

i) Reduce road user costs on the priority sections of the trunk and regional road networks. The Highway Development and Management Model IV was used to calculate road user costs on project road sections before and after the project. The ICR (data sheet and p. 13) reported that by project completion, road user costs were 18.07% less for the nine road sections in the Federation and 11.22% less in the Republika Srpska than the equivalent costs at the baseline before the project. Both savings exceeded the project’s 10% target. Such falls were due to the improved road conditions resulting from the project interventions. Efficacy is therefore rated as Substantial.

ii) Improve the institutional framework for road safety. The ICR (p. 15) stated thatthe legal and institutional framework for road safety has been improved and the capacity in both entities was also increased. The evidence provided by the ICR and the TTL on the achievement of this objective included:

A road safety agency was established in Republika Srpska,

Road Safety Strategies, Road Safety Program and Road Safety Laws were reviewed and improved to enhance the focus on results, and the revised Road Safety Law was adopted in 2011.

Pilot road safety measures in high-risk sections of road were completed, albeit with delays and with a reduced scope due to the budget and time constraints; A monitoring framework has been set up to understand the effect of the pilot road safety measures;

All road safety databases had been reviewed in order to propose information, indicators, procedures of data collection, storage and dissemination and interconnectivity between various databases.

Efficacy is rated as Substantial.

iii) Introducing innovation to road maintenance practices. The project’s introduction of an output performance based road maintenance contract was, for Bosnia and Herzegovina, an innovation for road maintenance and demonstrated by other countries’ experience to be more efficient than the traditional input type of road maintenance contract. At project completion, a hybrid type of road maintenance contract (a combination of the traditional method (Bill of Quantity) and the performance based method) was in its first year of operation in the Federation. The road users survey and measurement on the roads under the output performance based road maintenance contract suggested that the quality of the road sections improved through the project contracts was perceived by the road users to be better than those roads under the traditional contracts. However, the cost difference was not substantial between these two types of road maintenance contracts for this specific project. The maintenance contract for roads in Republika Srpska was signed right before project completion, therefore, no information could be provided on the effect of the contract. Since there was only limited implementation of output performance based road maintenance contract in one Entity, efficacy is rated as Modest.

5. Efficiency:

The Project Appraisal Document (PAD p. 21) reported that HDM4 model was used for the economic analysis of all the road sections proposed by the Borrower (about 350km) at project appraisal. The appraisal cost benefit analysis estimated the project’s Net Present Value (NPV) of 85 million (2005) Euros and an overall Economic Internal Rate of Return (ERR) of 73%. Out of the 350km, 240km of roads were selected to be financed by the project. However, the ICR reports (p. 6) that these appraisal results probably overstated the net benefits of the project because the cost had been underestimated at the appraisal stage.

At project completion, the same HDM4 model was used for the economic analysis of the293km of roads rehabilitated under the project, based upon the higher actual costs incurred. The average ERR was 59%. Overall, the efficiency of the project is rated as Substantial, given its high economic rate of return and the completion of road rehabilitation within the planned timeframe.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:

Rate Available?

Point Value

Coverage/Scope*

Appraisal:

Yes

73%

80%

ICR estimate:

Yes

59%

91.3%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project development objectives were substantially relevant at project preparation stage and were still relevant at project completion. The project design was relevant given that the results chain from the project inputs to outputs, intermediate outcomes and outcomes was clear and logical. Achievement of the first project development objective was substantial as road user cost (in this case, vehicle operating cost) was reduced by more than the targeted value at project completion, The institutional framework for road safety at project completion was also improved. The introduction of innovation in road maintenance was partially achieved; hence the rating for the achievement of the third project development objective was rated as modest. The project efficiency is substantial given that the road rehabilitation works was completed in a timely manner with high economic rate of return. The overall project outcome is satisfactory.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:

As noted by the ICR there is heightened awareness by the Government of the need to prioritize road maintenance to keep the road network in Bosnia and Herzegovina in good condition. Such awareness is the first step toward sustaining the project development objectives of reducing road users costs on the trunk and regional road networks and improving road safety. However, the country’s road network is possibly faced with inadequate funding for its maintenance, which will not only affect the sustainability of the first two project development objectives but also the achievement of the Output Performance Based Road Maintenance Contract under the project. The lack of maintenance funds is evidenced by the facts that: i) the public expenditure on road maintenance did not increase as the Government committed to due to the economic and financial crisis in the region; ii) even though the Government could have increased access to European Union (EU) resources, there is a shift of focus of EU resourcesaway from the trunk and regional road networks in the country.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

While the project design focused appropriately on improving road conditions, the priority of the road transport sector, some issues at the project preparation stage could have been better addressed: i) Cost estimates at appraisal. The ICR noted that as detailed designs were not available to the appraisal, the cost estimates were based upon preliminary and incomplete designs that, in hindsight, did not take the full costs into account (ICR p. 8). While this practice has not been uncommon with the Bank's projects, the mountainous terrain of Bosnia and Herzegovina means that unforeseen costs often arise, something that could be mitigated by increasing the physical contingencies of the preliminary designs, ii) The financial arrangements for road maintenance: Road maintenance was emphasized at project preparation and a legal covenant was provided to ensure, generically, that adequate funds had to be provided for road maintenance. The project itself could not provide this funding, and offered no specific technical support for how this was to be obtained. iii) The quality of the Project Appraisal Document: As noted by the ICR (p. 6) that the quality of the Project Appraisal Document could have been improved with more detailed descriptions of project activities and more accurate information on project costs and the project procurement plan. It was noted that the procurement plan outlined procurement activities of about US$30 million for component 1 while the Project Appraisal Document reported only US$24 million for the same component. This discrepancy contributed to confusion and delays of the procurement activities later on during project implementation. The project team gave an example that during the implementation, the implementation agency tendered the civil works of the project based on the procurement plan and it turned out that less budget was available for the tendered civil work activities, leading to the initial implementation delays.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:

Eightsupervision missions were carried out for the project, about two supervision missions per fiscal year. The project experienced some initial delays and the Bank's supervision team worked closely with the client to recover the lost time by providing technical support and restructuring the project in a timely manner. Subsequently, the roads rehabilitation work could be completed within the planned timeframe. However, it was noted that the first supervision mission happened 10 months after the project approval and it was within this time period that the lower design standard was applied for the road design in Republika Srpska which furthercontributed to the project implementation delay.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Government was committed to the project by bearing the project preparation cost in both Entities and accepting the project legal covenant of increasing the road maintenance expenditure by five percent annually. However, the covenant was not fulfilled due to the economic and financial crisis that the country experienced. The ministries of finance in both Entities were also active in helping the implementation agencies in resolving the issues during the project implementation. However, it is noted by the ICR (p. 8) that the complex administrative approval process within the Government delayed the project effectiveness by 9 months.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

The implementing agencies for this project were the Federation Road Directorate and the Republika Srpska Road Directorate. Both agencies made good efforts to implement the project activities. After the initial delays of project implementation, the main project activity, road rehabilitation, was completed in a timely manner. However, the road safety component was scaled down due to the time constraint. There were significant initial delays in the submission of project progress reports and project financial audit reports (ICR p. 18) but the late submission issue was fully addressed.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The M&E framework has indicators to measure project implementation progress and achievementof objectives. However, there were some shortcomings. At the output level, for example, there was no indicator to measure the progress of bridge rehabilitation activity. At the intermediate outcome level, the intermediate outcome indicators were a mix of output indicators (i.e, indicator No.2 and No. 4) and intermediate outcome indicators. At the outcome level, the PDO indicators covered all three project development objectives. Indicator No.2 to "establish effective road safety office", however, and indicator No.3 "Complete a satisfactory pilot performance based road maintenance operation" were not so easily measurable given that the criteria of achieving the "effectiveness" and "satisfactory" aspects of these two indicators were not made explicit.

b. M&E Implementation:

Data on the progress on achieving intermediate and final project outcomes were collected and reported through project progress reports. In addition, there were road safety studies to help establish the socioeconomic cost of road accidents. It was noted in the ICR, however, that there were initial delays in submitting these project progress reports (ICR p.18).

a. M&E Utilization:

As reported by the ICR (ICR p. 10), both entities utilized the indicators to report the impacts of their efforts to improve the roads network conditions under the project so as to push for the similar intervention on roads network out of the project scope. The road safety indicators and road safety studies were used forenhacing public awareness of, and political support for the road safety issues.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

Environmental: Environmental category was B. The Bank safeguard policy triggered was OP4.01. At project appraisal, the Environmental Management Plans were prepared by both entities and disclosed. Environmental units with experience of implementing previous Bank's projects were established and their capacity was reviewed by the Bank and found to be sufficient. As reported by the ICR, during project implementation, all identified potential project negative impacts were mitigated, the Environmental Management Plans were fully compiled and there were no formal or informal complaints (ICR p. 9).

Social: The PAD (p.24-25) reported that no Bank social safeguard policies were likely to be triggered, and the ICR (p. 10) reported that none were.

b. Fiduciary Compliance:

Procurement: there was substantial delay in carrying out the procurement tasks in both directorates, as a result of the higher actual cost for the component 1 civil work activities and procurement staff issues in the Republika Srpska Roads Directorate. The low threshold of US$500,000 (instead of US$1 million) for International Competitive Bidding may have potential pressure on the competitiveness of the domestic contractors.

Financial management: As reported by the ICR that the Bank's financial management policy was complied with, though there were initial delays in submitting audited financial reports.

c. Unintended Impacts (positive or negative):

d. Other:

12. Ratings:

ICR

IEG Review

Reason for Disagreement/Comments

Outcome:

Satisfactory

Satisfactory

Risk to Development Outcome:

Moderate

Moderate

Bank Performance:

Satisfactory

Moderately Satisfactory

The project preparation and supervision, as evidenced by the information provided by the ICR, could have been improved.

Borrower Performance:

Satisfactory

Moderately Satisfactory

The borrower's long approval procedure contributed to project implementation delays.

Quality of ICR:

Satisfactory

NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons: The key lessons drawn from the ICR are:1. Project site visits are essential prior to cost estimation of components. Also, the more preliminary the estimates, the higher physical contingencies should be in order to mitigate likely cost overruns; with this project, the bridges to be rehabilitated should have been visited. This could have led to a more correct assessment of needed repairs and better estimation of associated costs.2. To sustain institutional strengthening achievement, it is necessary to have programmatic projects; There was a series of Bank financed projects in this country to help build up the roads management capacity. For example, the road directorates in both FBH and RS have acquired the ability to use the road asset management system developed under the previous Bank financed Road Management and Safety Project in their decision making. A follow-on Bank project to the Road Infrastructure and Safety Project could help built more directly on experiences relating to road safety and the OPBM pilots under the RISP project. 3. Promotional activities/public campaigns are important components of the road safety activities; The road safety campaignscarried out under the project made people more aware of the impact of their actions and the need for behavior change within society;4. Exposure to international good practice can be useful in building local knowledge. For example, in the RS, it helped to bring new perspectives to the Prime Minister’s Office and the Ministry of Finance who now see more than before the need to improve road safety and reduce maintenance costs.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR provided detailed information on project preparation and implementation. It is also frank and transparent aboutshortcomings, such as the project documents quality, the cost under-estimation at project appraisal, theand implementation delays. However, there was discrepancy between the cost information in the ICR and the latest cost information provided by the project team. In addition, the ratings given by the ICR were not always consistent with the information provided. for example, the ICR noted shortcomings with project preparation, yet still gave a "Satisfactory" rating for the Bank's performance of quality at entry.