The project development objective is stated identically in the project Appraisal Document and the Financing Agreement as:

"to facilitate infrastructure investments that have strategic importance and/or special complexity by ensuring that proper feasibility work is carried out in a timely manner in order to enable the government to make sound investment decisions, and where appropriate, to proceed expeditiously with further preparatory steps and mobilize adequate financing for implementation."

Component 1: Technical Assistance (TA) for hydropower project preparation (cost at appraisal: US$3.30 million; at completion: US$3.86 million). TA for Khudoni Hydropower project (HPP) preparation, to assess its technical, economic and financial viability, and to determine how the hydropower project development would proceed in compliance with appropriate environmental and social safeguards, prioritizing on Dam safety. Should the Khudoni HPP prove to be economically attractive (and otherwise feasible), financial analyses would be undertaken, and issues addressed concerning ownership and power sales.

Component 2: TA for Preparation of Transit Corridor projects (cost at appraisal: US$2.60 million; at completion: US$1.50 million): TA for the preparation of transit corridor projects entailed two subcomponents:a) TA for the preparation of the East- West Highway improvement, sought to complete a feasibility study, preliminary design, and environmental assessment to upgrade the road from Natakhtari to Agaiani; b) TA for the preparation of other transit corridor projects was to be determined, depending on the availability of remaining funds, following satisfactory results of the above mentioned priority project components. projects' selection criteria would focus on strategic importance, complexity, and merit.

Under component 1, the Khudoni Hydropower project (HPP) study had three phases (and an optional fourth): Phase 1 studied the original dam design; Phase 2 developed and assessed alternative dam and hydropower concepts; Phase 3 studied the feasibility of the selected alternative, and also included an overview study of potential export markets for Khudoni power and of project financing options. Under a possible Phase 4 consultants were to develop detailed designs and specifications, undertake a prequalification process, develop bidding documents for dam and power plant construction, and assist during the bidding process, provided that the Government decided to proceed with the Khudoni investment project. At the time when conclusions from Phase 3 were available, the government decided not to continue with Phase 4 within the time frame of this project, as according to the Ministry the private investors would undertake detailed design and there was no need in duplicating the effort by funding Phase 4.

Under component 2, sub-component b) was not realized. The project Appraisal Document explicitly envisaged cancellation of this sub-component 2b if no savings resulted from other components. Close to mid-term of project implementation, no project savings were expected, so this sub-component was canceled the remaining funds were transferred to geological surveys under Component 1,

Project costs, financing, and borrower contribution: The project used US$3.86 million for component 1 (Khudoni HPP), almost 25% more than the planned US$3.30 million. Component 2 which covered transit corridor projects used only US$1.5 million, about 40% less than expected at appraisal. This was because sub-component 2b - that was intended to cover 'other' transit corridor projects - was dropped, and the unused funds were transferred to the Khudoni HPP component. For the entire project the cost at completion was US$5.36 million, which was about 12% short of the appraised amount of US$5.90 million which is partly explained by the Ministry of Energy's decision in late 2009 - well into the last year of the project – not to undertake a drilling program in the Khudoni dam area (that would have better established the geological/geophysical situation) due to (a) the need to mobilize additional funds outside the project, and (b) because the Ministry felt that future investors could finance the activity. Unused funds of about SDR 560,000 equivalent was ultimately returned to IDA. The borrower's actual contribution was US$0.38 million, less than half of the planned US$0.90 million.

Dates: The project was initially extended by 12 months till September 30, 2010 because of delays in implementation of component 1. The closing date was extended for a further four months – until January 31, 2011 – to conclude pending contractual issues with the Engineering Consultant and to incorporate the comments of the International Panel of Experts into the final feasibility report.

3. Relevance of Objectives & Design:

a. Relevance of Objectives:

In the years after exiting the centrally planned Soviet economy, Georgia suffered from continued lack of infrastructure investment. By 1999, the power and road infrastructure had deteriorated to the point of near-collapse; In 2000, the power sector could deliver electricity for only a few hours a day and parts of the country had not received electricity for months. Georgia’s highways were also in poor condition, which lengthened transit times, constrained the movement of goods and people, and raised costs for road users. In 2003, the Rose Revolution brought in a new government that was committed to addressing infrastructure problems as part of wider economic growth, and initiated steps for addressing power tariff reform, power sector legacy debts, corruption and power theft, and power sector privatization efforts.

In this context, the project development objective remains highly relevant to the government’s priority of developing its hydro power potential to achieve energy security, as expressed in the country’s 2004 energy sector Strategy and the “Main Direction of the Energy Sector” adopted by the parliament on June 7, 2006. Upgrading of the East- West Highway remains a national priority. Power and road infrastructure investment was seen to contribute to the goals of the Bank’s Country Partnership Strategy FY2006-09. The project development objective is consistent with the current Country Partnership Strategy FY2010-13: one of its pillars is restoring growth and competitiveness; and reliable and efficient power supply.

Relevance of project objectives is rated high.

b. Relevance of Design:

The results framework was clear and logical. The project financed technical assistance to assess investment feasibility and effectiveness of the Khudoni Hydropower project and the East-West transit corridor projects through technical and sector studies, engineering design, economic, financial, and environmental feasibility analysis; and by providing financial and legal advisory services where required. The project components were well defined and matched the project objective. Project design also envisaged a possible sub-component 2b for transit corridors beyond the East-West Highway corridor, and was kept flexible without jeopardizing the main project objective. Relevance of project design is rated high.

4. Achievement of Objectives (Efficacy) :

To facilitate infrastructure investments that have strategic importance and/or special complexity by ensuring that proper feasibility work is carried out in a timely manner in order to enable the government to make sound investment decisions, and where appropriate, to proceed expeditiously with further preparatory steps and mobilize adequate financing for implementation. Rated Substantial.

OutputsKhudoni HPP: The project helped prepare quality studies including detailed designs and economic and social safeguards arrangements, which would save prospective investors years of investment plans going forward. The international Panel of Experts recognized the feasibility of the project based on the techno-economic studies, and the international environmental/social Panel of Experts accepted that the safeguard (framework) studies undertaken were adequate for this stage of the project, although they had a series of critical comments on the safeguard studies that are made in the panel’s last report and which thus constitutes an important supplemental report for future investors in the hydropower project.

As planned, Khudoni HPP would be the second largest hydropower station in Georgia with an installed capacity of 700 MW. Khudoni HPP is located upstream of hydropower station Enguri, with an installed capacity of 1300 MW. Khudoni would have significant positive impact on the operation of Enguri. Khudoni HPP construction was halted in 1986 due to concerns about safety of the chosen design, and environmental impact. Also, the government's financial constraints of the government over two decades made it impossible to continue construction. A Strategic Environmental Assessment for the power sector was also undertaken during project implementation. Though this was not part of the original project design, it was considered important for improving the understanding and credibility of the Khudoni PPP, when being considered among alternative electricity supply-enhancing or demand-reducing projects,

OutcomesKhudoni HPP: At project completion, based on the studies prepared by the project, a Memorandum of Understanding (MOU) for project development was signed with an investor consortium. However, work on the Khudoni HPP has not yet commenced.

East West Highway sections: Investments in the East-West Highway, based on feasibility studies and designs, proceeded expeditiously. The East-West Highway studies financed under the project were utilized as inputs for the Bank- funded road investment projects. The ICR states that there were weaknesses in the design for Igoeti-Sveneti section. The preliminary design and environmental impact assessment for the Agaiani-Sveneti road section and the detailed design for Agaiani-Igoeti were used for the First East-West Highway project (IDA-42480). Detailed design, environmental management plans, resettlement action plans were used for the Bank-funded Second East-West Highway Improvement project (IDA-43730). In December 2006, the First East-West Highway Improvement project (FEWHIP) was approved and financed upgrading of the Agaiani-Igoeti section. In 2009, Additional Financing was approved to rehabilitate the Rikoti tunnel, currently under implementation. In December 2007, the Second East-West Highway Improvement project (SEWHIP) was approved; most of the works are completed and the project closed in June 2012. As a result, 37 km of this critical Georgian transport corridor have been rehabilitated and upgraded from 2 lanes to 4 lanes.

5. Efficiency:

The project did not estimate an economic rate of return, which is not mandated for technical assistance projects and is justified in this case given that the project development outcomes were not amenable to quantification. However, the ICR does not provide any other discussion on efficiency, or a benchmark for judging cost-effectiveness.

As discussed under "Costs" (section 5), the final project cost was 12% lower than the estimated cost, partly because of component 2b being dropped. The time overrun was 16 months, which was about 35% more than the originally envisaged 43 months. a significant portion of this delay can be attributed to procurement process and delays that could have been better managed (see section 9a). Overall, efficiency is rated modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:

Rate Available?

Point Value

Coverage/Scope*

Appraisal:

%

%

ICR estimate:

%

%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of project objective and design are both rated high. Efficacy rated substantial overall given the design quality issues experienced during the East-West Highway segments' implementation. Efficiency is rated modest due to the delays in implementation of the Khudoni HPP studies, which were at least partly under the control of the government. Overall outcome is rated as Moderately Satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

As discussed in the section on Efficacy (section 4), the East-West Highway studies financed under the project were utilized as inputs for the Bank-funded First and Second East-West Highway Improvement projects, of which the latter has been completed.

As for the Khudoni HPP, at project completion, a private investment consortium had signed a Memorandum of Understanding for investing in the Khudoni hydropower plant. The ICR states that it is possible that the construction decision for Khudoni could be pushed into a distant future as it is a relatively high-risk undertaking because of its complexity – technically, financially, environmentally and socially, as is often the case with large-scale hydropower projects.

The size of the completed Khudoni HPP will have a substantial effect on the national and regional energy balance. Therefore, due to complexity and size, the construction carries considerable risk, which is among the reasons why cascade developments like Namakhvani HPP, where investments can be broken into smaller segments and spread over time, recently have been perceived as more attractive for investors. An added complexity for Khudoni is its location upstream of the Enguri dam. The Enguri HPP has its reservoir in Georgia proper and the powerhouse and control room located in the conflict region of Abkhazia. The two power plants would need to coordinate their operations in order to enhance the economic results for Enguri, and also for dam safety reasons for Enguri.

On the basis of the above information, the risk to development outcome is rated significant, especially due to the lack of progress on the implementation of Khudoni PPP project.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:

At project preparation, the Bank appropriately took the following factors into consideration: adequate government commitment; comprehensive terms of reference (TORs) for the studies; availability of competent consultants; detailed review of consultant outputs; and wide consultations with stakeholders and donors. The choice of lending instrument was appropriate given the government’s urgency to move ahead with selected investments and their lack of funding to prepare necessary studies. IDA had been reviewing potential construction for the Khudoni hydropower station as part of formulating a power sector strategy since it was pre-assessed as the least-cost sizable power development prospect in Georgia; and IDA was active in the road sector and engaged in sector dialogue. To ensure efficient project implementation, the financial management team reviewed Georgia’s public financial management system and evaluated its suitability for implementing IDA-funded projects.

The ICR states that government commitment was displayed by placing the responsibility for project management at the level of Deputy Ministers in the Ministry of Energy (for component 1) and the Roads Department at the Ministry of Economy (for component 2). At the same time, the ICR notes that the Deputy ministers typically have many demands on their time, adding to the risk of slow decision-making, which was realized in the case of the component 1 (Khudoni HPP). In retrospect, the project missed identifying the risk associated with the proximity of the Khudoni project site to an area outside central government control, and the site of potential military activities.

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:

The World Bank conducted extensive project supervision, involving two infrastructure sub-sectors — energy and transport; seven supervision missions included both Bank staff and external consultants from the two sub-sectors. In addition, the project road sub-sector component was supervised in parallel with the current road portfolio. The task management and supervision was partly decentralized to the country office, which facilitated continuous project supervision and ensured high-quality and responsive Bank inputs. On several occasions supervision teams were reinforced by senior Bank staff responsible for Bank-wide oversight over dam safety and environmental and social safeguards, which heightened supervision quality and ensured compliance with environmental and social safeguards. In addition, the complexity of the proposed Khudoni hydropower project encouraged the Bank to undertake extra supervisory precautions. During project supervision, the Bank was actively involved in the substance of the Khudoni study and utilized its considerable expertise in hydropower construction for the benefit of the project. In hindsight, more time and resources should have been allocated to engage additional experts to produce the studies and for a thorough review of all outputs, and to strengthen the capacity of the Roads Department in monitoring the quality of the outputs.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Ministry of Energy and the Ministry of Economic Development were the Bank’s primary counterparts for implementation.

The engineering consultant initially faced delays in obtaining necessary data for the Khudoni HPP as the data was being held by the newly privatized hydropower project entity. The Ministry of Energy, which was technically the owner of the data, could have moved faster in making the data available to the engineering consultant.

The project was extended by one year in the expectation that it would help to the Ministry of Energy to seek additional funds from other sources to bolster Component 1, especially to execute Phase 4 for the Khudoni project (i.e. developing detailed dam and powerhouse designs and readying procurement documents for the subsequent investment phase). This would benefit Phase 3 (feasibility study) in that more precise estimates could have been developed for the dam construction based on better known geology. However, as Khudoni’s position in the priority list of large hydropower projects slipped, the Ministry of Energy decided to postpone the drilling and investigation program, and entrust it to a future investor. Delays in making this decision impacted the overall completion schedule.

For Khudoni HPP (component 1) the government assembled a panel of local engineers with prior knowledge of the Khudoni project to work in parallel with an international Panel of Experts mandated by Bank rules. Both panels provided comments to the reports, which provided comfort to the government and improved project ownership. The government led the public consultations on the environmental and social impact assessments for Khudoni, and the Bank attended all the consultations. Bank staff held supplemental public meetings with project-affected people. Consultations on environmental and social impact assessment for the highway were later done under the subsequent First and Second East-West Highway Improvement projects. The limited timeframe didn’t allow for a thorough review of the road designs and some weaknesses of the design of one section Igoeti-Sveneti were identified later during construction.

Despite some contractual issues with the Engineering Consultant took an excessively long time to resolve the overall Ministry of Energy performance was satisfactory. The Roads Department at the Ministry of Economic Development participated in implementing road-related project activities from the technical side. The ICR notes that the high stress level inflicted on staff and contractors for accelerated delivery and limited time frame and budget allocated for preparation of the studies impacted the quality of outputs.

To expedite project implementation, the expression of Interest for selecting engineers for the Khudoni studies was issued prior to project negotiations. However, engineer selection was delayed because, until project effectiveness, the government could use only local procurement procedures, which were inconsistent with the Bank procedures. The seven-person procurement evaluation committee for the Khudoni studies was too large, slowing the process. A evaluation committee, created by Presidential Decree, reviewed technical assistance proposals for Khudoni. The committee included five deputy ministers who had limited time to review proposals, making the process inefficient. During project implementation, the Bank requested creation of an expert working group to assist the committee with evaluation. The Minister of Energy created such a group, which speeded up procurement.

High-level Government attention to preparing priority road investment projects on East-West highway contributed to speedy procurement and implementation of the studies under Component 2A, effectively completed by end-2007.

Government and the Bank involved the Ministry of Culture and Ministry of Environment in the feasibility study review for the Khudoni HPP.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

The Ministry of Energy was responsible for the Khudoni study (Component 1) and the Ministry of Economic Development (MoED) was responsible for implementing the transport corridor studies (Component 2), with an MoED subsidiary, the Roads Department, managing the technical side of implementation.

The financial management function was split with the project Service Organization (PSO) and the Transport Reform and Rehabilitation Center (TRRC) being responsible for components 1 and 2 respectively. Both agencies had successful project implementation track records. In addition, TRRC was in charge of the ongoing road portfolio incorporating four projects and two additional financing operations. (PSO) had a track record of satisfactorily implementing previous Bank-funded operations (Power Rehabilitation project, Electricity Market Support project) and TRRC had previously implemented the Bank’s road investment projects. The ICR reports that the financial management and procurement performance of PSO and TRRC was satisfactory.

Pressure to speed up project implementation in the road sector stretched existing capacity and reduced attention to quality. Two investment projects were prepared based on the studies funded under the project. However, the intense schedule and high pressure to deliver strained the capacity of counterpart, the companies hired to prepare road designs, and the Bank team. The design companies were given a limited budget and a insufficient time to prepare designs. Subsequently, some design weaknesses became apparent during construction on the Igoeti-Sveneti section under the Second East-West Highway Improvement Project (SEWHIP). These issues and costs overruns experienced during SEWHIP primarily stemmed from the failure of the supervising engineer company to identify and address any weaknesses and issues in the designs early on in the project implementation.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

Project monitoring and evaluation was straightforward; the project envisaged technical assistance to develop studies for sound investment decisions, and achievement of the development objective was measured by delivery of those studies with adequate quality. The original project development objective indicator was formulated as: “sound investment decisions are made, and where appropriate further preparatory steps are taken and adequate financing is mobilized for investments. The indicator was revised as follows: "studies required to lead to sound investment decisions in the following areas are carried out promptly, efficiently, and are of high quality" (1) Khudoni Hydro Power Plant (HPP); (2) East-West Highway; and (3) other investment decisions (subject to availability of funds. Component 2b was cancelled for lack of funds and it was dropped from the set of indicators. The Ministry of Energy and the Ministry of Economy were responsible for monitoring the indicators.

b. M&E Implementation:

The indicators were tracked in terms of schedules and quality by the Ministry of Energy and the Ministry of Economy, and were reflected in the Bank's implementation supervision reports.

a. M&E Utilization:

The Monitoring system was not utilized beyond the project, as M&E had a simple design; indicators were linked to delivery of high-quality reports, which were then used for investment decision making for the roads sector, and are expected to be used for the hydropower project as well.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:

The Bank’s project team communicated extensively with Georgian NGOs that had local interest in the Khudoni area; that had particular interest in environmental issues, and that were overseeing Bank-financed projects in general. Also the Government communicated with these NGOs, although mostly through the public consultations that took place. The project team’s decision to develop a Strategic Environmental Assessment (SEA) was endorsed by the NGO’s, as adding value to the studies by placing Khudoni in the context of other electricity supply enhancing or demand reducing options The SEA was made publically available.

The project was designed to finance only preparation of follow-on activities; provision of goods or works was excluded so no negative impacts on the environment or society were expected from project implementation. However, since anticipated investments under the project umbrella would include components likely to rank environmental Category A or B, the project was classified as Category B. Investment-specific instruments of environmental due diligence could not have been applied to this technical assistance project. However a Safeguard Framework (SF) was prepared to meet Bank safeguard requirements for this type of operation. The project triggered OP/BP 4.01 Environmental Assessment. The OP/BP 4.01 guiding principles were incorporated during project environmental classification and in developing and disclosing all environmental documents produced during project preparation and implementation, in particular the public consultation process and especially for Khudoni hydropower plant-related environmental and social work. NGOs/civil society participated in developing TORs for studies, and discussing the reports produced. The World Bank OP/BP 4.12 Involuntary Resettlement was triggered because follow-on investments might potentially include resettlement. Therefore, the project developed resettlement action plans (RAPs). The RAP for Khudoni was developed into a framework on which to base a more detailed future RAP, because studies pertaining to Khudoni power plant have not yet been followed by investments and establishing a detailed RAP now is premature as it would be out of date in the medium to long term. Also, an Environmental Assessment (EA), Environmental Management Plans (EMPs), and Resettlement Action Plans (RAPs) were prepared under Component 2 (roads), and these documents and plans were used in the subsequent investment projects for the East-West Highway. World Bank O.P. 7.50 projects on International Waterways was relevant to the project because a future Khudoni hydropower plant would be located on a river that flows into the Black Sea. The Commission for the Convention on the Protection of the Black Sea against Pollution was notified about the project and the safeguard requirement was met. Overall, the project was in compliance with environmental and social safeguards.

b. Fiduciary Compliance:

The financial management arrangements in place at the TRRC and the PSO for implementation of the project were satisfactory and acceptable to the Bank. Financial management was rated satisfactory throughout the project. All procurement under the project was consistent with the Bank guidelines and in accordance with the methods and thresholds specified in the grant Agreement. The project procurement rating was satisfactory till September 2009. In 2009 the rating was downgraded to moderately satisfactory due to protracted contract payment disagreement between the Ministry of Energy and the contractor. The contractual dispute between the Ministry of Energy and the Engineering Consultant involved additional work undertaken by the Consultant without having signed a contract addendum and the additional costs and time for reasons outside the Consultant’s control, including Georgia’s armed conflict with Russia in August 2008. The contractual issues were eventually solved. The rating remained moderately satisfactory as the disagreement was only resolved after the closing of the project. The project team confirms that there were no qualified audits.

c. Unintended Impacts (positive or negative):

The practice of using a local panel of experts was effective for the Khudoni hydropower station and was replicated for other hydropower station schemes in Namakhvani and Mtkvari. The project has led to strengthening and mobilization of the local hydropower engineering capacity, this engineering capacity was later used for the design of other hydropower station throughout Georgia.

Uncertainties surrounding the finalization of the contract and start of implementation of the Khudoni PPP.

Bank Performance:

Satisfactory

Satisfactory

Borrower Performance:

Satisfactory

Moderately Satisfactory

Initial delays in the procurement process; and shortcomings in design features in the East-West Highway segment studies

Quality of ICR:

Satisfactory

NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons: The ICR provides several lessons, of which four are adapted and highlighted here. .

Allowance must be made for cost overruns in Hydropower studies, which are very likely because the geology of a dam site is typically not fully known at project start. This project was no exception to this experience, and additional funds were needed to top up the outlay for technical feasibility studies.

Large scale hydropower development must be set in the larger river basin context to account for possible multi-dam development in the future. In the case of Khudoni, the government already had elaborate river basin development plans, but they were over a decade old, and had to revisited to confirm the adequacy of the original Khudoni dam location.

Undertaking a Strategic Environmental Assessment (SEA) is prudent policy whenever new power generation is contemplated and past SEAs are non-existent or dated. Undertaking a SEA was originally not part of this project’s hydropower component, and the Government was originally of the opinion that sufficient studies of this nature had been made. However, it became clear that an updated SEA would be important to make a convincing case to civil society that environmental impacts are identified and addressed.

The Bank should ensure that the borrower commits sufficient time and resources for preparation of feasibility studies and designs for complex infrastructure projects. Any short-cuts in this regard are likely to impact implementation negatively and result in far greater costs later. In this project, only 6 months were provided for technical designs, when the scale of work would have justified 12 months. Similarly, the budget allocated for designs was only at about 1.5% of the construction costs against 5% that is allocated for projects of similar size and complexity,.

14. Assessment Recommended?

Yes

Why? It will be instructive to assess the barriers that are faced in taking forward technical feasibility studies for infrastructure projects, raising finance and onward to successful implementation.

15. Comments on Quality of ICR:

The ICR is generally frank and analytical in explaining the project experience. It provides the provides the necessary information to assess the project’s performance. The ICR discussion pays adequate attention to the project outcomes, which are somewhat difficult to pin down in the case of this technical assistance project. The ICR could have further elaborated on efficiency, by providing measures of cost effectiveness, e.g. using cost benchmarks for similar studies. Some more attention could have been paid to editing the ICR to make the writing style sharper and uniform.