The project development objective as stated in the Project Appraisal Commitment Document (PACD,page 4) was “to provide sustainable access to combined water and sewerage services to peri-urban quartiers [districts] with high levels of vulnerable households, i.e. with household incomes falling under US$295/month (150 percent of the poverty line of Dh 1687) or about US$60/cap/month. The project will benefit about 11,300 households (56,000 inhabitants).”

The statement of objectives in the Grant Agreement was: “to promote, following an output-based approach, the connection to water and sanitation services of about 11,300 low income households in disadvantaged peri-urban and rural National Initiative neighborhoods in the urban center of Casablanca, Tangiers and Meknes.”

This Review is based on the statement of objectives in the PAD, as it explicitly includes sustainability.

Since this project was financed and prepared by the Global Partnership on Output-Based Aid (GPOBA), there was no formal list of components as the operators were responsible for the process of delivering the required services. However, in order to ensure that the Bank could hold the service operators accountable for reporting, four components were discerned in the ICR from the PACD as described below. Regarding the cost information, since the four components are effectively processes, only a lump sum estimate of the total project cost was made at appraisal without an allocation of project costs among components. The ICR also contains only a total project cost figure at closure.

1. Assessment of demand for connections and marketing / outreach efforts to sign up households for the subsidized social connection plan: This component assisted the three selected utilities (Amendis in Tangiers, Lyonnaise des Eaux de Casablanca (LYDEC) in Casablanca and the Régie Autonome de Distribution d’Eau et d’Electricite de Meknes – RADEM) in undertaking market surveys to provide the GPOBA with demand projections. The component also supported operators in: a) conducting marketing activities to promote the social connection programs to involve households; b) providing semi-annual updates of demand projections; c) identifying the inconsistencies between actual signing and the submitted projections; and d) presenting a proposal for a new action plan to mobilize lagging customers’ demand.

2. Management and implementation of works as required for establishing functioning household connections: This component supported all utilities in arranging for pre-financing of works, household connection fees, and the design and construction of facilities that are necessary for social connections. This included measures to ensure that all the participating utilities had procurement policies in acceptable to the Bank.

3. Primary monitoring and household certification of outputs: This component aimed to ensure the active social connection accounts by collecting connection certificates signed by beneficiary households, and retaining inspection billing and collection documentation for at least two billing periods.

4. Provision of on-going services: This component aimed at assisting participating utilities in providing sufficient services to connected households as per the applicable regulatory provisions.

Project Cost:The actual project cost was US$7.04 million, including US$6.74 million for subsidy payments, US$300,000 for an independent audit firm and US$120,000 for project supervision.

Financing:The project was wholly financed by a Grant from the GPOBA, which is a multi-donor trust fund administered by the World Bank.

Borrower contribution:None was planned and none materialized.

Dates:The closing date was extended twice for 12 months each time, from the original end date of 12/31/2009 to 12/31/2010, then to 12/31/2011, in order to allow the full utilization of the Grant’s proceeds (total duration changed from 23 months to 47 months).

3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Objectives: High. The objective remains highly relevant to the Bank’s FY2010-2013 Country Partnership Strategy (CPS) for Morocco. The CPS specifically discusses the priorities of strengthening water and sanitation services under the core engagement of comprehensive rural development, and also of strengthening governance and accountability, thereby ensuring greater social and economic inclusion.

The objective is also highly pertinent to the ongoing National Initiative launched by the King of Morocco in May, 2005. This envisages promoting systematic and sustainable development by expanding access to basic infrastructure among the poorest.

b. Relevance of Design:

Design: Modest. The PACD does not include an explicit results framework demonstrating a causal chain linking inputs to outputs, outcomes and the achievement of objectives. However, the activities outlined in the project components outlined in Section 2c above could reasonably be expected to lead to some of the intended outcomes. These activities included outreach efforts to identify and interest eligible households, support for the marketing and client recruitment efforts of the operators, assistance in the implementation and management of the works (including prefinancing where necessary), and assistance to the utilities in providing services to the connected households in accordance with the regulatory provisions. Design was adapted to achieve the objective of delivering services in areas that may not be commercially attractive without a subsidy. It incorporated the results of an assessment of the technical and financial capacity and accountability of the three utility companies and of a survey to determine the required targeting mechanisms for the urban poor. It also incorporated a new way of reaching out to customers. All households in the selected pilot areas were eligible to participate in the connection subsidy program. Participation was to be encouraged through a communications campaign to raise awareness about the program and its conditions. Operator accountability was to be assured through the contracting of an independent verification auditor.

However, design did not include sufficient measures to ensure financial sustainability. First, no in-depth analysis was conducted during preparation of an affordable tariff for consumers. The PACD states only that "based on practical experience gathered by all three utilities with previous social connection programs it appears that monthly household payments of up to 150 Dh would indeed be affordable to a majority of households and reflect the willingness to pay for improved service for a sizeable portion of unconnected low income households." Second, there was no mechanism to mitigate the potential impact of exchange rate fluctuations on overall project costs, despite the recommendations of a consulting firm that such a mechanism (for example, escrow accounts denominated in Moroccan dirham) be established (ICR, page 25). Third, there was no mechanism to revise the subsidy amounts. This meant that operators would have to bear additional unanticipated costs of their respective pilots. It made operators vulnerable to exchange rate fluctuations and inflation in material costs, a vulnerability that was particularly marked during the last two years of the project when the highest number of connections were installed (ICR, page 25).

4. Achievement of Objectives (Efficacy) :

The project objective of providing sustainable access to combined water and sewerage services to peri-urban quarters with high levels of vulnerable households, can be broken down into two sub-objectives: (a) access to combined water and sewerage services to peri-urban quarterswith high levels of vulnerable households, and (b) sustainability of the service.

Outputs

The outputs are common to both sub-objectives:

Marketing activities were organized by the operators to promote the terms of the subsidized connection program and to encourage households to sign up to their respective programs.

According to the operators' closing reports (ICR, page 12), a total of 12,426 household connections were made, exceeding the original target of 12,025 and the revised target of 10,421 (set at the October 2008 mid-term review). 1,922 of these were not subsidized. 10,504 households were connected to subsidized water services and 9,036 households to subsidized sanitation services.

Based on survey results, the percentage uptake by targeted beneficiary households per operator and eligible area had, as of June 30, 2010, been exceeded in Tangiers (93%, target 85%), achieved in Casablanca (80%), and was below target in Meknes (75%, target 90%).

During the preparation phase, the Government established an enabling institutional and regulatory framework with the aim of ensuring a transparent and effective allocation of grant proceeds in the three cities and to administer the disbursement of funds (ICR, page 6).

Outcomes

(a) Access to combined water and sewerage services to peri-urban quarters with high levels of vulnerable households. Substantial.

According to the operators’ closing reports as of December 31, 2011, 55,741 persons had been connected to improved water and sanitation services, 18% in excess of the revised target of 47, 390.

The 6.0 m3 target value for average monthly household water consumption was achieved in the three cities: 10.0 m3 (in Casablanca-Lahraouine) and 6.2 m3 in Casablanca-Lemkensa); in Meknes 9.2 m3; and in Tangiers 10.0 m3.

According to the PACD (page 5), poverty targeting of the low-income peri-urban districts that would benefit from the project was based on the World Bank’s poverty maps within the National Initiative for Human Development zones.

Beneficiary households were targeted according to clear selection criteria that were set forth in the project's Operational Manual. According to the ICR (Table 3, page 9), these included: (i) location in the areas targeted by the interventions; (ii) maximum monthly household income; (iii) property title or proof of occupancy; (iv) connection authorization delivered by the municipality; and (v) the household's certified commitment to pay a specified monthly contribution.

The degree to which some of the criteria were applied varied -- the property title criterion does not seem to have been required by any of the three operators, and the maximum monthly income requirement was applied initially by RADEM and AMENDIS, but subsequently removed.

(b) Sustainability of the service. Modest.

Consumption bill collection ratios varied between 96% and 99% in Casablanca, and between 79% and 90% in Tangiers; they were 99% in Meknes. The target was a ratio "equal or superior to the operator's average collection ratio." There is no indication as to whether this target was met.

The collection ratio for monthly connection fee payments was over 95% in Casablanca but only 61% in Meknes. No figure is provided for Tangiers.

Costs of works and collection installations in Casablanca and Meknes were claimed to be considerably higher than forecast by the operators concerned (the cost per connection at project closure in Casablanca -- estimated on a sample basis -- was US$4,148, compared to an appraisal estimate of US$2,035). The ICR (page 17) states that the risk to completing the project was minimal, since the operators knew that there was no mechanism for adjusting the subsidy and that they would, therefore, have to bear all additional expenses not covered by the grant to complete the works and deliver the contracted number of connections. Nevertheless, the number of connections financed by the Grant was lower than foreseen and for those that were completed, operators had to finance a higher absolute and relative amount than expected. Operators also pointed out that the risk of a financing gap was significant and that this could jeopardize the expansion of the program until more details could be provided about the actual cost of providing services in the kinds of areas supported by the project. The summary of the Borrower's ICR ((ICR, page 25) states that "unit subsidy per connection became disconnected to real costs over time."

However, a significant deficiency of the M&E framework and the financial reporting procedures (see Sections 10 and 11 below) meant that it was not possible to determine the precise connection cost and hence develop a properly targeted subsidy policy. This was because the operators’ financial statements did not distinguish between costs related to ‘in-site” and “off-site” works and did not single out the costs of the connections subsidized under the project. Despite the efforts of the supervision team to address this issue, it persisted to the end of the project (ICR, pages 12-13).

According to the Borrower's ICR (ICR, page 43), the combined pilot operations in Tangiers are expected to show a 25% financial deficit by 2020, because (a) value added tax of 20% was not taken into account in calculating the subsidy; (b) the subsidy amount was never revised during implementation of the pilot projects, and (c) exchange rate fluctuations resulted in a shortfall of two million dirham.

There is no evidence that an in-depth analysis of consumers' ability and willingness to pay has been carried out and a corresponding range of tariffs calculated. According to a beneficiary survey of nine out of thirteen communes in the pilot areas, there is considerable variability between cities. In Casablanca, the entire surveyed population confirmed the affordability of the connection and their willingness to pay. In Tangiers, it is reported that the majority provided the same response. In rural Meknes, however, beneficiaries showed their willingness to pay for water only, and not for sanitation since sanitation was not covered by the subsidy and it was not seen as a priority. Further, in urban Meknes (which faced the highest monthly bills), an issue of affordability had clearly emerged by project closure (ICR, pages 20 and 30).

The absence of financing sources other than the subsidy may be a constraint on sustainability in some cities. In Meknes, the subsidy for the project pilots was higher because of this factor (by contrast, in Casablanca, the subsidy represented only 12% of the total financing requirement -- ICR, page 31).

5. Efficiency:

The ICR did not conduct any ex-post analysis to assess the economic rate of return for the project or for the three utilities. The project was successful in implementing the agreed upon investment program within the committed amount based on an assessment by an engineering consultant firm at the time of appraisal. However, there were two twelve-month extensions due to implementation delays and other administrative inefficiencies, for example, difficulties in meeting the beneficiary households criteria as set forth in the Operational Manual, including obtaining authorization for connections from the local authorities; the time needed to approve the lists of eligible beneficiaries; and issues related to works to be completed by third parties.

Efficiency is assessed as modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:

Rate Available?

Point Value

Coverage/Scope*

Appraisal:

%

%

ICR estimate:

%

%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of objectives was rated high and that of design modest. While the project substantially achieved the objective of enhancing the target population's access to combined water supply and sanitation services, there are a number of concerns affecting sustainability, which was rated modest. Efficiency is also assessed as modest.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The new subsidy mechanism, targeting poor households in peri-urban districts, benefited from strong support within the context of Morocco’s National Initiative for Human Development. However, the Government which took office after the 2011 election, was not familiar with the Output-Based Aid (OPA) approach, which required the project team to engage in policy dialogue with a view to obtaining the commitment of the Authorities. The outcome of this dialogue, and hence the continuation of OBA schemes in the water and sanitation sector, remains uncertain.

The emergence of a financing gap, projections of significant financial deficits for one of the pilot cities, the absence of a solidly based assessment of consumer tariffs, and evidence of inability or unwillingness to pay in some pilot communities, point to the necessity for an in-depth analysis of the costs of providing the service and the level of subsidy that would be required for its expansion.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:

Design benefited from extensive analytical work as part of the preparation of a Development Policy Loan for the water sector. Based on this analysis, an Output-Based Aid (OBA) approach was piloted to target the poorest segments of the population and to bridge the gap between capacity to pay and the cost of connection and service.

During the 18 month preparation phase, feasibility studies were conducted, and consultation meetings with potential implementing agencies and other stakeholders.

Some of the causes of delays in the initial implementation phase, including financial and procurement management and safeguards compliance issues, land tenure issues in the slum area of Casablanca, and coordination dimensions of in-site and off-site works, could have been mitigated through more careful preparation, together with provisions for training on the Bank procedures for the three utility companies and other stakeholders.

The Bank supported the project through nine supervision missions that were supplemented by the 17 Independent Verification Agency evaluation missions (see Section 10 below for a description of the Agency’s responsibilities).

At the time of the mid-term review in October 2008, the Bank retained a consulting firm to carry out an independent assessment of progress and to further explore the advantages of the OBA approach. As a result, the project team revised the original Operational Manual and identified a number of factors hindering implementation, in particular procurement, disbursement and legal aspects (ICR page 22).

M&E design weaknesses were only partially addressed. In particular, and despite the efforts of the supervision team to address the problem, inadequacies in key financial information provided by the operators persisted until project closure (see Sections, 9b, 10b and 11b below).

Weaknesses in the financial management function of the project unit could have benefited from more active support and assistance from the team.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Government took the initiative to introduce Output Based Aid (OBA) schemes in the water and sanitation sector with a view to enhancing the achievement of the National Initiative for Human Development programs. During the preparation phase, the Government managed to establish an enabling institutional and regulatory framework to ensure a transparent and effective allocation of grant proceeds in the three cities and to administer the disbursement of funds (ICR, page 6). The ICR also notes (page 23) that the Government fully supported the three pilot utilities, especially when land tenure issues had to be solved during implementation and also at the time of performance monitoring of the three operators. Commitment was also demonstrated through the high participation rate in all the coordination meetings by the Government officials during both the project preparation and implementation. However, the new Government which took office after the 2011 election was not familiar with the OBA approach, and its commitment to move the agenda forward by addressing the financing gap and extending service provision to other disadvantaged peri-urban and rural areas is uncertain.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

Three water providers were selected as implementing agencies, one in each of the three cities. Two of the operators were subsidiaries of international water companies(Lyonnaise des Eaux de Casablanca (LYDEC) in Casablanca; and Société des Eaux et d’Electricité du Nord (AMENDIS) in Tangiers), and the third was a public sector utility, RégieAutonome de Distribution d’Eau et D’Electricité de Meknés (RADEM) in Meknes. Implementation progress was slow during the first year of the implementation due to an issue concerning sanitation works and land acquisition outside the Bank project site in the case of LYDEC, and to compliance issues regarding procurement and financial auditing in the case of RADEM.

The project successfully achieved the targeted number of connections and the operators complied with their contracted service provision obligations. However, the ICR (page 13) reports that the operators did not always follow the Bank’s standard procurement guidelines, that a large number of audit reports were submitted with delay, and that some operators submitted delayed disbursement requests throughout implementation. The combined effect of these shortcomings led to two extensions of the closing date.

Although most weaknesses in financial management were addressed at the mid-term review, one of the three operators, RADEM, continued to submit audit reports late, which required an additional supervision mission focusing on financial management.

There was a key gap in the information provided by operators concerning the precise cost of subsidized connections which undermined not only monitoring but also the proper application of the subsidy policy (ICR, pages 12-13).

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The Ministry of the Interior was responsible for monitoring progress achieved by the operators against the key performance indicators agreed in the Operation Manuals of 2007. The quarterly and bi-annual progress reports were to be submitted by the three operators to the Ministry of the Interior and the Bank team together with the following information: i) the selection of beneficiary households in accordance with the eligibility criteria set forth in the Operational Manual; and ii) the guarantee of the quality of service standards, consistent with the provision of both the Concession Contracts or the public utility service agreement. The Ministry of Finance and Privatization was also engaged with the monitoring of the pilot project as a member of the Steering Committee.

As a requirement of the GPOBA an external auditor, the Independent Verification Agency, was assigned to enhance transparency and traceability of results (ICR page 10). The responsibilities of the Independent Verification agency were the following: i) to review, on a sample basis, at least 7% of the connections for which the operator requested payment of the first tranche of the subsidy; ii) to review, on a sample basis, the existence and quality of the connections for which the operator requested the payment of the second tranche of the subsidy; and iii) to gather the lessons to be learned from the implementation of the pilot project, and to identify issues and possible solutions. The Independent Verification Agency prepared the quarterly report that included the findings and recommendations of payment amount to each operator and submitted to the Ministry of Interior.

The M&E framework was not set up to furnish detailed information on the results of the pilots in each of the three cities (ICR, page 10). Further, the set of selected indicators was suitable to measure outputs, but less so to track outcomes (ICR, page 11) These deficiencies could only be partially addressed through the activities of the Independent Verification Agency.

b. M&E Implementation:

The Independent Verification Agency carried out seventeen supervision missions during the project implementation period and submitted progress reports to the Ministry of Interior and the Bank team. Prior to the mid-term review, the key performance indicators were informally revised on December 31, 2010. The target number of households was lowered in view of the reduced amount of grant due to currency depreciation and rising costs. This change, however, was not reflected in the results framework nor in the Implementation Status Reports. Three core indicators were added to measure the water quality and access to improved services in each city: i) uptake by beneficiary households per operator and eligible area; ii) the average residential monthly consumption bill; and iii) collection ratios for consumption bills per operator and, to the extent possible, collection ratios for connection fees (ICR page 11).

According to the ICR (page 15), “monitoring proved to be challenging throughout the implementation of the project, largely due to the lack of attention from the operators to comply with the reporting requirement even though the proposed reporting structure in place was, in theory, adequate. As this project was designed as a pilot, the relatively inadequate results framework made it a lesser relevant tool to get better insight into the sustainability of the services provided and the effectiveness of the subsidy policy, beyond the monitoring of the connection rates.”

A particularly significant deficiency of the M&E framework was the impossibility of determining the precise connection cost and hence developing a properly targeted subsidy policy. This was because the operators’ financial statements did not distinguish between costs related to ‘in-site” and “off-site” works and did not single out the costs of individual connections subsidized under the project.

a. M&E Utilization:

The ICR does not report on the utilization of data generated by the M&E system.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

The project was classified as Category “B.” OP 4.01 (Environmental Assessment) was the only safeguards policy triggered. A partial environmental assessment was conducted as a part of project preparation. According to the ICR (page 12), "[environmental] aspects were monitored closely throughout the life of the project and when deficiencies were noted, they were acted upon by the utility providers as substantiated by the project documentation (Aide-memoires, ISRs, quarterly and bi-annual reports, and IVA's reports).” The ICR concludes that "for the most part, compliance with environmental and social safeguards was rated satisfactory [in the Implementation Status Reports]."

b. Fiduciary Compliance:

Financial management. While the final audit report as well as previous audit reports, were submitted to the Bank by the utilities with delay, they were considered as satisfactory by the supervision team (ICR page 12). Nevertheless, the application of the subsidy policy was adversely affected not only by these delays, but also by the failure of the operators’ financial statements to distinguish between costs related to ‘in-site” and “off-site” works. Consequently, the costs of the connections subsidized under the project were not singled out.. As the ICR acknowledges (pages 12-13), “the lack of insight regarding the precise connection cost was an issue that was insufficiently addressed as a properly targeted subsidy policy was utterly dependent on having access to information on the cost of the connection.”

Procurement. Delays resulted from requiring the operators' compliance with both national procurement legislation (which required a two-stage bid opening) and Bank procedures. The slow submission of disbursement requests by some operators during the project implementation also caused delays (ICR page 13). There were no reported cases of misprocurement.

c. Unintended Impacts (positive or negative):

d. Other:

12. Ratings:

ICR

IEG Review

Reason for Disagreement/Comments

Outcome:

Satisfactory

Moderately Satisfactory

Relevance of objectives was rated high and that of design modest. While the project substantially achieved the objective of enhancing the target population's access to combined water supply and sanitation services, there are a number of concerns affecting sustainability, which was rated modest. Efficiency is assessed as modest.

Risk to Development Outcome:

Moderate

Significant

There are significant risks and uncertainties associated with the financial sustainability of the pilots and hence the extent to which they can be replicated. The approach of the current Administration in Morocco towards Output-Based Aid in the water and sanitation sector is also uncertain.

Bank Performance:

Satisfactory

Moderately Satisfactory

A weak M&E design for a pilot project is a significant weakness of Quality at Entry.

Borrower Performance:

Satisfactory

Moderately Satisfactory

Government commitment to the project's approach is uncertain. Gaps in the information provided by the operators make it difficult to apply a correct subsidy policy and to monitor results in the desired detail.

Quality of ICR:

Satisfactory

NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons: The following lessons are derived from the ICR with some adaptation:

- Social issues including possible land acquisition require identification during preparation, a thorough assessment of operators’ role, active involvement and cooperation of local authorities, and capacity building of local authorities based on the adequate strategy. This project experienced unexpected delays due to inexperience in dealing with the slum upgrading and land acquisition for works to be carried out.- A robustMonitoring and Evaluation system is an essential tool to measure, report and assess the project implementation capacity and its outcomes, especially for a pilot operation. In this case, M&E weaknesses impeded an adequate assessment of pilot outcomes. M&E design should incorporate procedures for ensuring that all key financial and other information, necessary for monitoring and for the proper application of the subsidy policy, will be available- The Output-Based Aid (OBA) approach can successfully involve a public utility provider: In the case of Meknes, the public utility proved that it can bear the performance risks involved in the OBA approach, thereby ensuring public service provision to neglected segments of the population.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR was generally clearly written and attempted to present an evidence based assessment of achievements using the performance indicators available. It was candid about the difficulties faced by the project, in particular the issues arising from weaknesses in the monitoring and evaluation framework. A fuller discussion of safeguards compliance would have been helpful.