Explanatory Memorandum to COM(2017)460 - Interim Evaluation of the Union Civil Protection Mechanism for the period 2014-2016 - Main contents
Please note
This page contains a limited version of this dossier in the EU Monitor.
| dossier | COM(2017)460 - Interim Evaluation of the Union Civil Protection Mechanism for the period 2014-2016. |
|---|---|
| source | COM(2017)460  |
| date | 01-09-2017 |
1. Purpose and Scope of the Evaluation 3
2. Main findings 4
3. Conclusions 7
3.1. Prevention 7
3.2. Preparedness 8
3.3. Response 9
3.4. Cross-cutting 10
3.5. The Way Forward 11
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL
on the Interim Evaluation of the Union Civil Protection Mechanism for the period 2014-2016
1. Purpose and Scope of the Evaluation
1. The Union's Civil Protection Mechanism (hereafter: the UCPM) constitutes a visible expression of European solidarity by ensuring a practical and timely contribution to prevention, preparedness and response to disasters of all kinds occurring inside or outside the Union. The UCPM facilitates the mobilisation and coordination of assistance provided by the Participating States1 with the overarching goal of protecting Europe's citizens, economy and the environment. The Emergency Response Coordination Centre (ERCC) and the European Emergency Response Capacities (EERC, also referred to as the Voluntary Pool of pre-committed capacities and trained experts from Member States) are core elements of the functioning of the Mechanism.
2. Under Decision No 1313/2013/EU of 17 December 2013 on a Union Civil Protection Mechanism2 (hereafter the Decision), the Commission is required to evaluate the performance of the UCPM and report to the European Parliament and the Council on the results obtained and the qualitative and quantitative aspects of the implementation of the Decision by no later than 30 June 2017. This report presents the results of the Interim Evaluation (hereafter the Evaluation) and is accompanied by a Staff Working Document covering the detailed findings of that evaluation.
3. An independent evaluation of the activities implemented under the Union Mechanism was carried out between November 2016 and June 20173. The fact-finding phase included a comprehensive document review and a targeted stakeholders' consultation involving, among others, civil protection authorities from 26 Participating States, 14 international partners (e.g. UN agencies, OECD, World Bank, Red Cross and Red Crescent, NGOs etc.) and officials from 15 Directorates-General and services of the European Commission. Three case studies were also conducted (two in-country and one desk review). In addition, an Open Public Consultation (OPC) was carried out online for three months (November 2016 – February 2017), with some 130 responders from all over Europe providing feedback on the UCPM. A report on the outcome of the OPC was published at the end of March 20174.
4. The Commission Staff Working Document on the evaluation builds on the evidence provided therein as well as feedback received during discussions with Participating States and the OPC. It also takes into account the results of the European Court of Auditors' (ECA) review of the UCPM5, the Commission's Report on progress made and gaps remaining in the Voluntary Pool6, as well as other reports publicly available. Overall, the Commission finds the design of the external evaluation adequate to the research questions and based on a satisfactory methodological approach. One limitation should be noted, namely the limited statistical base of the external evaluation. Most of the data gathered by the contractor during the independent evaluation's timeframe are of qualitative nature and refer to 'perceptions' of stakeholders on the UCPM performance. This limitation points also to the need to strengthen the monitoring system of the UCPM in particular in relation to including additional quantitative indicators of performance (as highlighted by the recent ECA report). The external evaluation has not proposed specific additional indicators to fill this gap. Nevertheless, within the limits of data-gathering and analysis, the evaluation findings are credible. The conclusions are well balanced and logical consequences of findings are linked to the available evidence.
2. Main findings
5. The evaluation addresses the five mandatory criteria set out in the Commission's Better Regulation Guidelines: (i) Relevance; (ii) Effectiveness; (iii) Efficiency; (iv) Coherence; and (v) EU added value. An additional criterion, (vi) Sustainability, was added to the scope of this evaluation. The main findings are summarized in the paragraphs below.
6. There is sufficient evidence to demonstrate the continued relevance of UCPM objectives to Europe's needs. The 2017 Eurobarometer survey on civil protection shows that a vast majority of European citizens support a common EU policy in the field of civil protection and agree that the EU should lead the coordination of disaster response to affected countries, both in Europe and in the rest of the world. Recent resolutions of the European Parliament7 have also called on the Commission to further support cooperation among European civil protection authorities. The stakeholders consulted also highlighted the flexibility of the UCPM to rapidly adapt and address newly emerging crises which were not 'traditional' civil protection emergencies (e.g. the Ebola outbreak in West Africa and the migrant crisis in Europe). The increasing number and scope of UCPM activations outside of Europe pose, however, the challenge of finding the right balance between being a framework of mutual assistance between European countries in the aftermath of natural and man-made disasters and evolving towards a more complex instrument capable of addressing crises of a multiple nature and with a global reach.
7. An analysis of the UCPM effectiveness showed encouraging progress under each of the specific objectives. The UCPM is fully on track to achieve objectives pertaining to disaster prevention framework (objective 1) and readiness for disasters (objective 2), based on the indicators set by the legislation and on feedback received by key stakeholders consulted. The UCPM's response to disasters (objective 3) has been timely and effective according to the vast majority of stakeholders consulted. Confirming one of the findings of the ECA review of the UCPM, the external evaluation found limitations in respect of consistent and robust quantitative data (e.g. speed, cost, needs assessments) to corroborate the stakeholders' very positive views on UCPM disaster response. Lastly, public awareness and preparedness to disasters (objective 4) have received slightly less satisfactory views from stakeholders when compared to the other objectives. In this respect, a directly attributable link between awareness of Union citizens of risks in their regions as the objective set by the Decision and activities carried out by the UCPM would be difficult to establish. Nonetheless, the Commission recognises that given the changing risk landscape in Europe, reinforcement of action related to citizen awareness would be relevant and timely.
8. Generally speaking, the stakeholders' perception of the UCPM's efficiency is positive. Emergency response and administrative costs associated to implementing the UCPM decision are considered acceptable by most Participating States. The external evaluation attempted to carry out analyses of costs for comparable response missions (e.g. aerial forest fires missions; transport of modules and in-kind assistance; etc.). The evidence cannot be considered fully sufficient to draw definitive conclusions. However, this confirmed the desirability of carrying out more detailed cost analyses and, possibly, determining benchmarks (or range of costs) for deployment and transport of assistance provided under the UCPM.
9. Internal coherence, that is coherence within the different pillars of the Mechanism, is deemed satisfactory by the external evaluation. Several cross-cutting activities of the UCPM (e.g. lessons learned, advisory missions, peer reviews, exchange of experts, etc.) provide for an integrated prevention-preparedness-response approach. External coherence, that is the linkages between the UCPM and other Union policy areas, has increased as compared to the previous framework (2007-2013). This was confirmed by interviews with a large number of Commission services. Focusing on a result-based agenda and milestones is seen as conducive to a more effective cooperation and it is felt that enhanced coordination mechanisms need to be considered in order to improve synergies between instruments funding prevention and preparedness activities, for instance in the Neighbourhood and Pre-Accession countries8. As far as response outside of Europe is concerned, the comparative advantage of civil protection intervention vis-a-vis humanitarian assistance needs to be carefully considered as the UCPM becomes increasingly activated for dealing with humanitarian and complex emergencies. In the same vein, a better articulation with other EU external policies (e.g. development aid) and tools is deemed necessary considering that the UCPM is increasingly intervening outside the EU (e.g. Ebola outbreak, Nepal earthquake, migration crisis) where recurrent disasters might interact with other risk drivers, such as conflict and fragility.
10. The existence of a single European 'hub' for information sharing and operational coordination, the introduction of common European standards for disaster response capacities and common guidelines on risk assessments, which have contributed to the development of an overview of risks in the EU in the framework of the UCPM legislation9, are perceived as having delivered high EU added value in the areas of disaster response, preparedness and prevention, respectively. Increased capacity and predictability of the EERC provided by the UCPM in the form of the Voluntary Pool would not have been possible without coordinated effort at EU level. Over the next years, introducing indicators to monitor the effectiveness and efficiency of the UCPM more adequately may provide additional evidence to appreciate (and communicate) the added value of EU-level action in civil protection.
11. Knowledge-sharing, expert exchange and training activities are seen as essential to creating a community of European civil protection practitioners and, ultimately, to ensuring sustainability of the UCPM's contribution to supporting safer and more resilient societies. The Disaster Risk Management Knowledge Centre, an online hub launched at the end of 2015, is providing services to enhance knowledge sharing and capacity building across the UCPM Participating States. However, at a system level, several issues are considered key to the future sustainability of the UCPM contribution. The financial model needs re-examining to ensure availability of capacities in the Voluntary Pool adapted to risk-based needs; an efficient strategy to build the European civil protection human resource capacity needs to be rolled out; and the role of the private sector in the implementation of UCPM activities should be clarified. In order to maximize the impact of prevention and preparedness projects financed by the UCPM, end-users' needs should inform design and monitoring of the projects. Further consideration should be given to evaluating their impact over a longer timeframe. Finally, recognising the importance of the role of regional and local authorities in disaster management, the work of the UCPM needs to be brought closer to local actors, while primarily supporting the role of the national civil protection authorities. Due attention would also need to be given to cross-border prevention and preparedness, thereby strengthening the risk management governance along the European-national-regional-local chain.
3. Conclusions
12. Taking into account the findings of the external evaluation report, its own assessment of the qualitative and quantitative evidence provided as well as operational experience and lessons learned, the Commission's overall conclusion is that the Mechanism is on a solid footing to achieve the specific and general objectives of the Decision. For the period 2014-2016 under evaluation, the performance of the UCPM has showed encouraging results in each of the three pillars (prevention/preparedness/response) and overall is viewed positively by the Participating States and the main stakeholders. At the same time, the evaluation also highlighted a number of challenges that, if addressed promptly, could increase even further the impact of UCPM supported action. These aspects are further highlighted in the accompanying Staff Working Document with recommendations (summarized below) encompassing the prevention, preparedness and response pillars of the UCPM. A general area identified for improvement is the strengthening of the UCPM results monitoring framework, including better measurability through the possible introduction of quantitative indicators and baselines, in support of a stronger focus on impact over time.
1. Prevention
13. Increase the focus of the annual prevention (and preparedness) projects that the UCPM finances every year. The programme, which has been running since 2000, has financed a large number of projects without always capitalising on their results. Given the relatively small size of financing (between 2014 and 2016 average EU funding was EUR 470 000), projects need to: target the needs of the end-users, incorporate measures for scaling up their results and achieve bigger impact. Cooperation between the UCPM and the EU climate change adaptation agenda should be enhanced.
14. Better articulate the linkages between the UCPM prevention (and preparedness) projects and broader EU programmes (e.g. Cohesion/European Regional Development Fund, European Investment Bank, etc.) building on work started with support of the Disaster Risk Management Knowledge Centre. A more strategic link with such programmes could help maximize the uptake of project results and leverage additional technical and financial resources for achieving more significant results in prevention.
15. Strengthen the link between prevention and preparedness/response activities. This recommendation, which had already been highlighted in 2014 by the ex-post evaluation of the previous Civil Protection Mechanism (2007-2013), remains relevant at the mid-term mark of the 2014-2020 Mechanism. The need to match response capacities with the major risks in Europe and global emergencies will represent the key challenge to ensure continued relevance of UCPM. The Overview of Risks in the EU10 underscored that forward looking assessments will need to increasingly take into account 'emerging' risks (e.g. cascading risks, such as technological accidents resulting from natural events as well as the loss of critical infrastructure, nuclear and industrial accidents due to the increased risks of terrorism and cyber-attacks). A performance-based approach to risk assessment could help go beyond the estimated physical damage to infrastructure and focus on the level of vital services required in the aftermath of a disaster, enhancing contingency planning and response operations by civil protection authorities.
2. Preparedness
16. Revise the capacity goals for the Voluntary Pool with an evidence-based approach. The capacity goals for modules and other response assets are defined by the UCPM Implementing Decision and should be revised against the main/critical risks Europe or other beneficiary countries are facing and prioritized accordingly11. Capacity goals for different profiles of individual experts (currently not set by the Decision) would be required in order to map out more clearly technical resources and gaps in Europe. A closer dialogue with the Participating States is critical in order to capture all national capacities, including those which are not registered in the Voluntary Pool but may still be deployable via the UCPM. Similarly, the spatial distribution of (natural and man-made) risks across and beyond Europe should inform the geographical distribution of certain response capacities, whilst ensuring the ability to provide adequate assistance under the auspices of the UCPM wherever it is needed.
17. Re-assess the system of incentives for pooling assets in the Voluntary Pool based on lessons learnt and other knowledge emerging from operations. The level of co-financing for the deployment of Voluntary Pool capacities or the mobilisation of buffer capacities (particularly outside of the EU); support to multi-national modules and response capacities; insurance coverage for health and third-party liability for experts deployed are, among others, incentives needed to strengthen the UCPM capacity to respond. The administrative burden involved in registering and certifying modules also requires further improvement, although this is being addressed by the Commission with a more streamlined and coherent planning process for exercises, certification and registration of assets in the Voluntary Pool.
18. Develop a more need-based, efficient approach for the training and exercise programmes. Shifting the focus of training from outputs (i.e. number of individuals trained) to competences allows to address needs that stem from operations. This is currently being addressed by the Commission with the forthcoming Strategic Framework for Training and Exercises, which is built around competences. Furthermore, a robust framework for evaluating the training courses administered as well as the individuals trained is considered as a priority in order to match the needs on the ground with the best technical resources available. Similar considerations were raised with regards to the exercises. In addition, considering different implementation models for the UCPM 'basic' courses, such as the Union Civil Protection Mechanism Introduction Course (CMI) targeting large audiences and specialized/advanced training (e.g. training for deployment; assessments; medical teams etc.), which are delivered to well-identified groups of experts, could help increase value for money. Finally, the Strategic Framework for Training and Exercises should also take into account needs stemming from emerging and evolving threats, such as CBRN and critical infrastructure risks or terrorist attacks, while avoiding duplication with existing training initiatives supported by the EU.
19. Expand the end-user side of trans-national early warning systems platforms, such as the European Forest Fire Information System (EFFIS) and the European Flood Awareness System (EFAS). These systems bring a very high EU added value by consolidating national data into Europe-wide forecasting tools that provides support to decision-making of emergency managers in the ERCC and in the Participating States. Developing the community of end-users would help complete the technology platform, allowing experts from Participating States to evaluate the performance of early warning systems and response operations with the objective of improving the practice. Such a development would make the tools more relevant for all countries within Europe, regardless of their level of preparedness.
3. Response
20. Promote a consistent collection of quantitative data and produce systematic analysis of response missions to improve future performance. This could help establish baselines/benchmarks (lower/higher bounds or a range) for speed and cost, allowing comparisons between emergency responses and contributing to enhance the efficiency of response missions. Needs assessments in the Requests for Assistance also need to be made available on regular basis and provided in a more standardised format.
21. Consider the pre-deployment of a small UCPM team ahead of the formal UCPM activation when facing imminent disasters (e.g. cyclones, river floods, etc.). This would help gain a better understanding of the needs on the ground and prepare the following deployment of the EU Civil Protection (EU CP) Team, increasing speed and effectiveness of UCPM response.
22. Further simplify the administrative and financial procedures for engaging response capacities under the UCPM. A number of operational quick fixes (e.g. integrating in CECIS the transport co-financing request form; revising the minimum threshold for individual transport grant applications; etc.) have been identified by the users and could be addressed rapidly. This would be helpful to further reduce response times, while increasing the incentive for ensuring well-coordinated deployment under the UCPM, rather than through bilateral channels.
23. Ensure the best match between operational needs and experts deployed in the EU CP Teams. Consulted civil protection authorities found the Terms of Reference produced by the ERCC for selecting experts (their use became increasingly systematic since 2015) helpful but raised some concerns on the selection process of the experts, which should be more competence-based.
4. Cross-cutting
24. There needs to be a continuous feedback loop between lessons learnt and the implementation of activities (including prevention and preparedness ones, in addition to response). Participating States have highlighted that some specific activities (e.g. training and exercises) would benefit from a stronger link with UCPM operations, for instance focusing on specific challenges arisen from the deployment of modules in the field (e.g. custom clearance in fragile and conflict-affected states; cooperation with host countries; integration in broader international frameworks for disaster response and recovery planning; etc.). To this end, there is a shared feeling that the lessons learnt process of the UCPM needs to be more structured and systematic from identification and throughout implementation (on both the Commission and Participating States sides), with regular monitoring and transparent reporting on the progress.
25. Ensure a good match between peer reviewers and the needs of reviewed countries (i.e. between profile/competences of the reviewer and specificities of the country/sector to be reviewed). Additional linkages could be explored between training and the deployment of experts on peer review missions. This may include a process to certify experts' technical and operational competences in order to better address the needs of the reviewed countries.
26. Prevention and preparedness ("advisory") missions should build on a well-formulated, result-oriented approach. A structured process needs to be put in place to prepare and implement these missions, be it in preparation for potential disasters (e.g. assessment of national emergency management capacity) or, in the aftermath of a disaster, to inform recovery and reconstruction strategies. Complementarity with other EU-funded programmes (in particular the Prevention Preparedness and Response to Natural and man-made Disasters (PPRD) and the Instrument for Pre-Accession (IPA) programmes) and/or Participating States engaged in third countries should be strategically considered, with a view to provide additional resources (technical and financial) needed for implementing recommendations.
27. Further increase the coherence between the UCPM and the neighbourhood policy instruments. At the policy level, Commission is committed to ensure coherence and consistency of its activities in the field of civil protection, in line with the international agreements (e.g. the Sendai Framework for Disaster Risk Reduction 2015-2030, the UN Sustainable Development Goals and the Paris Agreement). The Evaluation highlighted the need for increased coordination between the UCPM and Commission's activities in the Enlargement and Neighbourhood countries. In particular, the yearly Prevention and Preparedness Call for Proposals of the UCPM could take into account the disaster risk management activities financed under the PPRD- East and -South programmes. The IPA programme would benefit from further integration of its multi-countries response modules in the UCPM's Voluntary Pool and from joint efforts to strengthen the governance of the prevention activities (e.g. on the Disaster Risk Assessment and Mapping initiative).
5. The Way Forward
28. The Evaluation provides useful analyses and insights on the implementation of UCPM legislation, including on existing shortcomings and areas for further improvement. The Commission will start a reflection process on the recommendations highlighted above, particularly on those that could be implemented in the short to medium-term, as well as on those aspects that would require a modification of the UCPM framework in the future.
29. The UCPM will need to get closer to regional and local actors while continuing to strengthen, through the national civil protection authorities, the governance for disaster risk management both cross-border and along the European-national-regional-local chain. A new approach based on a network of European hubs for civil protection (and/or centres of excellence) could be explored.
30. In addition to the interim evaluation a number of key reports and documents have now been produced assessing different aspects of UCPM performance, including the abovementioned Court of Auditors report, the capacity gaps report and the EU overview of risks. These documents provide useful analysis and insight on implementation of UCPM legislation, on possible ways to strengthen it as well as on existing shortcomings and weaknesses. On that basis, the Commission will assess the appropriateness of making changes to the provisions of Decision 1313/2013/EU with a view to:
a. Strengthening effectiveness by providing realistic incentives, attaining simplifications and reducing administrative burden;
b. Matching current/emerging risks with response capacities;
c. Making full use of Europe's expertise and assets for preparedness and response.
1 EU Member States and Iceland, Montenegro, Norway, Serbia, the former Yugoslav Republic of Macedonia, Turkey
2 Article 34(2)(a) of Decision No 1313/2013/EU, OJ L 347, 20.12.2013, p. 924
3 Final report available at: https://ec.europa.eu/echo/sites/echo-site/files/ucpm_final_report.pdf
4 The report is available at: ec.europa.eu/echo/EUCPM-consult_en
5 European Court of Auditors, Union Civil Protection Mechanism: the coordination of responses to disasters outside the EU has been broadly effective, Special report No 33/2016, January 2017
6 Report from the Commission to the European Parliament and the Council on progress made and gaps remaining in the European Emergency Response Capacity, COM(2017) 78 final, 17.02.2017
7 Among others, European Parliament Resolution of 1 December 2016 on the situation in Italy after the earthquakes (2016/2988(RSP)).
8 This emerges as the main findings of recent evaluations, including: ROM Review of PPRD-East (2017); Evaluation of PPRD-South (2017); Evaluation of IPA II (2015).
9 Commission Staff Working Document, Overview of Natural and Man-made Disaster Risks the European Union may face, SWD(2017) 176 final, 23.5.2017
10 Commission Staff Working Document, SWD(2017) 176 final
11 This is in line with Article 14(2) of Commission Implementing Decision 2014/762/EU, which requires the Commission to regularly assess and revise the suitability of capacity goals 'on the basis of risks identified in national risk assessments or other appropriate national or international source of information'.
EN EN