Explanatory Memorandum to COM(2015)61 - Ex-post evaluation of the EC Civil Protection Mechanism and Civil Protection Financial Instrument for the period 2007-2013

Please note

This page contains a limited version of this dossier in the EU Monitor.

dossier COM(2015)61 - Ex-post evaluation of the EC Civil Protection Mechanism and Civil Protection Financial Instrument for the period 2007-2013.
source COM(2015)61 EN
date 18-02-2015
52015DC0061

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL on the ex-post evaluation of the Community Civil Protection Mechanism and Civil Protection Financial Instrument for the period 2007-2013 /* COM/2015/061 final */


REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL on the ex-post evaluation of the Community Civil Protection Mechanism and Civil Protection Financial Instrument for the period 2007-2013

Contents

1.

1. SCOPE AND PURPOSE OF THE REPORT


The European Civil Protection Mechanism (hereafter the Mechanism) was established in 2001 i to facilitate reinforced cooperation in civil protection assistance interventions. In January 2006, the Commission proposed to revise the Mechanism on the basis of past experience and to provide a suitable legal basis for future actions. Council Decision 2007/779/EC, Euratom i (hereafter the Mechanism Decision), was designed to deal with the increase in frequency and seriousness of natural and man-made disasters. In addition, Council Decision 2007/162/EC, Euratom i (hereafter the CPFI Decision) enabled the funding of activities aimed at preventive, preparedness and more effective response actions, particularly those taken by way of the cooperation between Member States and carried out under the Mechanism. The total amount for the actions and measures to be financed by the CPFI Decision was set at €189.8 million for the period 1 January 2007 – 31 December 2013. The Mechanism Decision and the CPFI Decision were repealed as of entry into force on 1 January 2014 of the Union Civil Protection Mechanism Decision i (hereafter the 2013 Decision).

Article 15(2)(d) of the CPFI Decision requires the Commission to submit to the European Parliament and the Council an ex-post evaluation report of Mechanism actions in the field of civil protection that received financial assistance in the period 2007-2013. Furthermore, Article 14 of the Mechanism Decision requires the Commission to evaluate its application and transmit the conclusions to the European Parliament and the Council. The Commission already carried out an interim evaluation of Mechanism actions that covered the period 2007-2009, the results of which were transmitted to the European Parliament and the Council at the end of 2011 by way of a Report from the Commission i, accompanied by a Commission Staff Working Paper i (together hereafter the Interim evaluation report).

This report presents the main findings of the ex-post evaluation of all Mechanism actions during the full period 1 January 2007 – 31 December 2013. An external consultancy was commissioned to independently evaluate the Mechanism actions, relying on extensive and comprehensive document review, observation of actions, case studies and a broad stakeholder consultation (Participating States, international partners, Commission services) by means of interviews and online surveys.

Overall, the design of the external evaluation was good and used a sound methodological approach. The evaluation relied on a large quantity of data extrapolated from reliable sources; however the use of quantitative data could have been more extensive. On the whole, the combination of evidence provided by different methodological tools was balanced. The observations of stakeholders other than Participating States, which are able to benchmark the Mechanism against other similar international systems (e.g. international partners, some Commission services) are well represented, which adds to the credibility of the findings. In some specific cases, the evaluation primarily relied on survey results and interviews, resulting in judgements that appear somewhat subjective. Nevertheless and within the limits of data and analysis, the findings are credible and the conclusions are well balanced, logical consequences of findings and linkable with the facts and data. The full external evaluation, with detailed qualitative and quantitative evidence, can be obtained at ec.europa.eu/echo/en/funding-evaluations

This report puts forward the Commission's position on the main conclusions and recommendations of the external evaluation. The following chapters outline the main evaluation findings assessed in terms of relevance, coherence, effectiveness, efficiency and EU added value of the Mechanism actions during the period 1 January 2007 – 31 December 2013, followed by the conclusions of the Commission.

2.

2. BACKGROUND TO THE MECHANISM ACTIONS IN THE PERIOD 2007-2013


3.

2.1. EU institutional changes


Prior to the entry into force of the Lisbon Treaty in 2009, EU civil protection actions and legislation, including the Mechanism and CPFI Decisions, were based on the catch-all provision of Article 308 of the EC Treaty, authorising the Council to act (by unanimity) where necessary to obtain a Treaty objective in areas where there the EC Treaty provided no other legal base. When the new Article 196 devoted to civil protection was introduced in the Lisbon Treaty, civil protection was formally recognised as a self-standing policy.

Until early-2010, civil protection was under the responsibility of the Commissioner responsible for Environment. It was then transferred to the Commission Directorate General for Humanitarian Aid and Civil Protection (hereafter DG ECHO) in view of better exploiting synergies and reinforcing the coherence of EU response operations.

4.

2.2. Overview of the Mechanism


The Mechanism supports the mobilisation of emergency assistance in the event of major disasters – any type of natural or man-made – inside and outside EU. At the end of 2013, 32 countries participated in the Mechanism: all 28 Member States of the European Union plus the former Yugoslav Republic of Macedonia, Iceland, Liechtenstein and Norway (hereafter together the Participating States).

The primary responsibility for dealing with the immediate effects of a disaster lies with the Participating State or the country where the disaster has occurred. Nevertheless, when the scale of an emergency overwhelms national response capabilities, a disaster-stricken country can request and benefit from the civil protection means and teams of the Participating States. By pooling civil protection capabilities of the Participating States, the Mechanism can ensure better protection, primarily of people, but also the environment and property, including cultural heritage.

Another main objective of the Mechanism was to support and complement the preparedness efforts of the Participating States. This includes a wide range of preparedness activities, such as training courses, exchanges of experts, simulation exercises and different types of cooperation projects. The period 2007-2013 saw also an increase in prevention activities.

5.

2.3. Main Mechanism actions


In the period 2007-2013 the Mechanism consisted of a number of elements, outlined below, intended to facilitate adequate prevention and preparedness as well as effective response to disasters at the EU level. The majority of these were financed via the CPFI Decision. Some were financed through additional funds granted by the European Parliament and Council i or other EU Instruments, such as the European Neighbourhood and Partnership Instrument.

Monitoring and Information Centre (hereafter MIC) was the operational heart of the Mechanism and played three key roles: i) monitoring (exchange of alerts and early warning information on upcoming disasters, facilitated by the Early Warning Systems i (hereafter EWS)); ii) information provision to the general public and to the Participating States via the Common Emergency Communication and Information System (hereafter CECIS); and iii) coordination of assistance provided through the Mechanism i. As of 15 May 2013, MIC was replaced by the Emergency Response Coordination Centre (hereafter ERCC), which since ensures 24/7 monitoring and immediate reaction to disasters all over the world. Over the period 2007-2013, the MIC/ERCC managed 223 Mechanism activations, of which 132 times outside the Participating States in a total of 69 countries. It also coordinated a total of 72 missions and deployed 246 experts and 64 MIC/ERCC liaison officers during these missions.

The training programme, comprising training courses, simulation exercises and an exchange of experts system, was designed to establish a common understanding of cooperation in civil protection interventions and to accelerate the response to major emergencies. In the period 2007-2013, 4 657 experts attended training courses, 882 experts from 29 Participating States took part in the exchange of experts programme and 31 simulation exercises were selected under the 2007-2013 calls for proposals out of the 58 proposals received.

Modules and Technical Assistance and Support Teams (hereafter TAST) were pre-defined, specific and interoperable emergency response units deployable at short notice. At the end of 2013 Participating States registered 150 modules and ten TAST in CECIS. Also funded were specific modules exercises, aimed at training modules likely to meet in a real emergency.

Prevention and preparedness projects were aimed at awareness raising and closer cooperation of Participating States in the field of civil protection. In the period 2007-2013 the priorities for prevention and preparedness projects were defined and the objectives of the call for proposals set on an annual basis (except in 2007 when there was no call for preparedness but only for prevention projects). A total of 76 prevention and preparedness projects were selected under the 2007-2013 calls for proposals out of the 371 proposals received.

A prevention policy framework was developed to complement EU policies in the areas of disaster response and preparedness. Various prevention activities were initiated and implemented to address the actions stipulated in the 2009 Commission Communication on a Community approach on the prevention of natural and man-made disasters.

Participating States were able to request support and co-financing for the transport of civil protection assistance to a country affected by an emergency. The Commission was responsible for supporting Participating States in: i) sharing or pooling of transport assets made available on a voluntary basis; ii) identification of transport assets available on the commercial market or from other sources (transport contractor); and iii) use of EU funding (grants or transport contractor) to pay for the necessary transport assets. A total of 122 transport grants were awarded in the period 2007-2013, resulting in total Commission net contribution of c. €10.9 million. The transport broker was used twelve times during the same period: nine times as a standalone solution and three times in combination with a grant.

Marine pollution related activities were also an important part of the overall Mechanism actions. In this area the Commission cooperated closely with the European Maritime Safety Agency (hereafter EMSA). During the period 2007-2013, the MIC/ERCC was activated nine times for marine pollution, with four activations leading to the deployment of a total of 13 experts and three MIC/ERCC liaison officers; five preparedness and two prevention marine pollution projects were co-financed, together with one marine pollution simulation exercise.

Pilot Project and Preparatory Action programmes ran during 2008-2010 and were financed through additional funds granted by the European Parliament and Council. Preparatory Action was established in order to test new ways of improving the EU's disaster response capacity: for the 2008 call for proposals 15 project proposals were received and five chosen, with total co-financing of c. €3.1 million; the 2009 call for proposals led to 15 proposals, of which seven projects were approved and received c. €6 million; the 2010 call for proposals resulted in 14 proposals, of which five projects were approved and received total funding of c. €7 million. The Pilot Project – EU Forest Fire Tactical reserve (hereafter EUFFTR) consisted of two fire-fighting planes – a supplementary resource designed to reinforce the overall EU fire-fighting capacity – and intervened in six of the total of nine 2009 forest fire emergencies.

Actions with third countries were designed to ensure that actions between Participating States and third countries were better coordinated to prevent, prepare and respond to disasters. These included: i) actions with candidate countries and potential candidate countries, namely the IPA Civil Protection Cooperation Programme I, which ran from December 2010 to November 2012 and consisted of a trainings, exchange of experts, exercises and workshops; ii) actions with European Neighbourhood countries, in which the EU cooperated with these countries through the Prevention, Preparedness and Response to Natural and Man-made Disasters Programme (PPRD South and PPRD East); and iii) actions with other third countries such as those with which a cooperation agreement was signed, as well as ASEAN and China.

6.

3. RELEVANCE OF THE MECHANISM ACTIONS IN 2007-2013


The independent external evaluation highlighted in particular the following points as regards the relevance of the individual Mechanism actions:

All stakeholders consulted in the external evaluation considered the MIC/ERCC and associated tools (CECIS and EWS) to be relevant and were satisfied with the monitoring, information management and coordination provided. Participating States considered the MIC/ERCC as crucial to ensuring: effective, efficient and rapid emergency response in the event of disasters; effective deployment of EU Civil Protection Teams; and complementarity of the EU interventions in disasters with Member States' and international response. One of the MIC's key shortcomings – lack of 24/7 coverage – highlighted also in the interim evaluation, was addressed by the transition from the MIC to the ERCC. The majority of Participating States considered the training courses to be relevant to the needs of their national civil protection authorities. The occupancy rates for the training courses were high throughout the 2007-2013 period, with on average 88.6% of all available spaces taken. However, the external evaluation also pointed out that there was no assessment of the optimum number of experts needed to be trained to cope with the volume of deployments in the framework of the Mechanism. The percentage of trained experts actually deployed was estimated at just over 10%. The simulation exercises covered a wide variety of scenarios which were highly relevant to Participating States. The majority (all but four) of Participating States took part in the exercises – 18 as the lead organiser – which further illustrated their relevance. However, during the period 2007-2013 the number of proposals received and simulation exercises co-financed varied and as a trend decreased: 2008 saw a record number of 18 proposals submitted, compared to 5 in 2013 (end of evaluation period); similarly 8 simulation exercises were co-financed under the 2007 and 2008 calls for proposals, compared to 2 in 2013. The results of the online survey of participants in the exchange of experts programme showed that 99% of those that partook in the survey were of the opinion that their competences and skills improved as a result of the exchange. A large majority of participants and hosts described the administration of the programmes as helpful, clear and competent in communicating details related to the organisation of the exchanges. Some stakeholders considered the volume of the exchanges as too limited and thought the programme should have the capacity to finance additional exchanges. In addition, there was a strong imbalance in Participating States' participation in the programme. The modules and TAST were deployed in a number of disasters inside and outside the EU, with 12 modules deployed in 2013 alone. Whilst most Participating States agreed that relevant guidelines and Standard Operating Procedures for modules were developed and implemented during the period 2007-2013, consultation with other stakeholders and the case study showed a need for more international and unified guidelines and standards for establishment, operation and common procedures of modules. Module exercises were seen by many stakeholders as essential and relevant to identify and address gaps in the operation of modules, thereby increasing the level of cooperation with other modules and improving the coordination of assistance. The prevention and preparedness projects selected under the 2007-2013 calls for proposals were in line with the specific objectives defined in the calls for proposals and were considered relevant for the overall objectives of the Mechanism. The selection process was regarded as adequate, although it was noted that its duration could be reduced. Finally, the process of monitoring projects was thought to be transparent and straightforward. Participating States, the European Parliament and the Council repeatedly called for action on prevention, hence the various activities undertaken to develop a prevention policy framework clearly responded to the needs of stakeholders. The interim report stipulated that the EU prevention policy framework should be able to address the various prevention aspects in different EU policy fields and facilitate further co-operation among Participating States. Evidence gathered suggested that civil protection has since been integrated with 13 other EU policies and that cooperation was aided through a number of expert meetings that resulted in the sharing of good practice. The recommendations have therefore been followed upon. However, prevention remained a high priority for stakeholders, as the development of a prevention policy framework was recent and therefore needs more time to mature. The development of the legal framework to enable transport support was highly relevant, as it answered a previously identified need. Although the uptake was slow at the start of the evaluation period, the increased use of the transport assistance during the evaluation period showed that by the end of the evaluation period it had become a well-established action. It also showed that the Commission has taken on board the recommendation of the interim evaluation to make the procedures and documentation requirements less cumbersome. The evidence suggested that the transport assistance positively contributed to the decision to provide civil protection assistance. The involvement of the Mechanism in marine pollution required close cooperation with EMSA. Marine pollution activities were found by the external evaluation to have contributed to three key Mechanism objectives: MIC/ERCC coordinated the EU responses to marine pollution responses nine times between 2007-2013; MIC/ERCC also facilitated the support to the affected countries, often in cooperation with EMSA, which provided expertise and logistical support; these operations contributed to the protection of human lives and the environment. The pilot project and preparatory action projects contributed to a strengthened EU response capability, by responding to critical needs arising from major disasters. Consulted stakeholders reported that the availability and readiness of the Canadair aircrafts that were part of the pilot project was a real asset in fighting forest fires. Equally, the preparatory actions allowed for testing innovative arrangements and the development of new modules that were subsequently deployed under the Mechanism. Actions in third countries corresponded to the needs of both the beneficiary third countries and the Participating States. For third countries, the actions contributed to enhancing their capacity to respond to disasters and to a better understanding of the Mechanism and its capabilities. For Participating States, building capacity in third countries, in particular neighbouring countries, may have contributed to reducing the need for Mechanism assistance in the future, preventing certain types of disaster from spreading (cross-border), or increasing the speed of the response in cases disasters.

Overall, taking into account its collective operational experience with the Mechanism and its positive evolution, the Commission agrees with the results of the independent evaluation that all Mechanism actions were relevant elements to an overall policy contributing to improved national systems for civil protection disaster management inside Europe and beyond. These conclusions are further supported by quantitative data i confirming, for example, the increase in the numbers of Mechanism activations, transport co-financing requests, module deployments and high levels of participation in exercises, trainings and exchange of experts.

7.

4. COHERENCE OF THE MECHANISM ACTIONS IN 2007-2013


The independent external evaluation highlighted in particular the following points:

94% of Participating States that responded to the online survey considered that the MIC/ERCC improved the coordination between the EU and the Member States with regard to civil protection assistance during interventions. This is a clear signal that the Commission has followed up on the recommendations of the interim evaluation that improvements in coordination between the MIC/ERCC and Participating States and international partners would be desirable. At the EU level, the MIC/ERCC was involved in the coordination of all types of disaster response and ensured full coherence amongst Commission services and other EU institutions in case of disasters (e.g. cooperation with DG SANCO in Haiti, DG ENER in Japan, EEAS in Pakistan, and EMSA in USA). All international partners interviewed considered the MIC/ERCC as a reliable partner and able to provide information and channel support. The MIC/ERCC collaborated extensively with other Commission services and international partners: more than 60 times during the period 2007-2013. The linkages between training courses were coherent by design, as the courses covered introductory, operational and management subjects and were interlinked with each other. A few minor remarks concerned specific areas (e.g. media management) that could be introduced in a coherent way throughout the different courses. The simulation exercises were coherent with other Mechanism actions, in particular with the training courses and module exercises, whose focus and modus operandi was different and generated complementary outcomes. Depending on the exercise scenario, a number of simulation exercises also allowed for the integration of third countries, which further enhanced the external coherence of the Mechanism. However, it was argued already in the interim evaluation that the coherence of simulation exercises could be improved if an overarching strategy or framework for the simulation exercise programme would be established. This recommendation is currently being implemented as a result of requirements introduced in the 2013 Decision. The exchange of expert programme was sometimes perceived by Participating States as an alternative to training courses rather than a complementing action and a forum for the exchange of knowledge and good practices. The programme was however perceived as creating opportunities to learn more about the role of the Mechanism and providing networking opportunities that can prove to be essential in an actual disaster. The pre-defined and interoperable modules represented additional capabilities that could be deployed at short notice and were thus important for achieving the Mechanism’s objective to facilitate cooperation in the field of civil protection. The module exercises were coherent in so far as they provided essential training and testing of capabilities and procedures with very realistic emergency scenarios. Prevention and preparedness projects selected were in line with the priorities established annually in the call for proposals. Actions implemented by the selected projects were also coherent with those funded through other EU instruments. However, it was suggested that the priorities for prevention and preparedness projects defined in the call for proposals were quite broad: defining more precise priorities could contribute more effectively to the Mechanism and to avoiding any potential risk of duplication of efforts with related EU funding instruments. The prevention policy framework was further integrated into other EU policies and was also coherent with international disaster risk management policies and agenda. In addition to existing efforts, it was suggested that further actions could be envisaged to further integrate prevention activities into national civil protection policies, linking prevention to preparedness and response. The different components of the transport assistance process (i.e. the pooling phase; identification of transport options on the commercial market; and co-financing) directly contributed to the key objectives of the Mechanism, which were to coordinate the provision of assistance and the deployment of this assistance when required. With regards to marine pollution, a series of common objectives were set for the cooperation between EMSA and the Commission (DG ECHO) and different mechanisms put in place to ensure a clear division of tasks and good coordination. The Commission also actively cooperated with four regional agreements developed in the EU sea-basins, a collaboration that was assessed as very strong and well organised by the external evaluation. In addition, the Commission participated in the Inter-Secretariat Meeting between regional agreements, the Commission and EMSA. These findings also confirm that the Commission has been successful in taking appropriate steps to implement the recommendations raised by the interim evaluation, namely that an improvement in the coordination with EMSA was required. One of the main objectives of preparatory action projects was to make equipment and other resources available to be deployed to respond to disasters, which was the case in a number of instances (e.g. Haiti earthquake). As for the pilot project, it contributed to reinforcing the overall EU fire-fighting capacity. Results achieved through the pilot project and preparatory actions were therefore considered by the external evaluation to be in line with their objectives, i.e. filling existing gaps in a coherent manner. The desk research and stakeholder interviews conducted as part of the external evaluation of actions with third countries concluded that the successful implementation of activities under the IPA, PPRD South and East programmes contributed to increased cooperation between candidate and potential candidate countries as well as neighbouring countries and the EU, thus fostering more coherent collaboration in prevention, preparedness and response actions.

The Commission's experience with Mechanism operations and the conclusions of the feedback and information sessions with all relevant parties post operations / exercises / trainings, support the results of the external evaluation, which found that Mechanism actions were complementary to each other and other actions at national, EU and the international level. Nevertheless, the Commission considers that there is further scope for a more comprehensive framing of all Mechanism actions and for enhancing opportunities for cross-action learning and will aim to address this by setting an overall strategic framework for trainings, exercises and lessons learned. Results of the Commission's on-going internal coordination between different services, as well as contacts with relevant external stakeholders, support the external evaluation conclusion that the overall Mechanism implementation was consistent with related EU and international actions and programmes.

8.

5. EFFECTIVENESS OF THE MECHANISM ACTIONS IN 2007-2013


The independent external evaluation highlighted in particular the following points:

The MIC/ERCC was considered an effective platform with the appropriate features and adequate resources to achieve its objectives and to fulfil its functions. All Participating States responding to the online survey considered that the MIC/ERCC successfully ensured communication between the EU level and the Participating States and almost all considered that the MIC/ERCC was accessible and able to react immediately 24/7. Different stakeholders remarked that the time needed by the MIC/ERCC to coordinate and deploy the EU assistance decreased over the years. Some criticisms came from a smaller group of Participating States and other stakeholders concerning: i) the selection of national experts to be deployed to emergencies (the view was that the MIC/ERCC should have a greater role in selecting experts in order to better align the skills with the actual needs in the field); and ii) lack of field experience of MIC/ERCC staff. The training courses achieved their objectives to improve the individual skills and competences of the experts and to establish a common understanding for cooperation in civil protection interventions. The evaluation suggests that the link between the skills required on the field and lessons learned from deployments and the training courses could be strengthened and that this process could be made more systematic. The simulation exercises have broadly achieved their objectives, in particular promoting better coordination and faster response times, and contributed to the overall Mechanism. A number of broader goals were also achieved by simulation exercises, such as exploring possibilities for collaboration between UNDAC and EU Civil Protection Teams, and further developing the operational and strategic collaboration between the humanitarian aid and civil protection on national and EU level. The exchange of expert programme was considered effective to the extent it fulfilled the objectives of the participants. Many experts reported that they had developed directly applicable specialist skills. However, their expectation to be deployed in EU CP teams following the exchange was not met for many of them. While the programme was overall highly valued by both participants and host organisations, the feedback process could be improved as it is pivotal to measuring its effectiveness. The interim evaluation recommended that the modules concept should be further developed, including through specialised exercises and developing standard operating procedures. These recommendations were fully implemented. The number of modules and TAST registered in the CECIS database increased over the period 2007-2013, reaching a total of 150 modules at the end of 2013. 16 different types of modules were registered, out of 17 defined. In order to increase the effectiveness of registered modules, stakeholders suggested further developing the minimum requirements and guidelines for modules. Most surveyed Participating States considered that the module exercises met their objectives to the extent they encouraged civil protection staff to take account of lessons learned and to introduce improvements to the operation of modules. However, it was suggested that more frequent opportunities to exercise modules, even on a smaller scale and with less complex exercises, would contribute to their sustainability in terms of skills gained. The results of prevention and preparedness projects met the objectives set in the annual work programmes and resulted in concrete actions, such as supporting the development of disaster prevention strategies and raising awareness on specific issues. Nevertheless, the impact and sustainability of prevention and preparedness projects stayed too limited: in particular, the implementation of results at national level was somewhat lacking, the results were not systematically promoted and made visible, and the transfer to other Participating States lagged behind expectations. The various prevention activities developed and implemented contributed to achieving the objectives stipulated in the 2009 Commission Communication and the Council Conclusions on prevention. Notably the activities contributed to more knowledge-based disaster prevention policies, to linking prevention actors and policies to the relevant preparedness and response actors, and to the mainstreaming of disaster prevention considerations into existing EU financial and legislative instruments. 77% of Participating States surveyed considered that the provision of transport support through grants effectively contributed to improving the response to emergencies. A number of stakeholders stated that the transport broker was adapted to situations requiring high levels of flexibility and that there is potential for considerable improvement in its use. Finally, experience from certain disasters suggested that EU co-financing limited to 50% significantly reduced the effectiveness of the instrument. This echoes one of the recommendations of the interim evaluation, which was to investigate the potential for different levels of co-financing. The Commission has since made progress in that respect, with the introduction of new, higher rates of co-financing in the 2013 Decision. The effectiveness of marine pollution actions depended directly on the good cooperation with EMSA. In all the operations involving EMSA this cooperation was considered effective. Equally, marine pollution prevention and preparedness projects were considered highly effective, delivering results otherwise not possible without the financial support of the Commission (DG ECHO). The evaluation of the marine pollution simulation exercise came out inconclusive, as the Commission and EMSA had different perspectives on its effectiveness. Most Participating States surveyed considered that the pilot project and preparatory actions contributed to more effective disaster response by complementing existing capacities rather than duplicating previous efforts and results. For example, compared to individual Participating States' solutions the pilot project offered crucial support in the rare but severe events of several simultaneous large forest fires. The effectiveness of preparatory action projects was limited to the extent that the dissemination of results could have been broader. Overall, collected evidence indicated that the objectives of the actions with third countries, namely IPA Programme and PPRD South and East Programmes, were achieved. Some stakeholders highlighted the need to focus more on prevention actions in the future, in line with the developments of the Mechanism.

The Commission's view is that, as evidenced by the Commission's operational experience and extensive feedback in carrying out or supporting many successful prevention actions, preparedness activities such as exercises, and response operations carried out in 2007-2013 period, the overarching policy objective of the Mechanism as a whole, i.e. facilitating reinforced cooperation, was fully achieved. The success of these actions is in some cases immediately visible, such as for example in the case of the pilot project that was able to support Participating States in fighting forest fires in the absence of other available response capacities, whilst at other times the results are less tangible immediately, and consist for example, of teams learning to work with each other in view of possible cooperation in a real life emergency. The Commission's view is supported by the results of the external evaluation, which concluded that the implementation of Mechanism actions was effective insofar as all individual Mechanism actions achieved their specific policy objectives.

9.

6. EFFICIENCY OF THE MECHANISM ACTIONS IN 2007-2013


The independent external evaluation highlighted in particular the following points:

1. A total of some €20 million was allocated in the 2007-2013 annual work programmes to fund the different functions of the MIC/ERCC, including maintenance and further development of MIC/ERCC, CECIS and EWS and deployments. The evaluation noted that the MIC/ERCC was particularly efficient in facilitating the provision of assistance in a coordinated way which allowed an efficient usage of resources. The switch from MIC to ERCC, which was staffed and hence better accessible 24/7, was another important factor of efficiency gains, especially with regard to planning and responding quickly to emergencies. One of the findings of the interim evaluation was that there was scope for operational improvements in EWS. Since then, there was considerable investment in EWS. The budget allocated to EWS increased from €0.7 million in 2009 to €1.6 million in 2013 and was used to develop a full multi-hazard coverage of near-real time EWS.

2. A total of some €29 million was allocated to training courses in the 2007-2013 annual work programmes. Between 2009 and 2010, the number of offered training courses doubled from six to twelve. Between 2010 and 2013 the training courses witnessed a steady increase of participants, while the amount of financial resources invested remained constant. Training participants confirmed that the participation in trainings created a common understanding of the Mechanism, which in turn generated efficiencies by providing a coordinated and thus more effective response.

3. The overall size of the budget allocated to simulation exercises in the 2007-2013 annual work programmes was over €20 million. Depending on the scope and complexity of the scenario the individual budgets varied greatly. Due to the specific characteristics of each scenario an efficiency comparison between simulation exercises was difficult. However, according to third-party financial and audit reports examined during the external evaluation, the budget of each simulation exercise was considered appropriate and proportionate to their individual objectives.

4. The total amount dedicated to the exchange of experts programme in the 2007-2013 annual work programmes amounted to €1.75 million. The results of the external evaluation show that some Participating States considered the overall capacity of the programme limited, commenting on the long waiting times for experts to be assigned to exchanges. The actual administration of the exchanges was described by experts as good and the selection process as appropriate and conducive to achieving the aims of the programme.

5. The 2007-2013 annual work programmes allocated a total of over €13 million to modules exercises. According to the external evaluation, whilst overall module exercises were perceived very positively, some stakeholders were not sure whether it was efficient to invest in preparing such a significant number of modules whilst the share of actually deployed modules under the Mechanism remained relatively small in recent years. In addition, module exercises were thought not to have included equally all types of modules, but mainly focused on a few.

6. The overall budget available for prevention projects was about €14 million and for preparedness projects some €10.5 million in the period 2007-2013. The majority of Participating States surveyed considered the size of the budget appropriate and proportionate to what the projects were set out to achieve. Most budgets of completed projects were proportionate with regard to the concrete outputs produced.

7. The consensus among the Participating States was that prevention is a cost-effective and legitimate way to reduce human and economic costs of disasters. However, it was noted that it was unclear whether the prevention activities had fully contributed to an enhanced capability manage disasters at EU level, and how prevention activities interacted with other Mechanism actions.

8. The 2007-2013 annual work programmes allocated a total of €62.9 million to transport assistance, resulting in Commission’s net contribution to transport assistance of around €11 million during this period. Two main observations came out of the external evaluation: the transport pooling phase mandated by the legal framework was thought to ensure the efficient pooling and combined transport operations, however, in view of the limited actual outcomes of pooling, the necessity of the pooling phase was not certain; and although the use of the transport broker was designed in a way which ensured cost-effectiveness its use was nevertheless minimal.

9. The 2007-2013 annual work programmes did not allocate separate funding for marine pollution actions. As accidental marine pollution was mainstreamed through the different actions of the Mechanism, it was integrated in the different budget lines and assessed as part of those.

10. The EU Budgetary Authority earmarked a total of €22.5 million for pilot project and preparatory actions for the period 2008-2010. The external evaluation noted that the budget available was adequate to achieve planned activities and that it was used to deliver concrete outputs, including enhanced response capacity, design of exercises and/or modules, training programmes and communication activities.

11. Actions with third countries were funded through different financial instruments. The IPA Civil Protection Cooperation Programme I was funded under the Instrument for pre-accession Assistance (IPA) for the period 2010-2012. PPRD South ran between 2009 and 2012 with a budget of €5 million, whereas PPRD East covered the period 2010-2014 with a budget of €6 million. In addition, the EU-China Disaster Risk Management project was launched in 2012 and was funded through the Development Cooperation Instrument, with the total budget amounting to €6 million. The external evaluation noted that due to the diversity of the actions with third countries and different funding mechanism, the overall efficiency of actions with third countries was potentially reduced due to the lack of an overarching framework for these actions.

Overall, the Commission agrees with the results of the external evaluation that Mechanism actions were designed to be cost-effective (e.g. in the case of training courses, the budget remained stable in the period 2010-2013 yet the number of participants benefiting from the courses increased) and were broadly implemented efficiently. Beyond the external examination of the quantitative and qualitative data, the Commission's operational experience with the Mechanism reveals a more nuanced picture. For example, the increased investment in EWS was relatively small compared to the budget allocated to other Mechanism actions, but resulted in a significant operational benefit for the Mechanism. Equally, the investment in training resulted in a large number of experts trained but not necessarily deployed on Mechanism operations; this finding underestimates the significant benefit of having Mechanism trained experts in national operations and their potential to spread the knowledge.

10.

7. EU ADDED VALUE OF THE MECHANISM ACTIONS IN 2007-2013


The independent external evaluation highlighted in particular the following points:

The added value of MIC/ERCC was manifold. For Participating States the MIC/ERCC represented a ‘one-stop-shop’ for responding to disasters, with clear added value over bilateral response. CECIS was considered by all stakeholders as a useful centralised information sharing platform with clear EU added value. For third countries and international partners, the MIC/ERCC enabled them to reach out and to request assistance through one single platform, making cooperation easier. EWS enabled Participating States access to state-of-the-art alert systems covering the full spectrum of hazards, as well as a stronger science/operational interface to enable innovation. The training courses demonstrated strong EU added value, resulting in participants not only gaining skills and knowledge about the Mechanism but also strengthening the network of civil protection professionals. It was argued by the external evaluation that this networking effect rendered cooperation and interventions easier. Based on the interviews, in at least five out of six cases the simulation exercises would not have taken place without EU co-funding. Exchange of experience and learning from best practices was one of the key results and also provided strong EU added value. In addition, exercises allowed for a re-creation of emergency scenarios and were considered by stakeholders as complementary to training courses. The exchange of experience and learning from best practices was one of the key results of the exchange of experts programme. The programme resulted in strong connections being fostered at national and European level, which made cooperation much easier. Without EU financial assistance, it was considered unlikely that such exchanges would have taken place. The EU added value from modules and modules exercises is obvious insofar as they directly supported the national response to disasters in affected countries. The testing of skills and procedures during module exercise created a common understanding for coordinated assistance. The prevention and preparedness projects achieved results which could not have been achieved by Participating States individually. Most projects involved several partners from different Participating States, thereby improving mutual knowledge and contributing to building trust across Participating States. The prevention activities demonstrated EU added value: prior to the development of a prevention policy at EU level, not all Participating States had well-developed policies aimed at prevention of disasters – the activities undertaken at EU level incentivised those Participating States to undertake additional efforts to develop such policies. Participating States also benefited from the exchange of practices and expert meetings at EU level. The transport assistance proposed by the Mechanism brought EU added value by directly supporting the deployment of disaster support assistance in affected countries. Concrete examples included cost-savings at Participating State and EU level by supporting the pooling of transport assets and flexibility when warranted by the situation in the field. The Mechanism was found to have delivered EU added value in marine pollution, primarily through bringing the marine pollution and civil protection communities closer together and supporting cooperation and collaboration where needed. Marine pollution prevention and preparedness projects were highlighted as a key value adding initiative. The pilot project and preparatory actions contributed to the development/availability of additional capacities which otherwise might not have been developed/made available by Participating States individually. In addition, cooperation between Participating States was strengthened. In some cases, the results of preparatory actions led to an overall assessment of capacity needs and to the use of standard procedures. All stakeholders confirmed the EU added-value of actions with third countries. The most important reported result was the reduction of the impact of disasters brought about by improving the civil protection capacities. Participation in the third country actions was also thought to have a positive impact on national policies. Most third country stakeholders interviewed also had first-hand experience with Mechanism activations and acknowledged their EU added value in third countries.

The Commission concludes, based on the results of the external evaluation, lessons learned from the large number of Mechanism operations in Europe and beyond, and feedback from countries requesting Mechanism assistance, that he Mechanism demonstrated EU added value on multiple levels, including by strengthening the cooperation between Participating States, addressing the need for appropriate prevention activities, facilitating the deployment of teams and assistance and offering a single ‘package’ of support, including civil protection experts and modules and TAST. Overall, the Mechanism constitutes a successful EU policy area with a growing demand from Member States and other stakeholders.

11.

8. CONCLUSIONS


Considering the findings of the external evaluation report and taking into account its own operational experience and lessons learned, as well as its own assessment of the qualitative and quantitative evidence and data, the Commission's overall conclusion is that the Mechanism actions that received financial assistance in the period 2007-2013 have performed very satisfactory and achieved their objectives. The overall evaluation of the implementation of the Mechanism observed generally very good results and clearly demonstrated the relevance, coherence, effectiveness, efficiency and EU added value of the Mechanism as a whole and of its individual actions. The Mechanism actions were successful in many respects, not least in improving coordination and cooperation and enhancing the Participating States' preparedness and response capacities; providing an efficient, rapid and effective response to emergencies; and providing access to transport resources for ensuring rapid response.

The Commission notes the significant progress the Mechanism underwent between 2007 and 2013, not least the move from the MIC to the ERCC, the increased focus on prevention policy and actions, and increased cooperation with third countries. Many of the ideas that were nascent at the start of the evaluation period are now fully fledged Mechanism actions delivering significant benefits to the Participating States, such as the transport assistance and the modules concept. The Commission also highlights the substantial progress made by the Mechanism since the Interim evaluation report with regards to its recommendations, all of which have been taken into account by the end of the evaluation period and have been or are in the process of being implemented, including as a result of the 2013 Decision.

In the light of the above, the Commission concludes that European cooperation and coordination in the field of civil protection made substantial progress since 2007 and established a good basis for continuing these positive developments. The external evaluation offers a number of positive examples that demonstrate the extent to which the Mechanism and the EU's disaster response coordination evolved and became better at achieving objectives. The CPFI financing used for this purpose has generated substantial EU value added.

The external evaluation, despite its positive assessment of the performance of the Mechanism and all of its actions, also highlighted some potential for additional improvements. Most significant recommendations included creating a more coherent framework for Mechanism preparedness actions, including exercises, training, projects, exchange of experts, and better planning, as well as more streamlined response procedures and higher transport EU co-financing rates. The Commission agrees that in a number of areas there is scope to further enhance and develop the Mechanism and welcomes the external evaluation recommendations.

The 2013 Decision provides the legal foundation for further significant progress in prevention, preparedness and response, and specifically addresses many of the issues identified in the external evaluation (e.g. the 2013 Decision calls for a strategic framework for simulation exercises). The Commission will therefore aim to take the majority of these recommendations into account in the already ongoing implementation of the 2013 Decision. Recommendations related to operations will be taken into account as part of the on-going work on improving the procedures (e.g. calls for proposals) or the running of programmes.

The Commission invites the European Parliament and the Council to take note of these evaluation findings.
C(2010)1206