Explanatory Memorandum to COM(2023)571 - Ex post evaluation of the Structural Reform Support Programme - Main contents
Please note
This page contains a limited version of this dossier in the EU Monitor.
| dossier | COM(2023)571 - Ex post evaluation of the Structural Reform Support Programme. |
|---|---|
| source | COM(2023)571  |
| date | 05-10-2023 |
Ex post evaluation of the Structural Reform Support Programme
Introduction
Contents
- Structural reforms are essential for modernising economies, encouraging investment, creating jobs and raising living standards. Reforms are complex processes by their very nature.
- In compliance with the Better Regulation guidelines, the SRSP ex-post evaluation is articulated around the following five criteria:
- General objective
- Effectiveness
- Cross-country dimension
- Facilitator role of the European Commission
- Effective implementation of reforms and coordination with other EU processes
- Technical Support Instrument
Structural reforms are essential for modernising economies, encouraging investment, creating jobs and raising living standards. Reforms are complex processes by their very nature.
In May 2017, Regulation (EU) 2017/825 on the establishment of the Structural Reform Support Programme (“the SRSP Regulation”) was adopted. It was meant as a tool to provide technical support to national authorities and help Member States design and implement institutional, administrative and growth-sustaining structural reforms.
The budget for the Structural Reform Support Programme (“the programme” or “the SRSP”) was EUR 222.8 million over 2017-20. The programme was managed by the Structural Reform Support Service (SRSS) which, on January 1 2020, became the Directorate-General for Structural Reform Support (DG REFORM).
Methodology
An independent external contractor conducted an evaluation study to inform the ex-post evaluation of the Structural Reform Support Programme. The study was carried out over a period of 11 months, starting on 11 October 2021. It included 826 SRSP-funded projects from 2017 to 2020, covering beneficiaries in all Member States.
The contractor deployed a complex methodology in line with EU Better Regulation guidelines to collect robust evidence, consisting of extensive desk research (including DG REFORM’s information extracted from its internal monitoring tool, with data up until November 2021), stakeholder consultation activities (interviews with key stakeholders, open public consultation, a survey of key stakeholders and focus groups), and in-depth case studies (30 projects in 8 Member States, and 1 multi-country project covering 17 Member States.)
The contractor carried out all tasks under the scrutiny of an inter-service steering group and the guidance of DG REFORM. The quality and the representativeness of the collected evidence was considered to be solid and meaningful, allowing us to draw conclusions and provide reliable and valid answers to the evaluation questions.
Main findings
In compliance with the Better Regulation guidelines, the SRSP ex-post evaluation is articulated around the following five criteria:
·Relevance
·Effectiveness
·Efficiency
·Coherence (consistency)
·EU added value (value that is additional to what would otherwise have been created by EU countries acting alone)
The general objective of the SRSP was to contribute to institutional, administrative and growth-sustaining structural reforms in the Member States by providing support to national authorities. The support was also designed to help non-euro area Member States prepare for joining the euro area.
Overall, according to available evidence, the implementation of the SRSP was successful in terms of effectiveness, efficiency and coherence. The programme was also found to be highly relevant in addressing the needs of beneficiary authorities and Member States and had ‘EU added value’.
The programme successfully achieved its general objective and has substantially advanced the reform agenda in Member States. The actual achievement of different types of reforms also depended on the wider efforts of the Member States and not only the results of other EU programmes and instruments.
Where the projects have not generated the full intended results and impacts, this tended to reflect wider challenges in the reform process rather than any inherent weaknesses in the projects or in the provision of technical support. More policy impacts are, however, expected to occur in future, as about one third of all evaluated projects were still ongoing during the ex-post evaluation study.
In addition, recommendations from some SRSP projects were integrated as milestones in the national recovery and resilience plans, ensuring there would be an impact when the milestones were completed. 1 More time would thus be necessary for the expected results and impacts to occur, due to the long logical chain between the provision of technical support and the expected results/impacts.
Active involvement by the beneficiary authorities and their ownership of the projects is key to achieving the results. This involvement is essential not only during the project implementation and follow-up, but also in the pre-implementation phase, in terms of defining the project objectives and timelines, creating ownership and commitment to change.
While absorbing technical support and implementing its recommendations, beneficiary authorities sometimes lacked adequate administrative capacity, which affected the smooth running of the technical support projects and the implementation of resulting recommendations. While the provision of technical support does not imply costs for the beneficiary authorities, sometimes they lacked sufficient financial resources to implement project recommendations and the related reforms.
Overall, communication activities were rarely included in project design, despite an increase in the number of communication initiatives in the 2019 and 2020 rounds of the programme. However, when engagement and communication with external stakeholders happened during the provision of technical support it was crucial, because the effective implementation of these activities contributed to increase the legitimacy, quality and sustainability of the reforms and reduce organisational resistance to change.
Efficiency
The SRSP proved successful in attracting many high-quality requests relative to the budget available. Moreover, the programme was able to support projects of increasing size and budget, thus increasing the potential for projects to achieve greater impact. Growing demand for technical support among Member States and a sufficient level of competition among the SRSP requests enabled the authorities to select high-quality proposals.
The programme’s stakeholders – including beneficiary authorities, coordinating authorities and providers – generally agreed that the financial contribution from the SRSP was proportionate to the needs and estimated scope of technical support and regarded SRSP projects as a user-friendly and cost-effective instrument. The costs of controls on total annual payments at DG REFORM were comparable to other EU programmes under direct management mode. DG REFORM achieved good results in executing the SRSP budget.
Most notably, the assessment and selection of technical support requests was completed on time and in line with the target set in the Financial Regulation, compared with other EU programmes. As one of the selection criteria for funding a project under the SRSP was the urgency of the reform, Member States expected each project to be running as early as possible, to ensure the reforms remained relevant and were implemented on time.
The stakeholders involved in the programme were generally positive about the clarity, transparency and user-friendliness of the SRSP’s administrative procedures and the proportionality of the administrative burden. One area for improvement is related to the need to further develop and refine project monitoring and evaluation practices.
DG REFORM efficiently and effectively coordinated the technical support given to the Member States with other Commission departments. However, according to the study, there could have been closer cooperation with other Commission departments in 2017-20 in disseminating and exploiting the outputs from the technical support, once the projects had been completed.
Coherence
The SRSP is evaluated to be a coherent policy tool with clear internal logic, well aligned with the objectives set out in the SRSP Regulation, with no major gaps, inconsistencies or overlaps in the internal structure of projects. The design of the programme allowed for considerable flexibility when selecting project activities that are best tailored to specific project objectives and beneficiary needs.
The structure and activities of the SRSP were adjusted over time to better respond to newly emerging challenges and to achieve its objectives in a more effective way, e.g., by introducing dedicated calls for requests that met the newly emerging needs of Member States. However, the discrete nature of the projects means that additional effort could be required to disseminate and exploit project outputs to wider audiences, so that greater impact can be generated.
Evidence also shows that the SRSP was highly consistent with other EU programmes and instruments (including ESF, ERDF, Horizon 2020, COSME, Fiscalis), as well as those of the Member States themselves. This was because it addressed different aspects of similar objectives, issues and target groups. Having common themes with other EU programmes, the SRSP-funded projects largely complemented those projects by providing tailor-made technical support for national authorities in specific sectoral areas.
In terms of consistent objectives, the SRSP reinforced wider reform agendas of the Member States and the EU, including by addressing country-specific recommendations (CSRs) made as part of the European Semester process. Generally, the stakeholders of the SRSP highly valued the programme’s contribution to the Semester process. In some cases, the SRSP-funded projects helped better define, clarify and operationalise the CSRs, and provide specific guidelines for Member States to implement them. In other instances, the results of SRSP-funded projects were used to formulate these recommendations.
Although overall there were sufficient mechanisms to ensure the linkages and integration between the SRSP and the European Semester process/CSRs, the supporting study for the ex-post evaluation showed that there was insufficient monitoring of how the SRSP-funded projects contributed to meeting the CSRs. There was also a lack of references to SRSP projects in the European Semester country reports.
EU added value
The SRSP delivered this by providing Member States with better access to international expertise than they would have experienced in the absence of the SRSP. Even in the limited cases when project benefits could have been realised without EU technical support, these would have been much smaller in scope or realised later in time. As a result, EU added value was attained in terms of the strengthened reform processes in the Member States and strengthened contribution to achieving EU policy objectives, not least those of the European Semester.
The SRSP had significant cross-border, EU-wide impacts going beyond single Member States. Such impacts included helping the Member States to implement EU policies and priorities nationally, including the promotion of European values, as well as helping them to share good practice among themselves. In general, these good practices and lessons learned were actively used by the beneficiaries after the end of the relevant SRSP-funded projects.
Relevance
The SRSP was found to be a highly relevant instrument, well-tailored to improving the administrative capacities of Member States to design and implement the reforms needed to address the challenges they face.
The beneficiary authorities and other stakeholders viewed positively the different aspects of the SRSP’s design and structure and its relevance for achieving the reform goals. At the same time, the objectives of technical support projects largely corresponded to the key reform goals and challenges faced by the Member States.
The technical support provided by the SRSP was highly relevant to addressing the needs of beneficiary authorities and Member States, both in terms of the implementation method (i.e. international technical support providers sourced by the Commission) and the nature of the support provided (i.e. tailored to the needs of each beneficiary authority). The programme – uniquely and in an effective and efficient manner – fulfils the need of Member States to access high-level international expertise when they are in the process of designing or implementing reforms.
At the same time, where the SRSP was used in an emergency capacity it was not very suitable for coping with reform or capacity-building on the ground, since it was restricted by the timetable laid down in the calls for requests.
Despite having received the SRSP’s contribution over many years, beneficiary authorities were still in need of the technical support provided by the programme. The growing need for technical support was also showcased by the increasing number of requests from Member States, as well as by the increasing average budget for the projects.
Main lessons learned
The following main lessons have been learned from the implementation of the SRSP. The application of these lessons can help improve the design and management of future programmes for providing technical support. The lessons are grouped by subject.Engagement of stakeholders
·Active communication with and involvement of external stakeholders was crucial during the provision of technical support: it increased the legitimacy, quality and sustainability of reforms and reduced organisational resistance to change.
·Engagement of beneficiary authorities was key in implementing technical support projects and in the reform process, to enable the smoother implementation of projects and their results.
·Close involvement of national coordinating authorities was important in monitoring the implementation of the technical support projects, discussing their progress and taking follow-up action to use the outputs after project completion.
·Providing access to international expertise for Member States that might not otherwise have had access to it was a key feature of the programme.
·Exchanges of civil servants provided a basis for greater use of peer-to-peer advice within national administrations in future. The accumulated knowledge of civil servants in the Member States represented an enormous pool of expertise and skills.
·Disseminating the results of technical support projects within and across Member States encouraged the sharing and replication of good practice.
·The transnational dimension of the programme – for example through projects that support several Member States and through better dissemination of results to wider audiences – was instrumental in strengthening the ‘EU added value’.
·Reducing the complexity of technical support projects made their implementation easier and more effective.
·The capacity to provide tailored solutions resulted in low administrative burdens for beneficiary authorities and ongoing guidance for them, and it helped facilitate and legitimise the reforms.
·The Commission played a valuable role in facilitating collaboration between potential beneficiary authorities in different Member States who had similar needs and priorities.
·Improved monitoring and evaluation contributed to a more efficient implementation of the programme and a better dissemination and exploitation of technical support outputs. A solid monitoring system may allow to assess the impact of actions and reforms as well as their sustainability over time.
·When in place, the monitoring of the SRSP’s contribution to the European Semester process and the fulfilment of specific CSRs in specific national contexts was significant to understand what worked well and what could be improved.
·Technical support was also crucial in areas where the subject of the request is less frequent, such as assistance in applying and implementing EU legislation.
In 2021 the Technical Support Instrument (TSI) replaced the SRSP as the main programme for delivering technical support in 2021–27.
The TSI was designed on the basis of several recommendations proposed in the SRSP mid-term evaluation and factoring in some of the conclusions of this ex-post evaluation, such as more active involvement of stakeholders, a tighter focus on the cross-country dimension and increased attention on monitoring and evaluation.
(1) The Regulation (EU) 2021/241 establishing the Recovery and Resilience Facility was adopted in February 2021; by that time, many SRSP-funded projects had achieved their expected results and produced recommendations that were instrumental in defining milestones and targets in national recovery and resilience plans.