Legal provisions of COM(2020)386 - Ex Post evaluation of the 2018 European Capitals of Culture (Leeuwarden and Valletta) - Main contents
Please note
This page contains a limited version of this dossier in the EU Monitor.
| dossier | COM(2020)386 - Ex Post evaluation of the 2018 European Capitals of Culture (Leeuwarden and Valletta). |
|---|---|
| document | COM(2020)386  |
| date | August 18, 2020 |
Brussels, 18.8.2020
COM(2020) 386 final
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS
Ex Post evaluation of the 2018 European Capitals of Culture (Leeuwarden and Valletta)
{SWD(2020) 165 final}
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS
Ex Post evaluation of the 2018 European Capitals of Culture (Leeuwarden and Valletta)
1.Introduction
This report is presented in accordance with Article 12 of Decision No 1622/2006/EC of the European Parliament and of the Council of 24 October 2006 establishing a Community Action for the European Capital of Culture event for the years 2007 to 2019 1 (here after, the Decision), which provides that the Commission ensures each year an external and independent evaluation of the results of the European Capital of Culture event of the previous year 2 and reports on that evaluation to the relevant EU Institutions and bodies.
The findings and methodology of the ex post evaluation are presented more comprehensively in the accompanying Commission Staff Working Document.
2.Background to the Action
2.1.The EU Action for the European Capital of Culture (ECOC)
Since the launch – at intergovernmental level – of the European City of Culture in 1985 3 , the scheme grew into a fully-fledged EU Action in 1999 4 . It is currently governed by Decision No 445/2014/EU 5 , but cities which were designated as ECOC for the years up to 2019 are regulated by Decision No 1622/2006/EC.
The ECOC Action is designed to highlight the richness and diversity of cultures in Europe and the features they share, thereby encouraging a greater mutual understanding among European citizens. It is also meant to stimulate a long-term culture-based development of cities in the broader perception of the term, which entails socio-economic impacts, strengthening of cooperation between cultural operators, artists and cities in Europe, as well as local (and foreign) citizens' involvement and participation in culture.
2.2.The selection and monitoring of the ECOC 2018
In accordance with the Decision, Malta and the Netherlands were the two Member States entitled to host an ECOC in 2018.
The two-phased selection processes (shortlisting and final recommendation) were carried out in parallel by the relevant authorities of these two Member States (i.e. their respective Ministries of Culture). The selection process starts with the publication of a call to trigger off applications from interested cities. A panel of thirteen members – six of whom nominated by the Member State concerned and the other seven by European Union institutions and bodies – assesses the bids submitted by candidate cities on the basis of the objectives and criteria laid down in the Decision.
Malta decided to proceed with its selection procedure one year ahead of the normal time schedule in order to finalize major infrastructural projects needed to host the ECOC year. It published its call for submission of applications in December 2010. The main stakeholders at national and local levels took an early decision that a single application would be submitted on behalf of Valletta but involving the entire territory of the Maltese islands. Valletta’s application was therefore the only one submitted by the deadline of 17 October 2011. Following the pre-selection meeting in January 2012, the city was short-listed and in November 2012, the panel recommended that Valletta be awarded the ECOC 2018 title in Malta 6 .
The Netherlands published their call in December 2011, following the normal time line. Five cities entered the competition by the deadline of 31 October 2012: Eindhoven, Leeuwarden, Maastricht, The Hague and Utrecht. At the pre-selection meeting in November 2012, three of them (Eindhoven, Leeuwarden and Maastricht) were short-listed. In September 2013, the panel recommended that the ECOC 2018 title in the Netherlands be awarded to Leeuwarden.
The Council of the European Union formally designated Valletta and Leeuwarden as ECOC 2018, respectively in May 2013 7 and May 2014 8 .
Subsequently, both cities were subjected to monitoring arrangements: under the auspices of the Commission, the progress in the cities' preparations was monitored and guided by a panel consisting of the seven independent experts appointed by the EU institutions and bodies. The panel had the additional task of ensuring compliance with the programme and commitments on the basis of which the two cities had been selected. Representatives of Valletta and Leeuwarden attended two formal monitoring meetings convened by the Commission, in September 2015 and March 2017. Upon completion of the monitoring process, the panel made a positive recommendation to the Commission to award a €1.5 million prize in honour of Melina Mercouri to each of the two cities. The prize – funded under the Creative Europe programme 9 – was paid to the two ECOC in the autumn of 2017.
2.3.The themes and focus of the two ECOC 2018
Valletta is the capital city of Malta and part of a wider metropolitan area of nearly 400,000 people, representing 90% of the country’s population. The area is heavily dependent on leisure tourism, whilst other key industries include medical tourism, electronics, textiles and film production. The country’s history has given Valletta a rich cultural and architectural heritage, which enabled the city to gain UNESCO World Heritage status in 1980.
Valletta 2018’s initial theme and motto was “Imagine 18”, intended to provide a much-needed boost to the city’s inward-looking cultural sector. The application was connected with a strategic development plan for Valletta and enjoyed the unanimous support of all 68 local Councils and cross-party political support in Parliament. It pursued four main objectives: Making careers of culture; Growing internationally; Establishing Valletta as a creative city; and Nurturing sustainable relationships with the environment. The initial concept was however dropped following a change of leadership after the country’s 2013 general election, in favour of “Valletta 2018: An island-wide fiesta”, the intention being for the ECOC to represent the whole of Malta and facilitate the participation of citizens across the territory. This theme was also chosen for its reference to Malta’s local traditions and fiestas bringing whole communities together.
Located in the northwest of the Netherlands, Leeuwarden is a city of 100,000 inhabitants in the region of Friesland, which has a population of 646,000 and a total of eleven historic cities connected together by water. Leeuwarden is also one of the oldest cities in the north of the Netherlands with a rich history dating back to the Roman age. The city and the region are relatively well served as far as cultural infrastructures and activities are concerned, but their cultural sector is often seen as being locally-focussed and inward-looking in terms of its cultural content, target audience and ambition.
Leeuwarden associated the Friesland region in its bid, a key element of which was “Iepen (open) Mienskip”. Mienskip is a Frisian word and tradition, associated with core values such as mutual respect, participation, grass-roots development, equality and civic responsibility. The adjective “Iepen” adds an important second element of “openess” and expresses the will to include also those not belonging originally to the “Frisian community”. The open mienskip approach was to dominate the development work of Leeuwarden-Friesland 2018, as well as the content of its cultural programme, with a key overarching principle being around using culture to help deal with society’s issues. Against this background, the cultural programme was organised around the following thematic lines: Nature and Culture; City and Countryside; Community and Diversity; Lab Lws (innovation laboratories); and Royal Frisians.
3.Evaluation
3.1.The terms of the evaluation
The evaluation explores the implementation and delivery of the two ECOC 2018 programmes throughout their lifecycle, from their early inception through to sustainability and legacy considerations.
Specifically, it assesses the relevance, efficiency and effectiveness of the two ECOC 2018. It also examines the EU added value and the coherence and complementarity of the ECOC Action to other EU initiatives. Finally, it draws conclusions emerging from the two experiences.
3.2.Methodology and limitations of the approach chosen
The evaluation and its methodology were designed to satisfy the requirements of the Decision, and contribute to develop a more detailed understanding of the performance and achievements of the ECOC Action. In particular, it constitutes a valuable opportunity to critically reconsider the past year in order to highlight lessons and recommendations for reshaping current wisdom and insights in the light of the new experiences of the host cities.
As for all previous evaluations, the intervention logic is based on a hierarchy of objectives corresponding to the Decision.
In order for results to be comparable, the methodology for this evaluation followed the approach for evidence gathering and analysis adopted in previous assessments of the ECOC Action 10 .
The evaluation was grounded in two types of data and respective sources:
-Primary data included either data collected during fieldwork or provided by each ECOC such as interviews and online questionnaires; interviews in particular sought to gain a variety of perspectives on each ECOC, including those of the management teams, decision-makers at local and national levels, key cultural operators, a range of partners involved in the delivery of the programme and a sample of organisations either leading or participating in the actual projects;
-Secondary data sources encompassed EU policy and legislative documents relating to ECOC, academic research on ECOC and the role of culture in cities development, the two ECOC 2018 original bids, internal reports linked to the application processes, monitoring and evaluation reports, studies and reports produced or commissioned by the ECOC, events programmes, promotional materials and websites, as well as key statistical data collected by the two cities on budgets and spend details, projects numbers and types, participation levels and audience figures, outputs and results;
-The evaluation does not include a wider public consultation. As explained in the roadmap 11 , the Action is considered to be local, and international participation is scattered within and outside Europe and is difficult to reach. This conclusion was reinforced by the limited insight offered by the public consultation conducted in the framework of the evaluation of the two ECOC 2017, as highlighted in the corresponding Staff Working Document adopted by the Commission in 2019 12 .
As was the case with all previous ex-post ECOC evaluations, the Commission maintains that the methodology chosen is appropriate to produce a report providing a reasonably solid basis on which sensible conclusions may be inferred regarding the ECOC performance.
However, as was already highlighted in last year’s report, the lack of baseline data to be integrated in a comparative study of the city prior to the win of the title, at the start of title-year and after the implementation of the ECOC year continue to be a limitation. These data are crucial to get a balanced perspective, supported by a cogent and ample data basis, of the actual impact of the ECOC Action on a city.
However the budget allocated to the evaluation work (approximately 75 000 € each year) – which is proportionate to the modest level of EU funding directly provided by the EU to each ECOC host (€1.5m Melina Mercouri Prize) – doesn't make it possible to have a before ('baseline') study and an after-picture ('ex-post') study. An additional consequence of the modest budget is that the primary evidence data gathering tends to be more of qualitative than quantitative nature; while qualitative data still holds a great importance in the evaluation, the lack of diversity of data sources translates into a lesser dependability, for instance, in the process of proving the objective outcomes and impacts of ECOC on widening participation in culture.
Therefore, the report and its conclusions are substantiated by an ample basis of qualitative data (e.g. the views and opinions of various types of stakeholders) more than by a comprehensive quantitative set of data.
The Commission can only reiterate in this report what it repeatedly expressed in its previous yearly reports on ECOC ex post evaluations, i.e. that it is fully aware of – and accepts – those limitations, which had also been cleraly identified and communicated in a Staff Working Document accompanying the proposal for a Decision establishing a Union Action for the ECOC for the years 2020 to 2033 13 .
As mentioned in its previous reports, with regard to this difficulty, a subsequent Commission's proposal and the Decision ultimately adopted by the European Parliament and the Council 14 foresee that the designated cities themselves – which are the main funders and beneficiaries of the ECOC Action and better positioned to have baseline data and gather primary data on the impact of the title – become the main implementers of the evaluation process.
This new obligation put on the ECOC-cities – instead of the Commission – to perform an ex post evaluation of their title-year will however apply only from the 2020 titles. For the ECOC 2019, the Commission will therefore continue carrying out its own evaluation with the limitations described above. At a later stage, the Commission will also carry out an overall evaluation encompassing several ECOC years, enabling to measure the long-term impact of the ECOC Action, as indicated in Decision No 445/2014/EU.
It is also worth underlining that the outcomes of the extensive local research activities commissioned by both Valletta and Leeuwarden fed into the Commission’s evaluation whenever possible.
In conclusion, despite the deficiency of quantitative data and other independent evidence, the Commission finds a sufficient solidity in the evidence gathered to support the evaluation and shares its overall assessment and deductions, which are considered to provide a generally true and informed picture of the ECOC 2018.
4.Main findings
4.1.Relevance
According to the findings of the evaluation, the objectives of both 2018 ECOC cities were relevant to the objectives of the ECOC Action, as set out in the Decision.
The programme of Leeuwarden-Friesland 2018 promoted cultural diversity and a greater understanding among European citizens, both in terms of content (with the cultural programme giving prominence to European themes, such as minority languages or the connection between city and countryside) and in terms of process (with 1,600 international collaborations covering 87 countries). The programme was also relevant to the ECOC’s own objectives, in particular those which were linked to enhancing the range and diversity of the cultural offer and to widening access to and participation in culture; indeed, the ECOC used culture as a vehicle through which to engage with local people and saw the open programme (with its bottom-up principles) as the central spine of the whole project rather than a specific and separate community ‘add-on’.
As far as Valletta is concerned, the ECOC programme featured much more extensive collaborations with European artists and cultural organisations compared to the cultural offering of Malta in previous years, including for example a new scheme of artistic residencies that enabled 50 international artists to perform in the country. It also explored relevant European themes such as migration. Finally, though the programme changed considerably from the application, it remained broadly consistent with three of the four original objectives with less focus being given in practice to the fourth objective relating to the environment.
4.2.Efficiency
Overall, the available evidence suggests that ECOC remains an efficient EU Action providing good levels of returns at EU level for a relatively modest EU investment: the awarding of the title itself has a substantial leverage effect on the amount of funding that host cities dedicate to designing and delivering the ECOC cultural programme, and it is a significant generator of interest and financing from a broad array of stakeholders, including regional and national authorities and private contributors. Moreover, the absolute value of the Melina Mercouri Prize, which is the only direct monetary contribution that host cities receive from the European Union, is modest (€1,5m per ECOC) in comparison to the overall costs of an ECOC: the operating expenditure of the ECOC 2018 were approximately of €104,6m for Leeuwarden-Friesland and €26,5m for Valletta.
At city level, the evaluation concludes that the efficiency of management arrangements varied between the two cities, with those in Leeuwarden-Friesland being generally strong, whilst Valletta went through considerable changes and lost important expertise, which ultimately impacted on the artistic direction and content of the cultural programme. The political dimension of the Valletta 2018 Foundation also attracted criticism locally and internationally. Last but ot least, Leeuwarden-Friesland raised sufficient funds to implement a sizeable cultural programme. Funding in Valletta was more modest but still sufficient, although the final operational budget was considerably lower than the one proposed in the application (€49,6m against €26,5m).
4.3.Effectiveness
The evaluation comes to the conclusion that the two 2018 ECOCs made a significant contribution to the achievement of the Decision’s objectives.
Both presented cultural programmes that were more extensive, diverse, innovative and international compared to the baseline cultural offering in previous years.
Both helped widen access to and participation in culture, though the evidence is stronger for Leeuwarden-Friesland with its “Iepen Mienskip” approach (5,4 million people attended ECOC projects and 60,000 local people actually took part in the ECOC as performers or volunteers) than in Valletta (overall audience estimated at 400,000 people) due, to a great extent, to the unstability in the latter’s leadership.
They also helped strengthen the cultural capacity of their cultural and creative sectors as well as their links with other sectors.
Moreover, the two ECOCs raised the international profile of both cities (6% of the audiences were international in Leeuwarden-Friesland while Malta registered a 14,3% rise in inbound visitors from 2017 to 2018) although Valletta suffered considerable adverse international publicity, as described above.
At the time of writing, Leeuwarden-Friesland had a less-developed plan for legacy. The Valletta Cultural Agency will be a clear mechanism to continue activities beyond 2018, not least the Valletta Design Cluster, which will help sustain the culture-driven regeneration of the city.
4.4.Coherence
The ECOC Action is coherent and complementary to the EU Creative Europe Programme, in that it promotes the objectives of Creative Europe and is distinct from the other activities supported by the programme. The ECOC Action is also coherent with and complementary to the European Structural and Investment Funds. Most notably in Valletta, some significant investments in cultural infrastructure that were implemented by the Valletta 2018 Foundation received co-financing from the ERDF (in particular the new MUŻA museum and the Valletta Design Cluster) and will enhance the legacy of the year by providing new venues for activities.
4.5.EU added value
As already mentioned and illustrated above, the ECOC Action has achieved an impact that would not have arisen through the actions of Member States alone.
The evidence from both cities shows that the ECOC provides titleholders with the impetus to implement more extensive cultural programmes than they would otherwise do. The title also helps them attract resources from both the public and private sectors and increase their international profile. The European Commission plays a key role in promoting the ECOC Action via publications, events and the Europa website, albeit with relatively limited resources. These activities support the ECOCs’ own promotional activities.
5.conclusions
The Commission concludes from the evaluation that the ECOC Action remains relevant at EU level as well as greatly valuable for host cities, and that it generates extensive cultural programmes with positive outputs and impacts which cannot, however, be fully assessed at the current evaluation stage, so shortly after the end of the ECOC year. The intention of the Commission is to assess these outputs and impacts in a broader perspective at a later stage within the framework of the long-term evaluation it will conduct as specified in Decision 445/2014/EU.
Another conclusion is that the programmes implemented by the two 2018 title-holders were consistent with the objectives of the ECOC Action. They reflected its European dimension through the themes addressed and international connections made. Furthermore, they involved local residents and stakeholders while widening access to and participation in culture. They also helped strengthen the cultural capacity of the local cultural and creative sectors and their links with other sectors.
These findings confirm those that emerged from all previous ECOC evaluations, i.e. that ECOC title-holders carry out cultural programmes that are more extensive and innovative than the cities' usual annual cultural offer, with a strong European dimension and involving local citizens as well as international visitors, in line with the objectives of the Treaty and the ECOC Action.
After more than ten years of similar yearly ex post evaluations, the evaluator produced a very limited number of recommendations, all of them addressed to the Commission. Surprinsigly, all these recommendations are related to the selection, monitoring and financing procedures introduced by Decision No 445/2014/EU. It is the Commission’s view that these recommendations, though offering interesting food for thought, are not clearly connected to and substantiated by the core findings of the ex post evaluation of the two ECOC 2018, which were subject to the procedures of another legal basis (i.e. Decision No 1622/2006/EC).
The Commission, while accepting these recommendations as valuable elements nourishing its overall reflection, considers that at this stage, it is too early to draw conclusions regarding the performance of the new procedures introduced by Decision No 445/2014/EU. It will however take account of the evaluator’s recommendations when preparing the above-mentioned longer-term evaluation under the new legal basis.
In particular, the Commission does not agree with the recommendation that it should consider advising panel members to invite no more than two applicants per Member State to progress from the pre-selection to the final selection stage in each competition, given the resources required to prepare a full application. First, the Commission underlines that the current practice is that the panel only puts forward cities that it considers to have a real chance of receiving its recommendation at the final selection stage. Second, in the case of the two ECOC 2018, the ex post evaluation showed that a limited number of applications without a real competition (as was the case in Malta) can result in a more problematic course than when a city is selected from more than just two pre-selected cities (as was the case in the Netherlands). Third, experience shows that though preparing a full application has indeed a cost, the bidding process in itself has its merits and value as this is the opportunity for cities to further reflect on how culture and Europe can shape their overall development.
The evaluator also recommends that the Commission should consider whether the procedures introduced by Decision No 445/2014/EU will be appropriate to a situation where a national Government is likely to be both the organiser of the selection process and one of the key players in an ECOC application even if indirectly, e.g. via a body under its control, as was the case in Malta. While recognizing the relevance of such a recommendation, the Commission would like to point out that such an occurrence is exceptional. In the guidelines it provides to Member States to help them prepare their respective competitions, the Commission will however introduce a warning to make that point clear and ensure that this is not repeated.
Regarding the recommendation that the Commission should review the effectiveness of the procedures introduced by Decision No 445/2014/EU and consider whether to introduce a written agreement with successful applicants, the Commission’s view is that at this stage, there is still too little first-hand experience (with only the two ECOC 2020 having been through the whole cycle of the monitoring process) to come to solid conclusions. This may be covered in the frame of the first evaluation to be carried under the new legal basis.
On evaluation and, more specifically, on the organisation of an evaluation seminar for title-holders, the Commission points out that in September 2019, it launched a capacity building scheme for the benefit of ECOC delivery teams. As part of this 30-month project, academic camps will be organised while MOOCs and webinars will be produced on various areas which are relevant to ECOC delivery teams, including monitoring and evaluation.
(1)
OJ L 304 of 3.11.2006, p. 1.
(2)
Full text of the evaluation at: https://op.europa.eu/en/publication-detail/-/publication/6312a17a-1b6a-11ea-8c1f-01aa75ed71a1/language-en .
(3)
Resolution of the Ministers responsible for Cultural Affairs, meeting within the Council, of 13 June 1985 concerning the annual event European City of Culture (85/C 153/02).
(4)
Decision No 1419/1999/EC of the European Parliament and of the Council of 25 May 1999 establishing a Community Action for the European Capital of Culture event for the years 2005 to 2019 (OJ L 166, 1.7.1999, p.1). That Decision was amended by Decision 649/2005/EC of the European Parliament and of the Council (OJ L 117, 4.5.2005).
(5)
Decision No 445/2014/EU of the European Parliament and of the Council of 16 April 2014 establishing a Union Action for the European Capitals of Culture for the years 2020 to 2033 (OJ L 132, 3.5.2014).
(6)
All panel’s pre-selection, selection and monitoring reports are available at: http://ec.europa.eu/programmes/creative-europe/actions/capitals-culture_en.htm.
(7)
Council Decision No 2013/286/EU of 17 May 2013 designating the European Capital of Culture for the year 2017 in Denmark and in Cyprus and the European Capital of Culture for the year 2018 in Malta (OJ L 162, 14.6.2013).
(8)
Council Decision No 2014/352/EU of 21 May 2014 designating the European Capital of Culture for the year 2018 in the Netherlands (OJ L 175, 14.6.2014).
(9)
Regulation (EU) No 1295/2013 of the European Parliament and of the Council of 11 December 2013 establishing the Creative Europe Programme (2014 to 2020) and repealing Decisions No 1718/2006/EC, No 1855/2006/EC and No 1041/2009/EC (OJ L 347, 20.12.2013).
(10)
See previous evaluation reports at: http://ec.europa.eu/programmes/creative-europe/actions/capitals-culture_en.htm.
(11)
See Ares (2018) 1933090 of 11/4/2018.
(12)
See SWD (2019) 213 final, chapter 4.
(13)
See SWD (2012) 226 final, point 2.4.4.
(14)
Decision 445/2014/EU of the European Parliament and of the Council, see footnote n° 5.