The Politics of Social Media Manipulation (deel 1) (bijlage bij 30821,nr.91) - Main contents

POLITIEK EN SOCIALE MEDIA MANIPULATIE

THE POLITICS OF SOCIAL MEDIA MANIPULATION

Edited by Richard Rogers and Sabine Niederer

The Politics of Social Media Manipulation

Edited by Richard Rogers and Sabine Niederer

The Politics of Social Media Manipulation

Editors: Richard Rogers and Sabine Niederer

Design: Federica Bardelli, Gabriele Colombo, and Carlo De Gaetano

Digital Methods Initiative, Media Studies, University of Amsterdam

© The authors, 2019.

This research was commissioned by the Dutch Ministry of the Interior and Kingdom Relations (BZK). The authors are responsible for all findings and conclusions presented in the report.

The content does not necessarily reflect the views of the Minister of the Interior and Kingdom Relations.

Dit onderzoek is uitgevoerd in opdracht van het Ministerie van Binnenlandse Zaken en

Koninkrijksrelaties. De verantwoordelijkheid voor de inhoud van het onderzoek berust bij de auteurs. De inhoud vormt niet per definitie een weergave van het standpunt van de Minister van Binnenlandse Zaken en Koninkrijksrelaties.

Table of contents

Nederlandse samenvatting: Politiek en sociale media manipulatie 3

• 1. 

  The politics of social media manipulation 14

  Richard Rogers and Sabine Niederer

  • 2. 

    Junk news on Facebook during the 2019 Dutch elections 43

    Stijn Peeters and Richard Rogers

  • 3. 

    Junk news in search engines: Exploring Google’s sensibility towards hyperpartisan sources during the Dutch elections 63 Guillén Torres and Richard Rogers

  • 4. 

    Twitter, junk news and troll-like users during the Dutch provincial and European elections 81 Sabine Niederer and Maarten Groen

  • 5. 

    The presence of fakeness in the Dutch political Instagram: Fake content, sources and followers 100

    Gabriele Colombo and Carlo De Gaetano

• 6. 

  Dutch junk news on Reddit and 4chan/pol/ 116

  Sal Hagen and Emilija Jokubauskaite

• 7. 

  Fake news and the Dutch YouTube political debate space 152

  Marc Tuters

• 8. 

  Conclusions: Mainstream under fire 167

  Richard Rogers and Sabine Niederer

Nederlandse samenvatting:

POLITIEK EN SOCIALE MEDIA MANIPULATIE

Politiek en sociale media manipulatie

Samenvatting

Van desinformatie en nepnieuws naar het bredere begrip junknieuws

Desinformatie en nepnieuws zijn hedendaagse fenomenen met een rijke historie. Desinformatie, of de opzettelijke introductie van foutieve informatie met het doel schade te berokkenen, kent verschillende recentelijke en historische voorbeelden van met name buitenlandse inmenging in nationale mediasystemen met als doel om verdeeldheid te zaaien. In de jaren 80 verspreidde de Russische campagne ‘Operation Infektion’ het gerucht dat het HIV-virus zou voortkomen uit lab-experimenten met biochemische wapens in de Verenigde Staten. Het verhaal werd opgepikt door gerenommeerde nieuwsmedia en zelfs in het avondjournaal uitgezonden.

Tegenwoordig omvatten nationale medialandschappen ook globale sociale mediaplatforms zoals Twitter en Facebook. In 2018 werd bekend dat een trollenfabriek in Sint-Petersburg met de naam ‘Internet Research Agency’ actief desinformatie verspreidde over de MH17 ramp op sociale media. Waar een dataset met de twitterberichten (‘tweets’) die vanuit deze trollenfabriek werden verzonden destijds werd vrijgegeven om onderzoek te doen naar mogelijke inmenging in de Amerikaanse verkiezingen, bleken de trollen juist het actiefst te zijn geweest op de dag van en de twee dagen na het neerhalen van MH17, overigens hoofdzakelijk in het Russisch en slechts marginaal in het Nederlands (Noorda en van de Ven 2019).

Nepnieuws, ofwel gefingeerde verhalen in de gedaante van een nieuwsbericht, zien we door de jaren heen telkens verschijnen bij de introductie van nieuwe mediatechnologieën in de journalistiek, zoals bij de ‘broadsheet’ kranten van eeuwen geleden tot de meer recentelijk in de ‘blogosphere’. De term nepnieuws kennen we ook als aantijging, waarbij nieuwsorganisaties wordt verweten nep -fake news- of zelfs leugenachtig te zijn- ‘der Lügenpresse’. Dit gebruik van de term nepnieuws kent een duistere geschiedenis die in de literatuur vooral wordt geassocieerd met autoritaire regimes of met populistische aanvallen op zogenaamde ‘elite media’.

Vandaag de dag spelen sociale media platforms een belangrijke rol bij zowel desinformatie als bij nepnieuws in beide betekenissen. Zoals besproken in theoretisch en empirisch onderzoek naar dit onderwerp, hebben sociale media de inmenging van buitenlandse desinformatie-operaties in nationale mediaslandschappen mogelijk gemaakt. Daarnaast hebben deze platforms de wijdverbreide circulatie van gefingeerde content gefaciliteerd, en bovendien ruimte geboden aan tendentieuze nieuwscommentatoren -veelal met een omvangrijk publiek- die de mainstream media als nepnieuws bestempelen.

Wereldwijd worden desinformatie en nepnieuws vooral gezamenlijk bestudeerd, maar het is relevant om deze fenomenen ook apart te bezien. Zoals in eerder onderzoek beschreven is in

Nederland vrijwel geen bewijs voor buitenlandse desinformatie te vinden en zijn voorbeelden van Nederlandse spelers die Russische desinformatie-tactieken toepassen ook maar zelden gedocumenteerd. In tegenstelling tot de situatie in de VS, waar het meeste onderzoek naar verricht is, is er in Nederland ook geen sprake van de opkomst van namaak-nieuwsorganisaties

  3 of ‘fake’ actiegroepen die doen alsof zij sociale groepen of doelen representeren. Sterker nog, wanneer men nauwe definities hanteert van desinformatie en nepnieuws is er nauwelijks zoiets te vinden in Nederland.

Maar definities van nepnieuws en met name het aanpalende ‘junknieuws’ worden vaak juist verruimd, en omvatten dan naast desinformatie ook complottheorie, clickbait, alsmede extremistische, sensationalistische, tendentieuze en politiek sterk gekleurde (‘hyperpartisan’) bronnen en verhalen. Daarnaast kennen sociale media het probleem van de ‘artificiële amplificatie’, waarbij accounts en content door middel van nep-volgers en -likes populairder lijken dan ze in werkelijkheid zijn en daarmee symbolische macht verwerven. Toen het beroemde artikel van Buzzfeed News in oktober 2016 vaststelde dat nepnieuws het in de VS in de aanloop naar de presidentiële verkiezingen beter deed dan mainstream nieuws op Facebook, omvatte de definitie waarop de studie gebaseerd was ook clickbait en politiek sterk gekleurde bronnen.

Wanneer we met die ruimere blik naar de situatie in Nederland kijken, treffen we wel degelijk junknieuws aan.

Eerdere studies hebben geconcludeerd dat Nederlanders grootverbruikers zijn van clickbait en zogenaamde junk content. Daarnaast is er sprake van een aantal zeer populaire tendentieuze en nieuws-achtige bronnen, vooral aan de rechterzijde van het politieke spectrum. Tenslotte zijn er al incidenten gerapporteerd met nepvolgers en -fans van Nederlandse politici en muzikanten. Uit onderzoek blijkt dat clickbait meer gelezen wordt dan mainstream nieuws, maar er is dan ook meer van in absolute aantallen. Nieuws aangeboden door commerciële en publieke partijen wordt nog altijd in grotere getalen genoemd, geliket en gedeeld dan tendentieuze bronnen, hoewel bronnen uit die laatste categorie zeker rondom verkiezingen veel circuleren. Het kunstmatig opvijzelen van online impact kan op korte termijn iemands imago een boost geven maar leidt tot mini-schandalen wanneer nepvolgers worden ontmaskerd door nieuwe online detectietools en datajournalistiek.

Of junknieuwsbronnen hun lezers ook overtuigen wordt in toenemende mate onderzocht. Waar een eerste lichting onderzoekers zich vooral richtte op de productie en verspreiding van desinformatie, richt een tweede golf zich vooral op de effecten ervan. Gegeven de ‘verhardende houding’ van het publiek rijst de vraag of de invloed van desinformatie en junknieuws ooit groter dan minimaal zal zijn.

In dat opzicht is de opkomst van extreme content (waaronder ‘extreme clickbait’) die wordt gecirculeerd op sociale media er een van bijzondere aandacht en zorg, met oproepen tot regulering van deze platforms tot gevolg. Een ander aandachtspunt is het mainstreamen van twijfel en wantrouwen jegens publieke instellingen en de media, dat gelijk opgaat met de opkomst van ‘alternatieve feiten’ en de infrastructuren die daaraan ten grondslag liggen. Dit wordt ook de ‘post-truth conditie’ genoemd, waar feiten in twijfel worden getrokken en gefactchecked en tevens naast de mainstream nieuwsmedia een concurrerende media ecologie wordt opgebouwd. Neemt de autoriteit van mainstream nieuws en kennisinstellingen af nu die concurrerende media ecologie in toenemende mate wordt gedeeld en geliket? Uit Amerikaans onderzoek is gebleken dat in de VS vooral oudere nieuwsconsumenten een gewillig publiek vormen voor dergelijke bronnen en daar ook nog eens relatief veel gebruik van maken.

  4 Het consumeren en delen van nepnieuws is ook het aandachtsgebied van initiatieven ter bevordering van mediawijsheid, middels quizzen, serious games en publieke campagnes. Door verhoogde bewustwording, vooral rond de verkiezingen, kan een al te grote impact van desinformatie en dubieuze content worden afgewend, zo is de gedachte. Vrijwillige en professionele fact-check initiatieven worden in de literatuur ook besproken, net als het (al dan niet automatisch) labelen van sociale media content die op juistheid gecontroleerd zou moeten worden. De vraag die hierbij vaker wordt gesteld is of het volume van junk content op deze platforms niet veel groter is dan de beschikbare capaciteit om het te reviewen. En daaraan gerelateerd is het de vraag wie de reviewers controleert.

Tenslotte worden er in de literatuur volop zorgen geuit over de beperkte toegankelijkheid tot de data van sociale media, ook bekend als de ‘locked platforms’ discussie. Toekomstig onderzoek zou zich kunnen richten op de mate waarin desinformatie en nepnieuws in al hun verschijningsvormen online nog altijd floreren. De vraag is dan of er voldoende capaciteit is om deze te monitoren zodat het algemeen gebruik van die bronnen kan worden gemeten, de overtuigingskracht kan worden vastgesteld en de bredere maatschappelijke implicaties kunnen worden gewogen.

Bevindingen in het kort

Deze studie bestaat uit een reeks empirische casussen omtrent het engagement met nep/junknieuws (in de rest van deze samenvatting kortweg aangeduid als junknieuws), evenals politiek sterk gekleurde (‘hyperpartisan’) en tendentieuze bronnen, in Nederlandse politieke arena’s op sociale media tijdens twee verkiezingen in 2019. Deze arena’s zijn afgebakend middels zoekopdrachten naar de namen van de politici en politieke partijen, en sociale kwesties. Sommige van deze kwesties zijn gerelateerd aan de verkiezingen (zoals EU) en andere zijn controversiëler van aard (zoals Zwarte Piet). Hier zijn de bevindingen samengevat, conclusies per platform kort beschreven, en tenslotte in perspectief geplaatst met een bespreking van de beleidsaanbevelingen.

Vraag: In hoeverre resoneren desinformatie en zogenaamd junknieuws in de online politieke arena’s op sociale media, voor en tijdens de Provinciale Statenverkiezingen en Europese

Parlementsverkiezingen van 2019 in Nederland?

• 1) 

  Wij hebben geen buitenlandse desinformatiecampagne of nep-actiegroepen gevonden, rond de Provinciale Staten- en Europese parlementsverkiezingen van 2019.

• 2) 

  Mainstream nieuws wordt op alle platforms meer gecirculeerd en verkrijgt meer gebruiksinteractie dan junknieuws, maar dit geldt niet voor alle specifieke kwesties in alle periodes (8 februari – 25 maart 2019, 26 april - 24 mei 2019 of langere perioden).

• 3) 

  Zwarte Piet en MH17 hebben aanzienlijke hoeveelheden tendentieus en nep/junknieuws en zijn doorlopend actief (niet aan een periode gebonden).

• 4) 

  Met betrekking tot sociale mediamanipulatie, zijn trol-achtige gebruikers actief omtrent meerdere politieke kwesties (op Twitter). We vermoeden ook artificiële versterking

  (nepvolgers op Instagram).

  5 5) Er is een opkomende tendentieuze en polariserende mediasfeer, die concurreert met de

  mainstream, in de zin dat ze niet alleen door trol-achtige sociale media gebruikers maar

  ook door reguliere sociale media gebruikers worden verspreid.

• 6) 

  Facebook heeft relatief de grootste hoeveelheid junknieuws vergeleken met de andere platforms, gevolgd door Twitter. YouTube is een belangrijke rechtse debat-arena voor ‘nepnieuws’ als maatschappelijke kwestie.

• 7) 

  De Nederlandse 4chan en Reddit refereren aanzienlijk minder aan junknieuws dan aan mainstream nieuws. 4chan is een incubator voor uiterst rechtse activiteit in Nederland, middels het verwijzen naar polariserende YouTube video’s.

Beknopte conclusies per platform

Facebook: een vruchtbare bodem voor junknieuws

De studie van Facebook is inhoudelijk gebaseerd op (data-)journalisiek van Buzzfeed News en

NRC Handelsblad, maar waar Buzzfeed opzettelijke desinformatie heeft aangetroffen in hun studie in de Verenigde Staten komt in onze data gericht op Nederland geen desinformatie voor.

Het onderzoek bevestigt wel de reputatie van Facebook als vruchtbare bodem voor junknieuws in vergelijking met andere platforms en toont aan dat ondanks de initiatieven die het platform heeft genomen om dergelijke content te weren en verwijderen, deze nog steeds op het platform circuleert. Sterker nog, waar de NRC concludeerde dat ‘hoogstens tien procent” (Kist &

Zantingh 2017) van de nieuwsconsumptie politiek sterk gekleurde en tendentieuze content betrof, zien wij in onze analyse (die iets meer dan een jaar later plaatsvond) dat dit aandeel is gestegen tot 25%.

Alhoewel een deel van het verschil in bevindingen kan worden toegeschreven aan verschillen in de criteria voor het categoriseren van de content, is te concluderen dat zelfs als junknieuws in de minderheid is, het zeker niet marginaal is. Junknieuws neemt toe in het Nederlandse medialandschap en wordt soms net zoveel gedeeld en geliket als mainstream nieuws. Het verdient aanbeveling om dergelijke interactie (ook wel aangeduid met de term ‘engagement’) met junknieuws nader te analyseren om te zien wat dit in de praktijk betekent. Interactie met een bepaald artikel -bijvoorbeeld door het artikel te delen- betekent niet per definitie dat die gebruiker de inhoud van het artikel steunt. Nu we zien dat junknieuws een belangrijke factor is op Facebook, is het van belang de motivatie van het deelgedrag beter te doorgronden om te kunnen vaststellen welke rol het speelt in het politieke debat.

Terwijl Facebook een stevige onderstroom kent van junknieuws waarbinnen sterk politiek gekleurd nieuws en een aantal complotwebsites circuleren, is er geen reden om aan te nemen dat junknieuws een grote rol heeft gespeeld in het nieuws over de verkiezingen in Nederland in 2019, zoals dit tijdens de aanloop van de Amerikaanse presidentsverkiezingen in 2016 in de Verenigde Staten wel het geval was. Onze studie toont aan dat er geen onmiddellijke aanleiding is voor zorgen over de rol van desinformatie bij Nederlandse verkiezingen, maar dat junknieuws wel een groeiende factor is die kritisch moet worden gevolgd.

Google Web Search: zoekopdrachten in spreektaal resulteren in junknieuws

De Google zoekmachine-studie draait om het lokaliseren van nep/junknieuws in de eerste

  6 twintig Google.nl zoekresultaten voor zoekopdrachten die te maken hebben met Nederlandse politieke partijen en hun belangrijkste kwesties tijdens de provinciale staten- en Europese parlementsverkiezingen van 2019. De zoekopdrachten zijn geformuleerd door de namen van de politieke partijen te combineren met sleutelwoorden voor bepaalde kwesties. Hierbij is gekozen om zowel ‘officiële taal’ te verzamelen uit de partijprogramma’s, als ‘spreektaal’ uit de commentaren op de Facebookpagina’s van de verschillende partijen rond de Provinciale

Statenverkiezingen, om deze ook onderling met elkaar te kunnen vergelijken. Onze resultaten tonen aan dat de junknieuws websites die aanwezig zijn in de Google.nl zoekresultaten voor politieke zoekopdrachten vrijwel uitsluitend politiek sterk gekleurde (hyperpartisan) bronnen betreffen, en niet zozeer bronnen die desinformatie, complottheorie of clickbait verspreiden. Er zijn geen nepactiegroepen of buitenlandse desinformatiebronnen gevonden.

Zoekopdrachten naar kwesties in combinatie met de namen van centrum-rechtse en rechtse partijen resulteren in meerdere mate in junknieuwsbronnen dan zoekopdrachten waarin centrum-links of linkse partijen genoemd worden. Op google.nl is er een aanzienlijke kans dat een zoekopdracht met naam van een rechtse partij en hun kwesties leidt tot junknieuws in de top zoekresultaten. Een andere bevinding is dat de aanwezigheid van junknieuws in de zoekresultaten niet stabiel is, maar varieert door de tijd heen. Voor de Provinciale

Statenverkiezingen zien we meer junknieuws in de topresultaten tijdens de dagen voorafgaand aan de verkiezingen, om tijdens de verkiezingen en direct daarna van de radar te verdwijnen

(zoals duidelijk te zien rond de kwestie migratie). Bij de EU-verkiezingen is het effect precies tegenovergesteld, en zijn junk nieuwsbronnen vooral prominent aanwezig in de topzoekresultaten op de dag na de verkiezingen.

Wanneer we de resultaten van de twee soorten zoekopdrachten (resp. gebaseerd op officiële taal uit partijprogramma’s en spreektaal uit Facebook commentaren) met elkaar vergelijken, zien we dat een zoekopdracht die is geformuleerd in spreektaal leidt tot een hoger percentage junknieuws in de resultaten, bijvoorbeeld waar het gaat om migratie. In andere woorden: spreektaal leidt tot gekleurder nieuws in de resultaten dan ‘officiële’ partijprogramma-taal.

Twitter: junknieuws en trol-activiteit rond polariserende kwesties

In de Twitter-studie hebben we gekeken naar de aanwezigheid van junknieuws, tendentieus nieuws en trol-achtige activiteit tijdens de campagneperiodes van de Provinciale

Statenverkiezingen en Europese Parlementsverkiezingen in Nederland in 2019. Tijdens de

Provinciale Statenverkiezingen is trol-achtige activiteit aangetroffen rond zowel politieke thema’s (zoals de verkiezingen zelf en de partijleiders) als polariserende kwesties zoals MH17, Zwarte

Piet en de aanslag in Utrecht. Trol-achtige activiteit verwijst naar een aantal indicatoren in twittergedrag, waarvan de belangrijkste een opvallend hoog aantal tweets in een heel korte periode is. Uit de analyse blijkt dat veertien trol-achtige twittergebruikers rond al deze thema’s actief waren, en maar liefst 29 waren actief rond vier van de vijf thema’s. Vier van deze trolachtige twitterprofielen waren nog altijd (of wederom) actief tijdens de Europese verkiezingen.

Deze trol-achtige gebruikersprofielen verspreidden vooral sterk politiek gekleurde en tendentieuze bronnen, gevolgd door complotwebsites.

  7 Rond polariserende kwesties is in beide campagneperiodes activiteit waargenomen waarin junknieuws een aanzienlijke rol speelt. Tijdens de Provinciale Staten campagneperiode waren de kwesties Zwarte Piet en MH17 actief, ook al vindt het Sinterklaasfeest in december plaats en was er in die periode geen MH17 herdenking. Tendentieuze en politiek sterk gekleurde bronnen worden door trol-achtige gebruikers het meest verspreid, gevolgd door complottheorie-bronnen die vooral rond tragedies (MH17 en Utrecht) circuleren. De pro-Russische site novini.nl, die op verhaal-niveau laveert tussen sterk politiek gekleurd, complottheorie en desinformatie, is door trol-achtige gebruikers rond alle kwesties gedeeld, in het bijzonder rond MH17 en slechts marginaal rond de politieke leiders. Tijdens de Europese Verkiezingsperiode was op verschillende momenten junknieuws zelfs prominenter aanwezig dan mainstreamnieuws rond de polariserende onderwerpen Zwarte Piet en MH17. Belangrijk hierbij aan te merken is dat dit vooral is veroorzaakt door een afname van mainstream media-aandacht voor deze onderwerpen in deze periodes, terwijl junknieuws-aandacht actief blijft en daar de polarisatie voedt.

Gebaseerd op deze resultaten lijkt de Nederlandse Twittersfeer geen algemeen desinformatieprobleem te hebben, al zijn er trol-achtige gebruikers actief die bepaalde stemmen en bronnen kracht bijzetten. Hoewel we geen professionele of grootschalige trollencampagne hebben aangetroffen, is er wel trol-achtige activiteit gesignaleerd die zich uitstrekte over meerdere kwesties. Deze trol-achtige activiteit is afkomstig van verschillende soorten gebruiksprofielen, zowel in de vorm van geautomatiseerde ‘bots’ als van schijnbaar semi-geautomatiseerde profielen die een combinatie tonen van automatisch retweeten en het posten van originele content. Rond polariserende thema’s blijft de activiteit in junknieuws en tendentieuze nieuwsbronnen stabiel

(hoe marginaal ook) in beide onderzoeksperioden, hetgeen suggereert dat deze kwesties in junknieuws het gehele jaar leven en niet alleen in het verwachte seizoen (november/december voor Zwarte Piet en juli voor MH17).

Instagram: rechtse mediasfeer en tekenen van artificiële amplificatie

De Instagramstudie onderzoekt vanuit drie invalshoeken de aanwezigheid (en afwezigheid) van desinformatie: op post-niveau, kijkend naar de circulatie van junknieuws in politieke arena’s, op bron-niveau door de volgers te vergelijken van Nederlandse politieke partijen en politiek leiders met die van junknieuwsbronnen, en op activiteit-niveau, middels de detectie van artificiële amplificatietactieken zoals nepvolgers van politieke partijen en leiders. In het algemeen troffen wij op Instagram een ‘gezonde’ politieke arena aan met slechts een klein aantal voorbeelden van junknieuws en artificiële amplificatie. De overgrote meerderheid van de meest ‘gelikete’ content in de Nederlandse politieke arena betreft geen junknieuws, al zien we rond rechtse politici en een aantal kwesties wel een kleine hoeveelheid politiek sterk gekleurde en tendentieuze content. Voor het publiek van de politieke partijen en leiders was ook op Instagram in beide periodes mainstreamnieuws prominenter aanwezig dan junknieuws. Kijkend naar gebruikers van het platform die actief zijn geweest rond politieke partijen en leiders, zien wij een schijnbaar authentiek publiek met vrijwel geen tekenen van artificiële activiteit.

Binnen een in het algemeen relatief gezonde politieke arena over vrijwel het gehele politieke spectrum, is er alleen bij uiterst rechts enige verdachte activiteit waargenomen. Zo zien we rond politieke entiteiten aan de rechterzijde van het politieke spectrum meer circulatie van junknieuwsbronnen. Daarnaast zien we alleen bij deze entiteiten enige signalen van artificiële

  8 activiteit. Ook wanneer we kijken naar volgers zien we dat de politieke partijen en politici aan de rechterzijde van het politieke spectrum een uniek volgers-signatuur hebben. Zij zijn ook kijkend naar co-volgers (dit zijn volgers die zowel een nieuwsbron als een politieke partij of leider volgen) relatief nauwer verbonden met junknieuws en politiek sterk gekleurde nieuwsbronnen dan partijen en politici elders op het politieke spectrum. De artificiële activiteit die we hier aantreffen is in lijn met het schandaal in 2015, wanneer na verwijdering van nepvolgers op

Twitter bleek dat ook in de Nederlandse politiek van dergelijke artificiële amplificatietechnieken gebruik was gemaakt.

Reddit en 4chan: YouTube video’s als nieuwsbron dragen bij aan polarisatie

Hoewel Reddit en 4chan vaak worden beschreven als alternatieve zones op het web, laten de resultaten van onze studie van deze platforms geen groot aandeel van alternatieve nieuwsnetwerken zien die desinformatie verspreiden binnen deze platforms in de Nederlandse context. Naast een paar voorbeelden waarin de pro-Russische site novini.nl een rol speelt en verdachte activiteiten vanuit één Reddit account, zien we hier geen gecoördineerde campagnes van kwaadwillende gebruikers die links posten naar desinformatie. Er is zeker junk content te vinden, maar vergeleken met de algehele activiteit blijft het aandeel daarvan marginaal. Met name Reddit lijkt goed weerbaar tegen de circulatie van desinformatie.

Wat wel uit het onderzoek naar voren komt zijn de verschillende typen junknieuws van politiek sterk gekleurde aard, die met name op 4chan/pol/ zijn aangetroffen. Dit zijn hoofdzakelijk links naar tendentieuze en politiek sterk gekleurde bronnen als The Post Online en De Dagelijkse

Standaard, maar ook complottheorie op NineForNews. Deze rechtse bias was te verwachten op 4chan/pol/ dat bekend staat als uiterst-rechtse hub, maar voor Reddit lag het minder voor de hand omdat we daar een politiek diverse set sub-Reddits als startpunt hebben genomen. Hoewel het problematisch is om nieuwsbronnen als “nep” te bestempelen, treffen we op die bronnen wel degelijk polariserende content aan. De artikelen die de meeste gebruiksinteractie vertonen gaan dan ook vaker over onderwerpen als migratie en de Islam dan over geopolitieke events als

Russische inmenging of MH17.

Desalniettemin blijft het vooral populair om te linken naar gevestigde nieuwsbronnen als

NOS.nl, op zowel 4chan/pol/ als Reddit. Dit gaat tegen de verwachtingen in, gezien het “fringe” karakter dat vaak wordt toegeschreven aan deze platforms en hun anonieme gebruikers waarbij ook het verwijzen naar marginale of alternatieve kennisbronnen kan worden verwacht. Onze bevindingen weerleggen deze aannames echter en laten zien dat mainstream bronnen minstens enige autoriteit genieten op deze online platforms. Belangrijk hierbij op te merken is dat wij niet hebben onderzocht hoe de mainstream websites zijn besproken. Zo kunnen we verwachten dat

NOS.nl op Reddit wordt behandeld als een betrouwbare bron, terwijl deze op 4chan/pol/ wellicht enkel wordt geridiculiseerd of zelfs bestempeld als “nepnieuws”.

Tenslotte is het belangrijk om reguliere nieuwsbronnen niet te beschouwen als de enige bron van nieuwsvoorziening op het web. Zo hebben we kunnen vaststellen dat YouTube is uitgegroeid tot een bijzonder grote speler in nieuwsconsumptie en -circulatie, met name op 4chan/pol/. Op

Reddit vormen “News & Politics” video’s op YouTube de op een na belangrijkste bron, terwijl ze op 4chan/pol/ waarschijnlijk alle andere nieuwsbronnen ver achter zich laten. Uit een kleine

  9 exploratieve studie van de YouTube kanalen die kunnen worden aangetroffen op beide platforms blijkt dat de Nederlandse Reddit vooral naar gevestigde bronnen verwijst, zoals

PowNed, Zondag met Lubach en NOS, terwijl op 4chan/pol/ juist aan alternatieve, buitenlandse en politiek sterk gekleurde informatiekanalen wordt gerefereerd zoals aan Mike

Cernovich en RT. Als deze trend zich doorzet zou niet-Nederlandse YouTube content weleens een polariserende of zelfs radicaliserende werking kunnen hebben op Nederlandse gebruikers van dit platform.

Beleidsthema’s: nepnieuws als morele paniek en de bedreiging van de mainstream

In de academische literatuur houdt een kleine maar groeiende groep wetenschappers zich bezig met de vraag hoe nepnieuws kan worden beschouwd als een morele paniek (Morozov, 2017;

Hirst, 2017). Die term verwijst naar momenten in de geschiedenis wanneer “weldenkende mensen” (in de klassieker van Stanley Cohen gedefinieerd als “editors, bishops and politicians”) een maatschappelijke conditie waarnemen die een plotseling verval van maatschappelijke standaarden en waarden veroorzaakt (1972). Wanneer nepnieuws door die lens van een morele paniek wordt beschouwd, rijst de zorg dat de traditionele journalistiek als steunpilaar van de democratie thans wordt uitgehold door sociale media en daar vervangen door clickbait van lage kwaliteit en politiek sterk gekleurde commentaren, die op zo’n wijze worden verpakt dat de consumptie ervan doet denken aan die van ongezond junkfood (vandaar de term: junknieuws).

De algehele gezondheid van de media als sociaal weefsel staat op het spel, omdat burgers die sociale media gebruiken als bron van politieke informatie worden benadeeld in de mogelijkheden om een oordeel te vormen over sociale kwesties en politiek in algemenere zin (Carlson, 2018).

Een tweede subset in de literatuur beschrijft hoe de media-aandacht voor nep- of junknieuws, en in het bijzonder de relatie daarvan met de groei van een uiterst-rechtse media-ecologie, hier juist ‘zuurstof’ aan geeft (Phillips, 2018). Philips roept op om te stoppen met aandacht geven aan het maar al te fascinerende subculturele milieu waarin extreemrechts zichzelf cultiveert, en juist aandacht te schenken aan de slachtoffers van deze praktijken. Er zijn voorbeelden van politici die extremistische of verdeeldheid zaaiende content circuleren, juist omdat ze het sterk verwerpen en openlijk willen bekritiseren, en daarmee toch die materialen ook zuurstof geven. Door het te delen dragen zij onbedoeld bij aan de circulatie van het extreme materiaal en wellicht zelfs aan de normalisering ervan. In lijn van deze literatuur, bevatten de beleidsaanbevelingen aandacht voor het identificeren van en reageren op de bedreiging van de mainstream, zowel vanuit sociale media platforms als de journalistieke praktijk, online content creatie en politiek leiderschap.

Uit onze studie blijkt dat er bepaalde platforms en onderwerpen zijn waar de bedreiging van de mainstream acuter lijkt dan bij andere platforms en onderwerpen het geval is. Hoewel het geen platform is waarop Nederlandse junknieuwsbronnen worden verspreid, is de Nederlandse 4chan een incubator van extremistisch sentiment, vooral waar het gaat om antisemitisme en antiimmigratie. De polariserende content is daarbij veelal afkomstig van YouTube. Andere platforms zijn om andere redenen problematisch. Nederlandse politieke arena’s op Facebook en Twitter, die je kunt afbakenen met zoekopdrachten naar politici, politieke partijen en maatschappelijke kwesties, hebben de grootste hoeveelheden junknieuws, maar het engagement met die bronnen is op die platforms nog altijd lager dan de algemene consumptie van mainstream nieuws. Binnen het junknieuws zijn politiek sterk gekleurde bronnen populairder dan desinformatie,

  10 complottheorie of clickbait. Voor polariserende onderwerpen zoals klimaatverandering, MH17 en Zwarte Piet behalen verhalen van dergelijke junknieuwsbronnen soms meer engagement dan verhalen uit de mainstream pers, omdat de aandacht in die junknieuwsbronnen stabiel blijft in tegenstelling tot het mainstreamnieuws.

Op Twitter bijvoorbeeld, tijdens de Europese Parlementsverkiezingscampagne, deed een pro

Nexit verhaal in De Dagelijkse Standaard, over het uit de EU treden van Nederland, het beter dan een artikel over datzelfde onderwerp in NRC Handelsblad. Ook zijn er sporen van een gepolariseerde media ecologie te vinden. Op YouTube is een alternatieve (rechtse) mediasfeer ontstaan, waar extreme YouTubers (‘micro-celebrities’) de scepter zwaaien. Instagram heeft ook een rechtse mediasfeer, die in de analyse gedetecteerd kon worden door gedeelde volgers van rechtse politici en sterk politiek gekleurde mediabronnen. Deze zijn vooral te typeren als ‘alt lite’, dat wil zeggen anti-establishment en anti-politieke correctheid, met content die ook als counterjihadist en anti-Islam gezien kan worden. Er zijn geen linkse equivalenten hiervan gevonden.

Om onderzoek te kunnen doen naar polarisatie, opruiing rond maatschappelijke kwesties en de publicatie, verspreiding of consumptie van extremistisch materiaal, is toegang nodig tot data van sociale mediaplatforms. Dergelijke platformdata moet beschikbaar worden gemaakt aan onderzoekers en waakhond-organisaties, zowel middels APIs als in publieke onderzoeksarchieven, waar ook verwijderde materialen beschikbaar in zouden moeten zijn evenals metadata omtrent de verwijdering.

Aanbevelingen

In tegenstelling tot de situatie in andere landen tijdens de Europese parlementsverkiezingen, is in onze studie geen buitenlandse desinformatie gevonden maar wel zogenaamd junknieuws, met name rond bepaalde kwesties, zoals Zwarte Piet, MH17, het klimaat en de Europese Unie (Peel,

2019). Ook bij het onderwerp nepnieuws, hier bestudeerd als kwestie, was dit het geval. Hoewel er een flinke hoeveelheid junknieuws werd aangetroffen, deed deze het in het algemeen niet beter dan mainstream nieuws. De grootste hoeveelheden junknieuws circuleerden niet rond algemene thema’s zoals politieke partijen en politieke leiders, maar juist rond specifieke en polariserende kwesties. Activiteit op sociale media rond deze kwesties vertoonde in sommige gevallen een duidelijke piek tijdens de verkiezingen of bleef constant gedurende de gehele periode. Hiermee is de vraag niet langer alleen of er junknieuws aanwezig is rond de verkiezingen, maar ook wanneer junknieuws zich manifesteert en voor hoe lang.

De onderstaande beleidsaanbevelingen zijn dan ook toegespitst op deze meer specifieke kijk op het fenomeen junknieuws. In plaats van een algemene aanpak doen wij aanbevelingen die zich richten zich op het herkennen en monitoren van polarisering van het medialandschap, een breed maatschappelijk debat omtrent junknieuws toespitsen op polariserende kwesties in plaats van desinformatie in algemene zin, trainingen ontwikkelen voor beroepsgroepen die zich bezighouden met het produceren van content, en het mogelijk maken van onderzoek en mediamonitoring met (thans ontoegankelijke) sociale mediadata. Met deze aanbevelingen richten wij ons tot beleidsmakers, Ngo’s, waakhonden, onderzoekers, journalisten, mediaproducenten en partijen in het maatschappelijk middenveld, die zich bezighouden met de bestrijding of bestudering van desinformatie. Daarnaast zien wij een belangrijke rol voor de sociale media

  11 platforms zelf bij het ontsluiten van hun data ten behoeve van onderzoek, monitoring en archivering.

• 1) 

  Mediamonitoring van polarisatie, ofwel het monitoren van de groei en het mainstreamen van polariserende media met extreme content.

Sociale mediaplatforms zetten software, gebruikers en content-reviewers in om extreme content te herkennen. Door platforms wordt steeds meer van dergelijke content verwijderd. Maar historisch bezien hebben sociale mediabedrijven wisselend aandacht besteed aan content moderatie en is ook de definitie van extreme content niet stabiel gebleken. Daarmee blijft het wenselijk om onafhankelijke monitoring te organiseren. Hier zien wij een rol weggelegd voor academisch onderzoekers, non-gouvernementele organisaties, overheidsdiensten op het gebied van extremisme en polarisatie, en media-waakhonden. Een training voor media monitoring zou door partijen in het maatschappelijk middenveld kunnen worden opgezet.

• 2) 

  Mediawijsheid-training voor professionele content makers -van journalisten tot digitale media producenten- zowel op het gebied van online bronkritiek als het omgaan met polariserende content online.

Nederland kent mediawijsheid programma’s, bijvoorbeeld gericht op senioren of op schoolkinderen (en via hen, hun ouders). Deze aanbeveling richt zich echter expliciet op het ontwikkelen van een mediawijsheid-training voor professionele content-makers zoals journalisten en redacteurs. Dit zou kunnen worden opgezet binnen bestaande of nieuwe mediawijsheid-programma’s, met betrokkenheid van inhoudelijk experts op het gebied van journalistiek, digitale mediaproductie en digitale geletterdheid, en gericht op een combinatie van bronkritiek en het omgaan met polariserende content online (zie ook aanbeveling 3). Een dergelijk programma is ook relevant voor docenten in hoger en wetenschappelijk onderwijs, beleidsmakers, en het maatschappelijk middenveld .

• 3) 

  Geen zuurstof geven aan extreemrechtse content.

In onze studie zien wij hoe tendentieuze nieuwsmedia ruimschoots circuleren tijdens verkiezingen en daarbuiten. De artikelen van deze bronnen worden gedeeld en geliket door trolachtige gebruikers maar ook door reguliere nieuwsconsumenten, zoals te zien op Twitter tijdens de Provinciale Statenverkiezingen. In mindere mate delen en liken trol-achtige en reguliere nieuwsconsumenten ook discriminerende, antisemitische, misogynische en xenofobische content.

Het is van belang om tendentieuze en extreemrechtse media niet aan elkaar gelijk te stellen, ook niet wanneer zij dezelfde standpunten presenteren. Dat tendentieuze media in opkomst zijn en mainstreamen betekent niet dat evenveel gewicht moet worden toegekend aan extreemrechtse media. De aanbeveling is om geen zuurstof (als in: aandacht) te geven aan extreemrechtse media – en daarmee hun content dus niet te delen, liken, reageren, becommentariëren, retweeten, of

(video-)debatteren. Iedere vorm van het delen van extreemrechtse content betekent het vergroten van diens zichtbaarheid, en levert wellicht zelfs een bijdrage aan het normaliseren

  12 daarvan. Deze aanbeveling gaat uiteraard op voor publieke omroepen, maar ook voor commerciële mediaorganisaties en tendentieuze nieuwsmedia. In plaats van journalistieke aandacht te besteden aan extreemrechtse subculturen zelf, kan men de blik beter vestigen op de slachtoffers daarvan (Philips, 2018).

• 4) 

  Bij polariserende kwesties zoals Zwarte Piet strekt het de aanbeveling om het maatschappelijk gesprek breed in te richten met ruimte voor burgerparticipatie.

Het onderzoek suggereert dat polariserende thema’s zoals Zwarte Piet het gehele jaar actief zijn en niet alleen in het verwachte seizoen. Wanneer we herkennen dat er toenemende polarisatie plaatsvindt in de maatschappij, biedt dit ook aanleiding om te verkennen hoe gemeenschappelijke grond kan worden gevonden. Nederland kent rond maatschappelijke en ethische vraagstukken de traditie van de Brede Maatschappelijke Discussie. Daarnaast zijn er eigentijdse vormen van burgerparticipatie die in het gesprek rond dergelijke polariserende thema’s en culturele contestatie kunnen worden ingezet. Hier zien wij een rol weggelegd voor bottom-up initiatieven en bestaande organisaties die zich bijvoorbeeld richten op het faciliteren en op gang brengen van een maatschappelijk gesprek en het ontwikkelen of toepassen van vernieuwende vormen van burgerparticipatie.

• 5) 

  Belangenbehartiging van onderzoekers en waakhonden voor het verkrijgen van toegang tot sociale mediadata en de creatie van onderzoeksarchieven voor content, inclusief inmiddels verwijderde materialen.

Sociale mediabedrijven dienen toegang te bieden tot hun data ten behoeve van onderzoek, monitoring en archivering. De kwestie van ‘gesloten platforms’ draait om de mate waarin sociale mediabedrijven hun platformdata verminderd toegankelijk maken voor onderzoekers, journalisten en non-gouvernementele organisaties. Als antwoord op zorgen van de overheid omtrent ‘dark posts’ (advertenties die alleen in iemands nieuwsfeed te zien zijn) en politieke advertenties zonder duidelijke afzender heeft Facebook recentelijk een politieke advertentie tool en API gelanceerd. Maar tegelijkertijd heeft Facebook onderzoekers de toegang gedeeltelijk of geheel ontzegd tot services zoals de Pages API en Graph Search, die veel worden gebruikt voor onderzoek. Wij zien een belangrijke rol voor de sociale mediabedrijven zelf bij het ontsluiten van hun data ten behoeve van onderzoek, monitoring en archivering. Overheidsinstellingen kunnen hier sturend in optreden.

  13

THE POLITICS OF SOCIAL MEDIA MANIPULATION

Richard Rogers and Sabine Niederer

The Politics of Social Media Manipulation

Richard Rogers and Sabine Niederer

Introduction: Influence campaigning in political spaces online and the question of persuasion

In reviewing the scholarship surrounding so-called fake news, one would out of necessity make a distinction between the dominant work on the art of influence campaigning and computational propaganda online and the consequences to date for its consumers, but also the few findings, often journalistic, in the relatively understudied case of the Dutch political space online, both on the web as well as in social media. Much work has been undertaken on the influence of Russian

(and Russian-style) influence campaigning in the US, and the presence or absence thereof during elections in Germany, France, Sweden and elsewhere. With respect to the Netherlands, the case studies have been reserved to the early Russian influence campaigning around the downing of the MH17 Malaysian airliner (beginning in 2014) and the suicide bombings in the Brussels airport and metro (2016), discovered through the use of Twitter data sets of Russian trolls, or influence campaigners. Other work has been performed on the existence of home-grown troll networks, operating in a predominantly right-wing media sphere, that are at times ‘pro-Russian’ but do not seem to have had foreign input.

Crucially, in the studies and journalistic treatments to date it is increasingly remarked that there has been a shift in Russian disinformation campaigning from inflaming conflict with the West to stirring it within the West. It is also argued that disinformation could be said to be ‘Russifying’, i.e., the borrowing of so-called Russian techniques by domestic actors. The campaigning, whether foreign or domestic, does more than create narratives that divide; it also employs computational means to inflate and amplify them through bot work, fake following, astroturfing, the creation of front groups and other artificial publicity tactics.

It is also argued that more attention ought to be paid to the rise of extreme and divisive media on social media platforms, where the point is often made that great emphasis is being placed on foreign disinformation when by comparison it performs poorly in European news spheres. The growth of ‘hyperpartisan’ news and commentary also may be viewed as an alternative fact or knowledge infrastructure, contributing to discussions of a post-truth condition and the contention that established institutions are under threat.

It is of equal importance to examine the critique on persuasion, or the extent to which the influence campaigning strategies, fakery and hyperpartisan sources have discernible impacts on their consumers, especially the voters. They appear to be minimal. Indeed, there is a small, but growing literature critiquing transfer models, also known as hypodermic needle or magic bullet theories which themselves could be considered part and parcel of the fake news hype and fascinations with so-called psyops activities such as in the Cambridge Analytica case. 1 Transfer

1 The Cambridge Analytica case or scandal refers to the illegitimate use of over 80 million Facebook users’ information to develop micro-targeted advertising (Cadwalladr and Graham-Harrison, 2018). It prompted US

Congressional and UK Parliamentary investigations, and also led to Facebook’s tightening its data access for academics and public scrutiny more generally (Bruns et al., 2018).

  14 models do not take into account the active filtering of media users or phatic sharing, it is argued, whereby one circulates dubious media more to connect with others or for amusement than to pass along substantive information. Such models also would discount hardened attitude, and studies finding that campaigns generally have minimal effects.

As for the measures to be taken, the literature both describes and occasionally questions factchecking and media literacy efforts because of the assumption that corrected information would assuage knowledge deficits, for attitudes often remain the same. Nonetheless, among the policy recommendations most frequently put forward are bolstering media literacy initiatives, together with flagging questionable content, manually and automatically, for further scrutiny. Social media platforms are facing regulation and are asked to address extreme content and create public archives.

One aspect of the literature review relevant to empirical work concerns the methods employed to demarcate political space online for the subsequent study of the scope and impact of fake news, junk news and computational propaganda – to use some of the terms for the larger phenomenon under study. Under consideration here are largely mixed (quanti-quali) techniques and digital methods from media studies and data journalism. These provide distinctive political space demarcation strategies for the web as well as social media per platform as well as approaches for cross-platform analysis. They query engines and platforms, measure significant political stories (in terms of engagement) and determine construals of dubiousness through news criticism, categorising significantly engaged-with stories into genres such as disinformation, conspiracy, clickbait, hyperpartisan and (automated) amplification. While often practiced on a story level, the determination of dubiousness also may be made through source criticism, according to the extent to which publishers’ output repeatedly accords with junk news definitions, discussed in the next section. It is also worth studying how signal-based or algorithmic determinations of fakeness comport with qualitative methods that are based on source (provenance) criticism.

Fake news, junk news and computational propaganda

Historically, fake news could be thought of as linked to particular novel publishing practices both “when old media were new” but also nowadays through social media platforms (Marvin, 1988;

Gitelman, 2006). The term ‘canard’, meaning unfounded rumour or story, refers to the contents printed in the French broadsheets of the eighteenth century; ‘scandal sheets’ are the British term for the same era of publishing (Darnton, 2010). In the U.S., in particular, ‘fake news’ as a term recently experienced a revival and travelled internationally, in the numerous senses in which it has been deployed historically: “news satire, news parody, fabrication, manipulation, advertising, and propaganda” (Tandoc et al., 2018). As directed towards contemporary social media platforms, the charge of fake news and similar terms often has been uttered as a lament after the introduction of new media technologies, where there are concomitant calls for new journalistic standards, as witnessed with the competing tabloids and their sensationalist, yellow journalism in the late 1890s and into World War I as well as the radio and newswire in the 1920s (Opper, 1894; Lippmann, 1922; McQueen, 2018).

15 With the rise of corporate public relations, the blurring of the distinction between the editorial and the advertisement sent over the wire or into the airwaves prompted the use of the moniker,

“news fakers” (McKernon, 1925; Lazer et al., 2018). Similarly, the contents of the early, unedited web, populated by self-publishers, and later the blogosphere, were often described as “too fresh to be true”, given the speed of news production and the potential for those looking for a scoop to convert unsubstantiated rumor into news (Hall, 2001; Rogers, 2005). More recently, the notion would be routinely deployed by satirical news sources such as Saturday Night Live! in the

US (Day and Thompson, 2012); in fact, The Daily Show, the progressive comedy news program, described itself proudly as a “fake news program” (Newman, 2010; Harsin, 2018). Parody, it should be recalled, was behind the origination of the most circulated fake news story during the

US presidential campaigns of 2015-2016, “Pope Francis Shocks World, Endorses Donald Trump for President” (Allcott and Gentzkow, 2017). While many definitions concentrate on a story’s falseness, fake news have the ‘trappings of news’ through the use of the news apparatus, including the style, formats and containers employed (Laquintano and Vee, 2017; Grinberg et al., 2019). Indeed, narrower definitions of fake news take as their point of departure how the sources “falsely claim to be news organizations” (Tucker et al., 2018).

Fake news also has been deployed politically as a barb against the free press when publishing inconvenient truths, or speaking ‘truth to power’ (Cary, 1955; Darnton, 2017). Since the midnineteenth century, labelling the news media generally and disagreeable reporting specifically as product of der Lügenpresse or the lying press is a discrediting ploy or even communication strategy, still practiced today by right-wing social movements as Pegida in Germany, chanting at street rallies Lügenpresse, halt die Fresse (lying press, shut your mouth) (Beiler and Kiesler, 2018). It was the German Unwort des Jahres (notorious word of the year) in 2014, in the competition organised by TU Darmstadt. Fake news is also a label, used in highly conservative political circles in the

US, for particular news sources, notably CNN, MSNBC, The New York Times, and The Washington Post; the designation is similar, albeit perhaps more extreme, to past portrayals of the agendasetting ‘elite media’ in contrast to conservative upstarts as Fox News (Marwick, 2018; Tripodi,

2018; Peck, 2019). In this respect, one could call the current situation just the latest fake news scare, or even moral panic (Brennen, 2017; Morozov, 2017).

When discussing the fake news phenomenon in relation to social media and other online sources, researchers at the computational propaganda project at the Oxford Internet Institute

(OII) often offer the umbrella term ‘junk news’, defined as “extremist, sensationalist, conspiratorial, masked commentary” (Howard et al., 2017). Other catch-alls include

“problematic information”, “information disorders” and “false news” (Jack, 2017; Wardle and

Derakhshan, 2017). Apart from sensationalist, conspiratorial and masked – features that have been a part of fake news ontologies for centuries – the OII definition emphasises another element, extremist, which cuts to the heart of contemporary concern for the phenomenon when studied not only as a practice of media and public opinion manipulation but also a trigger for societal unrest.

With respect to the growing anxiety over fake news as harbinger of unrest, one may refer to the distinctions made between a variety of information disorders, as well as the coinage of new terminology that captures excitable, Internet-related media and speech (Wardle, 2018). First,

  16 disinformation and misinformation are both false, but the latter is unintentionally so, whilst the former is fashioned for the purposes of intentional disruption and causing harm. A third term,

‘mal-information’ (a neologism), seemingly borrowed from malware or malicious software categorisations, has been introduced to describe harassment, toxic content and hate speech online (Wardle and Derakhshan, 2017). These are the tools for the so-called ‘weaponization’ of social media platforms to foment discord through seeding the news and public opinion with divisive content. Indeed, ‘extreme speech’ is a term that has been offered as a nuancing of the hate speech discourse as it is applied to online toxicity. It is meant to capture a form of charged language and cultural conflict that stops short of hate, and has emerged with social media, defined as “vitriolic exchange on Internet-enabled media” (Pohjonen and Udupa, 2017). Its rise has prompted social media companies as Facebook, Twitter and Alphabet (owners of YouTube) to expand their content reviewer pools as well as widen their internal mandates to identify and remove more than violence, pornography and hate (Gillespie, 2018). Google also installed a feedback system for its web search to report inappropriate autosuggestions, after reports of queries for the ‘holocaust’ autocompleting with ‘is a hoax’ (Solon and Levin, 2016; Hern, 2017).

As with new media technologies of old, social media platforms currently are said to enable the

‘supercharging’ or the acceleration of the spread of fake news (Bounegru et al., 2018). Two terms have been used to capture the web and subsequently social media as accelerationist media: clickbait and computational propaganda. Clickbait connotes titillating and sensational content and is formulaic in its presentation, often containing numbered lists (sometimes referred to as a

‘listicle’) as well as an ‘information gap’ that sparks curiosity, e.g., ‘twenty things you should not do when visiting Japan’. Facebook, in seeking to identify and downgrade clickbait in its news feed, defines it as “a posted link with a headline that encourages people to click to see more, without telling them much information about what they will see” (O’Donovan, 2018). Generally social media companies seek to operationalise substantive definitions into computational signals. Thus, to Facebook, brief attention (or short ‘time-on-site’) is a signal of clickbait, for readers, having been lured in to the “junk food of content consumption”, are subsequently dissatisfied with the low-quality content, and leave the page quickly (DeAmicis, 2014). Clickbait, often innocuous, has been combined with divisive content, however (Burger and Schenk, 2019).

‘Extreme clickbait’ was a part of the the story behind the allegedly apolitical Macedonian teens based in Veles, who used “spammy techniques” in optimising pro-Trump sites to make money, in the run-up to the US presidential elections of 2016 (Silverman and Alexander, 2016). Followup reporting has sought to debunk that narrative, finding that the clickbait campaign was orchestrated by political operatives (Wendling, 2018; Silverman et al., 2018).

Computational propaganda, the second term, refers to the “the assemblage of social media, autonomous agents and algorithms tasked with the manipulation of opinion” (Neudert, 2017).

The breadth of the definition is intended to capture the bots that amplify content, the advertising platforms that enable micro-targeting and personalisation of influence messaging, and the click farms that inflate the follower counts and engagement scores, granting posts higher “vanity metrics” and thus greater symbolic power through fake support (Rogers, 2018a). For computational propaganda, bots increase the spread or reach of the posts and inflate their metric counts (Woolley and Howard, 2016). “Low-credibility content” is spread disproportionately by

‘social bots,’ which refer to bots or autonomous agents tasked with influencing discussion and

  17 public opinion; such a finding has led to calls for curtailing their use (Shoa et al., 2018). As a part of the ‘assemblage’ of actors and software practicing computational propaganda, the work of software-assisted, political operatives has come under scrutiny, especially in the run-up to elections. Sock puppets, assuming the false identity of a grassroots organiser or a concerned individual, create and circulate political content, organise events and mobilise audiences, making interventions in the physical world through hashtags, internet memes and Facebook events

(Mina, 2019). ‘Front groups’ or even faux ‘hashtag publics’ also mobilise followings and organise demonstrations (see Table one); one notorious case concerned an anti-Islam protest and counter-protest in Houston, Texas, in 2016, where both groups were mobilised by Russian campaigners operating under the names of the Blacktivists and the Heart of Texas, respectively

(Shane, 2018).

A related term for fake content insertion for political ends is astroturfing. It is the artificial seeding of newspapers and other content providers with political (or corporate) advertising disguised as genuine citizen concern. Such content is a different category than sponsored political content, where there are regulations that mandate labelling it as ‘paid for by’ a particular candidate or campaign (Vaidhyanathan, 2017). Nonetheless there have been calls to have

‘masked’ political content unmasked and marked as sponsored, however much in the recent case of a pro-Brexit group, Britain’s Future, investigative journalists were long not able to unearth the funding source, despite the transparency of its being labelled.

Particular forms of native social media advertising have prompted the calls for further public scrutiny of political ads, and also perhaps an expansion of the definition of such. ‘Dark posts’

(aka ‘promoted posts’) on Facebook refer to micro-targeted advertisements, without a referral page anchoring the content for further investigation (Bump, 2017). Used by political operatives, including foreign influence campaigners, in the US in 2014-2017 and beyond, such campaigning tactics assemble ‘keyword publics’ algorithmically by querying the Facebook advertising platform for words such as ‘second amendment’ or other pro-gun terminology and sending advertisements to the news feeds of the tens or hundreds of thousands of those users determined to have such an interest (Angwin et al., 2017). These publics are targeted not so much because they are persuadable voters but rather to have them circulate and amplify messaging.

2016 Fake rallies planned and promoted

Date Fake rally Location

25 June March for Trump New York

9 July Support Hillary. Save American Muslims Washington, D.C.

23 July Down with Hillary New York

  18 2 Oct. Miners for Trump Several Pennsylvania cities

12 Nov. Show your support for President-Elect Donald New York Trump

12 Nov. Trump is NOT my president New York

19 Nov. Charlotte against Trump Charlotte, N.C.

Table one: Overview of 2016 Fake rallies planned and promoted, as listed in the US indictment of 13 Russian nationals concerning foreign election interference. Source: Parlapiano and Lee,

2018.

Apart from particular social media advertising products such as dark posts, other formats have been identified as energizing publics with divisive messages. ‘Image macros’ are photos with two lines of text, one opening and one closing line, that are a popular format for political messaging on Facebook and have been among the most shared and otherwise most engaged-with content on the platform (Renner, 2017). Indeed, in the data analysis of the most shared posts of the

‘fake’ (or astroturfing) activist group pages set up by the Russian Internet Research Agency

(Blacktivists, United Muslims of America, Being Patriotic, Heart of Texas, Secured Borders and LGBT United), the image macros and other meme material scored particularly well (Chen, 2015; Albright, 2017; Timberg, 2017).

Russian influence campaigning, Russification and the ‘hyperpartisan’ style

‘Dark globalization’ is a term put forward by the historian Timothy Snyder to refer to how knowledge of western societal problems provides opportunities to influence campaigners from abroad, or Russia in particular (2018). In the US Snyder refers to the complex of race, gerrymandering and the electoral college, and the capacity to target voters in specific geographical areas (such as counties in ‘swing states’) with racialist political messaging that amplify or provide ‘oxygen’ to viewpoints. There have been detailed analyses of the Russian influence campaign of 2014-2017 commissioned by the US Congress, both of which benefited from data provided by Facebook, Twitter and Alphabet (Google) that previously had not been made available for research (Howard et al., 2018; New Knowledge, 2018). They are a part of a litany of literature that has appeared since the commissioning by governments to study the

‘tactics’ of the influence campaigners as well as the contemporary art of propaganda and the development of counter-narratives more generally. These studies also have led to recommendations concerning how to combat the effects.

The New Knowledge study (and its coverage) emphasise the collective cognitive dissonance that effective propaganda achieves, introducing (and popularising) language from intelligence and counterintelligence work. Among the goals of the propagandists is to create “a wilderness of mirrors”, originally a phrase from a T.S. Elliot poem but mobilised by the intelligence community (Holzman, 2008). It refers to an environment where truth (and its establishment) are no longer self-evident (Groll, 2018).

  19 To achieve that goal, New Knowledge argues, one particular tactic is the creation of a similarly termed “media mirage,” or “interlinked information ecosystems designed to immerse and surround targeted audiences” (2018: 42). They are enveloped in an information “cacophony,” where stories from the press are repurposed, and given another author (‘blacktivists’), interpretation and tone. Here is one example, taken from an original newspaper story about how an “11-Year-Old Texas Boy Invents Device to Prevent Hot Car Deaths” (Dahlgren and Arkin,

2017). It was reworked as follows: “[T]hese are stories of Black children the media don’t want you to see”; “White people invent tools for killing, this Black child is inventing a tool for saving lives” (New Knowledge, 2018: 62). The divisiveness and the effectiveness ascribed to the sample post derives not only from the transformation of the feel-good news story into a contrived ingroup and out-group divide based on race. Note, too, the format used; the second sentence is a two-liner, to be cast into an image macro or meme, the popular format for sharing and further circulation of grievance, outrage as well as mockery. The story also brings together categories of fake news. It is both clickbait as well as extreme content (be it hyperpartisan or junk), as it invites the consumer to read more about the grievance. It is also packaged to be shared.

The purpose of such campaigning is to sow discord and enmity, but it is only one of a variety of tactics where the overall goal is to remove a sense of a collective and shared experience of the world, as analysts have phrased it, and reify group formation (Gessen, 2018). Apart from the creation of a media mirage, the other tactics listed are as follows: “targeting, asset development, cross-platform brand building, memetics, inflecting a common message for different audiences, narrative repetition and dispersal, repurposing and re-titling pages and brands, manipulating journalism, amplify conspiratorial narratives, sow literal division, and dismiss and redirect” (New Knowledge, 2018: 2). With respect to social media, as discussed above, targeting could refer to the audience segmentation available in platforms for advertising purposes, and memetics to the use of both the image macro to formulate a punchy message as well as to build the meme as an additive content container for narrative reinforcement.

It is worthwhile to mention that the expert studies are snapshots, but these as well as subsequent reporting have pointed to the ‘ongoing efforts’ of the influence campaigners, and its global spread. While social media companies – since the Cambridge Analytica and fake news scandals – have become more active in identifying and suspending accounts of known Russian and other state-sponsored trolls (e.g., Iran), similarly named accounts are active and can be traced to known networks of political operatives (New Knowledge, 2018; FireEye, 2018). New accounts are continually made (Vaidhyanathan, 2018); the Chief Technology Officer at Facebook speaks of

“blocking more than one million fake accounts every day, sometimes just when they are created” (O’Brien, 2019). The percentage of influence campaigner accounts in that large number is not given.

Recently, there has been growing concern not only about the ongoing efforts of Russian influence campaigners but also the uptake by other groups (or “domestic actors”) of the socalled ‘Russian playbook’ (Frenkel et al., 2019). Journalistic coverage was prompted by the announcement by Twitter that prior to the US Congressional elections of 2018 it removed accounts of Americans posing as members of state Republican parties (Harvey and Roth, 2018). Facebook also announced that hyperpartisan pages on both sides of the political spectrum in the

  20 US would be removed. Discussions of the ‘Russification’ of online political campaigning also historicised disinformation, pointing to the classic examples, such as the claim that the HIV virus was the leaked product of a US bioweapons lab; it was planted in news outlets beginning in 1983 by Soviet dezinformatsiya campaigners in ‘Operation Infektion’ and ultimately spread four years later to US TV news (Boghardt, 2009; Ellick and Westbrook, 2018). Comparing the time span of such a news spread to the dynamics of reach in the hybrid media system nowadays is how one may describe how the ‘platform press’ has supercharged fake news (Chadwick, 2013; Bell and

Owen, 2017).

In a well-cited article in the New York Times, Facebook, as a leading example of the ‘platform press’, was described as a “totally insane, unintentionally gigantic, hyperpartisan political-media machine” (Herrman, 2016). The author spends some time describing the manner in which

Facebook mixes posts in its news feed from both family members and faint acquaintances, but also discusses the presence of upstart media organizations and self-styled advocacy groups that only exist online, many only in social media. Most are described as ‘hyperpartisan’. These sources populating the platform with content are defined as “openly ideological web operations”. They also are successful, not just because more extreme and sensational content spreads faster than more sobering truth (Vosoughi et al., 2018). It is also because they employ employ formats that engage large numbers of users and learn from their engagement and reach. ‘Operating’ in a continuous feedback loop of metrics data, posts are optimised to perform well in social media.

The performance measures are based on the virality of posts, and those that work well are emulated. There are particular formats as well as styles that drive engagement. Memes and clickbait such as listicles, cliffhanger headlines and human interest stories are among the formats used, as mentioned above. The hyperpartisan style has a variety of substantive features, not all of which are equally applied, but many appear to be working well. Often anti-establishment as well as positioned as against or in competition with the truth-seeking and fact-finding of the mainstream media, the media operations post stories that are alternatives. These alternatives may be interpretations, facts and editorial commentary on events. They become media layers on the news. The presentation is often edgy, both in terms of being knowledgeably on trend but also sharp in tone. The posts are regular, and as such are part of the permanent updating culture, providing a competing ‘feed’ about what is happening in the world and in media.

The post-truth condition

There is a series of contemporary utterances that have contributed to public discourse about a post-truth condition. One is the satirical notion of ‘truthiness’ (Colbert Report, 2005).

Developed as political news commentary and comedy, it refers to having the appearance of being true, but without evidentiary basis. Another – ‘alternative facts’ – is of more recent, political coinage. It refers to the insistence by a member of the Trump administration that the number of attendees at the presidential inauguration in 2016 was higher than reported and measured by crowd science (Still, 2017). The clarification of the meaning behind ‘alternative facts’ is more to the point: “additional facts, alternative interpretation” (Nuzzi, 2017). Compared to truthiness, here facticity does not derive from appearance or perhaps authority but rather from other factmaking.

  21 In response to what is sometimes considered first-order objectivity battles, or disputes over matters of fact (Margolis, 1995; Latour, 2008), newspaper reporting with such headlines as “Here Are the Real [Facts]” as well as the work by fact-checking bureaus and initiatives are contesting fact claims with increasing urgency (Fandos, 2017). These are public debates about facts, inputs into which include fact-checking, a common practice of journalists and university research groups seeking to confirm the basis behind particular statements by politicians and others

(Graves, 2016). Recently, scholarship on the effectiveness of fact-checking has developed in at least two directions: the extent to which fact-checking corrects the record as well as factual beliefs, and whether it changes attitudes (Barrera et al., 2017). Both are part of the decades-long discussion and critique of the ‘information deficit’ and ‘diffusion’ models, which challenge ideas that providing correctives clears up controversies (Wynne, 1991; McNeil, 2013).

In the fake news scholarly discourse, it has been found that there are distinct audiences for

‘alternative facts’ and ‘fact-checked facts’ (Bounegru et al., 2018). Whilst there may be a correction to the record, the original audience may not have been exposed to it. Fact-checked stories also have similar circulation patterns to alternative facts; they are forwarded to likeminded audiences (Shin and Thorson, 2017). Though it does not tell the entire story about exposure, both the original as well as the fact-checking publications are outlets with distinctive audiences or subscriber bases, with fact-checking newsletters often with smaller, specialty circulations, though their visibility may increase as they are built into the Facebook interface. In the other strand of work, it is asked, does exposure to fact-checked facts change factual beliefs as well as attitudes? Here one set of findings is in keeping with the critiques of the effectiveness of fact-checking and the information deficit model more generally, for respondents saw their factual accuracy improve, but their attitudes remain unchanged (Nyhan et al., 2019). Fact-checking, however, could be understood as a documenting process that corrects the record by capturing a dubious story and committing it, and its debunking or exposure, to searchable databases and other media.

The post-truth condition, though, has been described as a competition with respect to not firstorder but second-order objectivity. In such a circumstance there is a rise of competing regimes of truth (Fuller, 2018). Expertise becomes ‘sectarian’ (Turner, 2001). The idea of the media mirage (evoked to describe effective disinformation campaigns) does not in itself create a competing truth regime or infrastructure. Rather, it introduces noise into an infrastructure. But when propagandists, or in a different reading of the contemporary situation, a right-wing media ecology, create an alternative news and information infrastructure, those efforts fit with descriptions of the post-truth condition (Benkler et al., 2017; Sängerlaub et al., 2017).

In other words, post-truth is a term that should not be construed as signifying hoodwinked (or radicalised) consumers, or the ‘wholesale cheapening’ of fact-making (Sismondo, 2017). Rather, in asking whether “we can have our facts back”, the debate concerns whether (or when) publics can agree on the ‘facticity infrastructure’ or even the modernist project of knowledge institutions (Marres, 2018). As a case in point, there are ideologically distinctive alternatives to Wikipedia

(such as Infogalactic, Metapedia and Conservapedia), producing encyclopaedias challenging not only what is known or settled fact, but also the sources rooting it (Fitts, 2017).

  22 Elections, disinformation, and the Dutch case

Three recurring topics are often discussed in the news and (commissioned) research on disinformation and fake news in the Dutch context. First of all, of particular concern are Russian trolls and their spreading of disinformation in the Netherlands. Secondly, there are the (non

Russian) fake accounts and fake fans that that inflate the popularity of a campaign or a prominent figure, granting them greater symbolic power. And thirdly, publications are addressing its discernibility and possible countermeasures. How to recognise it and combat it? Each of these discussions is often set against the backdrop of a changing news media landscape, whereby mainstream news is increasingly competing with more tendentious and hyperpartisan outlets, and digitization is leading to user-driven and algorithm-driven personalization. That may narrow the horizon of news that users encounter and perhaps increase fringe consumption, though in empirical studies such has not been found (Wieringa et al., 2017). Comparisons of the Dutch situation are also drawn with the US.

While digitization may be changing how people consume news, a study of online news behavior, disinformation, and personalization of the news by Rathenau Institute stresses that in the

Netherlands, the traditional news media still hold a firm and stable position in the media landscape (van Keulen et al., 2018). The study also finds that there is not (yet) widespread algorithmic personalization in Dutch media sites. And, in stark contrast to the current situation in the US, Dutch news consumers tend to use a variety of sources and have trust in the traditional news media (and less so in social media). Lastly, the report underlines that the

Netherlands does not have such a particularly polarised media landscape as the US.

Overall, there is a strikingly moderate tone of voice in the literature, both in news reporting and research reports. Since 2016, several studies have looked at disinformation practices in the Dutch political landscape, and each of them has concluded that neither is there any large-scale disinformation activity in the Dutch media nor does disinformation have a significant impact on Dutch citizens. However, in the Summer of 2017, Wilfred Rietdijk, a Dutch general and national security advisor, announced in an interview with Dutch newspaper de Volkskrant that the

Netherlands could no longer deal with the digital threat (van Zijl and Modderkolk, 2017). A

“landslide of fake news”, as the subsequent tabloid headline read, would lead the country into chaos and division (Jonker, 2017). Including breaches and intrusions in his threat assessment

(thereby widening the scope beyond disinformation), Rietdijk explained how Dutch companies are “in the line of fire” from “thousands of hackers from Russia, China, and countries such as

Iran and even Sudan” (van Zijl and Modderkolk, 2017). The general is not the first to warn of

Russian interference in the Dutch online space, though case studies were lacking, at least in the public domain.

Russian trolling and its perceived insignificance in the Netherlands

When the Minister of Internal Affairs, Kajsa Ollongren, warned the Dutch government of

Russian disinformation in the Netherlands, she initially was criticised for not having compelling examples (Pleijter, 2017; Kist and Wassens, 2018a). Two journalistic studies that have looked into Russian tweets have found activity in the Dutch online realm, however. A study by NRC

Handelsblad mined 200,000 tweets from Russian Internet Research Agency (IRA) accounts and found disinformation campaigning beginning in 2014 and another spate in 2016. The weekly

  23 magazine De Groene Amsterdammer combined the NRC Handelsblad data with larger collections of

Russian troll accounts, made available on the American public opinion analysis website,

FiveThirtyEight as well as the lists published by American Congress (van der Noordaa and van de Ven, 2018a). Both studies found a peak in trolling activity after the downing of MH17 in July of

2014. The NRC Handelsblad study finds that Russian trolls posted 57,500 tweets, most of which were in Russian and aimed to influence public opinion in Russia and Ukraine, and only four of the tweets were in Dutch (Kist and Wassens, 2018b). The study by De Groene Amsterdammer confirms that most tweets on MH17 were in Russian but finds more mentions of Dutch

“conspiracy theorists and activists”, indicating a shift from challenging Western narratives (for

Russian-speaking audiences) to seeking to stir conflict within the West.

A second event revealed more coordinated Russian troll activity in the Dutch language Twitter space (in Belgium and the Netherlands), and a further example of striving to foment unrest, albeit unsuccessfully (according to engagement measures) (van der Noordaa and van de Ven,

2018b). It concerned the spreading of anti-Islam content directly following the terrorist attacks in the Brussels airport and metro in March 2016, and in the two years after the attacks. This anti

Islam ‘campaign’ involved about 950 tweets in the Dutch language that were circulated by some 150 IRA-related accounts. These tweets were rarely retweeted, however. In the event, Russian trolls are more successful in the Netherlands with the circulation of English-language content.

While these tweets are not related to Dutch issues and focus on for instance the US elections, they have been shared widely by over 6,000 Dutch Twitter users with a total of 9.5 million followers (Kist and Wassens, 2018a).

Perhaps counterintuitively, there was only minimal Russian interference with the Ukraine referendum in the Netherlands in April of 2016 (NOS, 2017). There was the Russian video capturing fake Ukrainian far-right militia members threatening terrorist attacks in the

Netherlands and burning a Dutch flag, but it was readily recognised as propaganda (Bellingcat, 2016). Otherwise, only a handful of tweets propagating a ‘No’ vote was found in the larger set of tweets under study (van der Noorda and van de Ven, 2018a).

The NRC Handelsblad concludes its work on the Twitter data set by noting that it is possible there is larger scale Russian activity in the Netherlands; it should be studied beyond just Twitter to include other platforms with known troll activity, such as Facebook, Instagram, YouTube and Reddit. Indeed, especially after Trump’s victory in the US presidential elections of 2016, many news outlets pointed towards Facebook. As discussed in some detail below, a study by BuzzFeed

News compiled the most engaged-with posts in the nine months prior to the elections and found that fake news during that time was circulating more than mainstream news. Journalists from the NRC Handelsblad replicated the study’s general method for the Netherlands, but with a narrower definition of fake news. They determined that the one hundred most-shared political news articles from January and February of 2017, in the run-up to the Dutch general elections, did not contain fake news (Kist and Zantingh, 2017). Certain articles could be considered misleading or biased, they thought, for they exaggerated news facts or took them out of context. The themes that were most resonant during the campaign period in the Netherlands were immigration, Islam and Geert Wilders.

  24 Dutch fake followers and trolls

Until November of 2017 much of the reporting has insisted that the Netherlands – and the

Dutch elections in particular – have been largely unaffected by disinformation or fake news.

Much of the news coverage that speaks of it concerns ‘fake followers’. For instance, in 2015, there was a small scandal about Geert Wilders concerning a dubious increase in his followers on Twitter. Indeed, when Twitter addressed the issue of fake followers and follower count inflation through a mass removal of suspect accounts in 2018, Wilders as well as other Dutch politicians

(including from the political party Denk) saw their metrics decline (NOS, 2018). In perhaps the most well-known case, the Dutch singer-songwriter Dotan was found to have a fake following of 140 user accounts, which were used between 2011 and 2017 to like the musician on social media, edit the Wikipedia article on the artist, request his songs at radio stations and circulate heartwarming stories about him across social media platforms. One of the profiles declared how Dotan’s music helped her through a period of grief after a miscarriage; another tells how Dotan welcomed one fan’s terminally ill brother in a meet-and-greet, throughout which the singer held the boy’s hand. Both testimonials were false, as reporters of De Volkskrant found and Dotan later confirmed (Misérus and van der Noordaa, 2018a; 2018b).

In 2018 the first large-scale global study of computational propaganda was published, examining organised social media manipulation such as the use of fake followers in 48 countries, including the Netherlands (Bradshaw and Howard, 2018). The study describes the different computational tactics employed not so much by Russian influence campaigners but by political parties to influence voters and the elections. 2 It was found that the use of social media as an infrastructure for the spread of propaganda and disinformation has become widespread. Under examination is “cyber troop activity,” defined as “government or political party use of social media to manipulate public opinion” (Bradshaw and Howard, 2018: 9).

While in more authoritarian regimes, social media manipulation fits into larger scheme of voter suppression and election rigging, in “emerging and Western democracies, sophisticated data analytics, and political bots are being used to poison the information environment, promote skepticism and distrust, polarise voting constituencies, and undermine the integrity of democratic processes” (Bradshaw and Howard, 2018: 5). The tactics described include the use of three kinds of fake accounts. First, there is the creation of online commentator accounts that attack and troll genuine users, spread divisive content, or “[divert] conversations or criticism away from important issues” (Bradshaw and Howard, 2018: 11). A second tactic entails automated accounts or political bots to automatedly flood particular hashtags, and astroturf by faking a follower base. The bots also troll genuine users by reporting them and flag organic content thereby having both suspended until a human moderator checks them. A third tactic is the use of hybrid accounts, which are those that make use of automation (for the sake of speed and convenience) but are actively curated by human users, who commonly manage multiple fake accounts or sock

2 The research conducted a content analysis of news articles reporting on cyber troop activity in a sample of 48 countries, supplemented by an in-depth secondary literature review. To collect the news articles, the researchers used the following keywords in combination, in queries across Google, Yahoo!, Bing and LexisNexis: astroturf*; bot; Cambridge Analytica; Facebook; fake; fake account; disinformation; government; information warfare; intelligent agent; military; misinformation; persona management; pro-government; propaganda; psychological operations; psyops; social media; sock puppet*; troll*; Twitter (2018:8).

  25 puppets. This type of fake account is difficult to recognise, and thus to combat. The study finds that automation is the most recurring tactic, seen in 38 of the 48 countries that were shown to operate fake accounts.

Besides the use of fake accounts, other strategies involve the use of political ads and the involvement of search engine optimization and activity on chat applications and across social media platforms. Where Twitter is proven to be the platform most friendly for automation, the study finds “cyber troop activity on chat applications or other platforms (Instagram, LINE,

SnapChat, Telegram, Tinder, WeChat, WhatsApp)” in one-quarter of the countries under study (Bradshaw and Howard, 2018: 13). In the European countries in their sample, they find a distinct fake news footprint per country. In Germany, junk news is rather marginal and was mostly circulated by right-wing political actors during the 2017 federal elections. In Italy on the other hand, a large and active “ecosystem” of it is connected to populist political forces such as the

Lega Nord (Northern League) and the Movimento Cinque Stelle (M5S, 5 Stars Movement), which were at work during the 2017 constitutional referendum and the elections of 2018. Here, fake news connects national politics to Euroscepticism, conspiracy theory, aliens and pro-Putin propaganda. In the Netherlands, this analysis also finds that disinformation revolves around politician Geert Wilders and in particular the spread of his anti-Islam video, which was broadcast on television and shared in social media in the lead-up to the 2017 Dutch national elections. In particular, the study finds that automated accounts have amplified Geert Wilders’ campaign hashtags.

These results match the findings in a study that looked at troll-like behavior on Twitter, leading up to the 2017 Dutch general elections, where sock puppets were found (Bounegru et al., 2018). The study collected over 500,000 tweets mentioning at least one of the Twitter accounts of the

28 political leaders a month before the 2017 Dutch general elections. To retain the users that demonstrated troll-like behavior, it narrowed down the set to retain only the 25 users who consistently targeted one or more political representatives. 3 The analysis showed that there was a notable asymmetry in the distribution of targets of troll-like behavior and sockpuppetry across the political spectrum, where left-wing politicians are most often targeted by negative mentions, while right-wing politicians receive support. Troll content extended to reputable news sources which cited it at least thirty times. Among the cited troll accounts were fake news organisations with names as ‘Today in Syria’ and ‘WorldNewsPolitics’, political parties (including multiple fake accounts for the Republican party in Tennessee) and concerned citizens, most of whom were fiercely pro-Trump and anti-Islam (Kist and Wassens, 2017). In another analysis by the NRC

Handelsblad, a Dutch political party (DENK) also exhibited troll-like behaviour, including sockpuppetry on both Twitter as well as Facebook (Kouwenhoven and Logtenberg, 2017).

While Dutch news consumers have been found to use a variety of news sources, the Netherlands also has a steady junk news diet (Burger et al., 2019; van der Poel, 2019). From 2013-2017 Dutch Facebook users consumed more low-quality, commercially driven clickbait than mainstream news, as was found through engagement scores. As may be expected, there is also relatively more clickbait on Facebook than quality news.

3 By @mentioning them at least 100 times in a one-month period.

26 The consumption and forwarding of clickbait, extreme clickbait as well as other problematic information extends also to politicians and public figures. One Dutch researcher, Peter Burger, has a collection of instances when Dutch politicians have retweeted anti-semitic or otherwise disturbing content. In one example, a video purporting to show ‘Muslims vandalising Christmas market in Lithuania’ was actually a recording of an event that took place in Baltimore in the US

(Burger, 2016).

Figure 1. Cartoon that ridicules the fake news taskforce, stating: “internet trolls are best countered by internet hobbits.” Source: Reid et al. (2018).

Recognizing and countering disinformation in the Dutch online space

Various initiatives aim to detect and counter disinformation in the Netherlands and on an EU- level. The EU taskforce (East Stratcom Task Force) against disinformation was heavily criticised in the Netherlands after its project EUvsDisInfo mistakenly categorised articles by The Post Online,

GeenStijl and De Gelderlander as disinformation (van Keulen et al., 2018; Heck, 2018). (Figure 1 shows a cartoon about the fake news taskforce, stating internet trolls are best countered with internet hobbits.) In a sense the dispute stemmed from misreadings of single stories, perhaps without an appreciation of how settled some of the sources are in the Dutch media landscape, despite their tendentious style (in the case of The Post Online and GeenStijl). For its part, De

Gelderlander had taken over nearly verbatim a Russian storyline concerning the perpetrator behind the downing of the MH17 but did attribute it to its original source in a barebones reporting style. The flagged cases were removed from the EUvsDisInfo site after complaints by the Dutch media organization Persgroep (EUvsDisinfo, 2018).

Fact-checking as a journalistic practice has taken hold in the Netherlands. Many newspapers have implemented (or revived) a fact-checking section, often dedicated to checking statements made by political figures in interviews in newspapers or TV shows. There are also websites such as Hoaxmelding.nl and Nieuwscheckers.nl that compile lists of instances of false news on

Facebook and elsewhere. For their study of disinformation, Rathenau researchers analyzed these lists, comprising respectively 140 on Hoaxmelding (collected between 1 February 2014 and 18

December 2017) and 166 on Nieuwscheckers (between 3 February 2017 and 5 January 2018)

(van Keulen et al., 2018). They found that the items on the list of Hoaxmelding involved examples of unfounded warnings (65), polarizing disinformation (32) and fake crime news (31).

Additionally, there were several examples of clickbait, benign as well as malicious. The content

  27 steers users to advertising, “like-farming” and phishing sites (van Keulen et al., 2018: 38). Such posts contain human interest stories that are “painful on a personal level” (van Keulen et al.,

2018: 45). The researchers found that only 25% of the disinformation concerned political content and most clickbait serves a commercial goal, rather than a political one. On the list of items collected by Nieuwscheckers, the Leiden University-based initiative, less than half was found to have political content. Within the full set, the researchers found six examples of polarizing content. Otherwise, many of the posts concern factually incorrect, public statements by politicians, the investigation of which is how fact-checking is conventionally practiced.

Figure 2. “Detected and eliminated” fake news, with a warning issued by NU.nl and Nieuwscheckers. Source: NOS (2017a).

Fact-checking now extends well beyond unpacking politicians’ statements, and Facebook has entered into partnerships with many bureaus around the world, including the Netherlands, to explore and catalogue dubious content. In 2017 Nieuwscheckers partnered with Facebook and

NU.nl and celebrated their first collaborative “successful detection and elimination of fake news” that year when they flagged a tabloid-style, human-interest post about an Australian newborn weighing 20 kilograms (see figure 2). In February of 2019, however, Nieuwscheckers withdrew from the Facebook fact-checking initiative because of liability risks (Kist, 2019). Nu.nl continues to work with Facebook on fact-checking, and they do so on a paid basis, an issue raised repeatedly in the context of journalists’ being asked to address an issue of Facebook’s making on a voluntary basis.

The effectiveness of fact-checking as a strategy in the Netherlands is a different question. As mentioned above, fact-checks and fake news often have separate publics, and fact-checks may lead people to fake news, rather than away from it. A recent study in the Netherlands found that even when many people would agree with a fact-check, they are not interested in reading the fact-checking article, prompting the scholars to advise journalists to make the fact checks an engaging read (Hameleers and van der Meer, 2019). Another strategy to counter disinformation concerns a strand of media literacy that involves developing skills to recognise fake user accounts and disinformation. One is on a source level, the other on a story level. The Field Guide to Fake

News provides a method for the detection of trolling accounts by looking at their friends, or their profile information (Bounegru et al., 2018). There are also courses and training modules for fake

  28 news detection and fact-checking, such as those given by Radio Netherlands (RNTC, 2019). The other format is the fake news quiz, such as those by de Volkskrant (2016) and the Guardian

(2016), as well as the New York Times ‘deceptive Facebook post’ test (2018). These quizzes make it clear how challenging it is to recognise fake news. The Dutch serious game, titled Slecht Nieuws (‘Bad News’), invites players to create fake news and by doing so gain insight into the strategies behind it and become more astute in its recognition (NRC, 2018; DROG, 2018). It is part of efforts that study fake news as risk and ultimately seek to inoculate populations against it

(Roozenbeek and van der Linden, 2018).

Voting aid applications

Voting aid applications (VAAs), often called stemwijzers in Dutch, are generally websites that describe their purpose as helping undecided voters find the political party that best matches their preferences and positions. As such, in the context of the study of fake news, they could be regarded as a competing persuasion instrument, a preemptive measure against influence campaigning, or even a potential site that may include it, either through parody, hoax or hack.

The literature on VAAs takes up the Dutch and Belgian cases, countries that together with

Germany, Austria and Switzerland have upwards of half the voter population accessing them prior to elections. The work can be positioned broadly as pertaining to “the impact of internetbased applications on politics” and can be roughly divided into user studies, impacts of VAAs on the voters as well as the methods behind them (Hirzalla and van Zoonen, 2015: 88). To date these online voting aids have not been raised as recommended technology to combat fake news and influence campaigning per se, though they do furnish a rather personalised information experience that may be studied for its ‘influence’ effects, as discussed briefly below.

In studies of their usage, researchers have asked whether VAAs “mobilize the mobilized”

(Hirzalla and van Zoonen, 2015). And indeed, while VAAs have a heterogenous user base across demographics, interests, attitudes and behaviour (Vassil, 2011), there is an overrepresented subgroup of younger, mainly left-of-center, urban and well-educated male users who are politically active or knowledgeable. This imbalance could lead to the conclusion that those who would benefit the most from political advice are the least likely to seek it (Ruusuvirta, 2010).

A second set of literature concerns the impact of VAAs and assesses whether they have influenced the voting behaviour of its users, though it is not clear whether the quality, reach and graphical interfaces of the aids affected the extent of the influence. From those surveyed anywhere from 1% to 15% using DoeDeStemTest (in Belgium) as well as StemWijzer and

Kieskompas (in the Netherlands) reported having been influenced by the aids (Walgrave et al.,

2009; Hirzalla and van Zoonen, 2015). While research has found that the politically knowledgeable and engaged users that are common to use VAAs perceive them as useful, they are also among the less likely users to be influenced by them (Alvarez et al., 2014; Dumont and

Kies, 2012).

A third set of literature concerns the methods used by the VAAs. Here there is a distinction between the choice of the policy positions to include in the interactive system and the models underlying the advice. The very selection of the policy positions is a crucial factor in the voting advice given, where another set would lead to other advice (Walgrave et al, 2009). In general,

  29 VAAs are found to select policy positions according to their saliency (for the election period), and variability (in that different parties hold different positions) (Hirzalla and van Zoonen, 2015). The editorial process differs, where certain VAAs select their statements solely with experts such as political scientists or journalists (e.g., the Austrian VAA wahlkabine.at), while others co-create the formulation of VAA positions, workshopping them with party representatives in the case of the Dutch StemWijzer, or with an editorial board that consists of professional experts as well as first and second-time voters in the German “Wahl-O-Mat” (Garzia and Marshall, 2017).

As the voters register their political views, and in certain cases add weight to them, the software calculates the extent to which the voters’ preferences match the respective parties’ and presents its results as a ranked list, bar chart, grid or radar chart. Several studies concentrate on the workings and visual outputs of the different voting aids. Louwerse and Rosema (2014) dissect them by examining how many dimensions are taken into account when ranking the political parties. In their study, a one-dimensional model refers to the ranking the political parties based on the level of agreement with the voter and presents its findings as a ranked list or bar chart. A two-dimensional model places the political parties’ statements and the voters’ responses on a continuum from left-wing to right-wing and proposes its match accordingly. The more elaborate multi-dimensional model, employed by the Swiss smartvote application, plots the statements and responses onto eight policy dimensions and presents its results in a spider plot that is more complex to read (Louwerse and Rosema, 2014). In a comparative test of these models, researchers took a dataset from the Dutch Stemwijzer and found that the different spatial models would lead to very different matches (Louwerse and Rosema, 2014).

As mentioned above, the voting aids are rather popular in a series of European countries and could be considered not only as another information input but also as one that competes with campaigning. Though the influence (similar to campaigning) may again be minimal, it could be considered as another approach or counter-measure in the discussion of how to address the fake news problem.

Fake news may be pervasive, but is it persuasive?

If one were to divide the current period of fake news studies into waves, it could be argued that the first related to the definitional issues and the production side (as mainly discussed above), whilst the second is increasingly concerned with the study of its consumption (Boczkowski,

2016). In other words, fake news may be seen as “pervasive, but is it persuasive?” (Shaw, 1979). Why do people consume fake news, and do these readers have particular demographics or profiles? Which people deem these stories credible or at least have pass-along value? Are they persuaded or even persuadable? In the US and in a growing list of other countries social media platforms are increasingly a main source of news, and the manner in which they deliver news is different from a newspaper or similar package or container (Gottfried and Shearer, 2016;

Poynter, 2019). One receives single stories, rather than an entire newspaper, each shared by someone with whom the social media user has made a connection, most often directly. These can be friends (Facebook), followers (Twitter), connections (LinkedIn), etc. Stories arrive in the feeds algorithmically, meaning there is a filtering mechanism where certain of them are boosted, based on signals such as activity and increasingly trustworthiness, or the amount of given and measured meaningful engagement between individuals. Put differently, those who are close to

  30 the user (by some special measure) are the ones whose stories more likely will be seen (Eslami et al., 2015). Such observations have led to discussions of the re-application of the notion of the filter bubble, a term originally associated with a user receiving personalised (rather than universal) search engine results (Pariser, 2011; van Keulen et al., 2018; Puschmann, 2018).

Personalisation, however, has evolved from being the result of the information interactions of one user searching to engagement with an entire social network. As such it shifts the bubble from enveloping the individual to the group; it has prompted ‘bubble studies’ of not just social media news environments, but those of health, science, fashion and other areas of collective information production, sharing and recommendation (Pedersen and Hendricks, 2014;

Hendricks and Vestergaard, 2019). Indeed, fake news circulation and consumption are increasingly experienced as an issue for the environment (e.g., climate change and its sceptics), health (e.g., the anti-vaccination discourse) and a variety of other areas (Kitta, 2018).

Such findings have led researchers to define on the one hand the groups most likely to consume and share fake news together with the dynamics of their bubbles, and on the other the meaning, or sincerity attached to the sharing. In terms of the consumption of fake news, it could be said at the outset that there have been two widely cited findings about their significance from the journalistic arena. One found the most shared stories during the US presidential elections were fake news (see figure 3), and the other that Russian disinformation campaigns had a far greater spread than previously imagined as well as reported in testimony by Facebook before the US

Congress (Silverman, 2016; Timberg, 2017). These findings have since been put into a broader context and compared to ‘normal’ political campaigning and the development of messaging strategies, filtered through news. First, in the event, only a small fraction of the population consumed such ‘news’ (Allcott and Gentzkow, 2017). Given the limited exposure, the impact, if at all, would have paled in comparison to political TV commercials (Persily, 2017). There is the larger question, however, of whether the messaging would have anything but “minimal effects”

(Lazarsfeld et al., 1948). As has been repeatedly found, the net effect of campaigning, albeit by political elites, that persuades the prospective voter is exceedingly low or even zero (Kalla and

Broockman, 2017). The aim then is less to persuade than to “rous[e] the enthusiasm of existing supporters” (Panagopoulos 2016).

Though they may have begun as symmetrical studies of the right and the left, of the most circulated findings to date about the spread of fake news – both with respect to the contents as well as its consumers – ultimately all have overwhelmingly concentrated on the right, be it conservatives and the alt right in the US or other right-leaning, populist right or new right publics in Europe (Bounegru et al., 2018; Benkler at al., 2018). It is of interest to note for starters that both during the US presidential campaigning and thereafter the information spaces or spheres of the right contained far more fake, junk, disinformation or otherwise dubious stories and sources than the left (Faris et al., 2017). Thus, conclusions drawn about right-leaning publics sharing information should take into account that they are disproportionately exposed to such information; all else being equal, the right would share more of it (Marwick, 2018). In the empirical studies it was found that the right (most notably Trump supporters) consumed the most fake news. However, there seems to be an older, hard core of its consumers in the US during the run-up to the US presidential elections in 2016 – “the 10% of Americans with the most conservative information diets” (Guess et al., 2018). These are heavy media users, and

  31 ‘available audiences’, who have made time to consume media (Nelson and Taneja, 2018). Unlike the majority of the media-consuming public, they are far more likely to read niche rather than only establishment sources. There is, in other words, a normalcy to the consumption by those audiences of fringe materials.

Figure 3. The birth of the fake news crisis, or ‘fake news’ outperforms ‘mainstream news’ on Facebook, in the run-up to the U.S. elections in 2016. Source: Silverman, 2016.

The strand of work that considers why users share fake news should be prefaced by the distinction between “earnest and ambivalent” internet users (Hedrick et al., 2018). Much of the scholarship about internet culture has not considered that considerable cultural production and sharing are undertaken not to be part of participatory culture, connective action and other earnest forms of civic culture online but rather for unsympathetic amusement (aka ‘lulz’)

(Phillips, 2015). ‘Sharing’, a term that has mutated in digital culture from acting in a gift economy to a dominant form of so-called platform capitalism, could have been prompted these days as much by insincerity as by mindfulness (Barbrook, 1998; Belk, 2007; Srnicek, 2017). That is, the rationale for making and sharing could go “either way … complicating an easy assessment of authorial intent” (Phillips and Milner, 2018: 11). Such a tricky attribution of intent is especially troublesome in the spaces where vitriolic exchange as well as extreme speech and content are prevalent. It is difficult to disentangle whether one is sharing for amusement and to trigger a reaction, or for substantive reasons.

As has been found in the US context, the fake news stories most shared on social media resonate with particular grievances (about the bias of establishment sources) and resentments (concerning economic opportunity) that underlie certain societal divides (Marwick, 2018). Moreover, the stories do not stand alone in a mirror world of conspiracy theory but rather are contiguous with more mainstream conservative news, anchored by Fox News; they are more extreme as well as

  32 transgressive in their wording and presentation. Hence the notion of ‘hyperpartisan’, but also there is also reference made to tendentious, anti-establishment sources. Here the Overton

Window is appropriately referenced, meaning the bounds of current, acceptable public discourse, and the extent to which extreme speech in hyperpartisan and tendentious sources is moving established norms (Daniels, 2018).

Fake news studies: Digital methods and data journalism

As we come to shortly, one research strategy for measuring the prevalence of fake news story types and sources around national elections is to gauge their presence generally in scoping exercises, but also more specifically in the most engaged-with content in social media concerning elections, political parties, candidates and social issues. A more subtle analysis would examine the top stories for the penetration of fake news narratives, measuring mainstreaming. Moreover, through comparison of engagement with fake news, one also could determine which platforms are most susceptible (or amenable) to hosting and circulating such content. Facebook in particular has been held up as a fake news machine (Herrman, 2016). Empirically, it has been found to host (proportionately) more of it, whether narrowly or liberally defined, than other platforms (Guess et al., 2018).

In order to contextualise such measures it is worthwhile to consider the ways in which the scale, reach and impact of fake news have been studied to date with media analysis, or more specifically digital methods and data journalism. The methods generally could be considered mixed quantitative/qualitative approaches. They often begin in the journalistic arena, with the qualitative determination of the dubiousness of a set of sources and/or stories, and proceed with digital methods that design queries and collect data from platform APIs, media monitoring company dashboards, and social media companies that have furnished lists of banned trolls or user accounts. Indeed, with respect to the dubious source lists, Buzzfeed News’ original list of about 20 sources determined to be fake news inform a series of empirical studies (Silverman,

2016; Allcott and Gentzkow, 2017; Bounegru et al., 2018; Marwick, 2018; Grinberg et al., 2019). For studies of the Italian news space, the lists relied upon are from BUTAC, Bufale and

Bufalopedia (Fletcher et al., 2018; Butac, 2018). Hoaxwijzer’s list of 92 Dutch-language “false news” sites also informs certain of the empirical studies to date in the Netherlands (van Keulen et al., 2018; Wieringa, 2017). 4 But other work, such as the NRC Handelblad’s analysis of the extent of the problem of fake news in the Netherlands in the run-up to the 2017 national elections, looks at the sources afresh, making on-the-spot determinations of fakeness (Kist and Zantingh,

2017). These may conflict with previous listings. For example, Hoaxwijzer lists De Dagelijkse

Standaard as a “false news” site whereas the NRC Handelsblad did not determine it to be “fake news”, but the “extreme right news site” fell among those they called “misleading” because it reported that “1,000 crazy Muslims” had “torched” a church in Dortmund on New Year’s Eve when instead a firework had landed on its roof causing slight damage. The NRC Handelsblad determined that it did not meet its definition of fake news as a “fully fabricated story packaged as news”.

4 As other studies also found, the list is dated; as of April 2019, 40 of the 92 sites are offline. It remains useful as a list for older media corpora.

  33 Indeed, the question of detecting fake news, on a source or story level, is often placed at the feet of journalists, media organisations and fact-checking bureaus, where credibility and transparency may be rated (NewsGuard, 2019). Masked sources are penalised, for example. As mentioned above, for online stories, the determination of dubious content may benefit, too, from a genre analysis (Lüders et al., 2010). Disinformation, conspiracy, clickbait and (automated) amplification have styles (Rony et al., 2017). Disinformation tends to be a hard counterfactual presentation, conspiracy has multiple characters and plot entanglements, clickbait is a cliffhanger that is often painful on a personal level and (automated) amplification posts at particular intervals and in coordination, as malicious social bot detection projects have found (Ratkiewicz et al., 2011; Bessi and Ferrara, 2016; RoBhat labs, 2017). Other technical signatures of dubious news sites are of interest. For instance, empirical work on the types of cookies and third-party elements in mainstream and nominally fake news sites found distinctive types in each, with the mainstream sites using customised trackers and the other off-the-shelf (Bounegru et al., 2018).

With the lists of fake news sources either in place or determinations still to be made, the next step is to build a media corpus. Following Buzzfeed News’ method, many undertakings query media monitoring services (such as Buzzsumo, Com Score and Facebook’s Crowdtangle) for keywords and/or URLs, in order to build source sets of most engaged-with media and pull in engagement scores per story. Certain of the techniques also include further interpretative coding of stories, including grievance narratives (Marwick, 2018).

Whilst much attention has been directed towards Facebook, and the study of the election-related stories most engaged with on that platform, Twitter is often used as the preferred data source, given dedicated data sets (made available by Twitter or academic researchers) of accounts run by the Internet Research Agency (Farkas and Bastos, 2018). There is a series of studies that rely on

Twitter’s curated sets as well as on the data robustly collected and shared among data researchers, such as by Clemson University and FiveThirtyEight, mentioned above. In a form of crowd science, the publication on GitHub of the Clemson data set led to numerous studies; in the US widespread disinformation campaigning was found, as is known, but also more nichetargeting of politicians in such states as Maine (Roeder, 2018). As in the Netherlands, discussed above, the data were put to use in other countries that according to journalistic accounts had been previously understudied. For example, in Italy IRA trolls posted numerous pro-populist party tweets in Italian, joining the ‘cacophony’ or media ecology around the right, as also discussed in the OII work (Fubini, 2018; Fletcher et al., 2018). Twitter is also preferred given the general ease of use of data collection through its streaming and search APIs, intermediate services such as Hexagon Crimson for samples as well as the availability of historical data sets, albeit at a price.

Other approaches (considering consumption and persuasion rather than definition and production) should be touched on that rely on surveys, user data collection and experiments.

Numerous experiments have been performed on misinformation (Jankowski, 2018). Recently, for example, a representative sample of the population consents to having their online media consumption passively monitored, and subsequently surveyed thereafter (Guess et al., 2018). Or, there are experiments that show fake news to consumers, and subsequently provide fact-checks to them in order to determine whether the fact-checks should be ‘attitudinally congruent’ for

  34 them to be persuasive (Hameleers and van der Meer, 2019). In another experiment in the

Netherlands, commissioned by the newspaper, de Volkskrant, respondents were presented with fake news around one of four topics: vaccinations and autism, MH17, rape incidents in connection to migration, or Sylvana Simons (a politician and media personality) and discrimination. The study tests whether they became less certain about the facts after being misinformed (I&O Research, 2017; Kranenberg, 2017).

With respect to platforms other than Facebook and Twitter, YouTube and especially Instagram appear to be relatively understudied but significant, and Reddit and 4chan are being recognised as breeding grounds for some of the more outlandish and consequential content such as

Pizzagate (New Knowledge, 2018; Tuters et al., 2018). There are platform-specific approaches for building and analysing datasets for Instagram (through queries for hashtags and place names), as well as YouTube, Reddit and 4chan (Rogers, 2018b; Rieder et al., 2018). Google web search also has invited scrutiny, given the extreme content returned for queries such as the holocaust.

Buzzfeed’s work on detecting and analysing fake news on Facebook has been particularly influential in data journalism research and subsequent studies that build upon it, and thus is worthy of mention in some detail (Silverman, 2016). First, the researchers built a keyword list concerning elections (and especially controversial election topics), and subsequently queried those keywords in media monitoring software (Buzzsumo) that returns stories ranked by engagement scores. With the aid of the results, they built a fake news and hyperpartisan website list, which they merged with lists of the same that they curated previously through separate reporting, including on the infamous collection of about 100 websites created by the Macedonian clickbait makers, members of the same family of sites (with the same Google Analytics ID) of

WTOE 5 News that created the story about the Pope endorsing Trump, and a collection of hyperpartisan sites (Silverman and Singer-Vine, 2016; Silverman et al., 2016). They also curated a list of some 20 mainstream news sites. 5 (All the accompanying data Buzzfeed also made available through online Google spreadsheets, in keeping with emerging standards in data journalism.)

The engagement scores of the top mainstream news and top fake news stories are subsequently compared. In the first study of this kind and perhaps the beginning of what could be called the

‘fake news crisis’ for Facebook, it was found that the fake news stories outperformed by engagement scores those from the mainstream news in the three-month period before the US presidential elections, thereby leading to conclusions about the comparable “power of fake election news on Facebook” (see Figure 3) (Silverman, 2016). Follow-up reporting has considered the extent to which such news continues to resonate more on Facebook than mainstream news stories, despite incipient efforts by the company to curtail its impact. One of the major studies commissioned by the US Congress found that fake news and influence campaigning activity on Facebook and especially Instagram substantially increased after the US elections (Howard et al., 2018).

In April of 2019, some two and one-half years after Buzzfeed News story, we found that only 4

5 Buzzfeed’s list contains the following mainstream sources: New York Times, Washington Post, NBC News, USA Today, Politico, CNN, Wall Street Journal, CBS News, ABC News, New York Daily News, New York Post,

BuzzFeed, Los Angeles Times, NPR, The Guardian, Vox, Business Insider, Huffington Post and Fox News

(Silverman, 2016).

  35 of the 13 top-performing fake news and hyperpartisan websites are still online: World News

Daily Report, Burrard Street Journal, Twitchy and Breitbart. The others appeared to have been fly-by-night operations, which is another means of considering a source’s dubiousness. That is, the other 9 sites, including two Macedonian-made ones (Denver Guardian and World Politicus) and the highest-performing site (Ending the Fed) that spread the ‘Pope endorses Trump’ story are all gone.

Facebook’s adjustments

After the US elections in 2016, Facebook CEO Mark Zuckerberg initially argued that “the idea that fake news on Facebook influenced the election in any way, I think is a pretty crazy idea,” and put forward that such material amounted to a small fraction of its platform’s content (Isaac, 2016). Two years later Facebook’s work to quell fake news, together with its more stringent policies on (research) data access should be considered here. Addressing the fake news crisis, there has been an increase in those hired to perform ‘content moderation’, referred to as the janitors of social media, or even those doing the platform’s dirty work (Roberts, 2017). Facebook also installed a political ad transparency tool; it lists on the ad itself who has sponsored it, and there also is a political ad archive and an API (Hern and Waterson, 2018). No longer is the targeted individual the only one able to view the hitherto ‘dark post’.

With respect to algorithmic changes, in 2018 Facebook began a three-pronged strategy that would favour “meaningful connections” (family and friends), “trusted sources” (user-surveyed media) and “local news” in the news feed over more far-flung “businesses, brands, and media”

(Abbruzzese, 2018; Flynn, 2018; Gartenberg, 2018). It should be remarked that these are global initiatives, coming on the heels of well-reported Facebook-associated riots in Myanmar and Sri

Lanka but also the compilation of compendiums on the effects of fake news on Facebook all over the world, as the OII’s global study have shown, but also the numerous governmental and think tank (umbrella) initiatives such as disinfoportal.org.

Whether Facebook’s measures are working in some sense is unclear. The political ad library tool may show a source, but who is behind it may remain unclear as in the case of a pro-Brexit campaign group, Britain’s Future (see Figure 4), that spent hundreds of thousands of pounds on ads in the run-up to significant UK parliamentary votes (Waterson and Hern, 2019). Efforts by journalists to unmask the source behind Britain’s Future as well as other ‘dark money’ campaigners had for months been in vain (Monbiot, 2019). Significant political ads are also not in the archive, as ProPublica found, before its tool crowdsourcing Facebook ads and targeted individuals was purposively rendered inoperable by the company in what it called a “routine update” that would prevent illegitimate “scraping” (Merrill and Tobin, 2019). Similar tools by

Mozilla and Who Targets Me also broke, thus making the verification work a difficult prospect.

The news feed tweak to boost “meaningful connections” was initially critiqued for its capacity to exaggerate the importance of fake news, as was observed in Slovakia and elsewhere when dubious sources saw their engagement scores rise (Frenkel et al., 2018). The prominence of

“meaningful connections” and “local news” in the news feed, according to Buzzfeed, stirred as well as amplified the Gilets Jaunes protests in France, for their coverage on the local news made

  36 the anger groups (groupes colère) and their posts more prominent in the news feeds, as evidenced by engagement scores from Crowdtangle (Broderick and Darmanin, 2018).

Given the fake news crisis stemmed from the US elections, Facebook also has created specific initiatives for future elections that would put political parties and their positions on issues in a single, curated Facebook portal. One of the early projects was for Sweden’s national elections in 2018, which, it was found in a separate study (with Twitter data), suffered from ‘junk news’ quantities second in magnitude only to that surrounding the US elections, and much larger in fact than such materials around the German, French and Dutch elections in 2017 (Hedman et al., 2018; Kist and Zantingh, 2017). The Facebook elections project, rolled out in meetings with social media researchers in 2018, also coincided with their new academic ‘partnership’ project,

Social Science One. It makes available to researchers data sets such as all the URLs that have been posted to Facebook over the course of a year (King and Persily, 2018). At the same time, however, Facebook revoked approvals to research software (such as Netvizz and Netlytic) that made use of its Pages API, sparking academic protest about ‘locked platforms’ (Bruns et al.,

2018; Rieder, 2018). Seen as reactions to the Cambridge Analytica scandal, Facebook’s measures could be described as curating the datasets researchers can use. The datasets notably do not include Facebook pages themselves and their engagement scores – data that led to the very knowledge about the fake news crisis and the scope of the Russian influence campaign in the first instance (Albright, 2017).

Figure 4. Facebook political ad library tool, results for Britain’s Future, 13 March 2019.

Conclusions: Fakery and campaigning

The fake news phenomenon could be viewed as a revival of previous ones that typically have occurred when a new media technology is introduced that destabilises production, distribution and consumption of news and information, as was the case with eighteenth and nineteenth century broadsheets and tabloids (respectively) but also the radio and newswire of the twentieth century. The early web and the blogosphere also challenged existing news publication practices and were considered unedited spaces populated by self-styled authors, providing speedy news

  37 ‘too fresh to be true’. Now social media platforms disrupt the trustworthiness of established news and fact and reintroduce the idea of the web as ‘truthless medium’ (Marres, 2018).

The post-truth age, or condition, as it were, may be viewed in light of a conflict between what counts as ‘fake’ (on a source or a story level), but it has been described rather as a contest between facticity regimes, or even sets of sectarian expertise. Locating a network of so-called

‘fake news’ websites, for example, could be viewed as the discovery of an influence campaign, but it just as well can be seen as an ‘alternative facts’ media ecology. When it is a hyperpartisan, right-wing news ecology, as in the US in the run-up to the presidential elections of 2016, it could be described as a part of the contemporary post-truth situation, or, as been often related, a culture war.

Having the ‘trappings of news’ in terms of look and feel, fake news has been defined as consisting of distinctive types with varying intentionality. For instance, disinformation and malinformation (the neologism) are meant to harm, whereas misinformation may be just as false but its circulation unintentional. As a case in point, satirical stories and parody may become misinformation, such as the story about the Pope endorsing Trump, which outperformed (by engagement score) any other ‘news’ on Facebook during the US presidential election campaigning in 2016.

In both the public as well as scholarly discourses, there has been a swing from the hype of the fake news problem (perhaps well exemplified by the Dutch tabloid headline “landslide of fake news”) to its gradual debunking, e.g., “researchers say fears about ‘fake news’ are exaggerated”

(Ingram, 2019). Such a view has resulted from a series of studies not just on engagement but also on its consumption, including the rationale behind its sharing. Small, older populations appear particularly active, as do ‘heavy news consumers’ and ‘available audiences’, or those who have at their disposal time for fringe news consumption and spreading it among online friends. The vast majority of news consumption remains of the mainstream sources, however. The evidence that consumers have been influenced or persuaded is minimal.

Nevertheless, there appears to be agreement that social media platforms remain worthy of study not only as the new ‘truthless medium’ but for their capacity to accelerate (or ‘supercharge’) fake news distribution in a hybrid media system comprised of new and established media and media formats. Despite increased content moderation, automated detection work, and a reorientation of its news feed principles, does Facebook remain a fake news machine, comparable to the one during the US presidential campaigning? Indeed, Facebook, at first hesitant to admit an issue, has taken a series of measures since then that strive to produce more trustworthiness, such as boosting posts by friends and family, crowdsourcing trusted sources as well as favouring local news, though the effectiveness of these reengineered principles has been questioned. Indeed, continuing empirical research on the most engaged-with, political news on Facebook could shed light on the quality of the platform’s content delivery, however much data access may be restricted to researchers. It remains to be seen how ‘oversight’ research will be affected when

Facebook closes research APIs and instead curates data sets for researchers, rather than allowing them to create their own. Other oversight projects have been thwarted; in early 2019 Facebook’s

  38 “routine update” blocked the software by ProPublica, Mozilla and Who Targets Me that was collecting political ads and their targets, as mentioned.

The question of fake news as a campaign strategy – be it by Russian operatives, Russified domestic actors, hyperpartisan or tendentious right-wing media-makers, and others – also has been meticulously studied, with detailed ‘playbooks’ laid bare as tactics to create both a media mirage (where fact and fiction are difficult to disentangle) as well as competing truth regimes, offering counter-expertise as well as uncertainty. Governments around the world have commissioned studies, revealing the breadth and scope of the problem, explaining the playbook and putting forward policy recommendations such as increased media literacy and the regulation of political advertising on platforms, including ‘dark’ posts. Platforms are asked to create public archives, which also would benefit research as well as (data) journalism. Fact-checking also has gone global, though it often remains a small-scale enterprise practiced by bespoke bureaus, occasionally working in tandem with Facebook, checking posts that have been flagged by users, and weighing in on the question of fakery.

Finally, there are scholars in the US and recently in Europe putting forward the argument that studying Russian disinformation shifts the attention away from the home-grown hyperpartisan news ecologies that have been emerging over the past few years, particularly on the right

(Benkler et al., 2017; Benkler et al., 2018; Rone, 2019). The point also fits with the ‘dark globalization’ argument concerning how existing domestic divisions, displayed in this media, may be exacerbated by foreign operatives but are not created by them. To date the effectiveness of

Russian influence campaigning in Europe, in either sowing or exacerbating division, has yet to be compellingly demonstrated; the false and junk domestic news sources (e.g., the pro-Russian sources re-narrating the cause of the downing of MH17) also appear to have scant reach

(Fletcher et al., 2018). In a climate of heightened sensitivity towards dubious sources and stories, it remains to be seen whether they have staying power.

* * * * * *

Appendix

Governmental efforts and discussions of countermeasures

A first step for many national governments and other regional political entities that wish to counter disinformation is to install committees as well as task forces; it occurs across the globe, from the much publicised hearings by the US Congress and UK Parliament on the Russian involvement in the US elections and the Cambridge Analytica affair, to the task forces and other entities formed in many of the nearly 50 countries where influence campaigning has taken place

(Bradshaw and Howard, 2018). Following from these convenings, there have been national calls to regulate the “digital giants”, and the European Union, through its creation of a High-Level

Expert Group (EU HLEG) on fake news and online disinformation, has issued its recommendations for countering disinformation, including calls for transparency, media and information literacy, and tools for empowering journalism. In the European countries with recent or imminent national elections there has been even greater urgency, with Germany and

  39 France enacting legislation (online hate speech and ‘fake news laws’, respectively), and Sweden and Denmark engaging in awareness-raising as well as media literacy campaigns. Denmark installed a ‘digital ambassador’ (Gramer, 2017).

Below is a list of certain measures to counteract disinformation and fake news, gleaned from recent governmental documents and related materials. They include social media company regulation, codes of ethics, fact-checking and media literacy campaigning.

Social media company regulation

Many government committees agree that the large tech companies that have come to dominate the online realm, such as Google, Twitter, and Facebook, should be regulated, but caution overregulation in forms that would curtail expression and press freedoms. The starting point for the regulation of these companies to counter disinformation is to address political advertising on social media platforms. It can include the verification of those paying for political advertisements and disclosing them publicly. Additionally, all social media companies could be required to create public archives of advertisements so that among other ad types ‘dark posts’ may be studied

(Bradshaw, 2018). In fact, as said, Facebook has such an archive (and an API), but it also prevented watchdogs including Mozilla from verifying its collection techniques, equating their methods with illegitimate data “scraping” (Merrill and Tobin, 2019).

Relatedly, the EU HLEG proposes the development of a ‘European-wide code of practices’ that describes the roles and responsibilities of relevant stakeholders such as tech companies, and media organizations but also research organizations and fact-checking initiatives, based on key principles (2018). In short, they address the adaptation of political advertising policies (including sponsored advertisements and other forms of content), and the provision of access to data for research and fact-checking. They also propose the installation of advanced settings for users to customise their user experience, collaboration with news outlets to facilitate users’ access to trustworthy news, the facilitation of fact-checking and content flagging, and allowing users to

“exercise their right to reply” (EU HLEG, 2018: 32-33).

The UK Parliamentary report on fake news and disinformation speaks in an unusually piqued tone of the importance of regulating social media platforms and related tech companies, singling out Facebook as providing the “impression of working towards transparency”, but often

“obfuscating” how well it is capturing and archiving political ads (House of Commons, 2019:

85). Ultimately, they call for establishing an “educational levy” or charge on social media companies to fund digital literacy as a fourth pillar of the education system after reading, writing and maths (House of Commons, 2019: 87). There is also a recommendation that social media companies should develop means to distinguish between those sources regularly furnishing disinformation and those who do not, in a new system of “content regulation” (House of

Commons, 2019: 87). While carefully worded, that measures can count on the criticism that similar proposals have faced concerning the restriction of the freedom of expression, while not being effective measures against hateful or incendiary content (Access Now et al., 2018).

Nevertheless, legislation has been passed. Germany has established a law, NetzDG, that extends its hate speech legislation compelling social media companies (with more than two million

  40 registered users in Germany) to remove such speech rapidly or face hefty fines (Claussen, 2018). More controversially, France has new legislation which applies to ‘false information’; the law requires that three months prior to an election ‘false news’ be removed.

Detecting and removing fake content

The Reporters’ Lab at Duke University keeps track of fact-checking initiatives worldwide and has identified some 160 active initiatives (Duke Reporters Lab, 2019). In European countries, some fact-checking initiatives are attached to news organisations, but most are operating as notfor-profits (Wardle and Derakhshan, 2017; Graves and Cherubini, 2016). Many work in tandem with Facebook; as of January 2019 some 50 fact-checking groups, who are party to the

International Fact Checking Network Code of Principles, independently assess fake news flagged by users (Volpicelli, 2019). The expertise developed includes a variety of flagging and adjudication systems such as NewsGuard’s “nutrition label” that evaluates some 2,000 online news sources, or, as it relates, the sites that garner about 95% of engagement in the news sector

(2019).

Automation

Brief mention should perhaps be made of automation as offering methods for flagging dubious or false content, however much it is rarely recommended in governmental reports. With respect to fact-checking, if there are shared databases of ‘already fact-checked’ stories as well as sources, then software could cross-check suspicious ones against those already debunked or evaluated, as the UK parliamentary report mentions. The discussion concerning the need for human reviewers for content interpretation and curation remains pertinent.

Counter-narratives

In Germany the government chooses to actively participate in spaces where disinformation is spread. “On these platforms, the German Government provides both reliable information that can be fact-checked and a narrative based on this information” (German Federal Foreign Office, 2018). In that vein, rumoursaboutgermany.info is a website for collecting and countering disinformation about Germany spread by human traffickers. While Germany chooses to work with counter-narratives, others have criticised this approach. A Canadian intelligence report argues that developing counter-narratives is a “one event at a time approach” that “fails to address the source and methodology of information campaigns” (Canadian Security Intelligence Service, 2018: 66).

Media literacy and digital ‘hygiene’

The EU high level expert group on fake news and online disinformation makes a case for increased media and information literacy to counter disinformation, which should be

“implemented on a massive scale in school and teacher training curricula” (EU HLG, 2018: 26). This media literacy also should involve the development of tools and training modules for journalists. As a particularly relevant method, the group proposes “more powerful tools to be able to visually map online networks and connections to understand how disinformation is being created, spread and amplified” (EU HLG, 2018: 28).

  41 Some countries speak of ‘digital hygiene’ when referring to media literacy practices, for instance in France when making a case for the development of skills to assess the validity of the arguments and the reliability of the source. “This is a public hygiene measure—just as people in the 19th century learned to wash their hands” (Jeangène Vilmer et al., 2018: 179). In Sweden the word ‘cyberhygien’ is employed. The Swedish Civil Contingencies Agency has published a handbook for communicators in public sector organizations for the countering of disinformation, which includes strategies that range from source checking and recognizing a bot to choosing an appropriate response to disinformation. The Swedish Media Council developed a media literacy programme for young people, teaching them critical thinking and disinformation detection; it includes a set of educational materials on ‘source criticism’ (‘Källkritik’)

(Government Offices of Sweden, 2017; Swedish Media Council, 2019). Several recent reports stress the importance of better equipping journalists with tools and skills to recognise and avoid disinformation, mentioning the importance of fact-checking, critical source assessment and ethics (Jeangène Vilmer et al., 2018; Wardle and Derakhshan, 2017).

Investing in civil society and building public trust

A more general way forward that is presented in the literature is to invest in civil society, as it

“must remain the first shield against information manipulation in liberal, democratic societies”

(Jeangène Vilmer et al., 2018: 169). Such initiatives are specifically relevant around events such as elections, in which civil society can be supported through non-legislative, pre-emptive measures and multi-stakeholder collaboration of government with the industry, non-governmental sector, and regional actors (Haciyakupoglu et al., 2018). In Sweden, the aforementioned Swedish Media

Council is an example in which politicians and media professionals collaborate and meet regularly to discuss and counter disinformation and related challenges. Such regular, multistakeholder consultation both within and across European countries is among the recommendations often given (Brattberg and Mauer, 2018).

Guaranteeing participation in public debate by all

Lastly is the admonition issued in the 2017 joint UN declaration on “fake news” that emphasised the need for states to enable the participation of all in public debate. They should ensure that any efforts to quell or thwart the practices of fake news-making and spread as well as that of disinformation be handled within the context of the freedom of expression and the freedom on the press (McGonagle, 2017).

  42

JUNK NEWS ON FACEBOOK

DURING THE 2019 DUTCH ELECTIONS

Stijn Peeters and Richard Rogers

Junk news on Facebook during the 2019 Dutch elections

Stijn Peeters and Richard Rogers 6

Introduction: Facebook

Since 2016 online disinformation and so-called fake or junk news have been virtually synonymous with social media platforms, serving as their most significant conduits. The 2016 U.S. presidential elections and the British Brexit referendum of the same year opened a period of increased scrutiny of these platforms in how false or misleading information are published and amplified. Facebook, the single largest social media platform of the past decade, has been an obvious focal point. It has been the subject of a substantial and growing amount of studies that investigate its “challenge [to] journalism” (Johson and Kelling, 2018: 817), the persuasiveness of fake news shared on it (Allcott and Gentzkow, 2017) and the prevalence of it in the average user’s Facebook practice (Guess et al., 2019).

One of the first well-publicized reports on this topic, and the one that informed some of the subsequent research, was BuzzFeed News’ 2016 story on the prevalence of fake news in the three months leading up to the presidential elections that saw Donald Trump elected the 44 th president of the United States. The report, entitled ‘This Analysis Shows How Viral Fake Election News

Stories Outperformed Real News On Facebook’ (Silverman 2016), outlines user engagement with ‘fake news’, finding that in the last few weeks before the election it was engaged with more often than mainstream news.

Following this piece and other coverage on the prevalence of fake news on its platform,

Facebook repeatedly announced initiatives that were ostensibly intended to prevent it from happening again by employing third-party fact-checking organisations (Mosseri 2017a), giving

“more informative” content higher priority (Mosseri 2017b), providing more information about the authors of news content (Hughes et al., 2018) and increasing content moderation. Despite these changes, a few years after the 2016 U.S. elections the platform has still repeatedly been found in studies to be spreading problematic content. It has been criticised because of its role in spreading false and hateful content about minorities in Myanmar (Fink, 2018), live streaming the 2019 Christchurch mass shooting (Shead, 2019) and in inciting religious hatred in Bangladesh through viral content that is misleading (Haque et al., 2018: 1). In an analysis of social media use around the Mexican presidential elections in 2018, however, only “limited evidence of junk content on [Facebook]” was found (Glowacki et al., 2018: 4). Similarly, a 2017 analysis of social media usage by Dutch political parties found scant “dubious” content shared by Dutch political

Facebook pages (Wieringa et al. 2017, p.60), though their focus was Facebook pages associated with political parties rather than a larger Dutch Facebook sphere.

Facebook therefore remains an interesting object of study. It is both the platform most commonly associated with fake news as well as one that, at face value, has been relatively proactive in deploying initiatives against its spread. Additionally, existing literature is inconclusive with regards to the extent to which these measures have been effective, and there seem to be

6 The research reported here was undertaken in collaboration with Tim Groot.

  43 significant regional differences in the penetration of fake news in the discourse on the platform, and its effects. There is some existing research focused on the overall Dutch media sphere, most notable a study on fake news during the 2017 Dutch parliamentary elections by the NRC

Handelsblad, the national newspaper. The NRC Handelsblad notably found little evidence of the phenomenon; however, as both Dutch politics and Facebook’s platform have undergone changes since then, the two Dutch elections of 2019 – the provincial elections (provinciale statenverkiezingen) and the EU Parliamentary elections – present a useful case study through which one may investigate the extent to which disinformation and fake news in a broader sense play a role in this particular geographical context on the platform, three years after the 2016 U.S. elections, and two years after the previous major national Dutch elections.

While ostensibly regional in character, the Dutch provincial elections nevertheless have a “strong national component 7 “ (Hietbrink and van Voorst, 2011: 6) as they additionally determine the make-up of the Dutch senate, which is indirectly elected by the ‘provincial states’ (provinciale staten). As such they can serve as a national case study similar to that of the two other major case studies by BuzzFeed News and the NRC Handelsblad that serve as a kind of baseline for this one.

In addition to provincial elections, only two months later, in May 2019, the Netherlands took part in the EU parliamentary elections. Given the close proximity of these two elections, and their different character, they together provide an opportunity to explore disinformation and fake news in the media concerning Dutch politics.

In the following, we first discuss how their methods may be appropriated for this case study, through an adapted query list and a more well-defined typology of ‘mainstream’ versus ‘junk news’ sources, a term preferred over fake news, as we discuss in more detail below. We then analyse the results in terms of overall trends and a characterisation of the sites found in the junk news category. By way of wider contextualisation, these findings are further compared with results found in other case studies contained within this volume. Finally, we offer a characterisation of the platform-specific and cross-platform trends, and a qualification of the role junk news plays in Dutch political news coverage.

The BuzzFeed method: results so far

The two aforementioned journalistic analyses that have investigated discourse on Facebook in the context of national elections serve as a methodological starting point here. These are

BuzzFeed News’ landmark report into fake news in the lead-up to the U.S. presidential elections of 2016, and the NRC Handelsblad’s study of news shared on Facebook around the Dutch parliamentary elections of 2017, which was inspired by BuzzFeed News’ and to a large extent employed the same method.

Both of these studies used BuzzSumo, a commercial content aggregation and analysis platform, to track the most engaged-with articles shared on Facebook in the chosen time period.

BuzzSumo defines ‘engagement’ as a “sum of likes, comments, and shares attributed to an article” (Lee 2019). If the article is shared in multiple places (e.g., in multiple groups), the

7 Transl. from Dutch: “de statenverkiezingen hadden een sterk nationaal component” (Hietbrink and van Voorst, 2011, p.6)

  44 engagement score represents the sum of all engagement that BuzzSumo has gathered from the platform. After capturing this data through BuzzSumo, both BuzzFeed News and the NRC

Handelsblad categorised the results as of one of two categories, ‘mainstream’ and ‘fake news’. This simple typology has the advantage of providing clear results, though is potentially limited through its lack of nuance in terms of distinguishing between disinformation, conspiracy, clickbait, and hyperpartisan (as discussed in the introduction to this volume), or related terms as problematic information, misinformation and mal-information.

We adopt this basic method for our case study, but some refining is offered as the original description could be said to lack specificity in some areas. Particularly, with regards to what

BuzzFeed News considers ‘fake’, the report is somewhat ambiguous, but it does provide the source list in the form of open data. On the one hand, BuzzFeed News consistently refers to content as either ‘mainstream’ or ‘fake’/‘false’, implying that all of the content in that category constitutes articles containing untrue information. On the other hand, their definition of ‘fake’ is somewhat expansive in the sense that hyperpartisan sites such as Breitbart News are included in their ‘fake news’ category. Either way, the most engaged with content they found primarily consisted of such false stories as the Pope endorsing Donald Trump, Hillary Clinton selling weapons to ISIS, and a fabricated ‘leaked e-mail’.

While the NRC Handelsblad’s study broadly uses the same approach, its method differs in how it categorises the articles it found. Rather than focusing on disinformation, the NRC Handelsblad uses a broader category of “news that is taken out of context, strongly politically coloured, or has a strongly exaggerated headline” 8 (Kist and Zantingh, 2017). Approximately 10% of the content they found fit this description. This would include hyperpartisan outlets, even if they do not make false claims in their content. Their report notes that very little of the content they found was actual false news, or consciously misleading, but that approximately 10% of the content they found fit the broader description. Crucially, even with this broader definition their ‘nonmainstream’ category is far smaller than that of BuzzFeed News’ findings, and thus the NRC

Handelsblad answers its question of whether fake news (‘nepnieuws’) plays a role in Dutch elections with a resounding ‘no’. In spite of these different outcomes, in different contexts, both studies follow the same basic methodology of extracting results from a number of relevant queries from BuzzSumo, which we follow here.

BuzzFeed News’ method, as described in their report, is relatively straightforward: a list of queries is prepared, engagement for articles matching these articles is extracted from Facebook (via

BuzzSumo), the results are aggregated and divided into three-month periods, results are coded as either ‘fake’ or ‘mainstream’, and the relative prevalence of both categories is plotted over time

(Silverman 2016). More practically, this data was collected by BuzzFeed by querying BuzzSumo for a number of thematically appropriate queries. While no full query list is given, the examples include names of election candidates ([“Hillary Clinton”] and [“Donald Trump”]) and phrases reflecting topics of debate during the campaign, such as [Clinton AND emails]. They also included a number of “known viral lies” such as [Soros AND voting machine]. It should be

8 Transl. from Dutch: “nieuws [dat] uit zijn context werd gehaald, sterk politiek gekleurd werd gebracht of werd voorzien van een sterk aangezette kop” (Kist & Zantingh 2017)

  45 noted that the latter inclusion is somewhat asymmetrical for it means the search for more sensational and divisive subject matters is more precise and targeted than the search for mainstream news topics, thereby seeking fake news. In any case, the question of asymmetry is addressed in the case study at hand.

Query design: descriptions, issues and party leaders

Dutch provincial elections

We follow BuzzFeed News and the NRC Handelsblad in their general method in terms of query design, querying BuzzSumo in order to find the most engaged-with content on Facebook. We compiled a list of queries to search BuzzSumo following BuzzFeed News’ approach of mixing names of political leaders with issues that were particular to the given election campaign. This method also was used by the 2017 NRC study which queried “words like ‘elections’, ‘parliament’ and ‘polls’, and/or the name of a party, party leader, and/or widely discussed topics such as

‘health care’, ‘pensions’, ‘immigrants’ and ‘EU’” 9 (Kist and Zantingh, 2017). We used the NRC list as a starting point and adjusted it to fit the provincial elections rather than the national elections they studied.

A complication here is the dual local/national focus of the elections. While candidate lists differ per province, in televised debates, national rather than local party leaders participate, and they can generally be said to dominate media coverage (though some local broadcasters organise their own debates as well). In terms of media coverage, local leaders are simultaneously more numerous (as there are far more local leaders than national leaders) and much less significant (as news coverage and debates concentrate on national leaders). A national focus additionally was particularly apparent in the 2019 elections as polls indicated the cabinet risked losing a senate majority following the elections (Herderscheê and Meijer, 2019). For this reason, we limited our party-based queries to the last names of the political leaders of the parties that currently constitute the Dutch parliament, 10 as well as the name of the Prime Minister, representing the national government. 11

Additionally, we queried a number of political issues that were topics of debate during the election campaign. We looked at the manifestos of the larger Dutch parties and chose three themes that were both significant across all parties’ manifestos and had been the topic of media coverage during the ongoing campaign: [Klimaat] (climate), [Migratie] (migration), and [EU].

Finally, we queried two further general keywords, [verkiezingen] (elections) and [PS2019], a widely used hashtag and shorthand for the elections at hand.

The queries were undertaken to capture the election campaign period from 18 February 2019

(the start of the first full week of campaigning, marked by the launch of various voting aids and

9 Transl. from the Dutch by the authors: “termen als ‘verkiezingen’, ‘Tweede Kamer’ en ‘peiling’, en/of de naam van een partij, lijsttrekker en/of veelbesproken onderwerpen als ‘zorg’, ‘AOW’, ‘asielzoekers’ en ‘EU’” (Kist and

Zantingh, 2017).

10 [Asscher], [Baudet], [Buma], [Dijkhoff], [Jetten], [Klaver], [Krol], [Kuzu], [Marijnissen], [Segers], [Staaij], [Thieme], and [Wilders].

11 [Rutte].

  46 launch events hosted by a number of parties) to 5 March 2019 (five days after the elections), or five full weeks after the start of the campaign for the provincial elections

EU parliamentary elections

Using the same general strategy, another set of queries was run to find discussion pertaining to the EU parliamentary elections on 23 May 2019. As parties ran with national lists in this case, we queried the lead candidates for each party in addition to the current political leaders of all parties in the Dutch parliament. 12 Querying these again was necessary as national leaders played an active role in the election campaign, such as when Mark Rutte, the VVD Prime Minister, and

Thierry Baudet, the leader of the FvD, engaged in a televised debate on the eve of the elections.

We further queried general election-related phrases, as well as three themes that occurred across multiple parties’ manifestos: [klimaat], [migratie] and [privacy]. As the elections coincided with a government campaign seeking to make voters aware of the dangers of disinformation (Ministerie van Binnenlandse Zaken en Koninkrijksrelaties 2019), we also queried [“fake news” OR fakenews OR nepnieuws OR desinformatie OR junknieuws]. Finally, for this election we also queried the names of all parties for which one could cast a vote. 13

We queried these keywords using BuzzSumo, limiting ourselves to articles in Dutch, excluding

Belgian sources. As with the Dutch provincial elections, for the EU campaign we queried a similar 5-week period between 19 April and 23 May (election day). Finally, we removed irrelevant results such as those covering various Belgian election campaigns and those resulting from ambiguous keywords such as [Klaver], the name of a party leader but also the word for clover.

Outlet coding: fake and/or junk news?

An important question here is how one identifies a source as either mainstream or its counterpart, whether fake news, junk news or another terms (such as problematic information).

While mainstream appears rather straightforward to identify (though that also may shift in time), its counterpart is a fuzzier concept. BuzzFeed News described their fake news as emanating “from news websites that only publish hoaxes or from hyperpartisan websites that present themselves as publishing real news” (Silverman, 2016). Here both types of sites purport to be “news”, but not in the manner or with the substance that the mainstream publishes, given their hoaxes or hyperpartisanism, or strongly politically coloured.

Another notion is ‘junk news’, and it may be preferred because it avoids the other, historically fraught ‘fake news’ definition of the ‘lying media’, but is more ontologically flexible, at least as scholars have described it. While this term has been used as a synonym for ‘fake news’

(Venturini, 2019: 10), Marchal et al. (2018) employ it to capture a broader category of content that consists of “various forms of propaganda and ideologically extreme, hyperpartisan or

12 [“De Graaff”], [“De Lange”], [“in 't Veld”], [“van Dalen”], [“van der Spek”], [“van der Staaij”], [“van Lanschot”], [Asscher], [Azmani], [Baudet], [Berendsen], [Buma], [Dijkhoff], [Eickhout], [Eppink], [Hazekamp], [Hoekstra],

[Jetten], [Klaver], [Krol], [Kuzu], [Manders], [Marijnissen], [Rutte], [Segers], [Thieme], [Timmermans], [Tonça],

[Wierda], [Wilders].

13 [50Plus], [CDA], [Christenunie OR SGP], [D66], [Denk], [FvD OR “Forum voor Democratie”], [GroenLinks],

[“Jezus Leeft”], [PvdA], [PvdD OR “Partij voor de Dieren”], [PVV], [SP], [VVD].

  47 conspiratorial news and information” (2). This then would include BuzzFeed News’ notion, but also part of the NRC Handelsblad’s broader category of tendentious sites that may more often comment upon rather than deliver news, as we come to.

For their ‘Junk News Aggregator’, a Facebook junk news scraping project, researchers at the

Oxford Internet Institute identified a set of measures to define what qualifies as junk news, consisting of 1) a lack of journalistic standards; 2) tendentious style; 3) low credibility; 4) clear bias; 5) a mimicry of traditional news reporting aesthetics; or 6) aggregating content matching the first five criteria (Liotsou et al., 2019: 3). A source was then considered junk news if it satisfied at least three of the first five criteria, or the sixth. Herein lies the flexibility, but also the breadth of the definition that may be suitable for the current analytical purposes in the Dutch case.

In its report, the NRC Handelsblad concluded that propaganda or disinformation did not play a significant role in Dutch media. It also distinguished between mainstream and hyperpartisan sources, where the latter is news that is purposively taken out of context, exaggerated to promote a cause (i.e., tendentious) or strongly politically coloured. A number of Dutch outlets can be qualified as both ‘tendentious’ and strongly politically coloured, while also being embedded in the Dutch media landscape (and in that sense mainstream or mainstreaming). Originally a socalled ‘shock blog’, Geenstijl describes itself as tendentious, and gave birth to PowNed, a public

TV broadcaster with a similar signature style. Given its durability and link with the public broadcasting company, GeenStijl could be considered both tendentious and mainstream, or the hybrid category, tendentious-mainstream. Another case that is prominent in the BuzzSumo results we found is The Post Online (TPO). It is a right-wing media outlet and could fit the NRC’s definition as well as a broader definition of hyperpartisan sites as “openly ideological web operations” (Hermann 2016). Putting it in the same category as more fringe sites such as

Ninefornews.nl (a site promoting conspiracies and UFOlogy) or De Dagelijkse Standaard (a farright outlet that regularly publishes virulently anti-immigrant articles) would not do justice to the less extreme tone. Thus, we could dub it tendentious-hyperpartisan. In the analyses to follow here we show the results with tendentious as a separate category made up of these two sources.

In other studies to follow (on Twitter), the results are compared when the tendentioushyperpartisan source is categorised as tendentious or as hyperpartisan (see Niederer and Groen, this volume).

In the following we employ the fine-grained categorisation and continuum, distinguishing between ‘mainstream’, ‘tendentious’, ‘hyperpartisan’, ‘conspiracy’ and ‘clickbait’, occasionally linking the categories, as mentioned. These categories reflect the various sub-types of mainstream, tendentious and otherwise lower-quality content discussed in the introductory chapter. This also allows more nuanced categorisations of sites such as GeenStijl and The Post

Online. In the following analysis we offer this five-category coding as an addition to the binary

OII-based categorisation, as a way to illustrate the make-up of non-mainstream content found in the data. This categorisation resulted from a collaborative coding effort across all case studies found in this volume and provides a more detailed alternative to the binary ‘fake/junk’ versus

‘mainstream’ opposition found in, for example, the BuzzFeed News and NRC Handelsblad studies.

  48 In all we therefore elect not to reduce the sources to fake but rather use a more inclusive category of ‘junk news’, but then also pay special attention to the tendentious outlets. After identifying the sites using this typology, we further removed all other sites from the results that were either marginal or local. Marginal here refers to sites that received very low engagement scores in the BuzzSumo results and were not otherwise notable in terms of content or overall engagement. We also excluded local news sites, as our main concern for this analysis is outlets with a national or otherwise substantial reach; regional outlets conversely typically have a limited audience, and our list of ‘junk’ sites contained more nationally oriented outlets rather than regional ones. This left a ‘mainstream’ category containing national outlets, mostly deeply embedded in the traditional Dutch media landscape, such as various national newspapers, TV broadcasters and a number of online news sites and magazines.

Data analysis: overall and per-query trends

Dutch provincial elections

We used the annotated source list (or expert list) to code the results for the BuzzSumo queries, as discussed in more detail below. This allows for a per-query observation of the ratio between mainstream and junk sources. Next to these separate analyses we also calculated an average ratio, weighted by the relative engagement per category, on both an overall and a per-week basis. While our categorisation method is slightly different from BuzzFeed News’, this per-week analysis nevertheless allows for a trend comparison with the results of their over-time analysis of the U.S. 2016 presidential election campaign.

Figure 1. Engagement of mainstream (blue) and junk-like news (pink) articles found through provincial electionsrelated BuzzSumo queries, per week, between 18 February 2019 and 25 March 2019. Engagement scores have been normalised.

  49 Figure 2. Total Facebook Engagement of fake versus mainstream news. Results from election-related queries on

BuzzSumo, for the 20 most-engaged with articles during February and November 2018, per three-month period. Source: Silverman 2016.

Notably, the trendline found in our over-time analysis (Figure 1) does not match the one in

BuzzFeed’s study (see Figure 2). While BuzzFeed News’ data saw a clear increase of engagement of fake news in the weeks leading up to the elections, in our data junk news stayed relatively constant in terms of engagement and even decreased slightly during the last few weeks. There are, however, some differences between the two campaigns that complicate a direct comparison. The U.S. election campaign is typically far longer than Dutch election campaigns, especially in this case as the 2019 election was concerned with the provincial states and senate rather than the lower house of parliament (typically the most important Dutch election). While the U.S. campaign was analysed over a period of 9 months, the Dutch campaign and hence our data spans five weeks only. Additionally, BuzzFeed News’ data resolution is quite low (one datapoint per three months) while ours is more fine-grained (one per week).

Nevertheless, even considering these differences it is striking that the graphs indicate rather different dynamics. While the BuzzFeed data points to a clear ramping up of fake news content as the election date draws near, our data is more in line with the NRC’s earlier study and suggests a more constant but persistent undercurrent of junk news that is a consistent part of politically oriented media output. The above data is an aggregate of all queries performed on BuzzSumo, however. While in aggregate there is no clear trend, this could be the result of summing up the values, and more apparent trends exist in the results for individual queries.

  50 Figure 3. Per-query engagement of mainstream (blue) and junk (pink) articles found through provincial electionsrelated BuzzSumo queries, per week, between 18 February and 25 March 2019. Engagement scores have been normalised.

As can be seen in the overview in Figure 3, even on a per-query basis there are few clear trends with regards to the prevalence of junk news engagement. There is an interesting uptick in the prevalence of mainstream engagement for a few queries. Most notably, the data for [Segers], the leader of ChristenUnie (a centrist Christian party), shows a sharp increase in the last week of election campaign. This can however almost entirely be attributed to news coverage after the

  51 elections about the implications of the election results for the cabinet, of which Segers’ party is the smallest member. (Note the similar uptick for [Jetten], whose D66 party is the secondsmallest cabinet member.) Another notable bump in mainstream engagement occurs for a number of queries ([PS2019], [Buma], [Kuzu] and [Dijkhoff]) around the middle of the election campaign. A closer look at the articles responsible for this engagement reveals that this may be an indication of the campaign coming into full swing and hence increasing media coverage of it.

The oft-quoted and feared BuzzFeed News pattern of fake news outperforming mainstream news is thus not repeated on either an aggregate or query level in this case study.

What remains of interest is the relative performance of mainstream and junk news on a per-week and per-query level, particularly on a number of occasions where junk news briefly outperforms mainstream news in terms of Facebook engagement. For queries of politicians, it occurs most notably for [Baudet], [Kuzu] and [Wilders] during the first week of the election campaign, where the dominance of junk news is most pronounced. These politicians all lead relatively fringe parties: Baudet leads the far-right Forum voor Democratie (FvD), Kuzu the left-wing and immigrant-oriented DENK, and Wilders is the leader of the far-right Partij voor de Vrijheid

(PVV). A closer look at the junk news articles that are responsible for these surges shows that in all three cases, these are not articles primarily concerned with the elections themselves but rather coverage of other political issues (mostly around climate laws that were being discussed at the time) by hyperpartisan outlets like De Dagelijkse Standaard. The relative prevalence of this coverage is perhaps an indication that media had not yet started covering the election campaign in earnest, rather than a dominance of junk news in election discourse. Overall, while in some individual cases junk news outperforms mainstream news, these episodes are outliers and represent less of an overall trend than one for particular parties. There is one general exception to this rule, however, and it concerns the query for [migratie], or migration, where junk outperforms mainstream for most of the period. Also, [klimaat] or climate, has a week where junk news had more engagement that the mainstream. These are rather polarising issues, drawing attention from hyperpartisan outlets.

EU Parliamentary elections

A trend analysis of the EU parliamentary results (see Figure 4) shows a pattern not too dissimilar to the one found in the provincial elections data, similarly seeing junk news match the performance of mainstream news particularly in the beginning of the query period. Recall that during the provincial elections campaign junk news performed as well as mainstream news on two occasions. Though this trend is still notably different from the one found by BuzzFeed News, where junk news overtook mainstream news towards the end of the campaign, it is nevertheless a significant finding that suggests an increasingly robust position for junk news in the Dutch context.

  52 Figure 4. Engagement of mainstream and junk-like articles found through EU elections-related queries on

BuzzSumo, between 18 February 2019 and 25 March 2019. Engagement scores have been normalised.

A closer look at this second week of the EU campaign data shows that the junk news engagement can for a large part be attributed to an article in De Dagelijkse Standaard, which discusses a video posted by the political party Denk on their Facebook page, accusing the party of demonising Geert Wilders (of the PVV party). 14 This article’s engagement is responsible for about 36% of that week’s ‘junk’ engagement, providing a major boost.

More generally the relatively high engagement attained by junk sources can in many cases be attributed to a small number of high-performing articles. This matches the findings from the analysis of the provincial elections, where peaks in junk news engagement could similarly be attributed to a smaller number of well-scoring articles. While junk sources perform relatively well, especially in the earlier weeks of the data set, this success is thus attributable to a relatively small number of sources and articles rather than a broadly successful and diversified ecosystem or even a coordinated campaign.

Though the findings do not approximate those in the BuzzFeed News story, in the case of the EU election campaign it is noteworthy that indeed junk news does on one occasion match the performance of mainstream news, though not during the tail end of the campaign period as was the case in the BuzzFeed News data. Overall, junk news is roughly as successful during the EU campaign as it was during the provincial election campaign, and has a significant presence, though over the whole campaign mainstream news still easily outperforms it.

14 “Video! Kuzu (DENK) wil dat Wilders gestopt wordt, voordat hij een tweede Srebrenica-bloedbad kan aanrichten”: https://www.dagelijksestandaard.nl/2019/05/video-kuzu-denk-wil-dat-wilders-gestopt-wordt-voordathij-een-tweede-srebrenica-bloedbad-kan-aanrichten/

  53 Figure 5. Per-query engagement of mainstream (blue) and junk (pink) articles found through EU parliamentary elections-related BuzzSumo queries, per week, between 18 February and 25 March 2019. Engagement scores have been normalised.

Characterising sources

It is useful here to briefly discuss the sites that make up both categories of content. Our category of mainstream outlets (see Table 1) consists of well-known outlets with a national reach, which in practice translates to a number of national newspapers, public broadcasting organisations, national TV programmes and large online magazines. The junk category is comparatively more

  54 diverse; the typology we use covers conspiracy sites, hyperpartisan online sources (including independent self-styled journalists), and clickbait aggregators. Some of these are relatively large:

De Dagelijkse Standaard, a far-right weblog, appears in the top three of most engaged-with articles for 15 of our 19 queries. Some other junk sites appear to be more focused on a particular topic; this is especially apparent in the results for the provincial elections [Migratie] (migration) query, in which fenixx.org – a far-right site advancing the racist ‘race replacement’ theory – appears often, while it is far less prominent for the other queries, save for the [EU], in which it also appears occasionally. This site was also noted by the earlier 2017 NRC study as being especially prevalent in their ‘hyperpartisan’ category.

This ‘hyperpartisan’ category can then be seen to be comprised of roughly the same set of sites in both data sets (see Table 1 and 2). This could be considered to suggest a hyperpartisan news ecosystem of sites that enjoy a significant and stable readership. On the other hand, this ecosystem is notably top-heavy; for both data sets De Dagelijkse Standaard (DDS) is by far the most engaged-with site, almost four times as popular as the next site in the list. Following DDS is a number of far smaller but simultaneously more outspokenly far right blogs such as Stop de

Bankiers, Fenixx and JD Report. Fenixx here is further notable as a site that was also mentioned as a relatively prominent junk site in the 2017 NRC study. While we can thus identify a stable sphere of hyperpartisan news sites that drive significant engagement, the success of this sphere is still mostly reliant on De Dagelijkse Standaard, and with the exception of that site is quite marginal compared to the mainstream sphere.

Mainstream Junk news

Site Engagement Site Enagagement

telegraaf.nl 102117 dagelijksestandaard.nl 98414

nu.nl 46962 stopdebankiers.com 26429

rtlnieuws.nl 46849 fenixx.org 13024

wnl.tv 39975 jdreport.com 8564

nos.nl 37319 ninefornews.nl 5975

nrc.nl 16010 tpook.nl 4431

metronieuws.nl 14746 ejbron.wordpress.com 4126

pauw.bnnvara.nl 10130 opiniez.com 2777

evajinek.kro-ncrv.nl 7412 dlmplus.nl 2110

Table 1. Top 10 sites per category (Provincial elections), for all queries combined, sorted by overall engagement scores as reported by BuzzSumo.

  55 Mainstream Junk news

Site Engagement Site Enagagement

telegraaf.nl 232327 dagelijksestandaard.nl 225006

nu.nl 192962 stopdebankiers.com 46892

nos.nl 141440 fenixx.org 25852

rtlnieuws.nl 99820 tpook.nl 17453

wnl.tv 91211 jdreport.com 9199

elsevierweekblad.nl 31150 opiniez.com 8302

metronieuws.nl 28038 ejbron.wordpress.com 6427

nrc.nl 27195 reactnieuws.net 5565

joop.bnnvara.nl 22509 ninefornews.nl 2047

Table 2. Top 10 sites per category (EU parliamentary elections), for all queries combined, sorted by overall engagement scores as reported by BuzzSumo.

As discussed above, an alternative to the binary mainstream/junk opposition one may consider the data for both election campaigns in terms of a more detailed five-category perspective (see

Figures 5 and 6). For both the provincial and EU elections it is apparent that the largest nonmainstream category by far consists of hyperpartisan sources. The only other category that has a noteworthy impact are tendentious sources GeenStijl and The Post Online (which are both not included in the other, binary, categorisation in figures 1 and 4). Conspiracy and clickbait sources are present in the data but do not play a significant role compared to the other categories.

Figure 6. Engagement of mainstream, hyperpartisan, conspiracy and clickbait articles found for provincial elections-related queries on BuzzSumo, between 18 February 2019 and 25 March 2019. Engagement scores have been normalised. Geenstijl is considered ‘mainstream’ here, while The Post Online is classified as

‘hyperpartisan’.

  56 Figure 7. Engagement of mainstream, tendentious, hyperpartisan, conspiracy and clickbait articles found for provincial elections-related queries on BuzzSumo, between 18 February 2019 and 25 March 2019. Engagement scores have been normalised. Geenstijl and The Post Online are considered ‘tendentious’ here.

Figure 8. Engagement of mainstream, tendentious, hyperpartisan, conspiracy and clickbait articles found for EU parliamentary elections-related queries on BuzzSumo, between 19 April 2019 and 23 May 2019. Engagement scores have been normalised. Geenstijl is considered ‘mainstream’ here while The Post Online is classified as

‘hyperpartisan’.

  57 Figure 9. Engagement of mainstream, tendentious, hyperpartisan, conspiracy and clickbait articles found for EU parliamentary elections-related queries on BuzzSumo, between 19 April 2019 and 23 May 2019. Engagement scores have been normalised. Geenstijl and The Post Online are considered ‘tendentious’ here.

Dutch provincial elections EU Parliamentary elections Site Engagement Site Engagement dagelijksestandaard.nl 168668 dagelijksestandaard.nl 225006 stopdebankiers.com 35414 stopdebankiers.com 46892 fenixx.org 20757 fenixx.org 25852 jdreport.com 15679 jdreport.com 9199 ejbron.wordpress.com 5285 opiniez.com 8302 dailypaper.org 4887 ejbron.wordpress.com 6427 opiniez.com 4554 reactnieuws.net 5565 destaatvanhet-klimaat.nl 3912 xandernieuws.net 2009 pallieterke.net 3228 eunmask.wordpress.com 1296 eunmask.wordpress.com 2487 novini.nl 862

Table 3. Top 10 ‘hyperpartisan’ sites for both data sets, sorted by overall engagement scores as reported by

BuzzSumo.

An examination of the most engaged-with sites in the hyperpartisan category (see Table 3) further confirms that this category is the most influential one in the broader ‘junk’ (or ‘junk-like’) sphere, with the top ten sites mostly matching those found in the top 10 of ‘junk’ sites identified in tables 1 and 2. The top five is identical between all lists, and again De Dagelijkse Standaard is the most important site. Notably, as the campaign draws on, mainstream engagement can be seen to increase while junk news performance is relatively stable, meaning interest in mainstream news coverage increases towards the end of a political campaign, while junk news remains stable. Perhaps they serve different publics, though such a construal would require further work.

  58 Generally, the junk news sites, of which hyperpartisan sites are the largest constituent, can be characterised as right-wing, anti-immigrant, anti-EU and in some cases anti-Semitic or advancing conspiracy theories (the latter especially applying to ninefornews.nl and jdreport.com). This right-wing slant in our findings is consistent with other studies on junk news, including the 2016 BuzzFeed News analysis but also others that found that left-wing content was less prominent in that category (Silverman, 2016; Neudert et al., 2017:1; Alcott and Gentzkow, 2017:223). In this case study, next to the prevalence of hyperpartisan sites such as DDS the relatively large engagement of especially conspiracy sites is notable; ninefornews.nl, which is the 5 th -most engaged with site in our data, regularly promotes conspiracy theories ranging from UFO sightings to far-right theories such as Pizzagate and QAnon. The authors seem to be convinced that this is accurate accounting of events. Overall, the data show that junk news, consisting primarily of hyperpartisan and conspiracy theory sites, are a minor but constant and significant factor.

A cross-platform appraisal

This case study focuses on Facebook, but a similar analysis may be performed for other platforms. While Facebook has the dubious honour of being the platform with perhaps the strongest association with fake news, other platforms have their own affordances that could make them attractive for those seeking to spread forms of junk content. Just as this case study builds on the analyses of BuzzFeed News’ and the NRC’s, with a number of methodological tweaks, one could similarly move to other platforms as well, studying over-time engagement of junk and mainstream content respectively. Multiple case studies in this volume employ a method of this type.

The multiple platform analyses present an opportunity not only to investigate the prevalence of junk news on individual platforms, but also to perform a cross-platform analysis in order to investigate whether there are platforms that are particularly susceptible to junk content, or whether some platforms have perhaps succeeded in combating the spread of it, given that the phenomenon has been addressed for some time now, and the case studies in question take place in early to mid 2019. While we present such a comparison in this section, it should be noted that a direct comparison between platforms is complicated for a number of reasons.

One issue with a comparison between various platforms is that ‘engagement’ means different things depending on the features a platform offers for interacting with content. On Facebook, engagement means the sum of comments, likes (or reactions) and shares a post received. But

Reddit, for example, has no direct counterpart to some of those, as ‘shares’ are not a relevant concept on that platform. It simultaneously offers metrics Facebook does not use (including upvotes and downvotes). Moreover, on Facebook a dislike or angry reaction, for example, often would be counted as a plus engagement, whereas a downvote on Reddit reduces a post’s score.

More specifically, the case studies in this collection use different time periods and in some cases investigate, apart from election issues and leaders, certain polarised topics (such as MH17 and

Zwarte Piet) so as to seek disinformation or junk, as we discussed above in terms of asymmetrical querying. Such query design may be justified, given that previous studies of disinformation in the Dutch media context were borne of data curated by Twitter that consisted

  59 of Russian IRA trolls, and found activity around the downing of the MH17 airliner in 2014 as well as the terrorist attacks in the Brussels airport and metro in 2016. When examining on

Twitter the MH17 hashtag and keyword usage over the past number of years, one may find increased activity around elections (such as during the national elections of 2017), thus further justifying a renewed attention towards at least MH17 during the 2019 elections. Such asymmetrical querying of course complicates comparisons, as the ratio between mainstream and junk news engagement may be less balanced, given conspiracy and other sources’ continual attention to such themes. Differences in time periods also pose issues, as there may be particularly ‘junk-sensitive’ episodes from the past that are missing from the current analyses, and for analytical purposes have been removed from the comparison.

Nevertheless, provided one is aware of the limitations in such a comparison, the results of such an analysis for other platforms compared to the Facebook case study can provide an impression of the relative penetration of junk across different social media platforms. While in the rest of this volume there are separate case studies that investigate the individual platforms with methods similar to this one, the graph below presents a rough impression of the results across platforms, using data from this chapter and the other case studies.

Figure 10: Relative engagement of content categories across 4chan /pol/, Reddit, Twitter and Facebook. Geenstijl is considered ‘mainstream’ here while The Post Online is classified as ‘hyperpartisan’. 4chan and reddit data from 1 Dec 2015 until 1 June; Twitter and Facebook data from 18 Feb 2019 – 25 Mar 2019 and 19 Apr 2019 - 23 May 2019.

  60 Figure 11: Relative engagement of content categories across 4chan /pol/, Reddit, Twitter and Facebook. 4chan and reddit data from 1 Dec 2015 until 1 June; Twitter and Facebook data from 18 Feb 2019 – 25 Mar 2019 and 19 Apr 2019 - 23 May 2019.

What is striking in the cross-platform comparison of results in Figure 7 is that the two

‘mainstream’ social platforms, Facebook and Twitter, show a higher prevalence of junk content than 4chan and Reddit, the deep vernacular web platforms. This is interesting because the latter two – the “seedy underbelly” of the internet (Bergstrom 2011) – are often characterised as hotbeds of polarizing and alt-right political discussion, thus providing an environment where one could expect particularly hyperpartisan content to thrive.

One plausible explanation of this is that especially on 4chan’s /pol/, the ‘politically incorrect’ sub-forum that was investigated in this case, those posting may position themselves in opposition to mainstream sources. This positioning often goes hand in hand with linking to the sources in question, thus increasing the share of mainstream content in the overall picture for the platform. As such it underlines the notion that engagement does not necessarily indicate that one agrees with the engaged-with content, and in fact high engagement may be taken to indicate controversiality, as something polarizing that is hotly debated can be expected to be clicked on and scrutinised by many of those posting about it.

Conclusions: Absence of disinformation and junk news prevalence

As discussed, a detailed cross-platform comparison is complicated by the different methods used in each case study. While outside the scope of this research, further commensuration of these methods and results for a more thorough cross-platform analysis presents an opportunity for further research.

This particular case study, focused on Facebook, is informed by similar investigative (data) journalism originating with BuzzFeed News and the NRC Handelsblad. Our results are not strictly in keeping with theirs. It is of particular interest that the prevalence of intentionally false news

BuzzFeed found was not apparent in our data, indicating that this is far less of a problem in the

Dutch sphere than in the U.S. The data do seem to confirm the reputation of Facebook as an especially fertile ground for junk news in comparison to other platforms and indicates that despite its initiatives to combat such content, it is still endemic on the platform. In fact, whereas

  61 the NRC found that “at most 10%” 15 (Kist and Zantingh, 2017) of the engagement they analysed concerned hyperpartisan and tendentious content, in our analysis a little over a year later we find this share has risen to 25%.

While this difference between our findings and the NRC’s could partially be explained by the differences in the criteria used to categorise the content, it seems justified to conclude that even if junk news is in the minority, it is certainly not marginal, and seems to be a growing product in the Dutch media landscape, on some occasions matching the performance of mainstream news in terms of Facebook engagement. Though this case study is limited to the 2019 provincial and

EU elections, its findings suggest that a broader analysis of junk coverage of Dutch politics on Facebook is warranted. Such an analysis could also investigate what ‘engagement’ means in practice; as indicated by the cursory cross-platform analysis, engagement may not translate to agreement, and if junk news is such a factor on Facebook it is important to understand the motivations behind engaging with it if we are to understand the significance of it in the wider political debate.

A silver lining (so to speak) is that there was virtually no outright (foreign) disinformation in the data we found, and indeed across all platforms we investigated. While especially on Facebook there is a solid undercurrent of junk sites including hyperpartisan content, and a number of wellshared conspiracy sites which give promote highly dubious content, there is no imminent reason to expect so-called fake news affecting Dutch election coverage in the same way it appeared to for the 2016 U.S. elections. Overall, our Facebook case study indicates that there is no immediate cause for concern about disinformation about Dutch elections, but that junk news is a growing factor that warrants closer scrutiny.

15 Transl. from Dutch by authors of “hoogstens 10 procent” (Kist & Zantingh 2017)

62

JUNK NEWS IN SEARCH ENGINES:

EXPLORING GOOGLE’S SENSIBILITY

TOWARDS HYPERPARTISAN SOURCES

DURING THE DUTCH ELECTIONS

Guillén Torres and Richard Rogers

Junk news in search engines: Exploring Google’s sensibility towards hyperpartisan sources during the Dutch elections

Guillén Torres and Richard Rogers 16

Introduction: Search engines as junk source space

As key entry points to the web, search engines serve as a site for the consumption of information, including political information, and as such are a relevant space for the study of both the presence of disinformation and junk news as well as approaches to combat it (Bowden, 2016). Although they are described in the industry as ‘organic’, the output of search engines could be termed manufactured hierarchy (Hindman, 2008 Halavais, 2017). Sources are ranked per query, and as such certain ones are offered as more relevant than others, as if naturally. Such ranking practices are often considered inscrutable, since search engines generally do not provide a means to save and study query results, e.g., through offering an API that enables it or terms of service that allow it. In fact, the ranking algorithms are trade secrets jealously guarded by corporations.

Since 2009, Google and subsequently other search engines have personalized results, be it for the individual or the place where the search has taken place (Pariser, 2011; Puschmann, 2018).

Increasingly engines are thus both providing ranked political information but also tailoring it to user preferences and/or location (Martens et al., 2018).

When it comes to events, such as elections, search engines become providers not only of topical but also of timely information. These are particularly poignant moments to study the presence of disinformation and junk news. As cases in point, there have been occasions when sources that were otherwise insignificantly ranked rose to the top of engine returns during the ‘breaking news’ period of an event, such as in the immediate aftermath of the Las Vegas shootings in 2017, when a 4chan post misidentifying the shooter rose to the top of the results (Robertson, 2017). Google also prominently linked to rumours about the identity of a Texas shooter in 2018 as “a Muslim convert, member of Antifa or Democrat supporter” (Lomas, 2018). One could point to artificial manipulation, such as search engine optimization, as triggering the unexpected rankings and sudden presence of dubious information. With respect to the 4chan post, the gaming of the engine (if that were the cause) also may have been maliciously playful, introducing misinformation as an act of trolling. In the case of the rumours about the Texas shooter, the manipulation appears to have been hyperpartisan. Both spates of false news were not ‘corrected’ in the editorial sense of an erratum notice; rather, the dynamically published results are continually algorithmically tweaked so “good information” is said to ultimately prevail (Waters,

2017).

In the study of engine returns and hierarchies (through manual capture techniques) it is often pointed out that top placement matters, since engine users over the years have been browsing fewer and fewer result pages (Jansen & Spink, 2003; Dan & Davison, 2016). Thus, in the above

16 The research reported here was undertaken in collaboration with Anja Duricic, Lisa Fluttert, James Ingleby and Ziwen Tang.

  63 examples, the significance of the location of misinformation, rumour and extreme results relies on findings about how users gravitate to the top results, making them the most consumed and thereby particularly worthy of study. An additional research strategy for inquiries into fake and junk news presence concerns anticipatory search, also known as autosuggestion, which drives the user to particular search terms. It also has been studied for the offensive associations made by engines, such as the completion of “are Jews” with “evil” (Cadwalladr, 2016). Misogynistic autosuggestions also were documented in earlier cases which led to a UN campaign in 2013 discussed in a longer study of “how search engines reinforce racism” (Noble, 2018). Other extremist content has been similarly documented towards the top of Google results for the query “holocaust” (Hern, 2017).

The discovery by The Guardian journalist of the offensive associations with the word “Jews” and the resulting sites that surfaced (such as the neo-Nazi website, The Daily Stormer) have led to discussions of not just how Google’s algorithms could be tweaked, but also the reach of the sources producing and driving such information in the first place. Their presence (and top placement) could be interpreted as a proxy for the significance and audience of such material online, or as others have argued as evidence of a culture war, driven by the online boosting tactics of ‘culture hackers’ (Albright, 2016; Confessore and Wakabayashi, 2017). Whilst they may seek to correct the autosuggestions (and perhaps remove religious ones all together), companies such as Google are hesitant to delist such extreme websites, given free speech concerns, which also may arise if they are nudged downwards.

The present chapter studies the susceptibility of Google’s search engine to provide users with questionable information sources in the results for queries related to Dutch political parties during the Dutch provincial and European parliamentary elections of 2019. Our goal is to identify the presence of junk sources in the results for political queries. Thus, the research questions read as follows. Which rankings has the search engine assigned to fake and junk sources when querying political parties and their issues? Are there particular political issues and party spaces where fake and junk news are prevalent or entirely absent? Do the ranks and amounts increase as the elections draw near?

We divided the chapter into six sections: a brief reflection about the methodological challenges of studying search engines, the methodology for building our dataset, three sets of findings, and a discussion of the limitations and further steps. In all it was found that fake and junk news, specifically of the hyperpartisan variety, is rather pervasive in the search-demarcated political space, but far more so for certain actors and their issues on the far right of the political spectrum.

Studying personalization, junk news, or both?

The extent to which autosuggestions are personalized is understudied, but the personalization of results more generally has been the subject of numerous inquiries and methodological innovations that work around Google’s inscrutability through selectively scraping results or soliciting data donations. The findings from scraping batch-queried results have shown relatively low amounts of results affected by personalization in the Google search engine (Feuz et al.,

2011), and the same held for Google News (Haim et al., 2018), suggesting that original concerns

  64 regarding the ‘filter bubble’ may not be as well founded any longer. Where the second method is concerned, Algorithm Watch, the German NGO, created a browser extension for users to install that regularly would make political queries, such as for “Angela Merkel” (Puschmann, 2017). The results would be donated by the users to Algorithm Watch so as to enable a larger number of those under study than is normally the case. Here again the findings have shown low levels of personalization, but the study of the presence of certain junk sites (be they disinformation or another genre) could be pursued further. Another technique, discussed below, is to selectively scrape results in a manner that seeks to minimize personalization effects, thereby concentrating on the overall presence of junk sites rather than on whether particular users, in a filter bubble, are seeing more of them.

To begin to understand the amount and placement of junk news in search engine space, be it around events or even after an algorithmic tweak, a query routine is designed, and a window of activity is chosen. (Longer-term studies also may be undertaken, as in the Issuedramaturg project that followed 9/11 query results for years, but Google often changes its output formats, breaking automated tools (Rogers, 2013).) In order to reduce personalization, a research browser may be deployed, which is a clean instance of a browser with the user not logged in. City-level geographical personalization may be avoided through the use of advanced settings, choosing a particular region, such as the Netherlands. For projects as ours, geographical personalization is not viewed as a disadvantage in the sense of creating the conditions for a filter bubble to materialise.

A brief mention should be made of the search engine under study. Among them Google is the most popular, with the largest market share of users in most countries, certainly in the

Netherlands. As mentioned above, recently, the company has become entangled in the fake news debate through the appearance not of Russian disinformation sources (though that to our knowledge has not been studied in great detail), but owing first to the appearance of misogynistic and extremist content that the company previously defended as ‘reflective’ of societal concern rather than the product of algorithmic error or ‘culture hacking’. If one were to expand the number of search engines under study (to include Bing and Yahoo!, for example), one could triangulate results, and inquire further into the normalcy and regularity of misogynistic and extremist content present in the top results, though one could not control for algorithmic concentration or the extent to which the big engines’ algorithms are anyways similar. The extent to which the results reflect societal concern would remain an open question.

Studying Google results

The presence of Junk news within Google search engine’s result is a multi-causal phenomenon that may be credited to a number of factors. Among others, Google’s algorithm reacts and learns from users’ own consumption of junk sources. It is trained using varied datasets, and content producers’ attempt to game the search engine via Search Engine Optimization tactics (Finkel et.

Al, 2017). Given the inscrutability of Google’s tool, it is difficult to determine what could be causing the presence of junk sources in the Dutch websphere or others. Here, rather than attempting an explanation for the presence of junk news, we conduct a test of the tool’s sensibility to connect politically relevant queries with junk sources.

  65 As noted, the investigation relies on scraping as a method (Marres & Weltevrede, 2013), and takes as its point of departure the question of junk news in search returns rather than the effect of personalization in the creation of a filter bubble. The research seeks so-called junk news in search engine results, which has been defined as “extremist, sensationalist, conspiratorial and masked commentary” (Howard et al., 2017: 1). In keeping with Buzzfeed News’ definition of fake news (Silverman, 2016), we also seek (foreign) disinformation, hyperpartisan sources as well as clickbait, which itself may be extreme. In order to do so we rely on a list of sources expertly curated by other researchers in the project.

Generally, the research employs the ‘source distance’ approach, inquiring into how far from the top of the returns are the offending results (Rogers, 2013). More specifically, we investigate how fake and junk webpages are positioned in the first twenty Google.nl results of various queries of political parties and their most significant issues during the 2019 provincial elections campaign as well as that of the European parliamentary elections. Thus, this case examines the susceptibility of search engine results to fake and junk news, as defined above, rather than exploring the issue of fakeness and junk in themselves or the effectiveness of counter-measures.

Engine returns as political spaces

In order to demarcate a political space in search engine results, we designed a query protocol based on combining the names of political parties with specific issues associated with their respective political agendas. In that sense, the underlying assumption is that junk news may have a more significant impact when discussed in a specific political context, such as election campaigns, when voters gather information to guide their choice. To be able to collect the results of a large number of queries, we used the Search Engine Scraper by the Digital Methods

Initiative, a tool that allows one to scrape search engine results for a given query and commit them to a database for further scrutiny through visualization.

The first step of the methodology consisted in constructing a list of Dutch parliamentary parties and locating their websites as well as Facebook pages (see table 1). Another list was created pertaining to the parties participating in the European parliamentary elections. In the next step we identified the relevant keywords to build the queries; in the case of the Dutch provincial elections, these were sourced from both the party webpages and the party Facebook pages. For the parties competing in the European parliamentary elections, only the parties’ own websites were consulted. The party webpages provided the parties’ issue keywords. The Facebook pages furnished a more vernacular set of issue terms, as they contain issue keywords from users or citizens in the comment space. The aim of sourcing these two sets of keywords was to enable us to capture and compare the results for both official as well as more popular issue language.

The lists of party keywords were built by collecting the platform standpoints (standpunten) on the party websites. There is one list for the provincial elections and another for the European parliamentary elections. Most political parties mention between five and ten keywords on their platforms, and all were collected. A few parties (e.g., the Staatkundig Gereformeerde Partij, SGP) offered longer lists which were shortened on the basis of their key issues. In all, the political party issue space consisted of 158 keywords across the thirteen parties for the provincial elections, and four keywords across fourteen parties for the European parliamentary elections.

  66 The vernacular list was made through a close reading of the comments made under the posts of the Facebook pages of each political party. To build this list, the most commented posts around the days of the elections were close-read, and the most representative keywords related to the views expressed by the commenters were chosen. Identifying the most relevant issues in the comment space on Facebook proved to be problematic, given that the comments were often polarizing and emotive, especially those relating to the elections themselves. This citizenenriched political issue space consisted of five keywords for each of the thirteen parties, making a total of 65.

Dutch Provincial Elections European Parliamentary Elections Name of the Party Short Name Name of the Party Short Name Volkspartij voor Vrijheid en Volkspartij voor Vrijheid en

Democratie VVD Democratie VVD

Partij voor de Vrijheid PVV Partij voor de Vrijheid PVV Christen-Democratisch Christen-Democratisch Appèl

Appèl CDA - Europese Volkspartij CDA-EV

Democraten 66 D66 Democraten 66 D66 GroenLinks GL GroenLinks GL

Socialistische Partij SP ChristenUnie - Staatkundig Gereformeerde Partij CU - SGP

Partij van de Arbeid PvdA Partij voor de Dieren PvdD ChristenUnie CU 50Plus 50plus Partij voor de Dieren PvdD Jezus Leeft 50Plus 50plus Denk DENK Staatkundig Gereformeerde

Partij SGP Forum voor Democratie FvD

Denk DENK Van de Regio & Piratenpartij VR - PP Forum voor Democratie FvD Volt Nederland VN De Groenen GN

Table 1. List of Dutch political parties under study

The three lists of keywords were inputted in the Search Engine Scraper along with the name of each party. The results from the parties’ own websites were excluded. For example, for the political party D66 and the keyword onderwijs (education), the following query was made:

[onderwijs d66 -d66.nl]. Using the advanced search features of Google, maximum results were set to 1,000, and each day of our periods of interest (13-22 March and 22-24 June 2019) was queried separately. The date range included the run up to the provincial elections on 20 March and a short election aftermath period, and the days before and after the European Parliamentary elections on May 23. The searches were conducted in a clean browser, in the Dutch Google.nl domain, in the Dutch language, and in the Netherlands region (through the advanced setting).

The keyword and party were queried together so that the scraper tool delivered results that are

  67 related to election politics, rather than a general overview by querying each keyword in isolation. The keywords derived from Facebook were queried in the same format, using the same settings and date range.

The outputs of the Scraper tool are the top twenty ranked Google.nl results per query. The

URLs in our lists were then truncated to their host names so that they could be cross-checked with the list of known fake and junk websites curated by other researchers in the project. Here a formula was used that effectively linked the search engine results spreadsheet to that of the expertly curated list of fake and junk sites. The question for each source concerned its ranking per query and its presence or absence in the expert list. All query results (per party and per official or vernacular language type) were marked as fake and junk sites or not and listed in the order they were returned.

Party platforms

Foreign affairs Europese Unie, Europa, EU, Nederland en Europa, buitenland, internationale zaken.

Polarizing

topics Islamisering, Islam

Health zorg, menselijke zorg

Environment klimaat, natuur, milieu, dierenrechten, natuur en milieu

Finances belasting, economie, inkomen, pensioen, werk en inkomen, schone economie, eerlijk delen, werklozen, economisch beleid, overheid en bestuur

Safety and

security veiligheid, privacy, defensie, criminaliteit, rechtsstaat, terrorisme

Society waarden, gezin, respect, familie, samenleving, burgers, democratie, ouderen, onderwijs, goed onderwijs voor iedereen, vrijheid, verantwoordelijkheid, drugs

Future innovatie, duurzaamheid, schone energie, energie

Migration immigranten, migratie

Facebook

Foreign affairs Europa, EU, referendum

Polarizing

topics Islam, Moslim, racisme, discriminatie

17

Environment milieu, klimaat, kernenergie, energie

Finances belasting, bezuinigingen, pensioen, onderwijs

Society samenleving, democratie, toekomst crisis, vrouwen, vrijheid, Nederland, armoede

Faith Islam, Moslim, Christendom, Christenen, geloof

Migration migratie, immigranten, gelukszoekers, migranten

Table 2. List of categories and political keywords used in the study.

17 Islam is placed in both faith as well as polarizing topics categories, given how it is discussed as shorthand for a social issue.

68 We zoomed in on those queries in which fake and junk news showed up consistently, that is, for a minimum of four days within our period of interest for the provincial elections, and two days for the European parliamentary elections. The keywords that produced fake and junk news websites in their search results in the first case were then grouped thematically in the following categories: ‘foreign affairs’, ‘polarizing topics’, ‘health’, ‘environment’, ‘economy’, ‘safety and security’, ‘society’, ‘future’, ‘migration’ and ‘faith’, the most salient of which are described in some detail below (see table 2). For the case of the European parliamentary elections, the four keywords common to all parties were queried: Europese Unie (“European Union”), klimaat

(“climate”), migratie (“migration”) and economie (“economy”).

Political parties and issue keywords

Before analyzing the presence and positioning of junk news in Google search engine, we would like to discuss briefly the keywords obtained for the provincial election campaign from the official websites and Facebook pages of the political parties. Comparing the composition of the categories that emerged from each of the two political spaces allows for showing differences between the matters of concern as expressed by political parties and citizens or social media users. Whereas political parties included keywords that could be grouped under the categories,

‘future’, ‘security’, and ‘health’, that was not the case for the Facebook users. In contrast, ‘faith’ was present in the Facebook comment space, whereas it was largely absent from the party platforms (except for the SGP, with its long list). 18 There are also matters of concern common to citizens and political parties alike, such as ‘foreign affairs’, ‘economy’, ‘society’, and

‘environment’.

Within the shared concerns there are still differences between the way each political space is constructed by political parties or citizens. For example, within the ‘foreign affairs’, ‘economy’ and ‘society’ categories, parties tend to refer to a wider variety of issues in comparison to the concerns expressed by citizens, which are mostly focused on the European Union and the referendum. In the economy cluster, political parties address ten issues, whereas citizens are concerned with far fewer. The same holds roughly for the society cluster. Interestingly, this trend reverses in the environment cluster, where users tend to express concerns about nuclear energy, while the topic is not present in parties’ platforms (again, with the exception of SGP). Lastly, even though the usage of some keywords is more or less the same in party platforms and the

Facebook comment spaces, for others it differed, as is the case for onderwijs (education). By the political parties it is framed as a societal issue, whereas in the discourse on Facebook it is discussed in terms of citizens’ ability to afford it.

Visualizations were made to facilitate the analysis; they show at a glance the presence and ranked position of fake and junk news for each query over time. The columns represent the days in the timeframe studied (13-22 March or 22-22 May 2019). Red cells indicate the source as marked as fake or junk news. One also may read the distribution of it over time.

18 Islam was present in the party platforms (largely the PVV and SGP) but discussed in terms of a social issue.

  69 Figure 1. Presence of fake and junk news in Google.nl search engine results for political queries related to foreign affairs, 13-22 March 2019.

Political party standpoint space

The study of the political party standpoint space found overall that all fake and junk webpages that appeared can be subcategorized as hyperpartisan. The one exception fell within the environment cluster in the form of a conspiracy website in the top twenty. Three specific websites make up for the biggest amount of junk: De Dagelijkse Standaard, Stop de Bankiers, and

Opiniez. A second general observation is that queries related to right-wing parties returned hyperpartisan sources in a greater proportion than queries related to parties with other political orientations. In particular, the queries related to the FvD were the most populated by hyperpartisan sources. Thirdly, queries related to parties located at the center of the political spectrum seem to produce results with less questionable sources. In those instances where hyperpartisan websites appear among the top twenty results for center-oriented parties, the sites mainly do not make it to the top positions (though there were exceptions).

In the political party standpoint space, most junk appears to be associated with queries related to keywords within ‘foreign affairs’ and ‘polarizing topics’. The keywords within the foreign affairs cluster mainly relate to the European Union and the Dutch relationship with it. Issues related to political parties from the center of the political spectrum seem to be least connected to junk, as may be noted for 50Plus, Partij voor de Dieren or D66 (see figure 1). Contrariwise, parties that position themselves strongly against the European Union are linked to a high concentration of junk results. For example, when looking at FvD and PVV, we can see that hyperpartisan sites account for 37% and 47% respectively of all the returns discussing the European Union.

Moreover, the hyperpartisan sources are among the top five results throughout almost the entire time period.

  70 Figure 2. Presence of fake and junk news in Google.nl search engine results for political queries related to polarizing topics, 13-22 March 2019.

Regarding the keywords within the ‘polarizing topics’ cluster (figure 2), two related issues in the political party-demarcated space lead to search engine results with a large amount of hyperpartisan sources: Islam and Islamisering (Islamization). These keywords are only discussed by two right-wing parties, PVV and SGP, the latter from the religious right. In particular, when

PVV is queried together with the keyword Islamisering (Islamization), hyperpartisan websites appear at the top of the results throughout the entire time span, occupying even the highest positions. This changes only for three days (16, 19 and 20 March), when, however, the amount of junk increases overall.

The queries for environmental keywords (figure 3) also lead to significant quantities of junk.

First, it is of note that the amount decreased as the elections neared, though junk websites still maintained a prominent position among the first 10 results. Once again, queries mentioning right-wing parties such as FvD are more prone to produce junk sources within their results. The site, ninefornews.nl, was identified here as the only conspiracy website in the entire data set. It emerged in the 8th position when [milieu FvD -www.fvd.nl] was queried, meaning that it appears on the first page of Google results (with default settings at 10). The center party, D66, also registered signficant amounts of junk for the query climate (klimaat), as did the center-left labour party (PvdA).

The cluster related to economic issues, the second largest in our set of results, contains nine keywords such as belasting (taxes), eerlijkdelen (fair sharing) and werklozen (unemployed) (see figure

4). Queries including FvD are once again among the most prone to produce hyperpartisan results, particularly with the keywords economie (economy), pensioen (pension) and belasting (taxes).

When examining the remaining parties (with the possible exception of the PVV), the number of junk results is relatively low overall and rarely occupy the first position of the search engine results.

The ‘society’ cluster is the largest; twenty issues were identified in this cluster, ranging from concerns about warden (values) to drugs (see figure 5). Social issues produced a varied distribution of hyperpartisan sources overtime, with hyperpartisan websites appearing less prominently. Only three queries, two including the FvD and one the VVD, stand out in this cluster as junk-ridden: drugs, democracy and responsibility (verantwoordelijkheid).

  71 Figure 3. Presence of fake and junk news in Google.nl search engine results for political queries related to the enviornment, 13-22 March 2019.

Figure 4. Presence of fake and junk news in Google.nl search engine results for political queries related to the economy, 13-22 March 2019.

  72 Figure 5. Presence of fake and junk news in Google.nl search engine results for political queries related to societal issues, 13-22 March 2019.

Figure 6. Presence of fake and junk news in Google.nl search engine results for political queries related to future innovation, 13-22 March 2019.

  73 Lastly, the thematic cluster, ‘future developments’ (figure 6), with such issues as duurzaamheid

(sustainability), innovatie (innovation), and schone energie (clean energy), are addressed by parties located at the extremes of the political spectrum and the query results are populated by hyperpartisan sources. This is most evident in the case of FvD, where questionable sources appear nearly every day, and in four instances are returned in the top two positions, albeit not in close vicinity of election day, which remains largely unaffected by junk sources in relation to these terms.

Vernacular political issue space

Based on the keywords gleaned from the Facebook pages of the Dutch political parties, one new category was created (‘faith’) on top of the other six from the previous exercise. Generally, the results were similar. Queries mentioning right-wing parties such as PVV and FvD returned more hyperpartisan sources in the top 20 Google results, and the positions of these junk sources tend to be higher than in queries related to other parties.

In the vernacular political issue space, the foreign affairs cluster received the most junk news returns: nearly 25% of the top 20 results are considered hyperpartisan websites. Migration and polarizing topics have the second and third highest percentage of junk websites, with 19% and

17% of junk, respectively. Results associated with keywords such as immigranten (immigrants) and Islam were significantly populated with hyperpartisan sources. Issues related to environment, society, and finance obtained fewer junk returns (less than 10%). The faith cluster had the fewest hyperpartisan returns. 19

The environment cluster (which for many terms could have been merged with the future innovations one) is the largest one in the vernacular issue space. In it the keyword klimaat

(climate) was discussed by Facebook users on the pages of five different parties, and milieu

(environment) on three. For Groenlinks, D66, PvdA and VVD, the keyword klimaat features

(right-wing) hyperpartisan sources in the top results (see figure 7). From the election day onwards, fewer junk websites showed up in the top 20 results. The same pattern was observed in the ‘foreign affairs’ cluster (see figure 8). Hyperpartisan websites occupied the first position for five days during our 10-day research period for queries related to FvD and PVV. Queries for

FvD and EU were the most likely to return fake and junk webpages.

Regarding the keywords grouped under the ‘polarizing topics’ cluster, Islam was brought up by

Facebook users in the pages of four political parties: PVV, SGP, FvD and DENK. 20 Among them, results associated with FvD received the most junk webpage returns in the top 20 results, and the hyperpartisan results maintained the first position for seven days (figure 9). On election day, however, all the parties returned few to no such results in the cluster of ‘polarizing topics’, with the exception of the FvD. The number of junk sources and the rankings dropped dramatically on that day. Concerning DENK, although queries including it did not return many

19 Here again Islam is excluded, because it is considered a social issue, given the manner in which it is discussed in the vernacular issue space.

20 For the political party, DENK, Islam, while discussed as a social issue, also could be categorized as ‘faith’.

  74 junk websites overall, they were found in relation to the issues moslim (muslim), discriminatie

(discrimination) and racisme (racism).

Figure 7. Presence of fake and junk news in Google.nl search engine results for political queries related to the environment, using language from the comment space of the political parties, 13-22 March 2019.

Figure 8. Presence of fake and junk news in Google.nl search engine results for political queries related to foreign affairs, using language from the comment space of the political parties, 13-22 March 2019.

  75 Figure 9. Presence of fake and junk news in Google.nl search engine results for political queries related to polarizing topics, using language from the comment space of the political parties, 13-22 March 2019.

The migration cluster (figure 10) was more prominent in the discussion space on Facebook pages than in the platforms of the political parties. Keywords as migranten (migrants), migratie

(migration), immigranten (immigrants) and gelukszoekers (fortune hunters) were often mentioned in the comments on the Facebook homepages of political parties. (Gelukszoekers could be said to be a pejorative term for economic migrants.) In particular, results for three parties (FvD, PVV and

VVD) had junk webpages on the top 10 Google search results, and they were spread evenly over the ten-day research period. It is also noteworthy to see that there appears to be a decreasing tendency of junk news from the election day onwards in this cluster, both in terms of the amount as well as the rankings.

Figure 10. Presence of fake and junk news in Google.nl search engine results for political queries related to migration, using language from the comment space of the political parties, 13-22 March 2019.

European Parliamentary Elections

The results for the political party standpoint space during the European Parliamentary elections also show a consistent presence of junk news. Three major findings are worth mentioning.

  76 Firstly, as figures 11 and 12 make evident, the presence of junk sources in the Google search engine results was lower during the European parliamentary elections than during the provincial elections. In many cases, our queries combining issues with parties did not produce links to hyperpartisan material or only did so for one day. Only 25% of our queries returned junk for more than one day during the three-day period under research. Of the four keywords queried, the one that produced the least amount of junk in combination with party names was economie

(economy). In contrast, migratie was the most prone to returning junk, with queries related to five parties consistently returning junk websites between May 22 and 24 (see Figure 11).

Queries in combination with the political party DENK were particularly junk ridden. On the day of the election and the day after, almost half of the results provided by the Google search engine are problematic. It is particularly relevant that during the 24 th of May, hyperpartisan websites occupied the top seven positions.

A second relevant finding is that the presence of hyperpartisan resources is more prevalent in the day after the election than the day before, in contrast to what happened during the provincial elections. Although this is the case for all parties and keywords where junk sources were identified, it was especially prominent for the issues of migration, economy and climate

(keywords migratie, economie and klimaat). In most cases, junk sources also occupy the first positions in the results during the 24 th of June. We cannot answer the question of whether this phenomenon can be credited to the Google search algorithm reacting to an increase in searches related to the elections, to a surge in the activity of hyperpartisan websites after they took place, or to some combination. A close reading of the results in the highest positions, however, shows that the hyperpartisan sources behave as one would expect from any information provider during election season, if only keeping their radical tone; before the election they provide predictions about the results, on the day of the election they invite citizens to vote, and on the day after they discuss the results. An article by the De Dagelijkse Standaard that consistently occupied the top result for various keywords and parties consists of a reflection about the FvD and leader Thierry Baudet’s need to tone down their radical discourse in order to become a more powerful political force.

Lastly, whereas during the Dutch provincial elections the queries related to right-wing parties were more strongly connected by the search engine with junk sources, during the European elections this is not the case. For example, although queries performed in May involving FvD also produced results pointing to hyperpartisan websites – similarly to the results obtained in

March – it was those related to DENK which, in aggregate terms, produced more junk (i.e., 25 for FvD and 27 for DENK). However, the case of DENK is difficult to assess given that, apart from the name of a party, it is also a common Dutch word (“think”, in English). Although this does not change the fact that people looking for information about this party were very likely be exposed to junk sources, the content may not specifically relate to DENK. In fact, the highestranking result for the query [migratie DENK -bewegingdenk.nl] is an article in the hyperpartisan webiste De Dagelijkse Standaard that makes no specific mention to this party, but instead generally reflects on the results of the elections and specifically about the demise of the PVV.

  77 Figure 11. Presence of fake and junk news in Google.nl search engine results for political queries related to migration and European Union issues, 22-24 June 2019

Figure 12. Presence of fake and junk news in Google.nl search engine results for political queries related to

Cilmate and Economy issues, 22-24 June 2019

  78 Conclusions: Junk in search engine results

The goal of this research is to locate the presence and ranking of fake and junk websites within the first twenty Google.nl results of queries concerning Dutch political parties and their most significant issues during the 2019 provincial and European parliamentary elections. The keyword queries were built by combining political party names with keywords retrieved from political party platforms and party Facebook page comment spaces (in the case of the provincial elections). We clustered the keywords into categories to enable a comparative analysis. At the outset the research questions were as follows: Which rankings has the search engine assigned to fake and junk sources when querying political parties and their issues? Are there particular political issue and party spaces where fake and junk news are prevalent or largely absent? Do the ranks and amounts increase as the elections draw near?

Our results indicate that the junk websites present in the results of our Google.nl political queries are almost exclusively hyperpartisan, rather than sources spreading disinformation, conspiracy theories, or clickbait. Three websites, namely De Dagelijkse Standaard, Stop de

Bankiers and Opiniez, account for the largest portion of the junk sources identified. We did not find fake advocacy groups (nepactiegroepen) or foreign disinformation operatives. Furthermore, we found that queries involving right-wing parties were more prone to result in exposure to hyperpartisan sources than those associated with centre-left or left-wing parties. For most keywords, hyperpartisan websites appeared in the top positions, and certainly always within the first page of results. Our findings suggest that on Google.nl there is a considerably high probability that junk news is outputted on the first page of results when the queries concern right-wing parties and their issues.

Another finding is that hyperpartisan results spread unevenly during our research period. For the case of the Dutch provincial elections, they are more present before the day of the election and drop their presence and ranking, in some cases dramatically, on election day and in the immediate aftermath, for instance in the case of migration issues in the vernacular issue space. In contrast, during the European parliamentary elections this phenomenon reversed, and junk sources were more prominent the day after the elections.

Concerning the comparison between the two spaces, the vernacular space had the highest percentage of junk news returns, especially in the categories of foreign affairs and migration. In the political party standpoint space, the differences among themes is smaller. In other words, the queries designed with the political language of Dutch Facebook users were more likely to result in hyperpartisan results than the queries built with the standpoint language of political parties.

Although more research is needed in this regard, such a finding suggests that the discourse of normal citizens, or those commenting on party Facebook pages, is more politically contentious than that of political parties.

The results are indicative of the amount of hyperpartisan material in political space in Google.nl rather than conclusive, for they derive from a particular query strategy and not from multiple strategies and are only a snapshot from a particular event-related timeframe. The data set we built could also be read more closely, and additional fake or junk sources could be found, meaning that we could have undercounted (rather than overcounted).

  79 Given that our intention was to determine the sensibility of the Google search engine to junk news, the question remains whether the location of the results of politically charged queries can be credited to an optimisation effort on the side of hyperpartisan content generators, an overall susceptibility of Google’s search algorithm to provide questionable content to its users, consumers’ preference for low quality information, or some combination of the three.

While discussing the two latter hypotheses would require more space, a few words can be said about the first. A possible way to detect Search Engine Optimization strategies consists of using one of the many online services providing SEO analysis. However, given the proprietary nature of their methodologies, the results should be interpreted with caution. We submitted the three most recurrent junk websites we found during our research to the service SEO Tester Online, a tool that measures a website’s readiness to achieve top positions within search results. This tool provides analysis in four different categories: basic, related to the overall online presence of the website, content, which measures the richness of the keywords that trigger the website to pop up in search results, web performance, which measures how fast the website can be rendered in mobile and desktop devices, and social, which provides information about the websites engagement with users through social networks. De Dagelijkse Standaard obtained a score of

56/100, faring the lowest in their web performance, and the highest in its social engagement.

Opiniez obtained a score of 62/100, faring the best in web performance and the lowest in content, although it obtained an excellent score in the number of unique keywords. Stop de bankiers obtained the highest score, with 74/100, including a 100/100 in the assessment of keywords which may lead to the site through search engines. It also fared the best in social engagement and the lowest in web performance.

The reports produced by SEO Tester Online, which for all websites included a considerable amount of suggestions about how to improve the websites’ code, suggest that, at least currently, content producers are not actively seeking to artificially alter the performance of their sites.

Consequently, the presence of these junk sources in our analysis is more likely related to an inherent susceptibility of Google’s search engine or an organic result of users’ preferences.

Further research is necessary to paint a clearer picture regarding the increased consumption of junk news. One could repeat the work for longer periods of time in order to ascertain the extent to which the presence of these or other dubious source types is becoming more widespread or even disappearing in the top results for political queries, as is currently the case with clickbait.

Furthermore, specific sources could be monitored over time to track the performance of their content (and the changes to their code), in order to detect attempts to artificially increase relevance and thus the ranking assigned by search engine algorithms.

  80

TWITTER, JUNK NEWS AND TROLL-LIKE USERS

DURING THE DUTCH PROVINCIAL

AND EUROPEAN ELECTIONS

Sabine Niederer and Maarten Groen

Twitter, junk news and troll-like users during the Dutch provincial and European elections

Sabine Niederer and Maarten Groen 21

Introduction

In 2018 the Dutch daily newspaper De Volkskrant published an article entitled “The troll army of pop artist Dotan” (“Het trollenleger van popartiest Dotan”), which revealed how the Dutch singer-songwriter had made use of fictitious accounts pretending to be fans (Misérus and van der Noordaa, 2018a). The fake fans were highly active across social media platforms (including

Facebook, Twitter, and Instagram) where they circulated heart-warming stories about the artist, requested his songs on Dutch and German radio stations, and actively tried to edit the Wikipedia pages about the artist and his mother (who is also a Dutch singer). At the root of these activities were 140 accounts that the newspaper retrieved, at least one of which connected directly to the artist’s own Gmail account, and others to accomplices. Dotan’s case is perhaps the most-known example of artificially boosted accounts and content in the Netherlands, but certainly not the only known case of such behaviour. The politicians Geert Wilders and members of the political party Denk were found to have suspiciously inflated follower counts, which surfaced when

Twitter started deleting unvalidated accounts (NOS, 2018).

The present study builds on previous digital research in which the social media platform Twitter, used by over 326 million monthly active users accounting for 500 million tweets per day, is repurposed for social research (Omnicore, 2019). As with Dotan and the Dutch politicians mentioned above, it similarly looks into social media use and the question of manipulation, in particular in political spaces around elections. It studies troll-like and artificial boosting as well as the circulation of fake/junk and tendentious news sources during two election campaign periods in 2019. Initially intending to detect the possible presence of Russian disinformation in the

Dutch Twitter space, the study enquires into coordinated campaigning around divisive issues and ascertains the extent of homegrown junk news in Dutch political Twitter, including hyperpartisan, conspiracy and clickbait sources. So-called (and self-identified) tendentious sources such as Geenstijl.nl and TPO.nl are labelled as such, and one could argue that they are mainstreaming, given how they are shared, as we discuss below. These two sources are part of the “anti-establishment established source” set, and as such are closely related to an emerging right-wing media landscape (see Tuters, this volume).

In employing digital research methods and techniques, the analysis makes use of the platform’s own features and cultures of use, which offer built-in structurings of the content being shared

(Rogers, 2019). These are repurposed for social or political research. Hashtags can be repurposed as content categories or issue activity indicators, retweeting suggests “pass-along value”, and the

@reply and @mention functionalities network users and their content to fellow users and content (Niederer, 2018). Through an analysis of @replies, Twitter can be studied “as a conversation-maker, where one may explore the extent to which there is dialogue, or

21 The research reported here was undertaken in collaboration with Layal Boulos, Peter Fussy, Oana Patrici, Maria Stenzel Timmermans, Emile den Tex, Carlo De Gaetano and Federica Bardelli.

  81 broadcasting” (Honeycutt and Herring, 2009; Boyd et al., 2010). The @mentions may contribute to the inquiry of dominant voice - certain understandings of issues can be shaped by the actors most mentioned in a tweet corpus, and also by the actors that are the most vocal. Twitter can be studied as a social network of professional information-sharers (Java, 2007). It also can be considered a ‘rebroadcaster’ of (political) news, in which the platform’s built-in algorithms reinforce the issues and framings discussed there as so-called trending topics (Kwak et al., 2010). Furthermore, Twitter is often moving information faster than the news, and Twitter content in some cases becomes news (Niederer, 2018). As news and mass media sources strive to make their content “platform-ready” (Helmond, 2015), political news, other mass media content and new platforms become further entangled, forming a hybrid media system (Chadwick, 2013). Here, professional journalists include tweets in their stories, and when their work has been published, they may post a link to that article on Twitter and other social media, using the platforms both as a source of information and as a channel for the distribution of their own work.

Critiques of digital social research take issue with its dependency on the already problematic hegemony of proprietary social media platforms. On a methodological level, scholars warn of the sheer impossibility of distinguishing between the working logic of web platforms and exemplary ‘platform artifacts’ (Marres, 2015; Marres & Weltevrede, 2013; Rogers, 2013; Niederer, 2017).

How do we know whether the most-retweeted Twitter post is the most relevant, or the most

Twitter-friendly (Marres, 2015)? One way to approach this issue is to take into account the sociotechnical specifics of each platform, and to regard Twitter and other social media platforms as distinct windows on an issue. Rather than questioning the relevance of the platform for the elections, we then ask: how does Twitter present the elections? And how does this compare to how other social media platforms cover the topic? Such lines of questioning open up avenues for qualitative and empirical digital research across political events and social issues as they resonate online and offer insights into the cultures of use of the various platforms. In this present study,

Twitter can be seen to produce political subspaces around divisive issues, in which a relatively small number of highly active, troll-like users sow division and where fake/junk news at times outperforms mainstream news.

Troll-like user activity during the 2017 Dutch general elections

The present study follows from an earlier one, which itself concerned Dutch elections. In the lead up to the 2017 Dutch general elections for the national parliament, journalists revealed the use of sock puppets (i.e., false online identities assumed to deceive and influence opinion) by the political party Denk, in order to amplify their online messages and attack their political opponents on Twitter and Facebook (Kouwenhoven and Logtenberg, 2017). In an empirical study as part of the Field Guide to Fake News (Bounegru et al., 2017), we studied troll-like behaviour in Twitter, developing a research protocol for identifying and analysing political trolling, which in this case referred to repeated attacks of politicians on Twitter. It focused on the sources of troll-like activity (i.e., which user accounts target politicians?), their targets (who do these troll-like users address?), and the characteristics of these practices (what do troll-like users do?) (Borra et al., 2017).

The detection of user accounts engaging in political trolling behaviour starts by compiling a list of potential targets. The aforementioned study looked into the user accounts of 28 political party

  82 leaders participating in the 2017 elections. The users that @-mention them were queried. For the most-active users per @mention, their posts in which they @-mention the political leaders were qualitatively studied. In a next step, only those who @-mention one or more political leaders at least 100 times during a one-month period (8 February - 8 March 2017) were retained, and their tweets coded for being favourable or unfavourable of the politician. The study found an asymmetry in the troll-like behaviour across the political spectrum, as more left-wing politicians were being targeted by negative mentions while most right-wing politicians were receiving support (see figure 1). There are exceptions, such as Emile Roemer (SP) and Marianne Thieme

(PvdD), who in this time frame received only support by troll-like users, and Prime Minister

Mark Rutte (VVD) who received unfavourable mentions, in particular on his personal account though less so on his official @MinPres account.

Figure 1: Political party leaders as troll targets on Twitter during the 2017 Dutch general elections. Red indicates proportion of targeting. Source: Borra et al., 2017.

To classify the sources of political trolling, we used the same list of 24 highly active and troll-like users (mentioning political leaders at least 100 times in the one-month time frame), and collected their profile information (description, profile picture and banner) from the Twitter interface. If the profiles had a profile picture, Google reverse image search was used to check these images for authenticity. Then, using the Twitter API, the creation date for each of these accounts was retrieved, in order to assess whether accounts in our dataset had been created on the same date.

This analysis provided a more nuanced view of the user accounts responsible for the trolling behaviour. Of the 24 accounts still active at the time of study, three users appeared to be sock puppets created for trolling activities. They had very similar profiles and had been created within a short time-frame. Another six accounts in the data set promoted the same anti-Islam agenda, but were not determined to be fake accounts.

  83 To characterise the substance of the trolling practices, the study looked at the issues and the media sources that resonate in the set of tweets. To identify the issues, the hashtags used by the highly active and trolling users in their tweets (that @mention a political leader) were collected and analysed. Most tweets that include hashtags were found to mention the right-wing populist candidate Geert Wilders, and most hashtags referred to the issues in PVV’s political messages from 2017 (“Nexit”, “StopIslam” and “BanIslam”), as well as those pertaining to expressions of Dutch patriotism (Borra et al., 2017: 188). To assess which media sources were circulated by the troll-like users, the most-circulated URLs in the tweets were collected and categorised. For the

2017 general elections, the most-tweeted media sources by the 24 trolling users were the Dutch alt-right blog fenixx.org followed by the anti-Islam site Jihad Watch and the right-wing think tank

Gatestone Institute (Borra et al., 2017: 192). 22

Research questions and data collection

For the study presented in the next section, the main research question is to what extent fake/junk news sources and troll-like users are present on Twitter around both the provincial and the European parliamentary elections in the Netherlands in 2019. To answer these questions, we examine Twitter activity around the elections, the party leadership as well as political candidates, and zoom in on potentially divisive issues, including Zwarte Piet and MH17.

In addressing these research questions, queries were formulated to demarcate the political and issue spaces in Twitter (see table 1). The data was collected using the commercial social media monitoring tool, Coosto, and the Twitter Capturing and Analysis Toolkit developed by the

Digital Methods Initiative (DMI-TCAT). Coosto was used to retrieve data from both the provincial and European election periods, in order to conduct a comparative analysis of the engagement with mainstream and fake/junk news across political and issue spaces, and the presence of troll-like users in these spaces, as discussed in detail in the next sections. DMI-

TCAT, a tool that “provides robust and reproducible data capture and analysis and interlinks with existing analytical software” (Borra & Rieder, 2014), was used to analyse the engagement with fake/junk and tendentious news sources and the users responsible for this engagement.

While some collections (or ‘bins,’ in the terminology of the TCAT-tool) were created only for this study, others had been running for months prior, such as MH17, or in the case of Zwarte

Piet even years (with a bin that was created in December of 2017). The set for the Utrecht tram shooting was created on the day that event took place, 18 March 2019. For this study, the sets were limited to the provincial elections campaign period (18 February - 25 March 2019) and the

European election campaign period (26 April -24 May). The one exception was the Utrecht tram shooting set, which was only included in the Provincial Elections campaign period, as it took

22 It is important to note the ethical implications of these lines of empirical inquiry. With the right tools, it is relatively straightforward to assess which users are involved in attacking or amplifying political leaders. A qualitative analysis can subsequently give further insights in the authenticity of a particular account and give context to the political views it pushes by way of media circulation analysis. However, users who have created sock puppets may make mistakes, too. It can happen (as it did in the 2017 study) that a fake user account created for trolling purposes suddenly posts a proud story about his/her granddaughter, or a picture of him/herself in front of their newlyopened flower shop. It can be easy to then identify the people behind such activity, and publishing their names becomes a consideration of weighing public interest against ethical and possibly legal privacy implications. In the field guide, we thus mainly focused on networks and relationships between user accounts rather than singling out users. We redacted personal identifying information from the data set and the visualisations.

  84 place during that time frame.

Elections Topic Query

PS General Ps2019, Ps19, verkiezingen

EU General EUverkiezingen2019, euverkiezingen, ep2019, eu2019, euelections2019, verkiezingen, verkiezingen2019, EU, Europa, Europese Unie, europeseverkiezingen

PS Party leaders Mark Rutte, MinPres, markrutte, Geert Wilders, geertwilderspvv, Thierry Baudet, thierrybaudet, Jesse Klaver, jesseklaver, Rob Jetten, RobJetten, Liliane Marijnissenm, MarijnissenL, Marianne Thieme, mariannethieme, Gert-Jan Segers, gertjansegers, Lodewijk Asscher, LodewijkA, Tunahan Kuzu, tunahankuzu, Henk Krol, HenkKrol, Klaas-Jan Dijkhoff, dijkhoff, Sybrand Buma, sybrandbuma,Kees van der Staaij, keesvdstaaij

EU Party leaders SophieintVeld, Esther_de_Lange, mjrldegraaff, malikazmani, arnouthoekstra, TimmermansEU, petervdalen, BasEickhout, anjahazekamp, ToineManders, florens0148, atonca, paulbeasd, djeppink, sentwierda, RLanschot, MinPres, markrutte,

geertwilderspvv, thierrybaudet, jesseklaver, RobJetten, MarijnissenL, mariannethieme, gertjansegers, LodewijkA, tunahankuzu, HenkKrol,

dijkhoff, sybrandbuma, keesvdstaaij

PS and MH17 mh17 EU

PS and Zwarte Piet Zwartepiet, zwarte piet EU

PS and Climate klimaat EU

PS and Fake news Fake news, fakenews, nepnieuws, desinformatie, junknieuws EU

PS Utrecht utrecht, 24oktoberplein, gokmen tanis, gokman tanis

Table 1: Query overview showing the election campaign period (Provincial, EU, or both), the political or issue space and the query made resulting in Twitter data sets.

  85 Junk/fake news sources and troll-like users during the provincial elections on Twitter

During both the provincial and the European election campaigns we tracked the resonance of mainstream, junk and tendentious sources in Twitter. We did so around the potentially divisive issues of Zwarte Piet and MH17 and chose to include climate and fake news (as an issue).

Furthermore, we tracked the resonance of news sources for the political spaces of the (Provincial and EU) elections, as well as the party leadership, and political candidates. For each of the elections, we demarcated a five-week campaign period. For the Provincial elections this was 18

February - 25 March, and for the EU parliamentary elections in the Netherlands, 26 April - 24

May. Per political and issue space, and for each of the five weeks of the campaign, the mostshared links (up to a maximum of 500) were collected and coded (for mainstream or junk news of various types, using the aforementioned expert list). The engagement scores for the mainstream and fake/junk news source engagement per week were visualised as line graphs, as in the well-known Buzzfeed News study (Silverman, 2016).

Figure 2: Line graphs visualising the engagement with mainstream news (blue) and fake/junk news sources

(magenta) during the Dutch Provincial election campaign (left) and the European Election campaign period

(right). Visualisation by Federica Bardelli.

For both election campaign periods, overall the mainstream news outperforms fake/junk news (see Figure 2). When zooming in on the political spaces of the elections and the party leadership and political candidates, the mainstream news sources garner far more engagement than fake/junk news. A look at the top 500 most-engaged with links shows the rise and fall of mainstream hosts circulated in the issue space, and the relatively small but steady resonance of fake/junk news hosts, which during the provincial election campaign rises slightly in its last week.

  86 Figure 3: Line graphs visualising the engagement with mainstream news (blue) and fake/junk news sources

(magenta) for the issue of MH17 (top right) and Zwarte Piet (top left) during the Provincial elections, and the

EU elections (bottom right and left). Visualisations by Federica Bardelli.

Divisive issues: Zwarte Piet and MH17

Both for Zwarte Piet and MH17, there are instances in which junk/fake news outperforms mainstream news. The line graphs in figure 3 include a zoomed-in view that render visible the moments in which junk/fake news is more engaged with than the mainstream news. For the controversial topic of Zwarte Piet, during the Provincial election period mainstream news receives more engagement. Fake/junk news outperforms mainstream news in weeks three and four of the European parliamentary elections campaign. The article mostly responsible for this peak in week three is a short commentary on tendentious-hyperpartisan website tpo.nl about the proposal by Dutch politician Sylvana Simons (addressed to the Amsterdam Municipality) to ban the “racist caricature of Zwarte Piet” in the city of Amsterdam. (When one removes TPO from graph, the results remain the same apart from the one week in May during the European parliamentary election period where now mainstream news outperforms junk (see Appendix)).

  87 Thus in that one week in May, the ) In week four, an article on Cultuurondervuur.nu (culture under fire) entitled ‘Jerry Afriyie receives funding for anti-Zwarte Piet educational materials’ is responsible for the increased activity. In it activist Jerry Afriyie is described as a “Zwarte Piet hater” (cultuurondervuur.nu, 2019).

For the issue of MH17, during the Provincial elections campaign there are times in which fake/junk news outperforms mainstream news in terms of engagement. For the European parliamentary elections, the mainstream attracts more engagement, but during certain periods junk/fake news performs on a similar level of engagement as the mainstream. The peaks that occur during the Provincial elections are mainly caused by engagement with a piece from citizenjournalist Max van der Werff, on his website kremlintroll.nl, in which he demands rectification of an article in De Groene Amsterdammer (from August 2018) about Russian internet trolls (van der

Werff, 2019). Two other articles that attract engagement are from the hyperpartisan website jdreport.nl, questioning the integrity of the MH17 investigation, and in one Frans Timmermans

(who would win a seat for the PvdA in the EU parliamentary elections) is named as part of an

‘MH17-doofpot’, or cover-up (jdreport.nl, 2019). In week four of the Provincial elections campaign period, the Kremlintroll piece requesting rectification is particularly actively shared.

Simultaneously, the interlinked article with the actual critiques of the article from De Groene

Amsterdammer is receiving more engagement.

During the EU election campaign, mainstream news receives more engagement. It is important to note, however, that aside from a peak in mainstream news in week three of the campaign, its engagement level is equal to that of fake/junk news sources. Where in the mainstream certain events cause peaks in media coverage, it appears that for fake/junk news these divisive issues are continuous and year-round. Zwarte Piet may not be a subject matter in the mainstream news in

Springtime, but it remains a matter of concern and a source of engagement in fake/junk news media.

Troll-like users during the Dutch provincial and European elections on Twitter

For the Dutch provincial elections campaign period, a next step in the study is to look closely at the user activity related to the Dutch provincial elections and the political party leadership, as well as coverage of the potentially divisive issues of Zwarte Piet, MH17 and the Utrecht tram shooting. As a first step, the URLs (hosts) were extracted from the sets of tweets and checked against a collaboratively compiled expert list of junk/fake and tendentious news sources.

Similarly, the users active in each of the sets of tweets were checked against a list of flagged users. Here, we made use of existing lists from the previous project in The Field Guide to Fake

News (Borra et al., 2017) and expanded these lists. To do so, we extracted top users from the data sets of Zwarte Piet, MH17, Utrecht tram shooting, the Dutch provincial elections and the political party leadership and followed an extensive protocol adapted from the aforementioned study, and combined them with research on credibility metrics (Groot et al., 2019; Borra et al.,

2017). 23

23 For this particular study, to identify potentially suspicious users in the data sets, the top 15 most-active users in the set were selected, as well as the top 15 users who were highly active yet at the same time very low on visibility (i.e. rarely or not at all @mentioned). Then, the profiles of these user accounts were checked for the following red flags: mostly retweeting, or retweeting in several languages (as possible indicators of automation) which is of interest given

  88 With the Compare List tool (Borra, 2013), the study assessed whether any of the flagged users were active in one or more of the political issue spaces. Zwarte Piet had an initial list of 26 potentially suspicious accounts, five of which had already been taken offline at the time of inquiry. Of the 21 remaining each was classified as suspicious; one of which described itself as a retweet bot (in the user profile). For MH17, of an initial list of 26 potentially suspicious user accounts, two were inactive at the time of inquiry. Of the remaining user accounts, 13 accounts were flagged as suspicious, and 10 were not. For the Utrecht tram shooting, from an initial list of 23 potentially suspicious accounts, 10 were flagged as suspicious after examination. For the provincial elections dataset, the list of potentially suspicious users entailed 24 accounts, 17 of which were flagged according to our criteria and one of them described itself as a bot.

Subsequently, these flagged users were checked for activity in more than one issue. This would make sense for those data sets that are of related topics, such as the provincial elections and the political leadership. When users are active across distinct controversial issues such as Zwarte

Piet, MH17, and the shooting in Utrecht, which have in common their potential divisiveness, such multi-issue users and the content they circulate would be further scrutinised. In fact, 14 suspicious accounts are common to all of the five political issue spaces, and as many as 29 suspicious users are common to four of the data sets, pointing to a suspicious effort to fuel division during the election period.

Figure 4 gives an overview of the most-resonating hashtags, tweets, and sources as well as the most-mentioned and most-active users during the time around the elections (18 February – 25

March 2019). Flagged users as well as fake/junk and tendentious news sources are highlighted, with fake/junk being broadly defined to include hyperpartisan sources, as well as conspiracy, clickbait and disinformation. The overview shows that there is no disinformation resonating in the top 10 hosts per political and issue space. The top hosts are mostly (Dutch and international) mainstream news media. The hyperpartisan/tendentious site Opiniez.nl is among the top 10 hosts for Zwarte Piet and the provincial elections space, and geenstijl.nl is shared for MH17 and PS2019. Fake/junk sources are present across political and issue spaces around MH17, Zwarte

Piet, Utrecht, PS2019, and the Dutch party leadership. There are fake/junk news hosts that are common across all five issues: Ninefornews.nl, fenixx.org, tref.eu, ejbron.wordpress.com, drimble.nl (a particular story), and dagelijksestandaard.nl. Hyperpartisan and conspiracy sources are mostly circulated by flagged users. However, some hyperpartisan and tendentious sources are being mainstreamed, and circulated by regular (non-suspect) users. These include Geenstijl, which is considered part of the mainstream but retained here because of its self-proclaimed tendentious characteristics, tendentious-hyperpartisan host TPO, and hyperpartisan sources, De Dagelijkse

Standaard and Fenixx.

the wide distribution of easily acquirable retweet bots (McGarry, 2013); profile oddities such as suspicious user’s profile images, which were checked with Google Image search to assess their authenticity, a recently created account, or a high following count (of over 1,000), a username with over 3 numbers in it; and suspicious tweet frequency as tweeting over 200 times mentioning the issue, posting 20 tweets or more times per day; and, whether the user seems to mostly retweet more often rather than tweet his/her own content.

  89 Figure 4: Top hashtags, users, @mentions, hosts and tweets during the Dutch provincial election period of 2019. Visualisation by Carlo De Gaetano.

Looking at the time frame around the elections, flagged users are among the top most-active users across issues. In particular for Zwarte Piet and MH17, six of the top ten users are flagged accounts. The top @mentioned users vary per issue. Two flagged user accounts are among the top 10 @mentioned users in tweets about Zwarte Piet and MH17. When analysing the mostused hashtags across the issues, what stands out are that the top hashtags used in the MH17 issue space all seem to be Pro-Russian. Across the issue spaces of Zwarte Piet, MH17 and PS2019, we see the resonance of right-wing political party hashtags, such as PVV and FvD. Zwarte Piet contains hashtags both for pro-Zwarte Piet (e.g., ‘blokkeerfriezen’, referring to the Frysian counterprotest in Dokkum against anti-Zwarte Piet protesters of ‘Kick out Zwarte Piet’, which

  90 can be found in the data set with hashtag kozp, in which they blocked the highway to prevent anti-Zwarte Piet protesters from entering their town) and anti-Zwarte Piet, e.g.,

‘SamenTegenRacisme’, which translates as ‘united against racism’.

For the EU election campaigns, we similarly investigated the activity of flagged users in the political and issue space. For the political space, the top 1000 most active users were collected for the general EU election hashtags and the political leaders relevant to the EU election campaigns. For the issue spaces, the top 1000 most active users were collected on the topics of climate change, Zwarte Piet, MH17 and nepnieuws (fake news). These lists of top users were matched with the suspicious users list from the first part of the empirical study. Because some topics were more active than others, the activity of the top 1000 users varies per dataset. For the more generic EU set, the top 1000 users each posted more than 44 tweets in the EU election period. In comparison, in the Zwarte Piet dataset the top 1000 users each posted two or more tweets.

Of the flagged users list, eight users were active in all six issue spaces during the EU campaign period. Three users were active in five of the spaces and another three users in four of the spaces. Four of the eight users active in all spaces were also active in all the provincial election period datasets. From the users active in all datasets, the top user posted 2781 tweets. 2578 of those tweets were in the general EU and party leader dataset. This user is not only retweeting other content, but also posts his own content. The content in the EU Elections period can be characterized as anti-EU, anti-immigration, pro PVV/FvD and critical of all other parties.

The circulation of fake/junk and tendentious news during the provincial elections

To gain a better view of these troll-like, fake/junk and tendentious news activities, a next step zooms in on the circulation of these news sources during the campaign period in each of the political issue spaces. Visualised as network graphs, the analysis considers whether such news sources are circulated by flagged or regular (non-flagged) users. 24 Each host-user bipartite network graph includes a short overview of the user and host types per data set, clearly illustrating that the number of flagged users and the circulation of fake/junk or tendentious news sources is outnumbered by unflagged users and the circulation of mainstream news. Thus, these visualisations should be read as a zoom-in on a particular, small set of hosts that are of interest to the study of the presence and circulation of fake/junk and tendentious news and the users that circulate them.

In each issue space, hyperpartisan sources are circulated the most. And while the issue space of Zwarte Piet is dominated by the circulation of hyperpartisan sources being shared by flagged but also by regular users, the main fake/junk sources for MH17 are more diverse in composition.

Here, we see a mix of tendentious, hyperpartisan, as well as conspiracy hosts. For the Utrecht shooting, tendentious and hyperpartisan hosts are circulated the most, by flagged and regular users, making them appear as mainstream. The fake/junk and tendentious sources in both of the

24 Regular in this case in fact strictly speaking means not identified as suspicious. There could be suspicious users among them, which have not (yet) been identified as such.

  91 political spaces, PS2019 and the party leaders, revolve around mostly hyperpartisan and tendentious sources.

Figure 5: Gephi visualisation of Zwarte Piet host-user network during the Provincial elections campaign period, depicting only fake/junk and tendentious hosts and the users that circulate these sources. Visualisation by Carlo

De Gaetano.

The host-user network of the Zwarte Piet issue space (Figure 5) is dense and, as said, is dominated by the circulation of hyperpartisan sources such as dagelijksestandaard.nl, fenixx.org, cultuurondervuur.nu and opiniez.nl, and, at a slightly lower level, the tendentious source geenstijl.nl. These central nodes are the sources of choice for the majority of the flagged users,

  92 but also have been shared by regular users, who demonstrate a preference for the hyperpartisan source, dagelijksestandaard.nl. One clickbait host (tpook.nl), which can be found in the outskirts of the graph, stands out as being circulated by both flagged and regular users.

Figure 6: Gephi visualisation of MH17 host-user network during the Provincial elections campaign period, depicting only fake/junk and tendentious hosts and the users that circulate these sources. Visualisation by Carlo

De Gaetano.

The network visualisation of the MH17 fake news source circulation (Figure 6) shows a different source composition to that of Zwarte Piet which had hyperpartisan sources at its core. For

MH17, we see a more diverse set of sources central to the network: tendentious source geenstijl.nl, hyperpartisan/conspiracy source novini.nl, and a set of two other conspiracy hosts

(ninefornews.nl and niguru.co), which have been widely circulated by flagged users.

The flagged users in this issue space mostly circulate tendentious hosts, such as geenstijl.nl, and hyperpartisan and conspiracy sites, herstelderepubliek.wordpress.com and novini.nl. The source most circulated by regular users is the tendentious geenstijl.nl.

  93 Figure 7: Gephi visualisation of Utrecht shooting host-user network during the Provincial elections campaign period, depicting only fake/junk and tendentious hosts and the users that circulate these sources. Visualisation by Carlo De Gaetano.

In the issue space for the Utrecht shooting (Figure 7), tendentious and hyperparisan sources

(geenstijl.nl, tpo.nl and dagelijksestandaard.nl), populate the center of the network. Several smaller clusters of fake/junk news sources that have been circulated by regular users are evenly distributed on the periphery of the graph (e.g., drimble.nl (story-level), evendelen.net, dagelijksekrant.nl or hardwaarheid.nl). Only a minority of flagged users circulate clickbait

(tpook.ml, nietbarkie.nl) and conspiracy pages (martinvrijland.nl, ninefornews.nl, brekendnieuws.nl, ellaster.nl, wanttoknow.nl). It is important to note that overall the hyperpartisan and tendentious sources in this network have been circulated by both flagged and regular users, making them appear to be mainstream(ing).

  94 Figure 8: Gephi visualisation of PS2019 host-user network during the Provincial elections campaign period, depicting only fake/junk and tendentious hosts and the users that circulate these sources. Visualisation by Carlo

De Gaetano.

  95 The PS2019 (Provincial State elections) host-user network appears to be organised around two major hosts, hyperpartisan source opiniez.com and tendentious source geenstijl.nl (Figure 8).

The (marginal) presence of clickbait host aboutmedia.nl, is caused by the activity of only one regular user. Conspiracy hosts ninefornews.nl and dimplus.nl have been only marginally circulated by users who also shared other fake/junk hosts. Two recently created user accounts in the network (created in December 2018) demonstrate an uncommonly high number of tweets and likes. One of them has around 39,300 posts, and 31,900 likes within four months of existence, a level of activity that suggests automation and artificial inflation. 25

Figure 9: Gephi visualisation of Party Leadership host-user network during the Provincial elections campaign period, depicting only fake/junk and tendentious hosts and the users that circulate these sources. Visualisation by Carlo De Gaetano.

25 Their high number of likes is also inconsistent with the pattern of activity, which is mostly retweets and replies with GIFs or funny images.

  96 For the Party leadership network, the tendentious-hyperpartisan source tpo.nl and hyperpartisan source dagelijksestandaard.nl are the largest nodes in the network and are circulated by both suspicious and regular users (Figure 9). Smaller nodes of hyperpartisan sources, such as fenixx.org, opiniez.com and verenoflood.nu, are positioned slightly more towards the periphery of the network. A dense cluster of flagged users is situated in the heart of the network and has circulated mostly tendentious and hyperpartisan hosts as well as conspiracy hosts, such as ninefornews.nl or ellaster.nl. Regular users populate the rest of the network and have circulated mostly tendentious and hyperpartisan hosts (e.g. tpo.nl, dagelijksestandaard.nl and opiniez.com) and to a lesser extent, have circulated conspiracy hosts (e.g. donquijotte.wordpress.com or stoppasfamiliedrama.blogspot.com) which are visible in the margins of the graph.

Conclusions: Suspicious activity in divisive issue spaces

As emphasized in studies of the campaigning by the Russian Internet Research Agency as well as so-called home-grown actors, Twitter allows for easy automation, which makes the platform susceptible to abuse by bot and troll-like users (boyd et al., 2018; DiResta et al., 2018; Howard et al., 2018). We have identified suspicious activity during the Provincial elections of 2019, when looking at political issue spaces as well as divisive issues. In fact, troll-like users are central across political and issue spaces around MH17, Zwarte Piet, Utrecht, PS2019, and the Dutch party leadership. In particular, 14 flagged users were found to be active across all political and issue spaces, and the 29 that appear in four out of five, deserve scrutiny. Four suspect users active during the Provincial election period were also (or still) active in all issue spaces during the EU election period. Some of these users had already been flagged in previous research from 2017, which means they have been operating and engaging in new and existing issues for over two years. Overall, our study found that such flagged users tend to spread mostly hyperpartisan and tendentious sources, followed by conspiracy websites. This points to the prospect of a coordinated campaign to sow division, in which troll-like users include sock puppets, automated accounts, and semi-automated user accounts that post both retweets and original content.

Divisive issue spaces are active year-round. From 18 February - the beginning of the official campaign - to 25 March 2019, the issue spaces of Zwarte Piet and MH17 were still active, even though Sinterklaas takes place in December and the downing of the Malaysian airliner was not in the news, either through new developments or official memorial events. A significant number of the most active users in each issue during this period display troll-like behaviour through their high activity (30% in the case of Utrecht and 60% in MH17 and Zwarte Piet). Despite the activity, most of these users’ influence is still limited, however. Only two of them appear among the top ten most @-mentioned for each issue space.

At the same time, we identified at least three highly active new accounts that were created close to the elections with a clear purpose of disseminating divisive content, indicating how the platform may be employed around election time. When these troll or bot-like users are not aggressively attacking the opposition, they function as amplification machines for web news operations, ranging from tendentious sources such as Geenstijl and The Post Online to hyperpartisan sources such as De Dagelijkse Standaard, Opiniez and Fenixx. Repeatedly, we have seen how these tendentious and hyperpartisan sources are widely circulated by regular users who

  97 crowd out the flagged users (in a network clustering sense). The uptake of tendentious and hyperpartisan sources by such regular users leads to a “mainstreaming” of these hosts, in times of elections.

In all, suspicious users tend to spread mostly tendentious and hyperpartisan hosts, followed by conspiracy hosts, which appear in all datasets but seem to be more pervasive in tragedy spaces as MH17 and the Utrecht tram shooting. Clickbait domains resonate marginally in all of the studied spaces, while conspiracy host novini.nl, a host that can occasionally be classified (on a storylevel) as hyperpartisan or disinformation, were shared by users in all five data sets, resonating more prominently in MH17 and only marginally in the political leaders set.

During the EU election period, on several occasions, fake/junk news outperformed mainstream sources around the controversial topics, Zwarte Piet and MH17. On both issues, fake/junk news outperformed mainstream news in two of the five weeks. During these weeks, there is not a large increase visible in the engagement of fake/junk news sources compared to the other weeks.

Instead, the overperformance is mostly caused by a drop in the mainstream media attention for the topics on hand, while coverage persists on the junk news sources, fuelling the debate.

According to these results, the Dutch political Twitter sphere does not appear to have a “fake news” problem, though it is populated by some troll-like users, whose existence serves to amplify certain voices. While we did not find a professional or large-scale trolling campaign, the activity across issues in spreading divisive content was caused by various types of user accounts, both bot-like (as in: automated) and troll-like (as in: repeatedly engaging with divisive issues and targeting politicians) through what seems to be semi-automation. Divisive issues remain steadily

(even if marginally) active in fake/junk and tendentious news throughout the tested time frames, suggesting these issues are year-round rather than seasonal (as may be expected with Zwarte

Piet) or event-based.

  98 Appendix: Alternate figures

Alternate figure 2: Line graphs visualising the engagement with mainstream news (blue) and fake/junk news sources (magenta) during the Dutch Provincial election campaign (PS) and the European Election campaign period (EU), without the inclusion of tendentious-hyperpartisan sources. Visualisation by Federica Bardelli.

Alternate figure 3: Line graphs visualising the engagement with mainstream news (blue) and fake/junk news sources (magenta) for the issue of MH17 and Zwarte Piet during the Provincial elections (PS), and the EU elections (EU), without the inclusion of tendentious-hyperpartisan sources. Visualisations by Federica Bardelli.

  99

THE PRESENCE OF FAKENESS

IN THE DUTCH POLITICAL INSTAGRAM:

FAKE CONTENT, SOURCES AND FOLLOWERS

Gabriele Colombo and Carlo De Gaetano

The presence of fakeness in the Dutch political Instagram: fake content, fake sources and fake followers

Gabriele Colombo and Carlo De Gaetano

Introduction: fake followers, computational propaganda and the detection of fakeness on Instagram

Though Facebook has been labelled the “fake news machine” (Herrman, 2016) and Twitter studied as a matter of routine, owing to the availability of datasets, Instagram, when scrutinised, has been found to perform well as an outlet for fake, junk or hyperpartisan news circulation, artificially amplified engagement and other types of problematic content and users.

The relationship between Instagram and different types of “information disorders” (Jack, 2017) has been studied in connection with the Russian effort to influence the 2016 American elections. A study by New Knowledge (2018) found that Russian propaganda tactics played well on

Instagram. The report analyses data from a variety of platforms, in order to detect efforts by the Internet Research Agency (IRA) to spread disinformation and divisive content. The study found that Instagram, with “187 million engagements” of “116,000 Instagram posts across 133 accounts” (New Knowledge, 2018: 7), to be at the forefro int of an IRA operation, with better performing fake accounts and overall higher engagement than on Facebook.

The significance of Instagram, which “outperformed Facebook” (New Knowledge, 2018: 8) as a battleground in the Russian disinformation enterprise, is linked, according to the report, to two possible causes. First, since it is a platform designed around sharing visual materials, Instagram may be well suited for the so-called “image-centric memetic (meme) warfare” (2018: 8), that is, the weaponized use of image macros to stir conflict and foster division online. Second, the report states that the considerably high engagement of content from IRA’s accounts on

Instagram may also be the result of click-farm activity, and some of the accounts in the dataset appeared indeed to be linked to “a live engagement farm” (2018: 8).

With respect to the Dutch case, Russian influence has been studied mainly on Twitter, with the detection of trolling activities, especially in the aftermath of tragic and divisive events both in the Netherlands and in Belgium. For example, two journalistic studies found peaks in Russian trolling activity following the downing of MH17 in 2014 (Kist and Wassens, 2018; van der

Noordaa and van de Ven, 2018a), while another study uncovered a (rather unsuccessful) organized Russian effort in spreading anti-Islam content on Twitter after the 2016 Brussels airport attacks (van der Noordaa and van de Ven, 2018b). Despite the lack of empirical research regarding Russian influence on Instagram, one study from the NRC Handelsblad (Kist and

Wassens, 2018) suggests that a larger organized trolling activity may be found on other platforms beyond Twitter, including Instagram.

The use of computational means to amplify misinformation and hyperpartisan content on

Instagram has not been linked exclusively to the Russian propaganda operation in the West, but it has also been described as a domestic tactic, adopted by national campaigners as well. A

 100 comparative, global study of social media manipulation in 48 countries (Bradshaw and Howard,

2018) describes different computational tactics for political influence online, including the use of fake accounts to attack other users, automated accounts generating artificial engagement, and human-curated accounts that employ automation to be more efficient. With respect to the

Netherlands, the study found such automated activity to be mainly linked to the boosting of

Geert Wilders’ hashtags on Twitter. While the study describes Twitter as the platform where automation thrives the most, signs of “cyber troop activity” (Bradshaw and Howard, 2018: 13) are also to be found in other platforms, including Instagram, among 25% of the countries studied.

That fake following and artificial engagement flourish on Instagram may also be noted from reported cases in the news. As a case in point, in June 2017, the Russian journalist, Vasily Sonkin, posted an image of a vending machine, placed inside a shopping center in Moscow, that lets users buy Instagram followers and likes. The news that for the (extremely cheap) price of 50

Russian rubels (about eur 0.70) one could buy 100 fake Instagram likes was reported by numerous tech or news media outlet (Matsakis, 2017; Feldman, 2017; Tan, 2017).

There have also been efforts by Instagram itself to counter artificially amplified activities on the platform. In December 2014, Instagram announced a crackdown on fake (or improperly obtained) profiles, in the so-called “Instagram rapture” (Lorenz, 2014) that resulted in the deletion of hundreds of thousands of accounts. And later, in December 2018, a “Christmas crackdown” (Lorenz, 2018) resulted in the shutdown of 500 meme accounts, some of which with millions of followers, suspected of using stolen or traded profiles. On the same note, in

April 2019, Facebook filed a lawsuit against one company based in New Zealand, accusing it of providing “fake likes, views and followers to Instagram users” (Romero, 2019). The lawsuit is presented as part of a larger effort by the platform to prevent “inauthentic behavior” on

Instagram. While actions have been taken to cope with artificial engagement as well as fake or improperly obtained accounts, the platform has been apparently less active in limiting the spread of extremely colored or hyperpartisan content, but rather has become the “Alt-Right’s new favorite haven” (Sommer, 2018), offering refuge to extreme right-wing personalities, after their accounts are deleted from Twitter.

The presence of a large automated engagement infrastructure on Instagram is also indicated by the deluge of fakeness-detection tools offered by commercial services online. The detection of inauthentic automated activity on Instagram may work by fingerprinting one account’s follower base by nationality, and associate specific geographical locations, such as Brazil, Turkey or China, to suspected bot activity (Maheshwari, 2018). For example, among the available tools,

Hypeauditor, a “100% AI-powered” service to expose “fake followers and engagement” on

Instagram, flags certain countries, such as Brazil, as geographical locations that may signal the presence of fake followers (Komok, 2018).

Despite being understudied, specifically in the Dutch context, Instagram appears to be a platform prone to the presence of various instantiations of fakeness. Fakeness (also referred to as junk news) may refer to the presence of content that can be described as false as well as

 101 merely hyperpartisan and divisive, but deliberately pushed online in order to stir conflict in a political space, both from outside the country and from within. It can also concern various computational tactics (such as bot work, fake likes and fake following) employed as a means to artificially amplify that same content.

Studying the presence of fakeness in the Dutch political space on Instagram

As few studies have described Instagram as fertile ground for the circulation of content with varying degrees of fakeness, particularly for the distribution of inflaming content in the form of memes, but also as a well-performing infrastructure for the artificial amplification of engagement. In this empirical research project, we devised three complementary approaches for the assessment of fakeness in the Dutch political Instagram: fakeness with respect to the content shared on the platform, fakeness of the most relevant information sources within the space, and fakeness in the sense of inauthentic followers that may generate fake engagement (see figure 1).

In the first part of this study, we search for fakeness in the content shared on the platform, by asking to what extent the most liked content in a demarcated Dutch political space on Instagram can be defined as fake or junk (i.e., disinformation, conspiracy, clickbait or hyperpartisan)?

Second, as the estimation of fakeness also can be made through “online source criticism”

(Rogers and Niederer, this volume), we expand the work by detecting fakeness on Instagram at a source level. Here, we study the mainstreaming of fake sources by exploring the affinity of the follower bases of Dutch political entities with those of fake (or junk) news providers (flagged as such by experts), and we ask, to what extent do Dutch political entities share an audience with fake (or junk) news sources on Instagram? Thirdly, in order to account as well for the tactics of artificial engagement that political parties and news sources may employ on Instagram to boost their content, we search for signs of inauthentic activity in the follower bases in the Dutch political space and we ask, to what extent are efforts of artificial boosting (by means of fake followers) present around divisive topics on the Dutch Instagram?

In this research we employ a “digital methods” approach (Rogers, 2013), repurposing Instagramspecific features to gather data from the platform. In particular, we compile a list of hashtags and profiles in order to demarcate the Dutch political space on Instagram. Within this space, we collect and analyze most liked posts (i.e., posts that receive a certain amount of likes) to study fakeness in the shared content. Furthermore, we collect followers of the accounts of Dutch political entities, mainstream news sources, and fake news sources, to study the intersection between their audiences, and more generally to assess the degree of fakeness in the Dutch political space at the level of the sources. Finally, we rely on profile features (such as the 150 characters bio in one’s profile, or post captions), to evaluate the amount of fake followers of the Dutch political space.

 102 Figure 1. Diagram of the research protocol, showing the type of hashtags and accounts used for querying

Instagram, and the tools used to collect, visualize and analyze the data.

Detecting fakeness in the most liked content

The aim of the following analysis is to identify engaging content in the Dutch Instagram political space and observe the extent to which this space is fake – in the sense of how much engagement is generated by content that is either disinformation, conspiracy, clickbait or hyperpartisan.

To outline the Dutch political space in Instagram, we compiled a list of hashtags (see figure 2) that are used on the social network to identify the leaders of Dutch political parties (e.g.,

#markrutte), the 2019 Dutch provincial elections (#PS2019) as well as politically charged issues such as climate change (#klimaatverandering). We used the Instagram Scraper tool, 26 offered by the Digital Methods Initiative, to collect the 1,000 most recent posts per hashtag (data collected between the 25th and 28th of March 2019), together with their metadata (date of the post, media URL, caption, number of comments and number of likes). For each hashtag we selected only the 20 most liked posts, manually filtering out posts that are not relevant to the search criteria, or identical posts that prevent more diverse results from reaching the top 20. 27

In this curated list of most liked posts, we conducted a close reading by looking at post captions and embedded media (images and videos) to understand how political party leaders and

26 The tool is available at this link: https://wiki.digitalmethods.net/Dmi/ToolInstagramScraper

27 For example, we filter out posts about the Slovak professional footballer Marek Hamšík, who plays with the number 17 and is referred to in Instagram with the same hashtag of the Malaysia Airlines Flight 17 (#MH17). We also do not include in the dataset the identical posts of condolence messages for the Utrecht attack posted by Dutch national football team players with the hashtag #Utrecht.

 103 politically charged topics are discussed within the limits of the Instagram Dutch political space, and specifically to flag the presence of fake content.

As a result of this evaluation (see figure 3), we found that out of the 400 most liked posts within our dataset there are (only) 45 posts that can be flagged as fake or junk, 4 satirical posts, and 351 posts that do not appear to be junk. Looking at the engagement generated by these posts, fake content was liked 79,466 times, satirical content 37,532 times, and non-fake content 838,794 times.

In figure 4, the 400 most liked posts are divided in hashtag-dedicated columns, in which they are also ranked from the most liked post in the first row to the least liked one in the last. Fake content is flagged using three different colors: light blue for hyperpartisan content, magenta for conspiracy, and blue for click-bait. Satirical posts are color-coded in dark blue. Finally, columns are ordered from left to right according to the amount of fake content, calculated on the total number of likes for each hashtag.

The analysis shows that the #zwartepiet, #geertwilders and #tunahankuzu hashtags represent the most divisive political spaces, with respectively 56.1%, 42.8% and 42.7% of their total amount of likes directed to fake or junk content. Moreover, we find that the majority of the posts flagged as such can be considered hyperpartisan, mostly supporting and/or opposing particular right-wing ideology or figures, while only one post can be considered as clickbait, and one conspiracy. Generally, we did not find any trace of disinformation linked the content that receive most likes. The findings suggest that certain issues or political leaders, such as the Zwarte Piet debate and the leader of Denk political party, Tunahan Kuzu, draw more divisive content than others. Of the 20 most liked posts, however, we found no strong presence of fakeness or junk.

Hashtags related to Hashtags related to

Dutch political party leaders politically charged discussions

#markrutte, #rutte, #geertwilders, #wilders, #PS2019, #klimaatverandering, #immigranten,

#thierrybaudet, #baudet, #jetten, #utrecht, #zwartepiet, #MH17,

#tunahankuzu, #jesseklaver, #lodewijkasscher, #24oktoberplein

#alexanderpechtold, #gertjansegers,

#sybrandbuma, #mariannethieme

Figure 2. Lists of hashtags pertaining to political leaders and politically charged discussions used to demarcate the

Dutch political space on Instagram around the 2019 provincial elections.

 104 Figure 3. Proportions of most liked content shared around the 2019 Dutch provincial elections, categorised as fake, satire, and not fake. Data source: Instagram Scraper; data collection: 25-28 March 2019; pie-charts.

In general, we found a relative scarcity of fake or junk content in this high-engagement political space. In the top results for the Dutch provincial elections, #PS2019, we found only positive content, either celebrating preliminary poll results or encouraging people to exercise their right to vote. The hashtags, #24oktoberplein and #utrecht, returned mainly condolence posts and the news that the attacker was spotted and arrested. Almost all of the content we considered as fake or junk is hyperpartisan, mainly about particular right-wing ideology or figures. We found no presence of disinformation in the most liked results within the demarcated political space.

In order to ascertain the presence of fakeness on Instagram surrounding the 2019 European

Parliamentary elections in the Netherlands, we conducted a second hashtag analysis concerning content posted in the months before the election day (23 May). With the goal of demarcating the Dutch political space around the 2019 European elections, we compiled a new list of hashtags

(see figure 6) used to identify Dutch political parties (e.g. #fvd) and their leaders (e.g.

#thierrybaudet), the European elections (#EUverkiezingen2019, #EUverkiezingen), and various politically charged issues such as immigration (#immigratie, #migratie, #immigranten) and climate change (#klimaat, #klimaatverandering).

 105 Figure 4. 20 most liked posts per hashtag shared around the 2019 Dutch provincial elections, sorted from right

(most fake) to left (least fake). Data source: Instagram Scraper; data collection: 25-28 March; Image wall.

With the Instagram Scraper tool, we collected the 1,000 most recent posts per hashtag (data collected on the 22nd of May) and their metadata. For each hashtag we only retained posts shared after the 28th of March, in order to focus the detection of fakeness on the period prior to the European election, but after that of the Dutch provincial elections. For each hashtag we selected the 20 most liked posts, excluding those included in the dataset but not relevant to the search criteria. 28 Subsequently, in order to ascertain the amount of fakeness in the dataset, we looked at embedded media and textual captions and flagged each post as fake or junk (making the distinction among disinformation, conspiracy, clickbait and hyperpartisan content).

The analysis (see figure 7) confirmed the relative lack of fake or junk content in the Dutch political space, also around the 2019 European parliamentary elections: out of 452 most liked posts, we found only 41 that can be considered junk (specifically hyperpartisan), counting for less than 10% of the total amount of posts. Moreover, hyperpartisan posts score low even in terms of engagement, generating only the 4.66% of likes out of the total amount.

28 The query for some of the less popular hashtags returned less than 20 posts in the specified date range.

 106 Figure 5. Example the posts flagged as hyperpartisan or satire. Data source: Instagram Scraper; data collection: 25-28 March; Image wall.

In figure 8, most liked posts are organized in hashtag-dedicated columns. Columns are grouped by type of hashtag and sorted from right to left according to the amount of likes generated by hyperpartisan content. The analysis shows that the hashtags related to political parties attracting more divisive content are #pvda and #christenunie, with respectively 27.6% and 25.1% of likes directed to hyperpartisan content. Compared to the dataset around the provincial elections,

Geert Wilders (#geertwilders, #wilders) remains the political leader receiving the highest percentage of likes directed to hyperpartisan content (28.4%), followed by Jesse Klaver

(#jesseklever) with 13.4%, who instead scored low in terms of fake content in the previous analysis. We did not find traces of hyperpartisan content in the most liked posts around other political leaders. As it was the case with the hashtags used to refer to the Dutch provincial elections, #EUverkiezingen2019 and #EUverkiezingen present mainly invitations to exercise the right to vote. Among the issues under study, #zwartepiet remain the most divisive one, with

22.9% of likes directed to hyperpartisan content.

In general, we did not find evident signs of fakeness in the most-liked content around the 2019 European elections, except for few hyperpartisan posts. The finding is aligned with that of the hashtag analysis conducted around the 2019 Dutch provincial elections.

 107 Hashtags related to Hashtags related to Hashtags related to

Dutch political party leaders Dutch political parties politically charged discussions

#markrutte, #rutte, #cdavandaag, #pvv, #duurzaamheid, #klimaat, #geertwilders, #wilders, #socialistischepartij, #pvda, #klimaatverandering, #thierrybaudet, #baudet, #christenunie, #immigratie, #migratie, #jesseklaver, #jetten, #partijvoordedieren, #immigranten, #mh17, #mariannethieme, #50pluspartij, #groenlinks, #zwartepiet #tunahankuzu #fvd, #stempiraat, #voltnederland, #d66, #degroenen

Figure 6. Lists of hashtags pertaining to political leaders and politically charged discussions used to demarcate the

Dutch political space on Instagram during the months before the 2019 European elections.

Figure 7. Proportions of most liked content shared around the 2019 European elections, categorised as fake/junk and not fake. Data source: Instagram Scraper; data collection: 22 May 2019; pie charts.

 108 Figure 8. 20 most liked posts per hashtag shared around the 2019 European elections, sorted from right (most fake) to left (least fake) and grouped by type (elections, issues, political leaders, and parties). Posts flagged as hyperpartisan are colored in red. Data source: Instagram Scraper; data collection: 22 May 2019; Image wall.

Follower ecologies and the fakeness of sources

In order to detect the relevance of fake news sources within the Dutch political space on

Instagram, and to assess whether and how much fake news sources are becoming mainstream, we studied the overlap between followers of Dutch political entities, mainstream news sites and

Dutch-language “junk news” sites. 29 Specifically, we asked, to what extent followers of false news providers are shared with those of Dutch political entities?

First, we demarcated the Dutch political space on Instagram, by compiling three lists of profiles: a list of Dutch political parties and their leaders, a list of Dutch mainstream media outlets, and the profiles of Dutch information sources flagged as fake in the expert list. We then used the

API Instagram Follower Collector by Phantombuster 30 to collect the follower list of each Instagram account, and then, by creating a co-follower network, we looked at the amount of shared followers between the political entities and the suspicious Dutch information sources from the expert list.

29 We use the list of sites flagged by the hoax-wijzer (www.hoax-wijzer.be), which was edited and enhanced by

University of Amsterdam researchers, and is dubbed the ‘expert list’.

30 Phantombuster is an API store that “provides ready-made cloud APIs to collect data from various social networks and improve marketing strategies” (phantombuster.com).

 109 In mapping the follower network of the Dutch political space, we found three distinct follower ecologies (see figure 9). First, an ecosystem of followers of mostly established mainstream news organizations, such as the Dutch public broadcasting station, NOS. The follower bases of these news organizations are the largest in the network, which suggests that the Dutch mainstream news providers are still more relevant that those flagged as junk, at least in terms of follower count. Few sites from the expert list are close to (or part of) the cluster of mainstream news organizations, due to a relative high number of shared followers. Shared followers among mainstream news organizations and suspicious news sites may indeed suggest a special affinity among them, or rather be the signal of the mainstreaming of fake news providers.

A second ecosystem is made up of political parties and their youth organisations. The distribution of parties is laid out from left-wing parties to right-wing parties, whilst still being tightly clustered together. This may suggest that most followers either follow multiple parties on the same side of the political spectrum or follow all political parties regardless of political leaning. What can also be observed is the relative distance of the cluster of political parties to that of news organizations, suggesting that followers of political entities are mostly not shared with those of news organisations.

A third cluster is made up of far right-wing political entities, which are far from other political entities, closer to few hyperpartisan or clickbait sites and to few, less established, mainstream news providers. Within this cluster, the account of PVV leader Geert Wilders is surrounded by

GeenStijl, a tendentious, right leaning mainstream ‘shock blog’ and PowNed, the public broadcasting station that is an offshoot of GeenStijl. The official profile of FvD (Forum for

Democracy) and the youth organization of the same party are even more distant and isolated from other parties: they are surrounded by individual political commentators and share a high number of followers with the hyperpartisan news site, De Dagelijkse Standaard. This topology may suggest that although these parties and personalities share some followers with those from other sides of the political spectrum, they are mostly on their own and produce content consumed by a unique audience.

Fake followers and artificial engagement

In order to profile the follower base of the previously demarcated Dutch political space, we feed each account 31 (of political entities, but also of mainstream media, and of those from the expert list) in the HypeAuditor tool to check the authenticity of the accounts and look for signs of artificial boosting and fake followers. With HypeAuditor one can profile an Instagram account to determine the authenticity of its follower base. To assess the extent to which Dutch political accounts are employing artificial engagement tactics, we use reports from HypeAuditor, regarding the percentage of real followers, and their geographical origin. 32 The percentage of fake followers

31 HypeAuditor analyzes only accounts with more than 1,000 followers. For this reason, we limited the detection of fakeness to the accounts with more than 1,000 followers.

32 According to HypeAuditor, the geographical origin of one follower base is detected by analysing profiles biographies and place names in post captions (twitter.com/hypeauditor/status/1077143110432538624).

 110 returned by the tool is then used to rank each account from less fake to more fake (see figure

10). Furthermore, we zoomed in on those accounts with a higher percentage of fake followers, to observe their geographical provenance (paying particular attention to suspicious countries), as well as the segmentation of the follower base provided by HypeAuditor, which breaks down followers in “real people”, “influencers”, “mass followers” and “suspicious accounts” (see figure 11).

Figure 9. Follower ecologies in the Dutch political space, visualized as a co-follower network and manually annotated. In the network, accounts with higher amounts of shared followers (pink) are placed closer to each other. Data source: Phantombuster; data collection: 25-28 March; network graph.

Generally, we found that the majority of profiles do not have a suspicious follower base, with most accounts scoring higher than 70% in the real follower metrics provided by the tool. There are some accounts, however, that are suspect of having a fake follower base. For instance, the media entity Powned has 32.6% of suspicious followers. The clickbait site Prankster also scores

 111 relatively high in terms of fake following. Within the group of political entities, the personal account of Mark Rutte and the account of Geert Wilders have the highest number of fake followers. Strikingly, the “work” account of the prime minister, Mark Rutte, has a lower percentage of fake followers than that of his personal account. On the other hand, the account for the political party, Christenunie, has hardly any fake followers, just as the SGP (Reformed

Political Party) and that of the minister Gert-Jan Segers.

When we look closer to the nationality of the follower bases, we found no suspicious results, with most of the accounts followed by users based in The Netherlands. For both of Mark

Rutte’s accounts, the followers are mostly based in the Netherlands. On the contrary, Geert

Wilders account has 36% of his followers from Brazil. This raises some questions regarding the legitimacy of Geert Wilders’ follower base, for Brazil is often mentioned as one location that can signal the presence of fake followers (Maheshwari, 2018).

In all the follower analysis does not show an organised effort of artificial boosting within the Dutch political Instagram sphere, and it indicates, with the exception of Geert Wilders, a rather authentic follower base.

Figure 10. Degree of account fakeness according to report by the HypeAuditor tool. Accounts on the further right have more suspected ‘fake followers’ than accounts on the right side of the graphs. Data source: HypeAuditor; data collection: 25-28 March 2019; beeswarm plot.

 112 Figure 11. Visualisation of the follower base of Mark Rutte’s personal and work accounts and Geert Wilders account, based on results from the HypeAuditor tool. Each follower base is segmented based on “audience type” and geographical provenance. Popular suspicious countries, that may suggest an inauthentic follower base, are colored in red. Data source: HypeAuditor; data collection: 25-28 March 2019; pie charts.

Conclusions: Findings and limitations

The goal of the present research is to detect signs of fakeness in the Dutch political Instagram sphere. More generally, it can be considered an attempt at applying to the Dutch context the argument in the New Knowledge report (2018) that Instagram performs well in terms of fake content circulation and artificial amplification strategies. It also takes up the invitation from the

NRC Handelsblad study to inquire into other platforms than Facebook and Twitter for disinformation campaigning and computational propaganda.

The presence of fakeness (or lack of thereof) has been studied on three levels: at the story level (by looking at the circulation of fake content in high-engagement political spaces on Instagram); at the source level (by looking at the intersection between the follower bases of Dutch political entities and that of news sources flagged as junk); and also through the detection of artificial engagement tactics, specifically fake followers, among the profiles of Dutch political entities as well as Dutch information sources.

In general, we found a rather healthy political space, with only few evident signs of fakeness.

Most liked content in the Dutch political space proved to be fake to a very small degree,

 113 although we found a small amount of hyperpartisan and divisive content centered around rightwing figures and issues around the 2019 Dutch provincial and European elections. With respect to the alignment of the audience of Dutch political parties with that of (mainstream or fake) news providers, we found mainstream news organizations to be still more relevant in this political space, somehow confirming the argument that in the Netherlands “the vast majority of news consumption remains of the mainstream sources” (Rogers and Niederer 2019, this volume). Furthermore, the analysis of the follower base of Dutch political entities (and that of news sites, both mainstream and fake) revealed an apparent authentic audience with almost no signs of artificial engagement.

Within a relatively healthy political spectrum, it is at the extremes that fakeness surfaces. With the current research we have pointed out a special affinity between far right-wing political entities and some information sources that may be defined as fake and junk (or at least hyperpartisan). Furthermore, the few indications of artificial engagement we have found are located at the far right of the political spectrum, with Geert Wilders account being the most suspected of inauthentic activity.

In the co-follower analysis, we found right-wing extremes to have a unique follower base, not shared with other parties or mainstream news sites. Right-wing political entities are also relatively closer (in terms of shared followers) to suspicious sources (a few of them flagged by the expert list). Above all, Geert Wilders account is the closest (according to shared followers) to hyperpartisan news sources. Relatedly, Geert Wilders’ account is the only one of those under study that may reveal signs of artificial engagement, as suggested by a geographically dubious follower base. This finding resonates with the 2015 scandal about a suspicious increase of the follower count of Geert Wilders Twitter profile. In addition, the already mentioned comparative study of social media manipulation strategies by the Oxford Internet Institute (Bradshaw and

Howard, 2018) also refers to Geert Wilders as making use of various artificial boosting strategies in The Netherlands, reporting on an analysis by a social media analytics firm that in February

2016 found 26 fake accounts amplifying the #geertwilders hashtag on Twitter.

The determination of the relative absence of fake content, dubious sources and fake followers in the scope of the current research has a series of methodological limitations. First, in the search for fakeness in the shared contents, we collected data based on a limited list of hashtags related to Dutch politicians and controversial topics. One could repeat the analysis to include other politically charged issues. Furthermore, we have considered only the top 20 most liked posts per hashtag, whereas we could have also counted the number of comments per posts to analyze most-engaged-with content. Moreover, we could have included in the analysis a larger set of posts that do not necessarily make it to the top (because they receive fewer likes, or have less comments), in order to evaluate the presence of fakeness in less-engaged-with spaces. In addition, given that for data collection we made use of the DMI Instagram Scraper, which “scrapes Instagram to retrieve posts” (Digital Methods Initiative, 2019), this research is dependent on the limits of such scraping, including Instagram’s rate limits which are not documented and unknown security challenges (Instaloader, 2019). It is also not a platform that invites research through scraping. As others have pointed out, social media platforms are designed to increase a platform commercial value, rather than to meet researchers’ needs (Borra and Rieder, 2014). To

 114 overcome the limitations, one could use additional tools for data collection and compile a richer data set.

Secondly, we established the fakeness in the Dutch political follower base using the metrics provided by one single tool (HypeAuditor). We could have compared the results with those by other similar services (and auditing the auditors, so to speak). Moreover, we searched for signs of inflated engagement in the Dutch political space only by looking at followers’ demographics, while we could have paid attention to other signals such as patterns of repetition in posts comments. For example, to account for other tactics of artificial engagement on Instagram, one could perform a co-hashtag analysis 33 in a demarcated issue space, and detect signs of (semiautomatic) boosting, such as the use of long list of popular unrelated hashtags, deliberately added in the post captions to increase content visibility. 34 Moreover, one could trace back the users involved in this activity and profile them in order to evaluate their authenticity.

33 In addition to the most recent lists of posts, the Instagram Scraper tool returns a network of hashtag co-occurrence, that is a file that contains the hashtags used at least once together with the hashtag under study. For each pair of hashtags, the tool returns a numeric value representing the total number of posts in which the two hashtags appear together in the data set. A similar approach is largely used for empirical research on Twitter: with co-hashtag analysis one can get a sense of the relationship between sub-topics in a conversation (Borra and Rieder, 2014); or find additional and/or more “significant hashtags” (Rogers 2017) to be queried to expand a corpus of data; or spot hashtags practices aimed at enhancing the visibility of particular content (Wang et al. 2016), or overturning its original meaning through hashtag hijacking practices (Berg, 2017).

34 Unlike Twitter, which has a character limit of 280 characters, Instagram character limit is 2,200 characters, and users can include up to 30 hashtags in the caption and comment sections of the post. This results in certain users adding blocks of more or less related hashtags to the posts to enhance their visibility. Even if Instagram is applying countermeasures to block the use of certain hashtags (Drewe, 2016), there are several websites that provide lists of safe and popular hashtags that users can copy paste directly in their posts (for example tagblender.net).

 115

DUTCH JUNK NEWS ON REDDIT AND 4CHAN/POL/

Sal Hagen and Emilija Jokubauskaite

Dutch Junk News on Reddit and 4chan/pol/

Sal Hagen and Emilija Jokubauskaite 35

Introduction: The understudied, deep vernacular Web

Recent debates on online fake news and disinformation have largely been discussed with respect to the social media behemoths in the context of a “platformized” internet ecosystem (Helmond, 2018), with Facebook, Twitter, Instagram and YouTube in the spotlight. It is not without reason; given their gigantic user bases, open publishing and micro-targeting, they are vulnerable to disinformation campaigns and dubious information, not so unlike the Web itself. Regardless, they do not exist in a vacuum. On the fringes of the Web, yet difficult to characterise as marginal, are pseudonymous or anonymous platforms like Reddit and 4chan. Instead of publicfacing “e-celebs” or otherwise identifiable accounts, these spaces are characterised by “masked” users with distinctive subcultural styles, vernaculars and iconographies. The pseudonymous and anonymous users on Reddit and 4chan do not only congregate around shared interests or common goals, but also – and in some cases predominantly – around a deep understanding of shared subcultural knowledge and norms. The unconventional and sometimes downright esoteric cultural productions some of these groups create feed into community members’ selfimagination as “underground”, “countercultural”, or “internet native”. 4chan and (parts of)

Reddit can be associated with the term “deep vernacular Web” (Tuters and De Zeeuw, 2019), referring to online discussion forums that lack stable user identities and whose masked participants frequently transgress the boundaries of “mainstream” conventions, often through an entangled mix of sincere ideology and ironic play.

While Facebook, Twitter, Instagram and YouTube have already been studied in relation to issues of “fake news”, the abovementioned “virality-oriented subcultures” of the deep vernacular Web are also said to play a “crucial role in the system” of the circulation of various types of “junk news” (Venturini, 2019). 4chan and certain parts of Reddit have indeed been characterised as hotbeds for disinformation (Shiebel, 2017; Collins and Russell, 2018; Lagorio-Shafkin, 2018), trolling campaigns (Phillips, 2015), and conspiracy theories (Marwick and Lewis, 2017; Tuters et al., 2018). Despite their relatively marginal number compared to more mainstream platforms, users of 4chan and areas of Reddit are considered particularly skilled in “setting the agenda” of broader news media (Phillips, 2018). In a 2017 report, Marwick and Lewis highlight how an underground current of Internet subcultures associated with 4chan and Reddit “take advantage of the current media ecosystem to manipulate news frames, set agendas, and propagate ideas”

(Phillips, 2018: 1). In a more recent report, Phillips builds on this research by exploring how and why the false narratives of these online antagonists were amplified by major U.S. news outlets

(2018). As she identifies, journalists were keen on reporting the narratives with false information or dark undertones partly because of a fascination with their bizarre cultural phenomena or simply due to a lack of time required to decipher their problematic code language. The reporting, she argues, “amplifies” their overall presence. By 2019, there are now well-known byproducts of this cycle of the normalisation of false content emerging from fringe online spaces. To provide but one example, the “Pizzagate” conspiracy theory, originating on 4chan, presumed the

Clintons were maintaining a child sex trafficking ring, which led to media coverage and an actual shooting in a U.S. pizza parlour (Tokmetzis, 2018; Tuters et al., 2018). A related conspiracy theory, “QAnon”, gradually spread from 4chan to Reddit and mainstream news sources (Hagen

35 The research team includes Lucie Chateau, Gabriele Colombo, Ognjan Denkovski, Carmen Ferri and Holly

Foxton.

 116 et al., 2019), even sprouting international support groups including “QAnon Nederland”

(QAnon Netherlands).

The influence of fringe internet subcultures on the news ecosystem has mostly been scrutinised in relation to English-language spheres and U.S. politics. As such, it remains unclear to what extent the propagation of problematic content from the “deep vernacular Web” affects other news ecosystems such as the Dutch. Such activity already has caught the attention of Dutch media outlets. For example, the QAnon conspiracy was covered by major outlets like RTL

Nieuws (2018) and Algemeen Dagblad (Van Huet, 2018), while De Correspondent untangled the related Pizzagate conspiracy in some depth (Tokmetzis, 2018). De Volkskrant discussed Dutch users active in the far-right “politically incorrect” subforum of 4chan, /pol/, by observing an increasing prevalence of anti-Semitic conspiracies (Kranenberg and Bahara, 2018). In a broader sense, conspiratorial rhetoric native to the deep vernacular Web seems to be normalising in the

Dutch political and media discourse at large. For instance, the concept of “cultural Marxism” has increasingly appeared in Dutch news media (Van den Bos, 2018). It concerns a theory assuming a Marxist and/or Jewish network pulling the strings of European institutions – a narrative particularly popular on 4chan/pol/. Dutch politicians have subsequently flirted with such sweeping theories. For instance, the party Forum voor Democratie tweeted that Mark Rutte was a puppet of the Jewish philanthropist George Soros, 36 while the party’s leader Thierry Baudet supported the most conspiratorial aspects of the “cultuurmarxisme” debate, tweeting that the

European Union is “a cultural Marxist project aiming to destroy European civilisation”. 37 NOS, the public broadcaster, subsequently published an article framing George Soros as an “influential meddler with tentacles deep in world politics”, 38 which was later withdrawn after heavy criticism pointing out the framing’s commonalities with anti-Semitic rhetoric (Peek, 2018). While direct ties between such mainstream attention and fringe internet platform rhetoric are not to be drawn, each incident in its own right could be situated in the aforementioned dynamics of amplification in a Dutch context.

The falsehoods cooked up on the deep vernacular Web are hard to grasp through the concepts of “disinformation” or “fake news”, since their “fakeness” is broader than deliberately coordinated campaigns or clearly false information. Rather, they speak to what Muirhead and

Rosenblum (2019) call a “new conspiracism”, whereby sweeping accusations are made independent of evidence or coherent explanations, and complex phenomena are “explained” through “conspiracy without theory”. Such conspiracism is said to be dangerous since it delegitimizes the knowledge-making institutions at the foundations of democratic societies

(Muirhead and Rosenblum, 2019). Importantly, this conspiracism can be fueled by or work alongside a mix of foreign interferers, sincere believers and hyperpartisan actors.

To understand these broad range of actors and interests that stimulate the emergence of problematic information, the concept of “junk news” (or “pulpnieuws”) is more apt. Junk news shifts the focus from clear and coordinated falsehoods towards a broader notion of news crafted to be engaged with and to circulate, which, in turn, stimulates polarising or “simple” information that “saturates public debate” (Venturini, 2019). Junk news thereby forms an umbrella term for

36 The original Dutch tweet by Forum voor Democratie notes: “@MinPres [i.e. Mark Rutte] draait er niet eens meer omheen: De belangen van NDO’s (lees: Soros) gaan boven het beleid van de democratisch gekozen regering van

#Hongarije. Hoogste tijd dat deze loopjongen van het grootkapitaal nu van het toneel verdwijnt. Reken af met

#Rutte op 20 maart! Stem #FVD” (@fvdemocratie, 14 Sep. 2018).

37 The original Dutch tweet by Baudet noted: “Omdat de Europese Unie een cultuurmarxistisch project is dat tot doel heeft de vernietiging van de Europese beschaving” (@thierrybaudet, 19 Aug. 2017).

38 In their original article, NOS used the title “George Soros: invloedrijke bemoeial met tentakels ver in de wereldpolitiek” and noted: “De jood Soros steunt organisaties die regeringen openlijk bekritiseren [...]. Dat moet stoppen, zeggen tegenstanders” (Peek 2018).

 117 conspiracies, hyperpartisan slander, “ironic” falsehoods, low-effort clickbait articles, as well as deliberate disinformation. The circulation of these types of junk news has a plethora of reasons, but, as noted, “tightly-knit communities” (Zannettou 2017) and “virality-oriented” subcultures creating and engaging with this highly “shareable” content are said to be a crucial factor in their effectiveness (Venturini 2019).

Much has been said about the grassroots production of false narratives within spaces like Reddit and 4chan (Marwick and Lewis, 2017; Phillips, 2018; Tuters et al., 2018; Benkler et al., 2018).

However, a more elementary question is usually left untouched: what kinds of news sources do these actors rely on themselves? Zannettou et al. (2017) found that “‘fringe’ communities often succeed in spreading alternative news to mainstream social networks and the greater Web” (1), employing a statistical model (Hawkes process), indicating that fairly marginal spaces like

Reddit’s pro-Trump subforum r/The_Donald and 4chan’s /pol/ board are often first to post a URL to alternative news, that only later catches attention on Twitter. They furthermore traced which alternative sources where shared on Reddit, 4chan, and Twitter, showing that alternative news was shared more often on 4chan/pol/ and select subreddits than on Twitter, noting the popular use of breitbart.com, rt.com, infowars.com, and sputniknews.com across the three platforms.

What about junk news in a Dutch context? Is there a Dutch alternative “junk news” network within the Deep vernacular web, or do these platforms mostly rely on mainstream sources? If found, how vast is the presence of Dutch junk news on these spaces? In identifying linked-to websites, can signs of coordinated disinformation campaigns be discerned? Or are the types of junk news shared mostly hyperpartisan, clickbait, or some other “junk” category? These questions are of interest when applied to fringe and “extreme” spaces like 4chan/pol/ but can also aid in positioning more widely used yet still “alternative” spaces, like the largest Dutch subreddit, r/thenetherlands. The research reported here thereby begins with the question, where does Dutch junk news appear (if at all) on Reddit and 4chan/pol/? Subsequently, it asks, what kinds of junk news resonate? It concludes with a brief section on YouTube as a possible alternative news network by following the links to Google’s video platform.

Demarcating the Dutch spaces and Dutch junk news in the deep vernacular Web

Case studies: Reddit and 4chan

For Dutch cases of virality-oriented subcultures, we focus on Reddit and 4chan/pol/. Although less known than the likes of Twitter, Facebook, Instagram and YouTube, Reddit is one of the largest discussion sites globally, with Alexa metrics currently showing 234 million unique visitors per month. The platform is divided into different subreddits dedicated to the discussion of specific topics, such as r/tennis or r/politics. Posts on these subreddits can be “upvoted” or

“downvoted” by users. The higher the post’s score (upvotes minus downvotes), the higher it is placed in a ranked list of content and the more visibility it gains. In the comment section underneath every post, “redditors” discuss, debate, or simply joke around. Reddit’s Dutch user base seems to be growing (as is shown below), with the largest Dutch subreddit r/thenetherlands amounting to 236,000 “subscribers” at the time of writing. Its growing popularity makes it an increasingly important object of study in a Dutch context. This is heightened by the fact that

Reddit has been identified as a target of multiple Russian disinformation campaigns, with “at least a hundred” IRA accounts influencing the 2016 U.S. elections and campaigns continuing into late 2018 (Collins and Russel 2018; Lagorio-Shafkin 2018). Exploring whether such

 118 campaigns have also transpired within Dutch spheres of Reddit is thus part of the objective of this research.

Figure 1 . The frontpage of Reddit (retrieved 11-Jun-2019).

The second case study is 4chan, an infamous imageboard where users post anonymously within one of its subforums (called boards) dedicated to different topics like videogames or fitness.

4chan is ephemeral, meaning posts are deleted from the site after a few days or even hours. It is a visual environment conducive to the production of viral content and generation of junk news

(Venturini 2019). The space’s creativity extends beyond the generation of alternative theories, as 4chan is also infamous as the “birthplace of internet memes”, as well as a hotbed for nebulous political movements. The latter include “Anonymous”, the loose “masked” collective of geeks and hackers infamous for trolling and DDOSing the likes of the Church of Scientology and

MasterCard (Coleman 2014), as well as more recently the “alt-right”, once characterised as an

“amalgam of conspiracy theorists, techno-libertarians, white nationalists, Men’s Rights advocates, trolls, anti-feminists, anti-immigration activists, and bored young people” (Marwick and Lewis

2017, 3) but now arguably pertaining to the extreme side of those far-right actors. For this research, we chose to focus squarely on 4chan’s politics board, /pol/. This is the most relevant board in relation to the research questions, for it is currently among the most active boards on the website 39 and is a fertile ground for conspiracy theories (Tuters et al., 2018) and alternative news sources (Zannettou, 2017). 4chan/pol/ is a far-right space, identified as a recruitment zone for neo-Nazis (Wendling, 2018) and connected to various acts of extreme violence (Hankes and

Amend, 2018). This partisanship naturally affects the types of news shared on this platform. For balance, other partisan areas of the deep vernacular Web were also considered (e.g.,

8chan/leftypol/) but were ultimately found too insignificant in terms of Dutch activity.

39 At the time of writing, the website 4stats.io, which tracks activity on each 4chan board, lists /pol/ and /v/ (video games) as the most active boards, with almost 50 posts per minute and 120 thousand posts per day (taking the last 4 weeks as a benchmark). These numbers are supported by metrics from our own tools (Peeters and Hagen 2018).

 119 Figure 2. The index page of 4chan/pol/ (retrieved 11-Jun-2019).

Tools and timeframe

In contrast to mainstream platforms like Facebook and Twitter, data from Reddit and 4chan is rather accessible. For most of the data collection, we used 4CAT (Peeters and Hagen, 2018), a tool developed by the Digital Methods Initiative that captures data from a variety of sources, including 4chan/pol/ since November 2013. For Reddit, 4CAT makes uses of the Pushshift

API, which allows access to an archive of nearly all Reddit posts and comments (Baumgartner, 2018).

We chose a timeframe from 1 December 2015 up to 1 June 2019, spanning 4 1/2 years in total. Whereas most other studies in this volume present timeframes based on specific events, this long-term timeframe is more suitable here for multiple reasons. Firstly, as we will show, the activity in relation to junk news posting on these platforms was shown to be fairly marginal in comparison to more mainstream social media websites. A larger timeframe thereby aids to arrive at patterns in this relatively small stream of data. Secondly, to the best of our knowledge, it is the first time the presence of questionable Dutch news is researched on these platforms, so it makes sense to start with a high-level perspective on the object of study instead of limiting it to a particular case. Moreover, this timeframe includes a variety of major political events in the

Netherlands, including the general elections in 2017 and more recently the provincial and

European parliamentary elections in 2019.

Analyses: Haystack to needle and needle to haystack

As the research focuses on the presence of junk news linked to on Reddit and 4chan/pol/, it takes URLs as the primary research objects. To provide an overview of the types of news linked to, we decided to focus on domain names (sources) instead of links to individual articles (stories). To identify and categorise domains, we used two related approaches, referred to metaphorically as ‘haystack to needle’ and ‘needle to haystack’. The haystack to needle approach denotes a macro to micro inquiry where all domains posted were categorised in order to subsequently identify the presence of Dutch junk news within this larger pool of data. The needle to haystack

 120 does the reverse and starts from an expert list of Dutch junk news domains 40 and subsequently enquires into what is shared most often, where these sources appear, and, for Reddit, what kinds of users post them. The next two subsections describe these approaches in more depth.

Haystack to Needle

The haystack to needle approach moves from a high-level overview to the categorisation of particular linked-to domains, specifically by parsing (1) news from non-news, (2) Dutch news from non-Dutch news, and (3) types of junk news (mainstream/junk and types of junk). To do so, a Dutch sphere first had to be defined for Reddit and 4chan/pol/ from which an initial list of domains could be extracted. For Reddit, the full dataset of opening posts was filtered for a list of Dutch subreddits (thus excluding comments, for URLs appear less here and are less visible). The relevant subreddits were compiled from a set of “related communities” posted by administrators of r/thenetherlands 41 and supplemented through querying Dutch issues on Reddit. This resulted in a final collection of 182 subreddits (see Appendix I). On 4chan/pol/, all posts show a flag icon indicating the location of the IP address of the poster. To identify a Dutch sphere on /pol/, all posts with a country flag of the Netherlands were extracted. It is important to note that this only results in a partial sample of Dutch posters, since users can also choose to display a custom flag (like “Hippie”) instead of one based on geolocation, or they can spoof their IP addresses. In all, the dataset collected consists of over 2 million posts with Dutch country flags, forming a large enough sample to gauge the presence of Dutch junk news using the haystack to needle approach.

Having demarcated Dutch spheres on Reddit and 4chan/pol/, domains from URLs posted were extracted from all posts. For Reddit, this resulted in 3,489 unique domains. To qualitatively analyse a sample of this large dataset, the domains that were posted five times or more were retained. This resulted in a list of 372 domains. Similarly, domain names from the Dutch

4chan/pol/ posts were extracted using 4CAT, yielding 8,048 domains. 42 To arrive at a comparative sample, we kept the domains that were posted twenty times or more, resulting in

352 unique domains.

The two lists were then categorised according to whether (1) the domains were news websites,

• (2) 

  they concerned either news websites in the Dutch language or Dutch affairs, and (3) the category of news websites they would fall in. “News websites” here refers to a fairly broad selection of websites focusing on the production of news and opinionated columns which contain a section dedicated to timely updates. They include blogs on current affairs, special interest news, and websites of TV news programmes. Thereafter, the news sources were categorised as follows:

  ● Mainstream: Reporting by “established” general news outlets with a predominantly neutral tone of voice.

  ● Other mainstream: All other mainstream news websites concerning special interests, such as business or sports news.

  ● Disinformation: Sources deliberately publishing false information, often with harmful intention, and part of a network or campaign.

40 The expert list is comprised of an original list by De Hoax-Wijzer, edited to remove inactive sources, with additional sites added through qualitative analysis by University of Amsterdam researchers.

41 See: https://www.reddit.com/r/theNetherlands/wiki/related. Accessed 25 March 2019

42 This is a higher number than for Reddit because for 4chan, not only the first posts in a thread were kept, but also the replies since this matches 4chan’s infrastructure of more “horizontal” conversational threads.

 121 ● Hyperpartisan: Extremely coloured and “openly ideological” reporting and

editorialising from a far end of the political spectrum (Herrman 2016).

● Clickbait: Sources consisting mainly of articles with sensational headlines and gossip, often in the form of cliffhangers and listicles, with a financial incentive to gain

advertising revenue.

● Conspiracy: Sources mainly dedicated to propagating a range of explanations to events behind which are secret plots and multiple actor entanglements.

Two researchers categorised the domains and discussed the debatable cases with other researchers in this volume for higher intercoder reliability and used external sources like mediabiasfactcheck.com. Mostly, these discussions were held for websites that could be categorised with multiple labels or that fall between hyperpartisan and mainstream such as tendentious ones (Peeters and Rogers, this volume). A caveat to this method is that categorising websites on a source instead of story level results in stories being labelled, for example, as

“hyperpartisan”, even though the categorisation would differ on a story-by-story basis. Websites like The Post Online, for instance, contain stories from press agencies as well as tendentious and hyperpartisan ones. Despite this, the rigorous domain categorisation did allow preliminary overviews, which is why it was fitting for the “bird’s-eye” perspective of this research. To show these different categorisations (news or non-news, Dutch or non-Dutch, types of news), they were visualised in treemap diagrams using the software RAWGraphs (Mauri et al., 2017).

Needle to Haystack

Next, the needle to haystack approach was employed to analyse the prevalence of Dutch junk news in the entirety of Reddit and 4chan/pol/, now by starting with a list of URLs that were already identified as questionable. This list was constructed by combining an edited list by De Hoax

Wijzer (“Valse Nieuwssites”, n.d.) with websites found through engagement analysis by researchers in this volume (see Appendix II below). The list refers to Dutch domains known to present news of questionable validity, with an overwhelming partisan tone but also occasionally showing traits of conspiracism. The list was coded by the researchers who compiled it and contains the categories hyperpartisan, clickbait, conspiracy, and disinformation. 43 We fetched all the posts containing these domains with 4CAT, resulting in 1714 posts on Reddit and 443 on

4chan/pol/.

We then “scoped” how often junk news appears over time, plotting it as histograms. To compare these junk posts to all of Reddit, Google BigQuery was used. The total number of posts within subreddits where at least one Dutch junk domain appears was fetched to calculate the relative presence of junk news. Additionally, the “size” of the Dutch Reddit sphere and the entirety of Reddit was retrieved through fetching the total number of posts on Dutch subreddits and on Reddit overall. The data was mapped as circle-pack diagrams with RAWGraphs. For

4chan/pol/, we used 4CAT to fetch all posts (both opening posts and replies) mentioning one of the domain names from the expert list in the full timeframe. In order to identify temporal trends, the amount of posts with Dutch junk domains was plotted per month as histograms.

43 It also has the category tendentious-hyperpartisan, which seeks to capture sources like The Post Online that have stories from press agencies as well as hyperpartisan columns and other contributions that could be described as

“edgy”, anti-establishment and against political correctness. (Tutors, this volume).

 122 Characterising junk news propagation on Reddit

Finally, to characterise the kinds of actors propagating Dutch junk news and the effectiveness of their activities, various metrics were calculated for (further anonymised) junk news posters on

Reddit. A similar analysis was impossible for 4chan/pol/ owing to the imageboard’s anonymity and lack of “repurposable” objects to shine light on the posters. Taking the needle to haystack approach, 4CAT and the Pushshift API were used to retrieve all posts by Reddit accounts who posted a source from the expert list at least twice. The retrieved users were considered “junk news propagators” for the purposes of this research. The following metrics were calculated for the total corpus as well as for individual users:

● Subreddits most posted often in. ● Average score of all posts. ● Average score of posts referring to Dutch junk news domain. ● Most linked-to domains. ● Total posts with domains to Dutch junk domains. ● Percentage of posts linking to Dutch junk domains. ● Total posts by user.

The Reddit users’ pseudonyms were further anonymised, since not the identity but rather the characteristics of the users is of importance here. The first four metrics in the list above were plotted for the whole corpus in histograms and circle diagrams, while all metrics were also visualised in a matrix for the ten most active Dutch junk news posters, i.e., those who linked to the domains from the expert list most often.

Following and categorising YouTube links

YouTube emerged as one of the most popular websites linked to on “Dutch Reddit” and

“Dutch 4chan/pol/”. Since the video platform is often described as offering alternative news consumption, we also followed the links to YouTube videos in all posts in Dutch subreddits and 4chan/pol/ posts with a country flag of the Netherlands. Having collected these links, we used

4CAT’s “YouTube metadata” module (in turn using YouTube’s API) to retrieve metadata on the videos linked to, such as video title, views, and topics. We then plotted the thumbnails of the

1008 44 videos that were linked to most often on image walls with a custom Python script. To visualise what types of videos these concerned, we plotted YouTube’s “video categories”

(selected by the uploaders) on top of the image wall. We finally ranked the most-linked to

YouTube’s channels, derived from the full list of videos linked to on 4chan and Reddit, to gain a grasp of the type of video content posted.

Scoping Dutch junk news

This section explores the scope of Dutch junk news on both platforms under study. We do so by showing the volume of posts linking to one of the URLs in the expert list (i.e., the needle to haystack approach). These are then compared to the overall volume of (Dutch) posts on Reddit and 4chan/pol/.

44 We settled on the peculiar number of 1008 since we wanted a large sample and this number would make the image wall adhere to the common 18:9 screen aspect ratio.

 123 Scoping junk news on Reddit

First, as a way to ground the corpus, figure 3 shows the total amount of posts made on one of the Dutch subreddits (appendix I). Just like activity on Reddit in general, Dutch activity is increasing: in December 2015 there were just over 2,000 posts and comments per month, whereas in January 2019 this number had grown to 14,000 and seems to be rising.

Figure 3. Total amount of posts and comments on one of the Dutch subreddits (appendix I).

Data source: 4CAT and Puhshift; timeframe: from 1-Dec-2015 to 1-Jun-2019; line graph.

Does this increase in activity also mean an increase in Dutch junk news linked to on Dutch subreddits? As is evident in figure 4, the amount of posts linking to one of the domains from the expert list started at a maximum of just eighteen instances in 2016. Two subsequent spikes can be observed. The first one, in April to July 2017, speak to the “spammy” nature of some areas of Reddit, since one user frequently posted a Dutch junk news domain (ninefornews.nl) to an

English subreddit. The second spike is more varied, however, showing a range of websites like boinnk.nl, worldunity.me, and ninefornews.nl. Upon closer inspection, these were again posted by a single account, mrthirdeye, the closest one will find to a “fake news troll”, though its posts received little to no engagement (discussed in more detail in section 4). The subsequent dip in

November can be attributed to a content policy change in 2017, possibly leading to the banning of this malicious account (Alexander 2017). In subsequent months, junk news sharing increased somewhat compared to 2016 but remained fairly consistent with around fifty instances per month. An increase of posts linking to a tendentious-hyperpartisan website, The Post Online, can be seen in 2019, although no significant spikes during the 2019 Dutch provincial elections and

2019 European elections can be discerned. In perspective, these numbers do not seem extremely troubling, especially since most posts link to hyperpartisan sources instead of outright disinformation (see section 3), and furthermore do not receive a lot of engagement (see section

4).

 124 Figure 4. Frequency of posts linking to Dutch junk news domains on Reddit.

Data source: Google BigQuery; timeframe: from 1-Dec-2015 to 1-Jun-2019; Stream graph.

To further put the scope of Dutch junk news on Reddit in perspective, figures 5 to 8 contain circle pack diagrams that show its amount compared to the entirety of Reddit, as measured in terms of posting activity (excluding comments). Figure 5 shows the size of all subreddits where a link to a Dutch junk news source was shared at least once. The Dutch subreddits are tiny in comparison to non-Dutch subreddits (figure 5), given the dominance of English-language subreddits on the site. There are a few occasions when Dutch junk news was shared on very large subreddits, such as r/viral, r/news, and r/worldnews, as well as the infamous pro-Trump subreddit, The_Donald. Notably, however, in the Dutch subreddits, the proportion of junk news is very low as well. As will be touched on, only occasionally does a subreddit have over 5% of its posts linking to Dutch junk news. For the majority of subreddits, this figure is less than 1%.

 125 Figure 5. Dutch versus non Dutch subreddits in which Dutch junk news appears. Size of circle represents the overall number of posts in that subreddit within the timeframe, and colour represents the relative amount of posts with junk news. Data source: Google BigQuery. Timeframe: 1-Dec-2015 to 31-Jan-2019. Visualisation: circle pack diagram.

When zooming in on Dutch subreddits (figure 6), a clearer variation in the volume of junk news is observable. The subreddits where these sources appear are still in small amounts, with the highest percentages appearing in r/Forum_Democratie (5,37% of all posts), r/meerderheidnederland (5,67%), r/de_thierry (4,93%), r/Duindorp (13%), r/The_Wilders

(1,97%), and r/FreeDutch (3,04%). Most of these subreddits are related to right-wing political parties, ideologies or politicians, such as Geert Wilders or Thierry Baudet. These subreddits appear mostly because of the frequent posting of links to hyperpartisan websites such as De

Dagelijkse Standaard.

When compared to the overall Dutch sphere on Reddit (figure 6), quite a large area of the Dutch subreddits has at least some presence of junk news from the expert list. Still, the largest and most mainstream Dutch subreddits (r/thenetherlands, r/cirkeltrek, r/Amsterdam) contain a negligible amount. Dutch junk news can most notably be seen within already polarised or partisan spaces, such as the right-wing subreddits listed above. “Neutral” subreddits like r/thenetherlands seem fairly immune, likely because of a different user base and content moderation.

Lastly, figures 7 and 8 provide a zoomed-out visualization of the relative amount of Dutch junk news in the entirety of posts on Reddit. While some Dutch junk news appears in a number of both Dutch and non-Dutch subreddits, it pales in comparison to the total number of posts in other subreddits in the research timeframe. Moreover, even though some Dutch junk news appears on a number of large international subreddits (in turn, making the sphere appear large), the relative number of appearances of Dutch junk news in those subreddits is close to zero.

 126 Concluding, then, in terms of frequency, links to questionable Dutch-language news sources on

Reddit is a small issue outside of a few partisan subreddits.

Figure 6. Dutch subreddits where Dutch junk news appear compared to the size of all Dutch subreddits. Size of circle represents the overall number of posts in that subreddit, and colour represents the relative amount of posts with junk news. Data source: Google BigQuery. Timeframe: from 1-Dec-2015 to 31-Jan-2019.

Visualisation: circle pack diagram.

 127 Figures 7 & 8. All Dutch and non-Dutch subreddits where Dutch junk news appear compared to the size of all of Reddit. Size of circle represents the overall number of posts in that subreddit, and colour represents the relative amount of posts with junk news. Data source: Google BigQuery. Timeframe: from 1-Dec-2015 to 31-Jan-2019. Visualisation: circle pack diagram.

Scoping junk news on 4chan/pol/

4chan’s infrastructure allows less of a comparative approach than that of Reddit, but some metrics can shine light on the relative appearance of Dutch junk news on /pol/. First, to scope the Dutch sphere, the amount of posts with the country flag of the Netherlands is fairly stagnant since late 2015 (figure 9). Each month, around 40,000 “Dutch flagged” posts are made. The amount increased in March 2017, owing to the Dutch general elections. While these numbers are

 128 lower in comparison to Dutch users on mainstream platforms like Facebook, they are fairly comparable to Reddit, and a non-negligible number - a fairly significant insight considering the extreme political ideas present on /pol/. It is impossible to tell how many individual people these numbers of posts denote, however.

Figure 9. Line graph of posts with Dutch country flags on 4chan/pol/. Data source: 4CAT; timeframe: from 1- Dec-2015 to 01-Jun-2019; line graph.

Despite the frequent Dutch posts on /pol/, the amount of posts linking to Dutch junk news is quite low (figure 10). Links to Dutch junk news domains appear only around ten times per month. One significant spike occurs in March 2017, caused by links mostly to The Post Online and De Dagelijkse Standaard, again concerning the general election on March 15. Interestingly, a similar spike associated with the elections is absent from Reddit. Afterwards, however, the amount of posts linking to Dutch junk news drops, remaining low for both the 2019 Dutch provincial elections and the 2019 European elections. Considering the total amount of posts by Dutch

/pol/ users (averaging around 40,000 posts per month), the amount of references to junk news URLs should be considered negligible. This should not be equated with a lack of problematic news content, however, as is discussed below.

Categories of Dutch junk news

How sizable of a role do online news media play within 4chan/pol/ and Reddit? What types of domains are linked to when categorising news domains posted in these forums? What types of junk news can we discern? This section uses the haystack to needle approach to walk through a number of tree maps, each showing a different categorisation of the most-linked to domains.

First, the proportion of news websites is compared to non-news domains. After, the news websites are sorted by Dutch or non-Dutch. Finally, the categories of these news sources are outlined and discussed (mainstream, hyperpartisan, disinformation, etc.).

 129 Figure 10. Frequency of posts linking to Dutch junk news domains on 4chan/pol/. Data source: 4CAT; timeframe: from 1-Dec-2015 to 01-Jun-2019; streamgraph.

Firstly, figures 11 and 12 show which domains from the most-posted domain sample are categorized as “news”. For Reddit (figure 11), 21,6% of all posts on Dutch subreddits refer to

“news” websites. Notably, tweedekamer.nl appears 15,694 times, caused by the bot u/kamerstukken-bot posting parliamentary texts to the subreddit r/kamerstukken. Removing this bot increases the news proportion to 50% - quite a considerable number. Other non-news websites include reddit.com itself, often used to host images and text, and youtube.com and youtu.be, appearing 951 times cumulatively.

 130 Figure 11. Links to news (red) and non-news (blue) sources in posts in Dutch subreddits. Data source: 4CAT and Puhshift; timeframe: from 1-Dec-2015 to 01-Jun-2019; treemap diagram.

4chan/pol/ paints quite a different news/non-news picture. At 16.6% the proportion of links to news websites is lower than Reddit’s 50%. After twitter.com and en.wikipedia.org, a staggering

50% of URLs point to YouTube. Considering this major presence of Google’s video service, it is further scrutinised as an alternative news sphere in section five.

Figures 13 and 14 show the news domains on Reddit and 4chan/pol/, respectively, according to their origin (Dutch or non-Dutch). On Reddit, the domains shared on Dutch subreddits are almost exclusively of Dutch origin. This is likely due to content moderation in these spaces, requiring posts to be specifically about the Netherlands (e.g. on r/thenetherlands). The news sources on 4chan/pol/ (figure 4), on the other hand, are predominantly from Anglophone sources, such as The Daily Mail, The Guardian, BBC, and Reuters. This is fairly unsurprising considering 4chan/pol/’s designation as an English language space, unlike the Dutch subreddits. Still, it is worth noting that Dutch users on 4chan/pol/ are mostly concerned with English sources and are thus more internationally oriented in terms of news propagation than users on

Dutch subreddits. This also implies foreign news sources might significantly influence their news consumption. As shown below, English junk news is indeed posted by these Dutch “anons”.

 131 Figure 12. Links to news (red) and non-news (blue) sources in Dutch posts on 4chan/pol/.

Data source: 4CAT; timeframe: from 1-Dec-2015 to 1-Jun-2019; treemap diagram.

Figure 13. Links to Dutch (orange) and non-Dutch (blue) news on Dutch subreddits. Data source: 4CAT and Puhshift; timeframe: from 1-Dec-2015 to 01-Jun-2019; treemap diagram.

 132 Figure 14. Links to Dutch (orange) and non-Dutch (blue) news on Dutch subreddits. Data source: 4CAT; timeframe: from 1-Dec-2015 to 01-Jun-2019; treemap diagram.

Figure 15. Categories of news domains in posts on Dutch subreddits.

Data source: 4CAT and Pushshift; timeframe: from 1-Dec-2015 to 01-Jun-2019; treemap diagram.

 133 Next, we explore the types of news sources, and if “junky”, how they can be categorised. Figures 15 and 16 show the categorisation of the shared news domains as mainstream, other

(mainstream), conspiracy, disinformation, hyperpartisan, and clickbait, as defined in section 1.3.1. As is evident in the visualisations, mainstream or special interest (other mainstream) sources make up the largest share of URLs posted on both platforms: 99,6% for Reddit and 81% for

4chan/pol/. Despite the frequent characterisation of pseudonymous spheres like Reddit as

“alternative”, these results are thus somewhat counterintuitive since mainstream sources make up the dominant proportion links shared. On both sites, NOS.nl is the most-linked to news source, meaning the established source is highly relevant. For Reddit especially, the lack of problematic content in Dutch spaces is remarkable, as in these most-posted domains almost no websites from the expert list can be found, save for a few instances of dagelijksestandaard.nl and tpo.nl. Indeed, the platform and Dutch users show they seem to be inoculated against

“pulpnieuws”.

Figure 16. Categorised types of news from news sources posted 4chan/pol/.

Data source: 4CAT; timeframe: from 1-Dec-2015 to 1-Jun-2019; treemap diagram.

Dutch-flagged posts on 4chan/pol/ show a more problematic, hyperpartisan nature. Here, 21% of top news domains are “junk”, with hyperpartisan sources making up most of these. Some of these are foreign state-influenced and/or hyperpartisan, such as RT and Breitbart, and others are outright extremist, like the neo-Nazi website, The Daily Stormer. As alluded to above, Dutch junk news seems to play less of a role here. A few sporadic instances of far-right disinformation appeared in the Dutch posts (shown in orange), all originating outside of the Netherlands. These include links to Sputnik, the large Russian news website that has been known to propagate disinformation (MacFarquhar, 2016; EUvsDisinfo, 2017), as well as two far-right websites that post a large amount of Islamophobic stories, Speisa.com and The Gatestone Institute. In posts

 134 linking to Russian sources, like RT and Sputnik, the top URLs are usually referring to refugee slander, particularly in Sweden (see appendix IV). It is impossible to tell who posted these links considering 4chan’s built-in anonymity, but it could potentially point to foreign interference.

Characteristics of Dutch junk news propagation on Reddit

What are the characteristics of online actors who share Dutch junk news? And are their actions effective? While these questions are nearly impossible to answer for 4chan, considering its anonymity, Reddit does afford “natively digital” (Rogers 2013) objects to explore the characteristics and effectiveness of junk news propagators. This section will therefore discuss a range of metrics and lists concerning Dutch junk news on Reddit.

Figure 17. Mean Reddit posts scores by Dutch junk news propagators (users who posted a link to a Dutch junk news domain at least twice). Data source: 4CAT and Puhshift; timeframe: from 1-Dec-2015 to 01-Jun-2019; bar graph.

298 Reddit accounts were found within the timeframe that linked to domains from the expert list of Dutch junk news domains. Out of those, 193 accounts only posted a Dutch junk news URL once. Only sixteen accounts did so ten times or more, meaning there is a long tail of occasional junk news posters. When these junk news sources are linked to, they furthermore receive a lower score on average than other posts these propagators make (figure 17). To reiterate, Reddit scores are created by users’ “upvoting” or “downvoting” a post, with a high score meaning a post will move to a higher position on a subreddit, thus receiving more visibility. As can be seen in figure 17, posts to non-junk news by these propagators outperform posts linking to one of the sites in the expert lists, with the propagators’ mean score being 9.8 and the mean for their posts linking to a Dutch junk source being 5.6. This is mostly caused by automated, “spammy” posts. The median for each of these is 1 and overall, 1.24 of the 1.72 of junk news posts have a score of 1 or less (72%), meaning the Dutch junk news posts receive little visibility and approval on average.

 135 These low average scores do not mean that junk news stories are totally void of success, however. 33 of the 1,761 posts received a score of 50 or more. Reddit’s infrastructure stimulates a snowball effect of “rich get richer” posts, and some of these even scored higher than 1,000.

Zooming in on a URL instead of a domain level, it shows that most of these stories are hyperpartisan of tone. Table 1 shows the top three highest-scoring posts on Reddit linking to a domain from the expert list (see appendix V for the top 25). All three best-performing spots are “junky” and Islamophobic in tone. The first concerns a story by De Dagelijkse Standaard on rape and refugees. The second and third are both linking to the same story by Fenixx that framed a man who drove a car into a group of people at Amsterdam Central Station as a Moroccan terrorist, even though he was officially declared as unwell and confused. Interestingly, these stories are posted in English-language subreddits, notably the pro-Trump r/The_Donald and the now-banned r/CringeAnarchy, showing how junk news from the Netherlands spreads to foreign spaces.

subject url subreddit timestamp score

Amsterdam Square driver (terrorist) before declared a dagelijksestandaard.nl/201 confused and sick Dutch 7/08/onthullingnational is now revealed to werkelijke-naam-van-debe KHALID K. from amserdamse Casablanca. The media stationsrammer-blijkt-dus 22/08/2017 cover up doesn't stop! khalid-karmaoui/ The_Donald 13:31 1811

fenixx.org/2017/05/14/ja Japan ONLY Admits 27 pan-only-admits-27-

Muslim “Refugees”, Two muslim-refugees-two Already Arrested For Gang already-arrested-for-gang 15/05/2017 Rape. rape/ The_Donald 06:02 1211

fenixx.org/2017/05/14/ja Japan ONLY Admits 27 pan-only-admits-27-

Muslim “Refugees”, Two muslim-refugees-two Already Arrested For Gang already-arrested-for-gang 10/08/2017 Rape. rape/ CringeAnarchy 5:13 936

Table 1. The top 3 best performing posts linking to a Dutch junk comain on Reddit. Data source: 4CAT and Pushshift. Timeframe: 01-Dec-2015 to 01-Jun-2019.

In summary, the overall performance of Dutch junk news throughout Reddit is fairly marginal. Moreover, the high-scoring stories are usually hyperpartisan instead of clear-cut disinformation. Dutch junk news thereby can garner considerable engagement on Reddit, but it does not do so on a regular basis. In this sense, Reddit is more “resistant” to junk news than (for example) Facebook is said to be (Burger et al., 2019).

 136 Figure 18. Subreddits where Dutch junk news domains are most often posted in. Data source: 4CAT and

Pushshift; timeframe: from 1-Dec-2015 to 31-Jun-2019; circle pack diagram.

In which subreddits are Dutch junk news domains posted? Figure 18 shows that r/viral links to most Dutch junk news with 543 instances, but much of the prevalence is caused by a single

“spam” account, receiving no engagement whatsoever. More interestingly, r/Forum_Democratie, the unofficial subreddit for the currently the largest party in the Dutch Senate, comes in second with 312 posts to junk news sites. Other right-wing partisan and hyperpartisan subreddits appear further down the long tail, such as r/The_Wilders, r/FreeDutch, r/meerderheidnederland, r/The_Donald, and r/de_thierry. This is mainly caused by posts on these subreddits linking to The

Post Online and De Dagelijkse Standaard.

According to figure 19, the junk domains that are linked to often are mostly the well-known right-wing tendentious and hyperpartisan blogs, with The Post Online and De Dagelijkse Standaard ranking on top. The “alternative” news website NineForNews, which has been host to conspiracies and hyperpartisanship (Roermund 2017), is also amongst the most shared domains, but this is mostly due to an automated bot posting links to the website (u/ninefornews). As such, most of the shared junk news domains can be categorised as hyperpartisan, often with an

“alternative” right-wing stance. Signs of disinformation or coordinated Russian influence are

 137 fairly marginal, with Novini appearing 22 times, a website known for pro-Putin sentiment (Heck

2017). As such, from this needle to haystack method, partisan and hyperpartisan content is easy to be found, but coordinated disinformation or Russian meddling appears to be less of an issue within these spaces.

Figure 19. Most linked to Junk news domains on all of Reddit. Data source: 4CAT and Pushshift; timeframe: from 1-Dec-2015 to 1-Jun-2019; circle pack diagram.

Avg. score with %

Dutch Dutch Dutch junk Avg. junk junk Top Dutch Total

author source score posts posts Top domains junk domains Top subreddits posts

youtube.com: 416 welingelichtekringen.nl: boinnk.nl: 82 153 earth-matters.nl: 77

user1 1 1 13.5 591 rt.com: 125 stopdebankiers.com: 61 viral: 4390 4390

user2 1 1 100 294 ninefornews.nl: 294 ninefornews.nl: 294 news: 294 294

 138

russiawarinukraine: 6470

112.international: 9891 dagelijksestandaard.nl: meerderheidnederla unian.info: 6238 23 fenixx.org: 9 nd: 788

user3 1 1.1 0.1 70 liveuamap.com: 6210 politiek.tpo.nl: 4 oekraineukraine: 707 66860

dagelijksestandaard.nl: Forum_Democratie: i.redd.it: 431 22 1337 twitter.com: 352 tpo.nl: 19 FreeDutch: 83 user4 11.2 28 3.9 60 imgur.com: 106 politiek.tpo.nl: 12 The_Donald: 21 1525

twitter.com: 126 tpo.nl: 50 youtu.be: 60 opiniez.com: 10 Forum_Democratie:

user5 16 18.2 12.7 62 tpo.nl: 50 tpook.nl: 1 489 489

Forum_Democratie: geenstijl.nl: 52 tpo.nl: 28 263 twitter.com: 42 politiek.tpo.nl: 12 The_Donald: 11 user6 11.5 74.7 16.8 48 tpo.nl: 28 tpook.nl: 2 FreeDutch: 9 286

dagelijksestandaard.nl: Forum_Democratie: youtube.com: 118 13 954 twitter.com: 91 tpo.nl: 11 test_forum: 3 user7 8.4 9 3 29 i.redd.it: 64 opiniez.com: 3 JFVD: 2 961

youtube.com: 32 verenoflood.nu: 6 The_Wilders: 113 imgur.com: 14 politiek.tpo.nl: 5 The_Europe: 60

user8 25.6 23.7 7.7 19 twitter.com: 14 opiniez.com: 4 The_Donald: 46 246

youtube.com: 5859 news: 5813 gellerreport.com: 1698 worldnews: 5537

user9 1.5 1.5 0.1 19 bitchute.com: 1677 fenixx.org: 19 worldpolitics: 4232 22477

Table 2. Metrics of users who shared the Dutch junk news on Reddit. Data source: 4CAT and Puhshift.

Timeframe: form 01-Dec-2015 to 01-Jun-2019.

Finally, we highlight the Reddit accounts most active in propagating junk news to profile actor types. Table 2 shows various metrics on the ten accounts ranked by the amount of posts linking to one of the domains in the expert list. As indicated by total posts and average score, some of the accounts post frequently but receive no engagement. Most of these are “spam” accounts or automated bots. Interestingly, user 1, the aforementioned u/mrthirdeye68, has posted many URLs to Russian and pro-Russian websites as RT.com and novini.nl, as well as mainstream sources and hyperpartisan websites like the far-right website Red Ice TV. It is possible that user 1 is a Russian “troll”. It received no upvotes, however, and only posted links to the obscure subreddit r/viral, meaning it did not garner any engagement. As such, it is likely this user is an automated bot, or some hybrid. Other bots seem more effective, however. User 2, for instance, is the abovementioned ninefornews.nl bot, posting a hundred percent of posts to this website in the global news subreddit r/news. Of interest here is that user 2 does receive engagement, with a fairly high average post score of 570 and a junk news post score of 50. As such, Reddit is at least somewhat susceptible to manipulation, depending on the “strategy” of its users.

In terms of issues, it can be discerned that the most active accounts are either concerned with

Dutch right-wing parties or topics surrounding Ukraine. Despite their frequent linking to junk news websites, the most active accounts still link most often to platforms like YouTube and

Twitter. 45 A cohort of four right-wing partisans can be observed, who are most active on

45 Further research might scrutinise what YouTube videos or Tweets are linked to, for instance to identify further

“newsy” sources or influencers.

 139 r/Forum_Democratie and frequently link to websites like GeenStijl and The Post Online. Most of the accounts actually use the Dutch language, and, upon closer inspection, are also likely Dutch natives. As such we may conclude that there is not a significant attempt of foreign accounts meddling with Dutch affairs, with the possible exception of the now-banned u/mrthirdeye68.

YouTube as an alternative news network

Thus far, this text has handled “news” in the conventional sense of designated outlets publishing on current affairs. As discussed in the introduction, however, the consumption of both amateur and professional reporting increasingly occurs on social media. These modern, alternative ways of news consumption cannot be identified when the “needle” is formulated as traditional news outlets. As we have identified in section 3, URLs linking to YouTube are frequent, especially on

4chan. As discussed elsewhere in this volume, the video hosting site is host to various spheres of alternative news commentary and opinions, leading Zeynep Tufekci to describe it as “the Great

Radicalizer” (2018). Can we indeed outline an “alternative news network” working in tandem with 4chan/pol/ and Reddit? This section briefly touches on this question by visualising and categorising the most-posted videos on 4chan/pol/ and Reddit, as well as the most popular channels.

Figures 20 and 22 display the thumbnails of the 1008 most-posted YouTube videos on our

Dutch Reddit and 4chan/pol/ corpora. Figures 21 and 23 shows the “video categories” for each of these videos. For the top videos on Dutch subreddits, 161 are concerned with “People and

Blogs”, 129 with “Entertainment”, and 118 with “News & Politics”. From this, the type of content shared is fairly diverse. 4chan/pol/ is more concentrated on news and politics, with 196 videos categorised as such, with “People & Blogs” following at 95 and “Entertainment” at 64.

The number of missing videos for 4chan (the black labels) is notable, comprising almost half of the total, indicating 4chan’s extremism as well as YouTube content moderation. The amount of deleted videos is visibly less on Reddit.

If one takes the videos labelled as “News & Politics” as an indicator of a “news source”, as we categorised in the sections above, it becomes possible to quantify the role of YouTube as a news source on the two platforms. The “News & Politics” category comprises 11.7% of the stillonline videos for Reddit in the sample above, and 19.4% for that of 4chan/pol/. Considering the total amount of links to still-online YouTube videos in this timeframe – 7,667 for Reddit and

26,635 for 4chan/pol/ – one can estimate that around 896 “News & Politics” videos were posted on Dutch subreddits and 3,748 on 4chan/pol/ by users with a Dutch flag. 46 Comparing these numbers to those presented in section 3, for YouTube news videos would constitute the largest and second largest source of news content. On Reddit, they would form the secondlargest news source, only behind NOS.nl with 1,615 mentions. For 4chan, YouTube is by far the largest player in relation to news circulation, since the next most popular source, NOS.nl (861 mentions), comprises only one-third of the amount YouTube news videos. As such, the role of

YouTube as a new player in the circulation and consumption of news should not be understated.

Is this dominant presence of YouTube of great significance in the study of junk news? Table 3 shows the 25 most-occurring channels from all of the YouTube links in our two Dutch corpora. Here, the platforms differ significantly. On Reddit, some partisan channels can be discerned, like the one for Forum voor Democratie and PVV pers, but the list mostly consists of “established”

46 In reality, these numbers will be somewhat lower because not every YouTube URL points to videos (they can also refer to e.g .channels), although the vast majority in our corpus does.

 140

The EU Monitor enables its users to keep track of the European process of lawmaking, focusing on the relevant dossiers. It automatically signals developments in your chosen topics of interest. Apologies to unregistered users, we can no longer add new users.This service will discontinue in the near future.