Earlier this month, Twitter announced it would restrict free access to its API — the programming interface that allows third-party developers to communicate with Twitter. While the move certainly affects independent developers and startups building tools to make the platform fun and safe, it also creates a problem for students and academics who use Twitter for research purposes on a variety of topics.
Last week, the Elon Musk-led company sent an email to developers where it named that the basic level to access Twitter’s API – which costs $100 a month for “low-level use” – will replace outdated access levels such as Essential, Elevated, and Academic Research. At the moment there is hardly any information about what developers can do with that $100 a month. According to platformer Casey Newton, low-level enterprise API access could cost as much as $42,000 per month.
Affordability for researchers
For many people in the research community, spending hundreds or thousands of dollars each month may not be feasible.
“Earlier this week, a HateLab an undergraduate dissertation student had to change his thesis design from collecting data on twitter because he has no money to pay for it. His experience will be shared by thousands across the country and by millions worldwide. It’s really incredibly disruptive and will have a significant impact on the research ecosystem that relies on this data as the Twitter research pipeline from undergrad to professional has been disrupted by this change,” said Professor Matthew Williams of HateLab, which is part of the social data science from Cardiff University lab and studies of online hate speech.
Ironically, HateLab is listed as a success story on Twitter’s developer portal to use research for good. It has published multiple research papers on hate speech on the platform. The agency uses the social network’s academic research API. But that might cease to exist in the new version.
Twitter also failed to consider that $100 a month for basic access might be a lot for researchers in developing countries.
“The decision to end free access to Twitter API will have major implications for researchers studying hate speech online, especially independent researchers and researchers in developing countries. Its impact will be deeply felt in India, where hate speech is spreading at a very alarming rate on Twitter. Paying USD 100 per month and USD 1,200 per year is a significant financial burden for them,” said Raqib Hameed Naik, founder of Hindutva Watch, an India-based research organization.
The process isn’t even clear to institutions that may be willing to spend money. Rebekah Tromble, director of the Institute of Data, Democracy & Politics, said when they tried to fill out the corporate-level entry form, they were redirected to the academic research program. She also said that the person they contacted via Twitter is no longer with the company.
Impact of research API shutdown
Independent research has been a key factor in making Twitter more useful and less toxic. The company has shown multiple projects working in areas such as healthcare, online hate speech and climate change using Twitter data.
Earlier this year, the company launched the Twitter Moderation Research Consortium (TMRC), inviting members from academia, civil society, non-governmental organizations and journalism to study the platform’s governance issues. But since Musk took over, the program has stalled and the employees who worked on it have left.
The Tesla CEO himself used data from Botometer, a tool to measure bot followers on Twitter accounts, during the public spats leading up to the company’s takeover. The tool was created by Indiana University’s Observatory on Social Media. But the tool’s future may be jeopardized by the new API announcement.
Many research projects take into account a large number of tweets. They send hundreds of questions to the platform to study different topics. Twitter has not released any details about what might be offered in the $100 per month tier. But it probably won’t be enough for most projects. For reference, under the academic research track, Twitter API previously provided access to 10 million tweets per month and 50 requests per 15 minutes per app.
Kaicheng Yang, one of the researchers working on Botometer, expressed concern over Twitter’s move to allow shutter-less API access for academics.
Joshua Tucker, co-director of the NYU Center for Social Media and Politics, recently published a paper on Russian disinformation campaigns on Twitter during the 2016 US presidential election campaign. He said the campaign studied data from many thousands of tweets, so if the social network academics pay for that data, it would be difficult to conduct research on a large scale.
“This [move by Twitter] is just essentially a step in the wrong direction. We are at a time when major global legislative efforts are focused on making access to data easier for outside researchers, and this decision by Twitter will only make accessing data more difficult for outside researchers. This in turn means more blind spots about the impact of the platforms on society for policymakers, the press, civil society and business,” he said.
The past few days, a lot of researchers have pointed out that free Twitter API access also aids in crisis response to natural disasters such as the recent devastating earthquake in Turkey and Syria. Earlier this month, a group of independent researchers wrote an open letter to Twitter asking the platform to keep free API access open.
“Twitter’s new CEO Elon Musk has pledged to make the platform more transparent and reduce the prevalence of spam and manipulative accounts. We commend and support those priorities,” the letter said.
“In fact, the independent research community has developed many of the most advanced techniques used to manage bots. API access is a critical resource for that work. Twitter’s new barriers to data access will actually reduce the transparency that both the platform and our societies so desperately need.”
Without independent investigation, the company could become ignorant of disinformation and hate speech on the platform. The EU has already given the company a ‘yellow card’ for missing data in its disinformation report. Several companies have signed the Code of Practice, which promises to provide data to researchers, among other things.
While non-compliance with the research code has no legal ramifications, it could carry more weight from next year when the Digital Services Act (DSA) comes into effect. Separately, the bloc’s high commissioner, Josep Borrell, criticized Twitter for restricting access to its platform by making its API paid. The implications of that change are broader than expected – and rapid earnings gains could become a more significant long-term issue.
You can contact this reporter via Signal and WhatsApp at +91 816-951-8403 or im@ivanmehta.com by email.