latest-news-headlines Market Intelligence /marketintelligence/en/news-insights/latest-news-headlines/plan-to-open-source-twitter-algorithm-raises-transparency-revenue-questions-70156943 content esgSubNav
In This List

Plan to open-source Twitter algorithm raises transparency, revenue questions

Podcast

MediaTalk | Season 2 | Ep. 27 - College Football Preview & Venu Injunction

Podcast

Next in Tech | Ep. 181: Lighting up Fiber

Podcast

MediaTalk | Season 2 | Ep. 26 - Premier League Kicks Off

Podcast

Next in Tech | Ep. 180 - Datacenters and Energy Utilities


Plan to open-source Twitter algorithm raises transparency, revenue questions

SNL Image

Elon Musk plans to open-source Twitter's algorithm.
Source: Pexels, S&P Global Market Intelligence

Elon Musk's plan to open-source Twitter Inc.'s algorithm may not do much to increase transparency, but it could prove a lucrative business opportunity if enough people can understand it.

As part of his larger plan to take Twitter private, Musk has suggested open-sourcing its algorithm as a means of reducing manipulation and mistrust on the platform. "Open source is the way to go to solve both trust and efficacy," Musk said in a May 15 tweet.

Open-sourcing, in essence, is the practice of making an application's source code publicly available on a hosting site. Popular examples of open-source software include Mozilla Foundation's Firefox browser or the programming language Python. In the case of Twitter's algorithm, it means making the micro-blogging platform's processes for recommending content to users available for all to see.

Some call the idea a positive for the health of Twitter but also note that opening the platform's black box, so to speak, will make sense to only a small minority of users.

"If it's truly just source code and system diagrams, it's only going to be accessible to a very small number of experts," said Patrick Hall, principal scientist at artificial intelligence-focused law firm bnh.ai and professor of data ethics at The George Washington University.

SNL Image

Opening up to open source

Hall is not against the idea of open-sourcing Twitter but questions whether it would actually increase transparency for users.

"There is a very big difference [between] saying 'This is the technical specification of my algorithm' [and] 'This is how it impacts you as a consumer,'" Hall said in an interview with S&P Global Market Intelligence.

Millions of tweets are sent every day, making it difficult to understand the algorithm due to the constant presence of new data, Hall said. "It's very hard to understand machine learning algorithms without understanding their data," he said. "And if that volume of data is hard to deal with ... that again limits meaningful transparency."

Transparency has become a hotly debated subject as some conservatives, including members of Congress and government officials, have alleged underlying political biases in Twitter's algorithm. Democrats, meanwhile, have pushed platforms to use machine learning and algorithms to do a better job censoring hate speech, misinformation and other harmful content.

Demystifying the data

What lies under the hood of a social platform's content ranking algorithm is so complex that even in-house engineers at companies like Twitter may not understand all of its inner workings, a former employee of Facebook parent Meta Platforms Inc. told Market Intelligence, requesting anonymity due to connections they have in the tech industry.

To get an intuitive view of the algorithm, an engineer or related expert would have to focus on understanding its objective function, they said.

In social media, content recommender algorithms have an objective of optimizing engagement metrics. For instance, they could be directed to maximize ad revenue on certain content or to minimize "likes" and "shares" with content deemed low quality.

If engagement with a particular tweet is deemed valuable to Twitter, the algorithm may calculate the probability of a user liking, commenting, retweeting or direct-messaging it. The result of that value calculation would then determine whether or not said tweet is displayed on a user's feed with the hopes that, if predicted correctly, the user will engage.

The former Meta worker said that people are seeking to trust Twitter more, and making its content algorithm open source would be good for society as a whole. "People want to be able to trust that Twitter isn't biasing to a particular political direction" or favoring certain governments, they said.

Twitter did not respond to requests for comment on how the company's algorithms operate.

Building a business case

Even if open-sourcing does not increase transparency for the bulk of users, it could represent a monetization opportunity, according to Michael Schrage, a research fellow at the Massachusetts Institute of Technology focusing on model risk management, and author of the book Recommendation Engines.

Open-sourcing parts of the algorithm can be ad-supported in a way similar to Google LLC recommending ads to users upon the display of search results, Schrage said. This can help drive more revenue to the platform.

He also suggested accounts can pay to gain further insights about who engages with their content, such as a news outlet seeking to better understand its Twitter audience.

"There's not a shred of doubt in my mind that Elon Musk is looking at ways of turning Twitter into a freemium service," Schrage said. The micro-blogging platform has struggled to grow its earnings in recent years, with EBITDA dropping to $32.5 million in the first quarter, versus EBITDA of $183.2 million in the year-ago period.

SNL Image

Dashboard dilemma

When asked about interpretability concerns, Schrage said a dashboard can be implemented to grant better visibility into the underlying complexity of how the algorithm works.

Even with a dashboard, actionable insights may be hard to glean, said Mico Yuk, chief data evangelist at London-based software company Count and inventor of a business intelligence dashboard methodology.

"You'll have 10 people look at [the algorithm's dashboard] and then interpret it in 10 different ways, and there won't be any congruency," she said in an interview.

A dashboard could work, but it would serve better for specific tweets from one person's feed and not Twitter's entire recommendation algorithm, the ex-Meta employee said. They cited complexities and concerns with obfuscating source code.

Hall of bnh.ai said dashboards that explain algorithms only work well when the underlying design of the algorithm is interpretable from the start.

Putting it out there

Looking ahead, Musk will have to work out how to deploy Twitter as an open-source platform, sources say.

Yuk from Count says that mainstream open-source sites like GitHub would not be able to contain the complexity of the algorithm. She said that Musk might create a team responsible for communicating the algorithm to mass audiences.

If deploying the algorithm is not done correctly, those who have better technical understanding of it can "game the system" to favor certain content, she said. That can cause more divisiveness and a lack of political discourse on the platform, she added.

Representatives for Musk could not be reached for comment.