A new study by Mozilla has found that similar recommendations continue to appear even when users tell YouTube they are not interested in certain types of videos. “dislike,” “stop recommended channels” and “remove from watch history” buttons were largely ineffective in preventing similar content from being recommended.
The report found that even in the best case, these buttons still allowed more than half of the recommendations through to content similar to what the user said they were not interested in. In the worst case, the buttons did little to nothing to block similar videos.
To collect data from real videos and users, Mozilla researchers recruited volunteers to use the foundation’s RegretsReporter. This is a browser extension that overlays a generic “stop recommending” button on the YouTube videos that participants watch. On the back end, users were randomly assigned to a group so that each time they clicked the Mozilla-placed button they sent a different signal to YouTube – dislike, disinterest, no channel recommendation, delete from history, and a control group – that sent no feedback to the platform.
Using data collected from more than 500 million recommended videos, the research assistants created over 44,000 pairs of videos – one “rejected” video, plus one video that was subsequently recommended by YouTube. The researchers then performed their own pairwise evaluations or used machine learning to determine if the recommended video was too similar to the video the user rejected.
Compared to the baseline control group, sending “dislike” and “not interested” signals was only marginally effective in preventing bad recommendations, preventing 12 percent and 11 percent of bad recommendations, respectively. The “don’t recommend channels” and “delete from history” buttons were slightly more effective – they prevented 43 percent and 29 percent of bad recommendations, though the researchers say the platform still doesn’t provide enough tools to direct unwanted content.
The researchers wrote, “YouTube should respect the feedback users share about their experiences as meaningful signals about how people want to spend their time on the platform.”
YouTube spokeswoman Elena Hernandez said the actions were intentional, which is because the platform doesn’t try to block all content related to a certain topic. But Hernandez criticized the report, saying it did not consider how YouTube’s controls are designed.
“It’s important that our controls don’t filter out entire topics or opinions that could negatively impact viewers, such as creating echo chambers,” Hernandez told The Verge, “We welcome academic research on our platform, which is why we recently expanded data API access through our YouTube researchers program. the Mozilla report doesn’t take into account how our system actually works, so it’s hard for us to gather many insights.”
Hernandez said Mozilla’s definition of “similar” doesn’t take into account how YouTube’s recommendation system works. The “Do not recommend channel” button prevents the channel from being recommended in the future. The company said it does not seek to stop recommending all content related to a particular topic, opinion or speaker.
