Researchers at MediaFutures found that consumers of algorithmically recommended news articles mostly choose news based on topical preference matches. In a conjoint experiment, lead author Erik Knudsen, along with MediaFutures director Christoph Trattner and associate professor Alain Starke, compared the relative importance of seven features for predicting news reading behavior in news recommendation systems in Norway.
There are several features affecting whether a user chooses to consume a news article. However, only some of them were so far considered in conjunction when analyzing the efficiency of news recommender systems. These systems mostly rely on similar-item recommendation without knowing what attracts the user the most. Researchers assumed a match between the users’ interests in a topic and the topic presented in news content could be more effective than a mismatch. In his latest research publication, MediaFutures’ researcher Erik Knudsen confirmed this.
“Topical preference seems to trump similarity based on demographic similarity, general popularity, and recency, all of which are also often used in news recommendation”, Knudsen writes in his paper.
Abortion or meat eating
The goal of this study has been to assess the relative importance of different features in news recommender systems. Whilst most studies conduct offline or online evaluation, Knudsen performed a controlled experiment in the form of a conjoint analysis.
“We know that users make multidimensional decisions. It is rarely just one thing that influences our choice. Therefore, it is hard to find out which one feature influences the user the most in the decision-making”, Knudsen says.
They found that there are other types of news recommender system features that are also important like geographical relevance for instance. In his experiment he compared the importance of topical preference match, recency of the article, popularity among similar users, general popularity, headline topic, proximity to the user, and the estimated reading time of the article. Amongst these are both news article features and news recommender system features.
“News article features such as headline, teaser, reading time etc. are things which influence users regardless of whether you have a news recommender system or not. Recommender system features in contrast are for instance about the user’s proximity to the location of news. To apply these the system first needs some personal information such as the location of the user”, Knudsen explains.
The experiment was based on the data of the Norwegian Citizen Panel (NCP) where participants were asked to choose between pairs of different news headlines with different news article features and news recommender system features.
Providing direction for designers
The proof that topical preference match is the major feature affecting the selection of news in news recommender systems can be used to guide designers in creating news recommender systems.
“This indicates that news recommender systems that focus on surveying users’ topic preferences, and recommend stories based on the answers from such surveys, will likely have a higher chance or success-rate in terms of predicting clicks or reads”, Knudsen writes.
He argues that it is important to explore if their findings are also true for other domains. MediaFutures researchers have conducted similar studies on food recommenders, but more areas are to be explored. Another important step is to test the results in a more realistic situation.
“We should implement the feature to either real news sites or news sites that look like real to learn more”, Knudsen says.
Furthermore, the findings are relevant for offline evaluation approaches to generate an algorithmic approach which resonates with users in online evaluation.
Knudsens paper has been presented at INRA at ACM Recsys 2023.