Do you remember the presidential debates between Biden and Trump in 2020? Let me help you. These debates were chaos, theatrics, purely a stream of thoughts hard to follow as a watcher. Trump was interrupting both Biden and the moderator, changing topics rapidly and dropping names. For watchers it was a cognitive supertask to follow and understand the associations and relations made in these debates.
Naturally, we would go to our smartphone and look for smaller chunks of this debate, explained or summarised in a digestible form. But there is a better way to help us understand political debates. Developed and tested by PhD Candidate Peter Andrews together with researcher Njål Borch and Professor Morten Fjeld, the AiModerator improves understanding of video content by analysing it and providing contextually relevant information during the debate on the same screen.
This tool transforms how audiences interact with political debates by overlaying a dynamic, touch-based interface onto live or recorded video streams. Rather than moderating the debate directly, AiModerator acts as a co-pilot: analysing, synthesising, and presenting relevant information to viewers in real time.
Political debates overshadowed by theatrics, wrong accusations and fast content jumps can be made more accessible and reliable by adding this AI-based co-pilot.
A multimodal conversational agent (MCA) on one and the same screen
Young children were not much affected by Covid-19? This could have been instantly verified and been proven to be a false claim by Trump in the first presidential debate. The AIModerator is a fact-checker, can compare candidates’ position on key issues, links debate topics to official party platforms, clarifies terms, acronyms and reference, gives the option to dive deeper into related subjects, allows users to compare their views with those of the broader audience and offers emoji-based feedback for real-time audience engagement. AiModerator is a political debate co-pilot acting as a Multimodal Conversational Agent (MCA).
Key Features of AiModerator:
- Fact Checker: Instantly verifies claims made by candidates, offering evidence-based assessments.
- Stance Checker: Compares candidates’ positions on key issues to highlight differences and similarities.
- Platform Checker: Links debate topics to official party platforms, providing broader political context.
- More Information: Clarifies ambiguous terms, acronyms, and references used during the debate.
- Explore Topic: Enables users to dive deeper into related subjects as they arise.
- Opinion Poll: Allows users to compare their views with those of the broader audience.
- React: Offers emoji-based feedback tools for real-time audience engagement.
AiModerator is powered by advanced Natural Language Processing (NLP), Computer Vision (CV), event detection, API communication, and event management enabling it to extract and deliver hyper-contextualised information aligned precisely with the ongoing discussion. Users can fact-check statements, explore party policies, and better understand candidate positions, all without interrupting the flow of the debate.
How It Works:
AiModerator processes raw video feeds using WhisperX for transcription and speaker identification, while YOLOX tracks the candidates visually. Detected events are transmitted via TCP to a Unity-based interface optimized for touch interaction. When users engage with the system, it queries a database and uses GPT-based modules to deliver relevant, accessible insights.
For more details, read the paper here.
By combining AI with intuitive design, AiModerator redefines how we experience political debates, making them more transparent, informative, and engaging for all. The idea was to lower mental effort, assisting the user in retaining their attention on the video.
Relevant for all elections
AiModerator was tested on the second 2020 presidential debate between Donald Trump and Joe Biden during the Autumn of 2024. The debate was chosen because of its known concentration on rhetoric, inconsistent nature and lack of focus. They tested the prototype by conducting a user study with young adults, a group that frequently disengages from politics.
Their findings demonstrate that AiModerator significantly improves subjective understanding and provides a highly engaging experience compared to second screening. AiModerator sets the user into an objective viewing state, but developers should proceed with caution. Increased objectivity can reduce emotional engagement, which is highly important for active political discourse.
Their future research will optimise the balance between several attributes mentioned in the discussion of the paper to improve user experience. They will consider personalisation and enhanced collaborative viewing methods while being cautious of potential issues related to polarisation, such as echo chambers and filter bubbles. Developers and designers can reference their findings and methodology for academic projects, news broadcasting, and political analysis. The AIModerator could be beneficial for watchers of upcoming elections, like the national election in Norway in September.
