Real-Time Presenter’s Assistant: Empowering Presenters and Elevating Audience Experience through Adaptive Q&A Support

In this project, we focus on enhancing public speaking, particularly during Q&A sessions. Recognizing that public speaking anxiety can significantly impact personal and professional development, our goal is to alleviate this anxiety and improve the performance of presenters. We have identified challenges that presenters face, such as dealing with unexpected questions and maintaining audience engagement. To mitigate these challenges, we developed a natural language processing (NLP) system that provides accurate answers and relevant facts, helping presenters to confidently address questions while maintaining an engaging speech.

The system integrates Big Bird for question answering, a DistilRoBERTa-based sentence transformer for selecting relevant facts, and GPT-4 to synthesize the outputs. The responses are generated based on the content of the presentation and the audience question (transcribed using speech-to-text). The final result generated by the system is displayed on the screen of a HoloLens, allowing easy access while maintaining eye contact with the audience. We designed two versions of the system: one that provides keyword answers and relevant facts for presenters who prefer improvisation, and another that synthesizes answers into a complete script for those who desire more structured support.

Check out the manuscript here