NLP and sentiment analysis

The objective of this project was to gauge user sentiment by processing the interactions between a digital avatar and a user. The input data consisted of both textual and visual information, which were processed simultaneously. In the context of the textual modality, the aim was to infer the user’s emotional state from the dialogue and identify keywords that triggered further conversation management. The visual aspect of sentiment analysis involved categorizing user facial expressions into different emotional states and monitoring these expressions over time to detect changes in emotional state during interactions with the avatar. This task included the implementation of various NLP pipelines, which included the deployment of machine learning models for entity recognition and classification. All these components were intricately linked with a dialogue manager to ensure appropriate responses based on the user’s emotional state.