REAL-TIME MUSIC RECOMMENDATION SYSTEM

Abstract

An intelligent, real-time, Music Recommendation System (S) Moodify, based on real-time facial emotion recognition and EEG validation. The language-specific System (S) uses facial emotions to detect the emotion of the user and EEG signal to validate the change in emotion enabling music recommendation based on the preference of the user. Real-time expression of the user is captured through a built-in camera and the captured expression is processed using a custom-built CNN model which is trained on the CMU Multi-PIE Face Database. Thereafter, music is played from a music streaming platform based on the recognized emotion, the songs chosen from user preference of songs in the desired language. The system projects a customized user experience through language settings chosen from six regional and foreign languages. The system (S) recommends emotion-based songs to the user in the preferred language with high accuracy within 2-3 seconds to relieve the stress of a person instantly.

Specification

Description:This application is an improvement of the invention claimed in the specification of the main patent application no. 202341007344 titled "Music Recommendation System and Method Thereof" filed on 6th February 2023.

STATEMENT OF INVENTION AS DISCLOSED IN THE PARENT APPLICATION NO. 202341007344:
The invention as disclosed in the application titled "Music Recommendation System and Method Thereof" (parent application) relates to a music recommendation system which is based on speech and body gesture emotions. More particularly, the invention discloses a music recommendation system that recognizes emotions of a user from both body gestures such as facial expressions and speech and recommends appropriate songs based on the recognized emotion of the user by using few-shot meta-learning based Siamese Neural Network. Emotions of the user are derived by combining both speech and body gestures by using element-wise multiplication to counter the issue of dimensionality. The derived emotions are recognized through an inpu , Claims:1. A Music Recommendation System (S), Moodify based on real-time facial emotion recognition, the system (S) comprising:
- a Facial Emotion Recognition (FER) Module for capturing and processing real-time facial expressions of the user (U),
- a song extraction (SE) module for extracting songs from music API,
- a Music Recommendation (MR) module for recommending music based on the recognized emotion of the user (U),
- a Language Selection Module (LSM) with customized language selection option,
- an EEG signal validation (ESV) module for recording the EEG signals of the user (U),
wherein, music in the user preferred language is recommended based on real-time facial expressions of the user (U), instantly and the change in emotions validated through the EEG signals of the said user (U).

2. The Music Recommendation System (S), as claimed in claim 1, wherein facial expressions captured by the Facial Emotion Recognition (FER) module are used as input by the system (S) to recommend songs for enhancing the mood of the

Documents