image
image
user-login
Patent search/

A Novel Deep Learning based Technique for Driver Drowsiness Detection

search

Patent Search in India

  • tick

    Extensive patent search conducted by a registered patent agent

  • tick

    Patent search done by experts in under 48hrs

₹999

₹399

Talk to expert

A Novel Deep Learning based Technique for Driver Drowsiness Detection

ORDINARY APPLICATION

Published

date

Filed on 25 October 2024

Abstract

ABSTRACT A Novel Deep Learning based Technique for Driver Drowsiness Detection Every year, a significant number of road accidents occur due to driver drowsiness, resulting in numerous fatalities and injuries. To address this critical issue, the proposed invention presents a novel deep learning-based technique for real-time detection of driver alertness levels. This system categorizes a driver's state into three distinct levels: awake, moderately drowsy, and maximally drowsy. By employing a hybrid model that combines a stacked autoencoder for feature extraction with a hyperbolic tangent Long Short-Term Memory (TLSTM) network enhanced by an attention mechanism, the invention effectively improves classification accuracy and precision. The system utilizes a diverse set of biopotential signals and physiological biomarkers, including electroencephalography (EEG), facial electromyography (EMG), pulse rate, respiration rate, galvanic skin response (GSR), and head movement, to assess the driver’s drowsiness levels comprehensively. The stacked autoencoder autonomously extracts relevant features from these signals, which are then analyzed by the TLSTM network to predict the driver's alertness state. The inclusion of the attention mechanism allows the model to focus on the most impactful features, resulting in improved performance metrics such as accuracy, precision, recall, and F1-score. In addition to its immediate applications in enhancing road safety, the proposed drowsiness detection system is designed for seamless integration into existing vehicle safety frameworks. The invention's adaptable architecture allows for future enhancements, enabling the incorporation of additional parameters such as electrocardiography (ECG) signals and various vehicle dynamics. By effectively identifying and managing drowsiness-related risks, this innovative approach holds the potential to significantly reduce fatigue-related incidents, making it a valuable tool for ensuring safety in transportation and related sectors.

Patent Information

Application ID202431081553
Invention FieldBIO-MEDICAL ENGINEERING
Date of Application25/10/2024
Publication Number44/2024

Inventors

NameAddressCountryNationality
Dr. Anisha Halder RoyAssistant Professor, Institute of Radio Physics and Electronics, University of Calcutta, 92 Acharya Prafulla Chandra Road, Kolkata, West Bengal, 700009, IndiaIndiaIndia
Prithwijit MukherjeePhD Scholar, Institute of Radio Physics and Electronics, University of Calcutta, 92 Acharya Prafulla Chandra Road, Kolkata, West Bengal, 700009, IndiaIndiaIndia

Applicants

NameAddressCountryNationality
Dr. Anisha Halder RoyAssistant Professor, Institute of Radio Physics and Electronics, University of Calcutta, 92 Acharya Prafulla Chandra Road, Kolkata, West Bengal, 700009, IndiaIndiaIndia
Prithwijit MukherjeePhD Scholar, Institute of Radio Physics and Electronics, University of Calcutta, 92 Acharya Prafulla Chandra Road, Kolkata, West Bengal, 700009, IndiaIndiaIndia

Specification

Description:A Novel Deep Learning based Technique for Driver Drowsiness Detection

The present invention relates to the field of driver safety, and more particularly to a novel deep learning-based technique for detecting driver drowsiness. The invention utilizes a hybrid model based on a stacked autoencoder and hyperbolic tangent Long Short-Term Memory (TLSTM) network, coupled with attention mechanisms, to assess and predict different alertness levels (i.e., awake, moderately drowsy, and maximally drowsy) in drivers.

BACKGROUND
[0001] Background description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.
[0002] Road accidents are a leading cause of death and injury globally, with millions of fatalities and injuries occurring every year. A significant percentage of these accidents can be attributed to human error, particularly driver drowsiness. Drowsiness impairs a driver's reaction time, attention, and decision-making ability, resulting in hazardous situations on the road. This issue is especially prevalent in long-distance driving and among professional drivers, where fatigue builds up over time. While there have been efforts to reduce accidents through technological advancements in vehicle safety systems, a reliable and efficient method to detect driver drowsiness in real-time remains a challenge.
[0003] Several driver drowsiness detection systems currently exist, including those that monitor vehicle-based parameters like steering patterns, lane departure, and sudden braking. Some systems focus on facial recognition technologies to track eyelid movements and yawning. However, these methods have limitations, such as inconsistent accuracy due to varying environmental conditions, driver characteristics, and behavioral differences. Furthermore, these systems often fail to detect different levels of alertness and provide only binary assessments of whether the driver is awake or drowsy. These limitations call for a more comprehensive and accurate method of drowsiness detection that accounts for physiological and neural signals, which are more direct indicators of a driver's alertness.
[0004] Deep learning has emerged as a powerful tool for handling complex datasets, especially in fields requiring pattern recognition, such as biopotential signal processing. Stacked autoencoders are capable of automatic feature extraction from raw data, while Long Short-Term Memory (LSTM) networks excel at modeling time-series data, making them suitable for analyzing physiological signals over time. In this invention, a hybrid deep learning model is proposed that integrates a stacked autoencoder with a hyperbolic tangent LSTM (TLSTM) network and attention mechanism to predict a driver's alertness levels. This novel approach addresses the need for high accuracy in real-time detection and classification of various drowsiness levels, surpassing conventional machine learning models.
[0005] Biopotential signals, such as electroencephalography (EEG) and facial electromyography (EMG), offer a deeper insight into a driver's physiological state, including brain activity and muscle movements. Similarly, biomarkers like pulse rate, respiration rate, and galvanic skin response (GSR) provide real-time information about the driver's stress, fatigue, and drowsiness levels. The integration of these signals into drowsiness detection systems offers a more reliable approach compared to external behavioral monitoring. Advances in signal processing and machine learning now enable the use of these biopotential signals to predict varying levels of drowsiness with high accuracy, which can lead to enhanced driver safety.
[0006] The importance of real-time driver drowsiness detection cannot be overstated, as it directly impacts road safety and accident prevention. The invention introduces a robust and adaptable solution that can be integrated into modern vehicles and driver-assistance systems, offering superior performance compared to existing technologies. As future advancements in artificial intelligence and sensor technologies continue, this system could evolve to include additional parameters such as electrocardiography (ECG) signals and vehicle-specific data like wheel angle, speed, and brake pressure, further enhancing its reliability. This deep learning-based technique provides a promising avenue for reducing road accidents caused by drowsy driving and has the potential to be widely adopted in various industries where fatigue management is critical.
[0007] All publications herein are incorporated by reference to the same extent as if each individual publication or patent application were specifically and individually indicated to be incorporated by reference. Where a definition or use of a term in an incorporated reference is inconsistent or contrary to the definition of that term provided herein, the definition of that term provided herein applies and the definition of that term in the reference does not apply.
[0008] As used in the description herein and throughout the claims that follow, the meaning of "a," "an," and "the" includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein, the meaning of "in" includes "in" and "on" unless the context clearly dictates otherwise.

OBJECTS OF THE INVENTION
[0009] It is an object of the present disclosure to develop a novel deep learning-based technique that can detect different levels of driver alertness, including awake, moderately drowsy, and maximally drowsy states, with high accuracy.
[0010] It is an object of the present disclosure to employ a combination of biopotential signals such as EEG, facial EMG, and biomarkers like pulse rate, respiration rate, galvanic skin response (GSR), and head movement to evaluate a driver's drowsiness level.
[0011] It is an object of the present disclosure to develop a hybrid model that integrates a stacked autoencoder for automated feature extraction and a hyperbolic tangent LSTM (TLSTM) network with an attention mechanism to improve the classification accuracy of the driver's alertness levels.
[0012] It is an object of the present disclosure to design a drowsiness detection system that can be seamlessly integrated into modern driver-assistance systems or retrofitted into existing vehicles. The adaptability of the system ensures that it can be widely adopted across different vehicle types and industries where fatigue management is essential.
[0013] It is an object of the present disclosure to incorporate additional parameters, such as ECG signals and car-specific data (e.g., wheel angle, vehicle speed, and brake/accelerator pressure), in future iterations.
SUMMARY
[0001] The present invention presents a novel deep learning-based technique for driver drowsiness detection.
[0002] This invention presents a novel deep learning-based technique for detecting driver drowsiness using biopotential signals and biomarkers. By leveraging a hybrid model that combines a stacked autoencoder for feature extraction and a hyperbolic tangent Long Short-Term Memory (TLSTM) network with an attention mechanism, the system effectively classifies different levels of driver alertness, including awake, moderately drowsy, and maximally drowsy states. The model analyzes electroencephalography (EEG), facial electromyography (EMG), pulse rate, respiration rate, galvanic skin response (GSR), and head movement to predict the driver's drowsiness level. The proposed technique achieves high accuracy, outperforming traditional machine learning models with respect to classification precision, recall, and F1-score.
[0003] The primary objective of this invention is to enhance road safety by providing a real-time drowsiness detection system that is both accurate and adaptable. It can be integrated into existing vehicle safety systems, offering significant potential for reducing accidents caused by driver fatigue. Additionally, the invention's flexible design allows for future enhancements, such as incorporating electrocardiography (ECG) signals and vehicle-specific data, making it useful in various sectors where fatigue management is critical. This approach represents a major advancement in AI-driven drowsiness detection, contributing to improved safety and accident prevention in transportation and beyond.
[0004] One should appreciate that although the present disclosure has been explained with respect to a defined set of functional modules, any other module or set of modules can be added/deleted/modified/combined and any such changes in architecture/construction of the proposed method are completely within the scope of the present disclosure. Each module can also be fragmented into one or more functional sub-modules, all of which also completely within the scope of the present disclosure.
[0005] Various objects, features, aspects and advantages of the inventive subject matter will become more apparent from the following detailed description of preferred embodiments, along with the accompanying drawing figures in which like numerals represent like components.
BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The accompanying drawings are included to provide a further understanding of the present disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the analysis of the present disclosure.
[0015] Figure 1: A Novel Deep Learning based Technique for Driver Drowsiness Detection.
DETAILED DESCRIPTION
[0016] In the following description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. It will be apparent to one skilled in the art that embodiments of the present invention may be practiced without some of these specific details.
[0017] If the specification states a component or feature "may", "can", "could", or "might" be included or have a characteristic, that particular component or feature is not required to be included or have the characteristic.
[0018] Exemplary embodiments will now be described more fully hereinafter with reference to the drawings, in which exemplary embodiments are shown. This disclosure, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those of ordinary skill in the art. Moreover, all statements herein reciting embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure.
[0019] various terms as used herein are shown below. To the extent a term used in a claim is not defined below, it should be given the broadest definition persons in the pertinent art have given that term as reflected in printed publications and issued patents at the time of filing.
[0020] The invention focuses on a novel deep learning-based technique designed to detect varying levels of driver drowsiness in real time, ensuring road safety by preventing fatigue-related accidents. The system employs a hybrid model that integrates a stacked autoencoder and a hyperbolic tangent Long Short-Term Memory (TLSTM) network enhanced by an attention mechanism. The stacked autoencoder is responsible for automatic feature extraction from raw data, while the TLSTM network processes time-series data to predict different levels of drowsiness. The model classifies a driver's state as either awake, moderately drowsy, or maximally drowsy. This multi-level classification enables more precise detection, enhancing the system's utility in real-world applications.
[0021] The invention relies on a combination of biopotential signals and physiological biomarkers to assess a driver's alertness. These signals include electroencephalography (EEG) for brain activity monitoring, facial electromyography (EMG) for muscle movement detection, and additional physiological indicators like pulse rate, respiration rate, galvanic skin response (GSR), and head movement. Each of these signals provides crucial information about the driver's physical and mental state. The combination of such diverse data points allows for a comprehensive understanding of the driver's drowsiness levels. By focusing on internal physiological responses, the system overcomes limitations associated with external behavior-based detection methods, such as facial expression monitoring, which can be affected by environmental factors.
[0022] The hybrid model consists of two main components. First, the stacked autoencoder automatically extracts meaningful features from the input signals, reducing the need for manual intervention in the feature selection process. This automated feature extraction is critical in handling complex and high-dimensional data generated from biopotential signals. Second, the hyperbolic tangent Long Short-Term Memory (TLSTM) network processes these features to classify the driver's alertness state. The attention mechanism incorporated into the TLSTM model allows the system to focus on the most relevant features, improving classification accuracy by emphasizing key states that signal drowsiness. This attention-enhanced hybrid model offers superior performance compared to conventional machine learning models, demonstrating high precision, recall, and F1-score.
[0023] The proposed system is designed to be easily integrated into existing vehicle safety mechanisms. It can be implemented in modern driver-assistance systems to provide real-time feedback and alerts to drivers, thereby reducing the risk of accidents due to drowsiness. Additionally, the system's architecture is adaptable for future improvements. Parameters such as electrocardiography (ECG) signals, wheel angle, vehicle speed, and brake/accelerator pressure can be included to enhance the model's detection capabilities. These future enhancements will further improve the system's reliability and applicability in various sectors, including transportation, healthcare, and occupational safety, where fatigue detection is crucial for preventing accidents and improving overall safety.
[001] Data Acquisition (100): The first step involves the collection of biopotential signals and physiological biomarkers from the driver. This includes electroencephalography (EEG) to monitor brain activity, facial electromyography (EMG) for facial muscle movements, and other indicators such as pulse rate, respiration rate, galvanic skin response (GSR), and head movement. These signals are captured using wearable sensors or integrated devices in the vehicle, ensuring continuous monitoring of the driver's physiological state.
[002] Preprocessing of Data (101): The acquired signals undergo preprocessing to remove noise and artifacts. This step includes filtering techniques to clean the data, normalization to standardize the input ranges, and segmentation of the signals into relevant time windows. Preprocessing is crucial to enhance the quality of the data and ensure that the subsequent analysis is based on accurate and reliable inputs.
[003] Feature Extraction (102): In this step, the stacked autoencoder model automatically extracts meaningful features from the preprocessed data. The autoencoder compresses the input signals into lower-dimensional representations while retaining essential information. This feature extraction process reduces the dimensionality of the data, enabling more efficient processing and analysis in the following steps.
[004] Classification with TLSTM Network (103): The extracted features are then fed into the hyperbolic tangent Long Short-Term Memory (TLSTM) network for classification. The TLSTM analyzes the time-series data to predict the driver's alertness state. The attention mechanism integrated into the TLSTM model allows the network to focus on the most significant features related to drowsiness, enhancing classification accuracy. The model categorizes the driver into three states: awake, moderately drowsy, and maximally drowsy.
[005] Output Generation and Alert System (104): Based on the classification results, the system generates real-time alerts indicating the driver's current drowsiness level. If the driver is detected to be moderately drowsy or maximally drowsy, the system activates warning mechanisms, such as visual alerts on the dashboard or audible alarms, prompting the driver to take necessary actions, such as taking a break or pulling over.
[006] System Feedback and Learning (105): The final step involves collecting feedback on the system's performance over time. The model continuously learns from new data inputs to improve its accuracy and adaptability. User feedback, combined with additional data collected during driving sessions, allows for the refinement of the feature extraction and classification processes, ensuring that the system evolves and maintains high performance in various driving conditions.

Figure 2: Hybrid Classifier model
Figure 2 shows a deep learning architecture for drowsiness detection. It consists of a stacked autoencoder network, which processes an input data sequence through three autoencoders to extract features. The output is passed through two layers of a Temporal Long Short-Term Memory (TLSTM) network, where each layer has multiple TLSTM units. An attention mechanism is applied to the features from the TLSTM layers, which are then passed through a fully connected layer and a softmax layer to classify the drowsiness state as awake, moderately drowsy, or maximally drowsy.

Figure 3: Deep leaning‐based alertness detection model
The figure 3 illustrates the process flow of a deep learning-based alertness detection model. It starts with subject selection, where physiological signals like EEG, EMG, pulse rate, respiration rate, GSR, and head movements are recorded. The recorded data is processed and augmented to create a dataset, followed by designing a hybrid model. The model is then trained and evaluated for its performance, with the final output classifying the alertness levels of the subject as awake, moderately drowsy, or maximally drowsy.
, Claims:I/We Claim
Claim 1: A stacked autoencoder for automatic feature extraction from the collected signals.
• A TLSTM network with an attention mechanism for classifying the driver's alertness level into awake, moderately drowsy, and maximally drowsy states.
• A processor configured to output a real-time assessment of the driver's drowsiness level based on the classified data.
Claim 2: The system of claim 1, wherein the attention mechanism enhances the TLSTM network's ability to focus on critical patterns in the biopotential signals for improved classification accuracy.
Claim 3: The system of claim 1, wherein the classification accuracy of the model for detecting awake, moderately drowsy, and maximally drowsy states is 99%, 98.3%, and 98.6%, respectively.
Claim 4: The system of claim 1, further comprising a feedback mechanism to alert the driver in case of detected drowsiness.

Documents

NameDate
202431081553-COMPLETE SPECIFICATION [25-10-2024(online)].pdf25/10/2024
202431081553-DECLARATION OF INVENTORSHIP (FORM 5) [25-10-2024(online)].pdf25/10/2024
202431081553-FORM 1 [25-10-2024(online)].pdf25/10/2024
202431081553-FORM-9 [25-10-2024(online)].pdf25/10/2024
202431081553-POWER OF AUTHORITY [25-10-2024(online)].pdf25/10/2024
202431081553-REQUEST FOR EARLY PUBLICATION(FORM-9) [25-10-2024(online)].pdf25/10/2024

footer-service

By continuing past this page, you agree to our Terms of Service,Cookie PolicyPrivacy Policy  and  Refund Policy  © - Uber9 Business Process Services Private Limited. All rights reserved.

Uber9 Business Process Services Private Limited, CIN - U74900TN2014PTC098414, GSTIN - 33AABCU7650C1ZM, Registered Office Address - F-97, Newry Shreya Apartments Anna Nagar East, Chennai, Tamil Nadu 600102, India.

Please note that we are a facilitating platform enabling access to reliable professionals. We are not a law firm and do not provide legal services ourselves. The information on this website is for the purpose of knowledge only and should not be relied upon as legal advice or opinion.