SIGN LANGUAGE RECOGNITION SYSTEM FOR NUMERICAL GESTURES

Abstract

Disclosed herein is a system (100) for real-time recognition of numerical sign language gestures that comprises a camera (102) configured to capture video frames of a user's hand movements, a microprocessor (108) connected to the camera (102) and configured to process hand gestures, further comprising a data input module (112) configured to receive video frames from the camera (102), a pre-processing module (114) configured to convert the continuous video frames into discrete frames, process them, and isolate the hand gestures by applying segmentation techniques, a feature extraction module (116) configured to extract relevant gesture features from the pre-processed video frames, a gesture classification module (120) configured to classify the extracted features into corresponding numerical values using a convolutional neural network, an output module (122) configured to collect and transmit the recognized numerical gesture for display, and a user device (110) configured to display the recognized numerical gesture.

Specification

Description:FIELD OF DISCLOSURE
[0001] The present disclosure generally relates to assistive communication system, more specifically, relates to a real-time sign language recognition system for numerical gestures based on deep learning models.
BACKGROUND OF THE DISCLOSURE
[0002] Assistive communication systems are designed to aid individuals who face challenges in communicating due to disabilities, such as hearing loss, speech impairments, or other cognitive conditions. These systems utilize various technologies to facilitate communication, ensuring that individuals can interact more effectively with others. The goal of such systems is to enhance the quality of life for people with communication barriers by promoting inclusivity, independence, and access to essential services.
[0003] Building on the advancements in assistive communication systems, there is a growing need for a sign language detection system tailored for numerical gestures. Numbers play a crucial role in everyday interactions for counting, labelling, and measuring objects, as well as in tasks like managing time, finances, and quantities. A dedicated system for recognizing numerical gestures can greatly enhance communication, providing individuals with hearing impairments a more effective, accessible, and autonomous way to convey numerical information.
[0004] Traditional systems are often slow, prone to error, and lack real-time capabilities. Many systems rely on human interpreters or manual inputs, limiting their scalability and portability. Some existing systems use external sensors, such as hand gloves or motion sensors, to capture hand gestures. While these systems can provide good accuracy, they are often intrusive, expensive, and require users to wear or carry additional devices. This dependency on hardware can create discomfort and inconvenience for users, reducing practicality and adoption rates.
[0005] Early sign language recognition methods using rule-based computer vision techniques suffer from limitations such as low accuracy, poor generalization to different users, and sensitivity to environmental factors like lighting and background noise. These systems typically rely on handcrafted features and simple image processing techniques that fail to capture the full complexity of human gestures. Moreover, many traditional systems focus on a limited set of gestures, neglecting numerical gestures or requiring additional training to recognize them, limiting their practical utility in applications that require both alphabetical and numerical sign language recognition.
[0006] Additionally, traditional systems often lack real-time recognition due to the computational complexity of traditional machine learning models. They either process the data offline or have significant latency, rendering them unsuitable for dynamic and fast-paced environments such as live communication or instruction. Traditional gesture recognition systems are often rigid in terms of recognizing variations in gestures across different individuals, such as hand sizes, orientations, and movement speeds. This lack of adaptability makes them prone to errors when used by a diverse set of users, affecting the system's overall robustness.
[0007] The present invention overcomes the limitations of the prior art by providing a real-time sign language recognition system specifically designed for accurate and efficient recognition of numerical gestures (0-9) using deep learning models. Unlike conventional systems that struggle with real-time performance, the present invention employs advanced convolutional neural networks (CNNs) to achieve real-time efficiency, high accuracy and low latency in recognizing numerical hand gestures.
[0008] The present invention operates through non-intrusive computer vision methods, eliminating the need for additional wearable devices or specialized hardware, making it portable, cost-effective, user-friendly, and accessible on common computing devices such as smartphones, tablets, and personal computers. This enables seamless communication for individuals with hearing impairments in everyday settings. Additionally, the present invention is adaptable to different users, handling variations in hand sizes, gesture speeds, and orientations with reduced errors. The continuous updating capability ensures that the system remains effective and can evolve to meet changing user needs.
[0009] Thus, in light of the above-stated discussion, there exists a need for a real-time sign language recognition system for numerical gestures.
SUMMARY OF THE DISCLOSURE
[0010] The following is a summary description of illustrative embodiments of the invention. It is provided as a preface to assist those skilled in the art to more rapidly assimilate the detailed design discussion which ensues and is not intended in any way to limit the scope of the claims which are appended hereto in order to particularly point out the invention.
[0011] According to illustrative embodiments, the present disclosure focuses on a real-time sign language recognition system for numerical gestures which overcomes the above-mentioned disadvantages or provide the users with a useful or commercial choice.
[0012] The present invention solves the above major limitations of a real-time sign language recognition system for numerical gestures.
[0013] An objective of the present disclosure is to provide a real-time sign language recognition system for numerical gestures that enhances communication accessibility for individuals with hearing impairments.
[0014] Another objective of the present disclosure is to leverage deep learning models, specifically convolutional neural networks (CNNs), to accurately recognize numerical hand gestures to ensure high recognition accuracy and efficiency in real-time applications.
[0015] Another objective of the present disclosure is to provide a system that continuously updates and retrains based on new numerical gesture data, allowing the system to improve recognition accuracy and adapt to diverse users and changing gesture patterns over time.
[0016] Another objective of the present disclosure is to provide a system that can adapt to different hand sizes, shapes, and gestures, making it suitable for various users without the need for customization.
[0017] Another objective of the present disclosure is to provide a system that is user-friendly, highly efficient, and portable to ensure accessibility, ease of use, and convenience across various environments.
[0018] Yet another objective of the present disclosure is to provide consistent accuracy in gesture recognition across various lighting conditions and backgrounds.
[0019] In light of the above, in one aspect of the present disclosure, a system for real-time recognition of numerical sign language gestures is disclosed herein. The system comprises a camera configured to capture video frames of a user's hand movements. The system also includes a microprocessor connected to the camera and configured to process hand gestures, wherein the microprocessor further comprises: a data input module configured to receive video frames captured by the camera, a pre-processing module configured to convert the continuous video frames into discrete frames, process the discrete frames and isolate the hand gestures by applying segmentation techniques, including background subtraction and noise reduction, a feature extraction module configured to extract relevant gesture features such as hand orientation, shape, and finger positioning from the pre-processed video frames, a gesture classification module configured to classify the extracted features into corresponding numerical values (0-9) using a convolutional neural network trained on a dataset of numerical hand gestures, an output module configured to collect the classification results and transmit the recognized numerical gesture for display. The system also includes a user device configured to display the recognized numerical gesture in real-time communicably connected to the microprocessor via a communication network.
[0020] In one embodiment, the pre-processing module ensures robust performance and high accuracy of numerical gesture recognition under various environmental conditions, including different lighting and background scenarios.
[0021] In one embodiment, the pre-processing module performs gesture segmentation by detecting and isolating the hand gesture region of interest, applying contour detection to identify the hand, and generating a threshold image for subsequent feature extraction.
[0022] In one embodiment, the system further comprises a training and testing module configured to split the processed data into training and testing datasets, and train the convolutional neural network model on the training data.
[0023] In one embodiment, the training and testing module further includes mechanisms for iterative refinement based on feedback from the testing dataset to improve model performance.
[0024] In one embodiment, the system further comprises a database configured to store the processed data as part of the training dataset for the convolutional neural network, to enable efficient retrieval and updating of the dataset with new data for model training and real-time gesture recognition.
[0025] In one embodiment, the database comprises a labelled training dataset of images depicting various users performing numerical gestures under diverse lighting and background conditions to enable the convolutional neural network model to generalize effectively across a wide range of real-world environments.
[0026] In light of the above, in another aspect of the present disclosure, a method for real-time recognition of numerical sign language gestures is disclosed herein. The method comprises capturing video frames of a user's hand movements via a camera. The method also includes processing hand gestures via a microprocessor comprising of several modules. The method also includes receiving video frames captured by the camera via a data input module. The method also includes converting the continuous video frames into discrete frames, processing the discrete frames, and isolating the hand gestures by applying segmentation techniques, including background subtraction and noise reduction via a pre-processing module. The method also includes extracting relevant gesture features such as hand orientation, shape, and finger positioning from the pre-processed video frames via a feature extraction module. The method also includes classifying the extracted features into corresponding numerical values (0-9) using a convolutional neural network trained on a dataset of numerical hand gestures via a gesture classification module. The method also includes collecting the classification results and transmitting the recognized numerical gesture for display via an output module. The method also includes displaying the recognized numerical gesture in real-time via a user device.
[0027] These and other advantages will be apparent from the present application of the embodiments described herein.
[0028] The preceding is a simplified summary to provide an understanding of some embodiments of the present invention. This summary is neither an extensive nor exhaustive overview of the present invention and its various embodiments. The summary presents selected concepts of the embodiments of the present invention in a simplified form as an introduction to the more detailed description presented below. As will be appreciated, other embodiments of the present invention are possible utilizing, alone or in combination, one or more of the features set forth above or described in detail below.
[0029] These elements, together with the other aspects of the present disclosure and various features are pointed out with particularity in the claims annexed hereto and form a part of the present disclosure. For a better understanding of the present disclosure, its operating advantages, and the specified object attained by its uses, reference should be made to the accompanying drawings and descriptive matter in which there are illustrated exemplary embodiments of the present disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
[0030] To describe the technical solutions in the embodiments of the present disclosure or in the prior art more clearly, the following briefly describes the accompanying drawings required for describing the embodiments or the prior art. Apparently, the accompanying drawings in the following description merely show some embodiments of the present disclosure, and a person of ordinary skill in the art can derive other implementations from these accompanying drawings without creative efforts. All of the embodiments or the implementations shall fall within the protection scope of the present disclosure.
[0031] The advantages and features of the present disclosure will become better understood with reference to the following detailed description taken in conjunction with the accompanying drawing, in which:
[0032] FIG. 1 illustrates a block diagram of a real-time sign language recognition system for numerical gestures, in accordance with an exemplary embodiment of the present disclosure;
[0033] FIG. 2 illustrates a workflow depicting the step-by-step process for the execution of the real-time sign language recognition system 100 for numerical gestures, in accordance with an exemplary embodiment of the present disclosure.
[0034] FIGS. 3A and 3B illustrates hand gestures 300 showing the recognition of different numerical values by the real-time sign language recognition system, in accordance with an exemplary embodiment of the present disclosure; and
[0035] FIG. 4 illustrates a flowchart of a method, outlining the sequential steps for recognizing numerical sign language gestures in real-time, in accordance with an exemplary embodiment of the present disclosure.
[0036] Like reference, numerals refer to like parts throughout the description of several views of the drawing.
[0037] The real-time sign language recognition system for numerical gestures is illustrated in the accompanying drawings, which like reference letters indicate corresponding parts in the various figures. It should be noted that the accompanying figure is intended to present illustrations of exemplary embodiments of the present disclosure. This figure is not intended to limit the scope of the present disclosure. It should also be noted that the accompanying figure is not necessarily drawn to scale.
DETAILED DESCRIPTION OF THE DISCLOSURE
[0038] The following is a detailed description of embodiments of the disclosure depicted in the accompanying drawings. The embodiments are in such detail as to communicate the disclosure. However, the amount of detail offered is not intended to limit the anticipated variations of embodiments; on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure.
[0039] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be apparent to one skilled in the art that embodiments of the present disclosure may be practiced without some of these specific details.
[0040] Various terms as used herein are shown below. To the extent a term is used, it should be given the broadest definition persons in the pertinent art have given that term as reflected in printed publications and issued patents at the time of filing.
[0041] The terms "a" and "an" herein do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced items.
[0042] The terms "having", "comprising", "including", and variations thereof signify the presence of a component.
[0043] Referring now to FIG. 1 to FIG. 4 to describe various exemplary embodiments of the present disclosure. FIG. 1 illustrates a block diagram of a real-time sign language recognition system 100 for numerical gestures, in accordance with an exemplary embodiment of the present disclosure.
[0044] The system 100 may include a camera 102, a microprocessor 108 comprising of several modules, and a user device 110.
[0045] The camera 102 acts as a primary source of visual input in the system 100 and is responsible for capturing live video frames of a user's hand movements. The camera 100 continuously records the user's gestures in a defined area and ensures that hand movements are accurately captured. By capturing video frames in real-time, the camera 100 enables dynamic gesture recognition and aids the system 100 in maintaining a steady flow of input data to achieve instantaneous classification. The camera 102 in the system 100 may be any high-resolution device capable of capturing detailed video frames, such as a high-resolution webcam or a built-in device camera in smartphones, tablets, or laptops.
[0046] The microprocessor 108 in the system 100 is responsible for processing the data received from the camera 102 to accurately recognize the user's hand gestures using a convolutional neural network (CNN) model. The microprocessor 108 is equipped with several modules including, a data input module 112, a pre-processing module 114, a feature extraction module 116, a gesture classification module 120, and an output module 122 each of which contributes to different aspects of gesture recognition.
[0047] The data input module 112 is configured to receive the video frames captured by the camera 102 and make them accessible to the following modules within the microprocessor 108 for processing. The primary role of the data input module 112 is to pass the continuous video frames to the next stages, ensuring that the data flows continuously from capture to processing.
[0048] The pre-processing module 114 plays a crucial role in preparing the raw data for analysis. It converts the continuous video stream into discrete frames, which are then processed to enhance visibility and isolate the hand gesture. The pre-processing module 114 performs essential segmentation techniques such as background subtraction, to differentiate the hand from the surrounding area, and noise reduction, to eliminate visual artifacts and shadows that may interfere with recognition accuracy. By creating clean, discrete images of the gesture in each frame, the pre-processing module 114 enables subsequent modules to analyse the gestures more effectively. The pre-processing module 114 pre-processes the images by resizing them, converting them to grayscale, normalizing pixel values, and applying augmentation techniques.
[0049] In one embodiment of the present invention, the pre-processing module 114 ensures robust performance and high accuracy of numerical gesture recognition under various environmental conditions, including different lighting and background scenarios. It normalizes brightness and contrast to handle different lighting scenarios and uses background subtraction to isolate the hand from complex environments.
[0050] In one embodiment of the present invention, the pre-processing module 114 performs gesture segmentation by detecting and isolating the hand gesture region of interest, applying contour detection to identify the hand, and generating a threshold image for subsequent feature extraction. The pre-processing module 114 identifies the region of interest (ROI) where the hand is likely located within the frame, utilizing techniques such as skin colour detection and background subtraction. Once the general area of the hand is determined, the module applies thresholding to obtain a binary image. During thresholding, the pixels within the hand's region are assigned one value (typically white), while the remaining areas are assigned another value (typically black). The pre-processing module 114 then detects contours in the binary image to accurately outline the hand's shape and selects the contour with the largest area as the hand gesture. The module then returns the threshold image and the contour of the hand gesture for subsequent feature extraction.
[0051] In one embodiment of the present invention, the pre-processing module 114 uses the OpenCV library for image processing and hand gesture segmentation.
[0052] The feature extraction module 116 is responsible for isolating key features from the pre-processed frames that are relevant to identifying specific gestures. These features include details such as hand orientation, shape, and finger positioning. By extracting and highlighting these attributes, the feature extraction module 116 translates the processed image data into information that represents the gesture characteristics, simplifying the classification task. These extracted features serve as inputs for the gesture classification module 120, providing it with a refined dataset to ensure accurate recognition.
[0053] In one embodiment of the present invention, the system 100 further comprises a training and testing module 118 configured to split the processed data into training and testing datasets, and train the CNN model on the training data. The training dataset is used to teach the CNN model by allowing it to learn patterns and features associated with different numerical hand gestures (0-9). Once trained, the CNN model is validated and tested using the testing dataset to evaluate its generalization capabilities and ensure it performs accurately on new, unseen data.
[0054] In one embodiment of the present invention, the training and testing module 118 further includes mechanisms for iterative refinement based on feedback from the testing dataset to improve model performance. Based on the feedback from the testing dataset, the training and testing module 118 refines the model's performance by adjusting internal parameters or re-training it with the additional data. This iterative process helps the system 100 fine-tune the gesture recognition model, improving its accuracy and robustness in recognizing different hand gestures under various conditions. The process continues until the model achieves a high level of precision in identifying numerical hand gestures in real-time.
[0055] In one embodiment of the present invention, the system 100 further comprises a database 106 configured to store the processed data as part of the training dataset for the CNN, to enable efficient retrieval and updating of the dataset with new data for model training and real-time gesture recognition. The database 106 ensures that the system 100 can easily access and update key data components to enhance performance, streamline the training process, and support real-time gesture recognition.
[0056] In one embodiment of the present invention, the database 106 comprises a labelled training dataset of images depicting various users performing numerical gestures under diverse lighting and background conditions to enable the CNN model to generalize effectively across a wide range of real-world environments. The diverse dataset of labelled images ensures that the CNN model can accurately recognize gestures despite variations in lighting, background, and hand positioning. By including images from multiple users, the model generalizes across different hand sizes, orientations, and movement speeds. This variety enhances the model's robustness, adaptability, and accuracy in real-world environments, reducing errors and improving reliability.
[0057] The gesture classification module 120 plays an important role in the numerical gesture recognition capability of the system 100. This module utilizes the pre-trained CNN model stored in the database 106 to recognize numerical gestures based on learned patterns and features. Based on the input provided by the feature extraction module 116, the gesture classification module 120 performs classification to determine the numerical value associated with each gesture. By comparing the extracted features against the trained patterns, the gesture classification module 120 accurately labels the gesture and generates a result in real-time.
[0058] In one embodiment of the present invention, the system 100 utilizes the American Sign Language convention, to interpret and classify numerical gestures accurately.
[0059] The output module 122 collects the classified gesture from the gesture classification module 120 and transmits it to the user device 110 for real-time display. The output module 122 ensures that the recognized numerical gesture reaches the end-user without delay, maintaining the responsiveness of the system 100. The output module 122 also ensures compatibility with various user devices, providing flexible communication options to support different types of displays.
[0060] The user device 110 is connected to the microprocessor 108 through a communication network 104, which can be either a local network or a broader internet connection, facilitating data transfer. The communication network 104 may include wired or wireless connections such as ethernet, Wi-Fi, or cellular networks, depending on the system's deployment environment. The user device 110 is designed to display the recognized numerical gesture in real-time and can be a smartphone, tablet, or computer, offering flexibility for the system's use across various platforms. The user device 110 receives data from the output module 122 and visually displays the recognized gesture, providing immediate feedback to the user. The user device 110 is essential for enhancing accessibility, as it enables individuals to interact with the system 100 conveniently and receive instant, visual recognition of their gestures.
[0061] FIG. 2 illustrates a workflow 200 depicting the step-by-step process for the execution of the real-time sign language recognition system 100 for numerical gestures, in accordance with an exemplary embodiment of the present disclosure.
[0062] The workflow 200 begins at step 202, with the camera 102 detecting the presence of a hand within its field of view. The camera 102 streams video frames, which are then transmitted to a microprocessor 108 that processes the data to recognize hand gestures. The microprocessor 108 integrates several modules. The data input module 112 is configured to receive video frames from the camera 102 and transmitting it further processing. The pre-processing module 114 is responsible for converting the continuous video frames into discrete frames, applying segmentation techniques, and isolating the hand from the background. Hand detection algorithms, including skin color detection and contour finding, are applied by the pre-processing module 114 to isolate the hand from the background. The feature extraction module is responsible for extracting relevant features from the pre-processed video frames. At step 204, the pre-processed and extracted images are then labelled and captured in a database 106 in the directory of the system 100.
[0063] At step 206, after collecting the labelled training data, the training and testing module 118 is responsible for training the system 100 using the CNN model. The model is trained with the training and testing module 118, which adjusts internal weights using loss functions and optimization algorithms. At step 208, the CNN model is stored in the database 106 of the directory for future use once trained. The trained model is stored in the database 106 for quick loading during real-time hand gesture recognition.
[0064] At step 210, the detection of hand gestures is done by the gesture classification module 120, which predicts the numerical gesture (0-9) based on learned features. The output module 122 is responsible for displaying the predicted gesture in real-time on the user device 110. The system 100 continuously updates the gesture classification module 120 based on incoming frames from the camera 102. At step 212, the predicted numerical gesture label is displayed on the user device 110, typically in the top-left corner of the video stream window. For instance, if the hand gesture corresponds to "4," the system will display "four" in the corner of the window. As the camera 102 continues to capture frames, the system 100 continuously updates the prediction based on the current hand gesture detected in each frame. This allows for dynamic and real-time recognition of gestures as the user performs them.
[0065] FIGS. 3A and 3B illustrates hand gestures 300 showing the recognition of different numerical values by the real-time sign language recognition system 100, in accordance with an exemplary embodiment of the present disclosure.
[0066] The hand gestures 300 as illustrated in FIG. 3A and FIG. 3B are captured within a ROI 302, 306. The ROI 302, 306 defines the area where the system 100 focuses its gesture recognition process. A rectangular box marks the ROI 302, 306 on the screen, serving as a boundary for where the hand must be placed for recognition. The system 100 isolates this area from the rest of the background, ensuring that only the hand inside the box is processed for gesture recognition. In FIG. 3A, within the ROI 302, the user forms a closed fist, representing the numerical gesture for zero. The system 100 applies segmentation, feature extraction techniques, and thresholding to generate a first binary image 304. Using the first binary image 304, the system 100 detects the contours, analyses the hand gesture, and classifies the hand gesture into a numerical value. The closed loop formed by the fingers in the fist is identified by the system 100 as a distinctive feature representing the "zero" gesture.
[0067] Similarly, in FIG. 3B, the user extends four fingers while keeping the thumb bent inward, representing the numerical gesture for four. The system 100 applies segmentation and feature extraction techniques, along with thresholding, to obtain a second binary image 308, detects the contours, and recognizes the shape of the hand gesture. The system 100 identifies the extended fingers as key features, classifies the gestures into numerical values, and recognizes the "four" gesture based on the number and positioning of the fingers within the second binary image 308.
[0068] FIG. 4 illustrates a flowchart of a method 400, outlining the sequential steps for recognizing numerical sign language gestures in real-time, in accordance with an exemplary embodiment of the present disclosure.
[0069] The method 400 may include, at step 402, capturing video frames of a user's hand movements via a camera, at step 404, processing hand gestures via a microprocessor comprising of several modules, at step 406, receiving video frames captured by the camera via a data input module, at step 408, converting the continuous video frames into discrete frames, processing the discrete frames, and isolating the hand gestures by applying segmentation techniques, including background subtraction and noise reduction via a pre-processing module, at step 410, extracting relevant gesture features such as hand orientation, shape, and finger positioning from the pre-processed video frames via a feature extraction module, at step 412, classifying the extracted features into corresponding numerical values (0-9) using a CNN trained on a dataset of numerical hand gestures via a gesture classification module, at step 414, collecting the classification results and transmitting the recognized numerical gesture for display via an output module, and at step 416, displaying the recognized numerical gesture in real-time via a user device.
[0070] While the invention has been described in connection with what is presently considered to be the most practical and various embodiments, it will be understood that the invention is not to be limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims.
[0071] A person of ordinary skill in the art may be aware that, in combination with the examples described in the embodiments disclosed in this specification, units and algorithm steps may be implemented by electronic hardware, computer software, or a combination thereof.
[0072] The foregoing descriptions of specific embodiments of the present disclosure have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed, and many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described to best explain the principles of the present disclosure and its practical application, and to thereby enable others skilled in the art to best utilize the present disclosure and various embodiments with various modifications as are suited to the particular use contemplated. It is understood that various omissions and substitutions of equivalents are contemplated as circumstances may suggest or render expedient, but such omissions and substitutions are intended to cover the application or implementation without departing from the scope of the present disclosure.
[0073] Disjunctive language such as the phrase "at least one of X, Y, Z," unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and/or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require at least one of X, at least one of Y, or at least one of Z to each be present.
[0074] In a case that no conflict occurs, the embodiments in the present disclosure and the features in the embodiments may be mutually combined. The foregoing descriptions are merely specific implementations of the present disclosure, but are not intended to limit the protection scope of the present disclosure. Any variation or replacement readily figured out by a person skilled in the art within the technical scope disclosed in the present disclosure shall fall within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
, Claims:I/We Claim:
1. A system (100) for recognizing numerical sign language gestures in real-time, the system (100) comprising:
a camera (102) configured to capture video frames of a user's hand movements;
a microprocessor (108) connected to the camera (102) and configured to process hand gestures, wherein the microprocessor (108) further comprises:
a data input module (112) configured to receive video frames captured by the camera (102);
a pre-processing module (114) configured to convert the continuous video frames into discrete frames, process the discrete frames, and isolate the hand gestures by applying segmentation techniques, including background subtraction and noise reduction;
a feature extraction module (116) configured to extract relevant gesture features such as hand orientation, shape, and finger positioning from the pre-processed video frames;
a gesture classification module (120) configured to classify the extracted features into corresponding numerical values (0-9) using a convolutional neural network trained on a dataset of numerical hand gestures;
an output module (122) configured to collect the classification results and transmit the recognized numerical gesture for display; and
a user device (110) communicably connected to the microprocessor (108) via a communication network (104) and configured to display the recognized numerical gesture in real-time.
2. The system (100) as claimed in claim 1, wherein the pre-processing module (114) ensures robust performance and high accuracy of numerical gesture recognition under various environmental conditions, including different lighting and background scenarios.
3. The system (100) as claimed in claim 1, wherein the pre-processing module (114) performs gesture segmentation by detecting and isolating the hand gesture region of interest, applying contour detection to identify the hand, and generating a threshold image for subsequent feature extraction.
4. The system (100) as claimed in claim 1, wherein the system (100) further comprises a training and testing module (118) configured to split the processed data into training and testing datasets, and train the convolutional neural network model on the training data.
5. The system (100) as claimed in claim 1, wherein the training and testing module (118) further includes mechanisms for iterative refinement based on feedback from the testing dataset to improve model performance.
6. The system (100) as claimed in claim 1, wherein the system (100) further comprises a database (106) configured to store the processed data as part of the training dataset for the convolutional neural network, to enable efficient retrieval and updating of the dataset with new data for model training and real-time gesture recognition.
7. The system (100) as claimed in claim 1, wherein the database (106) comprises a labelled training dataset of images depicting various users performing numerical gestures under diverse lighting and background conditions to enable the convolutional neural network model to generalize effectively across a wide range of real-world environments.
8. A method (400) for recognizing numerical sign language gestures in real-time, the method (400) comprising:
capturing video frames of a user's hand movements via a camera (102);
processing hand gestures via a microprocessor (108) comprising of several modules;
receiving video frames captured by the camera (102) via a data input module (112);
converting the continuous video frames into discrete frames, processing the discrete frames, and isolating the hand gestures by applying segmentation techniques, including background subtraction and noise reduction via a pre-processing module (114);
extracting relevant gesture features such as hand orientation, shape, and finger positioning from the pre-processed video frames via a feature extraction module (116);
classifying the extracted features into corresponding numerical values (0-9) using a convolutional neural network trained on a dataset of numerical hand gestures via a gesture classification module (120);
collecting the classification results and transmitting the recognized numerical gesture for display via an output module (122); and
displaying the recognized numerical gesture in real-time via a user device (110).

Documents