Consult an Expert
Trademark
Design Registration
Consult an Expert
Trademark
Copyright
Patent
Infringement
Design Registration
More
Consult an Expert
Consult an Expert
Trademark
Design Registration
Login
A Wearable Gesture-To-Speech Translation System And A Method Thereof
Extensive patent search conducted by a registered patent agent
Patent search done by experts in under 48hrs
₹999
₹399
Abstract
Information
Inventors
Applicants
Specification
Documents
ORDINARY APPLICATION
Published
Filed on 14 November 2024
Abstract
The present invention is related to a wearable gesture-to-speech translation system. The invention relates to a wearable gesture-to-speech translation system (100) that enables individuals with speech disabilities to communicate through hand gestures. The system includes a microcontroller (10) interfaced with a plurality of flux sensors (120) that detect variations in hand gestures by measuring resistance changes corresponding to the degree of bend. The sensor data is converted and processed to interpret specific gestures, which are then matched to pre-recorded vocal messages stored in an APR9600 voice module (130). The voice module plays these messages through a speaker (150), providing audible communication, while an LCD display (140) presents a text output of the interpreted gestures. The flux sensors are arranged on a glove for capturing complex gestures, and the system is programmed via an Arduino IDE for customizable gesture mappings.
Patent Information
Application ID | 202441088163 |
Invention Field | COMPUTER SCIENCE |
Date of Application | 14/11/2024 |
Publication Number | 47/2024 |
Inventors
Name | Address | Country | Nationality |
---|---|---|---|
Dr. A. BALAMANIKANDAN | Associate Professor, Department of Electronics and Communications, School of Computing, Mohan Babu University (Erstwhile Sree Vidyanikethan Engineering College), A. Rangampet, Tirupati-517102, INDIA | India | India |
Ms. R BINDU MADHAVI | UG Scholar, Department of Electronics and Communications, School of Engineering, Mohan Babu University (Erstwhile Sree Vidyanikethan Engineering College), A. Rangampet, Tirupati-517102, INDIA | India | India |
Mr. PUDI ABHIRAM REDDY | UG Scholar, Department of Electronics and Communications, School of Engineering, Mohan Babu University (Erstwhile Sree Vidyanikethan Engineering College), A. Rangampet, Tirupati-517102, INDIA | India | India |
Mr. PINJARI AMEER SOHEL | UG Scholar, Department of Electronics and Communications, School of Engineering, Mohan Babu University (Erstwhile Sree Vidyanikethan Engineering College), A. Rangampet, Tirupati-517102, INDIA | India | India |
Mr. PAMALA VIJAY KUMAR | UG Scholar, Department of Electronics and Communications, School of Engineering, Mohan Babu University (Erstwhile Sree Vidyanikethan Engineering College), A. Rangampet, Tirupati-517102, INDIA | India | India |
Applicants
Name | Address | Country | Nationality |
---|---|---|---|
MOHAN BABU UNIVERSITY | IPR Cell, Mohan Babu University (Erstwhile Sree Vidyanikethan Engineering College), Tirupati, Andhra Pradesh, India - 517102 | India | India |
Specification
Description:The present invention relates to wearable gesture-to-speech translation system and a method thereof. Figure 1 illustrates the structural components of wearable gesture-to-speech translation system. The present invention is a wearable gesture-to-speech translation system (100) designed to help individuals with speech disabilities communicate through hand gestures. At the core of the system is a microcontroller (110) interfaced with a plurality of flux sensors (120) that detect variations in hand posture. These sensors measure resistance changes according to the degree of bend in the user's hand, capturing specific hand gestures. The microcontroller receives this sensor data and processes it through analog-to-digital conversion, interpreting each gesture into a corresponding digital signal that the system can translate into a specific output.
An APR9600 voice module (130) is incorporated to store, record, and play back pre-recorded vocal messages associated with the interpreted gestures. This module enables both sequential and random access of messages, allowing flexibility in playback. Once a gesture is interpreted, the APR9600 retrieves the appropriate vocal message and sends it to a connected speaker (150), where it is audibly output as speech, effectively enabling hands-free communication. This allows the user to communicate in real-time by simply performing specific gestures that the system translates into spoken language. Additionally, an LCD display unit (140) is integrated into the system, visually outputting each interpreted gesture as text, providing immediate visual feedback for the user or others nearby.
The flux sensors (120) are arranged on a glove worn by the user, allowing detection of multiple degrees of hand movement. This configuration enhances the system's accuracy in translating complex gestures into corresponding speech outputs, as each gesture is associated with distinct resistance changes that the system can precisely detect and interpret. Furthermore, the microcontroller (110) is an Arduino device, which can be programmed via an Integrated Development Environment (IDE). This IDE allows customization of gesture mappings, providing flexibility to adapt the system to different users' needs by assigning specific gestures to particular vocal messages.
The LCD display unit (140) in the system serves as a liquid crystal display designed to show the text corresponding to each vocal message, making it accessible to both the user and those around them. This feature supports enhanced communication, as it allows people in the user's vicinity to see a visual representation of each spoken message, adding a layer of clarity to the user's expression. The APR9600 voice module (130) also includes customizable message storage and playback duration, permitting users to adjust sample rates as needed to optimize both audio quality and storage space. This adaptability ensures the system remains effective and user-friendly in various settings.
The system includes a power management module that efficiently controls power supply to the flux sensors, microcontroller, voice module, and display unit, thus supporting extended battery life. This feature is essential for a wearable device, as it ensures the system can be used continuously without frequent recharging. This gesture-to-speech translation system (100) offers a practical, inclusive communication tool for individuals with speaking disabilities, making it easier for them to communicate through a combination of audio and visual outputs.
In the wearable gesture-to-speech translation system, the flux sensors (120) are strategically attached to a glove worn by the user. Each sensor is positioned along key joints of the fingers and hand to capture a broad range of hand movements and degrees of bend. These sensors function as the initial input devices, detecting resistance changes as the user moves their hand. Each flux sensor is wired directly to the microcontroller (110), transmitting the resistance-based analog signals corresponding to various gestures. This layout ensures that complex hand movements are accurately captured as data for interpretation.
The microcontroller (110), an Arduino device, is the system's central processing hub. It receives the analog signals from the flux sensors and uses analog-to-digital conversion (ADC) to translate these signals into digital data. Each specific pattern of sensor data is recognized as a particular hand gesture, which the microcontroller then maps to a pre-defined vocal message. The microcontroller is programmed via an Integrated Development Environment (IDE) to enable easy customization, allowing specific gestures to be mapped to particular vocal outputs. This flexibility is essential for personalizing the device based on individual user needs or language preferences.
Once the microcontroller identifies a gesture and assigns it to a vocal message, it transmits the command to the APR9600 voice module (130). This module is responsible for storing, retrieving, and playing pre-recorded messages. The APR9600 accesses the designated message either sequentially or randomly, based on the interpreted gesture, and sends it to a connected speaker (150) for audible output. This speaker, positioned for clear sound projection, converts the stored audio message into speech, effectively conveying the user's intended message in real-time.
Simultaneously, the microcontroller sends a signal to an LCD display unit (140) to show the text corresponding to the spoken message. The LCD screen provides immediate visual feedback of the interpreted gesture, which is beneficial for the user and those around them. This combination of audio and visual output enhances clarity in communication, especially in noisy environments where the audio might not be as easily heard.
To support all components with reliable power, a power management module is integrated within the system. This module distributes power efficiently to the flux sensors, microcontroller, voice module, and display, conserving battery life for continuous wearable use. This comprehensive connectivity structure allows each component to interact seamlessly, creating a user-friendly, responsive device capable of real-time gesture-to-speech translation.
The Arduino Mega 2560 is a microcontroller board based on the ATmega2560 chipset. It features 54 digital I/O pins, 16 analog inputs, and 4 UARTs (hardware serial ports), making it ideal for more complex projects that require a larger number of pins and more communication channels. The board operates with a 16 MHz crystal oscillator, providing stable performance, and includes a USB connection, power jack, ICSP header, and a reset button. To begin using the Mega 2560, simply connect it to a computer via USB or power it through an AC-to-DC adapter or battery.
This board can be programmed using the Arduino IDE, which simplifies the development process. The ATmega2560 is preloaded with a boot loader, allowing for easy code uploads without the need for an external programmer. The board uses the STK500 protocol for
communication and can also be programmed directly through the ICSP header if preferred. For more advanced use, the firmware source code for the ATmega16U2 chip is available in the Arduino repository, providing further customization options. The Arduino Mega 2560 can be powered either through a USB connection or an external power supply, with the board automatically selecting the power source. External power can be supplied via an AC-to-DC adapter (commonly referred to as a wall-wart) or a battery. The adapter connects to the board's power jack via a 2.1mm center-positive plug. For battery-powered operation, the leads from a battery can be inserted into the GND and Vin pin headers on the power connector. The board is designed to operate with an external supply voltage ranging from 6 to 20 volts. However, if the supplied voltage is less than 7V, the 5V pin may output less than five volts, potentially causing instability. On the other hand, a supply voltage greater than 12V can cause the voltage regulator to overheat and damage the board. Thus, the recommended voltage range for stable operation is between 7 and 12 volts.
The Mega 2560 features several power-related pins, including the VIN pin, which is used to provide an external power source, bypassing the 5V USB supply. The 5V pin outputs a regulated 5V from the board's regulator, which can be sourced from either the DC power jack, the USB connector, or the VIN pin (7-12V). It's important not to supply power directly to the 5V or 3.3V pins, as doing so bypasses the regulator and may damage the board. The 3.3V pin provides a regulated 3.3V output, with a maximum current draw of 50mA. Other pins include GND for ground connections, and IOREF, which gives the voltage reference for the microcontroller's operation. A properly configured shield can use this voltage reference to determine the correct power source and enable voltage translation for working with 5V or 3.3V systems. In terms of memory, the Mega 2560 is equipped with 256 KB of flash memory for code storage, with 8 KB reserved for the boot loader. It also has 8 KB of SRAM and 4 KB of EEPROM, which can be read from or written to using the EEPROM library. For input and output, the board's pins are mapped to the Atmega2560 ports, which allows for extensive interfacing with external components.
The APR9600 is a versatile single-chip voice recording device that offers non-volatile storage and playback capabilities for 40 to 60 seconds of audio. It supports both random and sequential access to multiple pre-recorded messages, giving designers flexibility in how they manage and use stored content. The sample rate is user-selectable, which allows customization of both audio quality and storage time to meet specific application needs. This device is ideal for various applications, including portable voice recorders, toys, and a wide range of consumer and industrial products.
The device integrates several components, including an output amplifier, a microphone amplifier, and Automatic Gain Control (AGC) circuits, making system design simpler and more efficient. APLUS, the company behind the APR9600, achieves its high storage capability through proprietary analog/multilevel storage technology. This process utilizes advanced Flash non-volatile memory, where each memory cell can store 256 different voltage levels. This technology allows the APR9600 to reproduce voice signals naturally, without the distortion that often results from encoding or compression methods. Internally, the APR9600 includes several key stages to ensure high-quality audio performance. The analog input section features a differential microphone amplifier with integrated AGC, which is ideal for applications that require microphone input. The microphone signal is processed through an internal anti-aliasing filter, which adjusts according to the selected sampling frequency to ensure that Shannon's Sampling Theorem is satisfied. After filtering, the signal is ready to be clocked into the memory array for recording and playback.
Liquid Crystal Display (LCD) technology is widely used in modern devices like scratch pad displays and smaller personal computers, offering significant advantages over older technologies like Cathode Ray Tube (CRT) and gas-plasma displays. LCDs are thinner, more energy-efficient, and consume far less power than LED and gas-based displays because they operate by blocking light rather than emitting it. LCDs come in two primary types: passive matrix and active matrix. The passive matrix LCD has a grid of conductors at the intersections of each pixel, where current is sent through two conductors to control the light for individual pixels. In contrast, an active matrix, also known as a Thin-Film Transistor (TFT) display, has a transistor at each pixel intersection, which allows for more precise control over the pixel's luminance with significantly lower power consumption. This makes the active matrix LCD more advanced, though some passive matrix LCDs may use dual scanning to improve performance by scanning the grid twice with current, a feature not found in earlier technologies.
Among the different types of displays, the 16x2 LCD module is a commonly used and essential component in various electronic circuits and devices. Compared to seven-segment displays, LCDs are favored for their affordability, programmability, and versatility. Unlike seven-segment LEDs, which are limited to numeric characters and simple symbols, LCDs can display custom characters, graphics, and even animations, providing far greater flexibility for designers and users. This makes LCDs particularly useful in applications that require a rich and dynamic visual display.
A flex sensor, also known as a bend sensor, is a device designed to measure the amount of deflection or bending. It works by detecting the change in resistance as the sensor material bends. The sensor typically consists of a flexible strip, often made from materials such as plastic or carbon, where the carbon surface is arranged along the strip. When the strip bends, the resistance of the carbon layer changes, and this resistance is directly proportional to the degree of bend. As a result, the flex sensor is frequently used as a goniometer and is sometimes referred to as a flexible potentiometer due to its ability to measure angles and bends. The design of the flex sensor typically involves two terminals: Pin P1 and Pin P2. Pin P1 is connected to the positive terminal of the power source, while Pin P2 connects to the ground (GND). The sensor is powered by a DC voltage supply ranging from 3.3V to 5V. It does not require polarized terminals, meaning it has no specific positive or negative sides like diodes or capacitors. This simplicity in the design makes it adaptable for use in various electronic applications where precise measurement of bending or deflection is required.
Fig. 2 illustrates the method for translating sign language gestures into audible speech. The method for translating sign language gestures into audible speech begins with the capturing of hand gestures using flux sensors attached to a glove worn by the user. These sensors are placed at key points on the glove, particularly along the fingers and joints, where they can detect changes in hand posture and movement. When a user bends or moves their hand to form a specific gesture, each sensor experiences a change in resistance due to the flexion. This resistance change is unique to each gesture, creating distinct sensor readings that represent various hand postures in sign language. The sensors then translate these physical variations into analog signals that convey the exact degree of bend in the user's hand. Once the analog signals from the flux sensors are generated, they are transmitted to a microcontroller, such as an Arduino, where the data undergoes analog-to-digital conversion. This conversion is crucial as it enables the microcontroller to work with the continuous analog input data by converting it into digital values that represent the precise positioning of the hand. The microcontroller, as the central processing unit, uses this digitized data to begin interpreting each distinct signal pattern as a specific gesture. This step ensures that the physical movements captured by the sensors are accurately translated into digital information that can be processed further.
Following digitization, the microcontroller processes the digitized data to interpret specific hand gestures. This interpretation involves comparing the incoming data patterns with pre-defined gesture mappings stored within the microcontroller's programming. Through this process, each gesture is assigned a unique identifier, making it possible to match it with a specific audio message. The microcontroller's program, designed in an Integrated Development Environment (IDE), enables these mappings to be customized, allowing for flexibility in which gestures correspond to particular spoken messages. By analyzing these sensor inputs, the microcontroller can accurately recognize various hand gestures and prepare them for the next step in the translation process.
Each interpreted gesture is then assigned to a pre-recorded vocal message stored in an APR9600 voice module. This module contains multiple pre-recorded audio messages corresponding to different gestures, acting as the system's audio library. When a gesture is interpreted, the microcontroller sends a command to the APR9600 module to retrieve the vocal message linked to that particular gesture. The module can play the messages either sequentially or in random access mode, depending on the system design. This ensures that each interpreted hand gesture is effectively translated into the correct vocal message, ready to be audibly output. To complete the translation, the system generates an audio output through a speaker connected to the voice module. This speaker, positioned for clarity, converts the pre-recorded audio from the APR9600 module into audible speech, effectively vocalizing the user's intended message. This feature allows individuals who rely on sign language to communicate verbally without needing an interpreter, making the system highly accessible for users with speech disabilities or those who communicate using gestures.
Alongside the audio output, the method includes displaying a visual representation of the interpreted gesture as text on an LCD screen. This step provides real-time feedback by showing the corresponding text message on the display unit, giving both the user and surrounding individuals a visual confirmation of the spoken message. This dual output-both audio and visual-enhances the clarity and accessibility of the communication, ensuring that the user's message is easily understood in a variety of environments, even when audio alone may be insufficient.
The wearable gesture-to-speech translation system offers numerous advantages, particularly for individuals with speech disabilities or those who communicate using sign language. It enhances communication accessibility by bridging the gap between sign language and spoken language, allowing users to interact more easily in environments where others may not be familiar with sign language. By converting hand gestures into audible speech, the system enables real-time, seamless communication, which is especially beneficial in situations where an interpreter is unavailable or immediate communication is necessary, such as during emergencies or casual interactions. Additionally, the system provides dual output through both audio and visual feedback. While the speaker vocalizes the message, the LCD screen displays the corresponding text, ensuring that the message is comprehensible to both the user and those around them, even in noisy environments.
The system is customizable, allowing specific gestures to be mapped to desired vocal messages, which offers flexibility and adaptability to the user's needs. Its portable, wearable design makes it easy to use throughout the day without requiring bulky equipment, while the power management module ensures energy efficiency, supporting long battery life for continuous use. This provides users with independence in communication, allowing them to express themselves without relying on interpreters. Furthermore, the system is easy to operate, with minimal setup required, making it user-friendly even for individuals who are not familiar with complex technology. Overall, this system enhances communication for individuals with speech and hearing impairments, significantly improving their ability to interact socially and independently.
, Claims:We claim
1. A wearable gesture-to-speech translation system (100), the system comprising:
a) a microcontroller (110) interfaced with the flux sensors, wherein the microcontroller receives sensor data and processes it through analog-to-digital conversion to interpret specific hand gestures;
b) a plurality of flux sensors (120) configured to detect variations in hand gestures by measuring resistance changes corresponding to the degree of bend in a user's hand posture;
c) an APR9600 voice module (130) configured for recording, storing, and sequentially or randomly playing back pre-recorded vocal messages associated with the interpreted gestures;
d) a LCD display unit (140) configured to visually output the interpreted gesture as text; and
e) a speaker (150) connected to the voice module for audibly outputting the vocal messages;
2. The system of claim 1, wherein the flux sensors are arranged on a glove to detect multiple degrees of hand movement, capturing complex gestures and providing accurate translation into corresponding speech outputs.
3. The system of claim 1, wherein the microcontroller is an Arduino device programmed to interpret and assign specific gestures to corresponding vocal messages, utilizing an integrated development environment (IDE) to enable customization of gesture mappings.
4. The system of claim 1, wherein the display unit is a liquid crystal display (LCD) configured to show text corresponding to each vocal message, providing visual feedback of the interpreted gesture.
5. The system of claim 1, wherein the APR9600 voice module allows for customizable message storage and playback duration, enabling users to adjust sample rates for optimized audio quality and storage time.
6. The system of claim 1, further comprising a power management module to control power supply to the flux sensors, microcontroller, voice module, and display, enabling extended battery life for continuous wearable use.
7. A method for translating sign language gestures into audible speech, the method comprising:
a) capturing hand gestures using one or more flux sensors attached to a glove, wherein the sensors detect variations in hand posture based on resistance changes due to bending;
b) transmitting the sensor data to a microcontroller, wherein the data undergoes analog-to-digital conversion;
c) processing the digitized data within the microcontroller to interpret specific hand gestures;
d) assigning each interpreted gesture to a pre-recorded vocal message stored in an APR9600 voice module;
e) generating an audio output through a speaker connected to the voice module corresponding to the interpreted gesture; and
f) displaying a visual representation of the interpreted gesture as text on an LCD.
8. The method of claim 7, further comprising the step of allowing sequential or random access of vocal messages within the APR9600 voice module to customize playback according to specific gestures.
9. The method of claim 7, wherein the microcontroller's processing includes the use of predefined gesture mappings programmed through an Arduino IDE to assign each gesture to a specific vocal message, enabling dynamic adjustments in gesture-to-speech mappings.
10. The method of claim 7, further comprising the step of adjusting sample rates on the APR9600 module to optimize the quality and storage time of pre-recorded vocal messages, enabling high-quality audio output based on the user's preferences.
Documents
Name | Date |
---|---|
202441088163-COMPLETE SPECIFICATION [14-11-2024(online)].pdf | 14/11/2024 |
202441088163-DECLARATION OF INVENTORSHIP (FORM 5) [14-11-2024(online)].pdf | 14/11/2024 |
202441088163-DRAWINGS [14-11-2024(online)].pdf | 14/11/2024 |
202441088163-EVIDENCE FOR REGISTRATION UNDER SSI(FORM-28) [14-11-2024(online)].pdf | 14/11/2024 |
202441088163-FORM 1 [14-11-2024(online)].pdf | 14/11/2024 |
202441088163-FORM FOR SMALL ENTITY [14-11-2024(online)].pdf | 14/11/2024 |
202441088163-FORM FOR SMALL ENTITY(FORM-28) [14-11-2024(online)].pdf | 14/11/2024 |
202441088163-FORM-9 [14-11-2024(online)].pdf | 14/11/2024 |
202441088163-REQUEST FOR EARLY PUBLICATION(FORM-9) [14-11-2024(online)].pdf | 14/11/2024 |
Talk To Experts
Calculators
Downloads
By continuing past this page, you agree to our Terms of Service,, Cookie Policy, Privacy Policy and Refund Policy © - Uber9 Business Process Services Private Limited. All rights reserved.
Uber9 Business Process Services Private Limited, CIN - U74900TN2014PTC098414, GSTIN - 33AABCU7650C1ZM, Registered Office Address - F-97, Newry Shreya Apartments Anna Nagar East, Chennai, Tamil Nadu 600102, India.
Please note that we are a facilitating platform enabling access to reliable professionals. We are not a law firm and do not provide legal services ourselves. The information on this website is for the purpose of knowledge only and should not be relied upon as legal advice or opinion.