AR SCRIBE: ENHANCING LEARNING FOR CHILDREN WITH DYSGRAPHIA THROUGH AUGMENTED REALITY AIR WRITING

Abstract

6. Abstract of the invention: Title: AR SCRIBE: ENHANCING LEARNING FOR CHILDREN WITH DYSGRAPHIA THROUGH AUGMENTED REALITY AIR WRITING Dysgraphia being one of the Specific Learning Disabilities (SLD) impacts and affects the writing skills that require a complex set of motor and information processing skills. Ultimately leads to a heavy impact on the self- esteem behavior of children and has various other challenges in their period of schooling. Since handwriting is a basic academic but Complex skill that is acquired over a period of time it takes time to treat and overcome this challenge. The practical difficulties are to identify the students with 15 Dysgraphia as early as possible. Lack of understanding about Dysgraphia among parents is a major hindrance to their child’s development and progress. Early diagnosis or therapy helps the student to overcome this. The main objective is to develop a learning and development therapy platform or online therapy system to provide air writing therapy for identified dysgraphic children (aged 8 to 12 years) at an early stage. Augmented reality is used to make air writing therapy a possible one. The v system is the integration of learning, practice, and tests which leads to improvement in their handwriting. The system automatically analyzes the progress of a child in due course and informs the child’s improvement to their teachers and parents via SMS and mail. The system’s ultimate goal is to build good self-esteem achieve great academic potential and improve writing skills like other normal children. Keywords - Learning difficulties, Specific Learning Disability, self-esteem,, air writing therapy, Augmented reality, dysgraphia, Machine Learning Techniques Dated this Date 09 of October 2024 Signature of the applicant Dr N Saravanakumar' Principal PSG Institute of Technology and Applied Research Coimbatore, Pin Code- 641062 Dr. N. Saravanakumar Principal, PSG Institute of Technology and Applied Research

Specification

FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
COMPLETE SPECIFICATION
(See section IO and rule 13)
1. Title of the Invention:
"AR SCRIBE: Enhancing Learning for Children with Dysgraphia through
Augmented Reality Air Writing"
2. Name of the Applicant:
PSG Institute of Technology and Applied Research
Address for communication:
The Principal
PSG Institute of Technology and Applied Research
08-Npv-2024/134559/202441085881/Form 2(Title Page)
Avinashi Road, Neelambur,
Coimbatore - 641 062, Tamil Nadu.
Phone No: 0422 3933400
&
Nationality:
INDIAN
3. Preamble of the Description:
The following specification particularly describes the invention and the manner in
which it is to be performed.

4.
Description:
Title: "AR SCRIBE: Enhancing Learning for Children with Dysgraphia through Augmented Reality Air Writing"
Field of the invention
The present invention is related to the Field of specific learning disability (SLD) especially focused on dysgraphia which is a neurological condition affecting writing abilities and poses significant challenges to children in their academic development. AR. Scribe, an online therapy application combines cutting-edge augmented reality technology with educational theories tailored for dysgraphic children. AR scribe is integrated with machine learning algorithms and image processing techniques which helps to assess student performance and improvement. One of the best therapies, air writing therapy, which is used in the traditional method of teaching is inculcated as the main method of the learning process. The invention is more particularly related to enhancing the holistic learning process through air writing in an augmented reality environment. The stakeholders of the application are dysgraphic students, parents, tutors, and educators.

Background of the invention
A specific learning disability (SLD) is a disorder that interferes with a student's ability to listen, think, speak, write, spell, or do mathematical calculations. Students with Dysgraphia learning disability may struggle with writing. Dysgraphia characteristics are Illegible handwriting, Inconsistent use of letters (either uppercase or lowercase), difficulty with spatial planning on paper, spelling errors, etc. Among the age group of 6 to 10 years, around 5% to 33% of students have dysgraphia. The effects are difficulty in concentrating on writing, taking notes during a class, low self-esteem, and anxiety. Causes for Dysgraphia may be developmental disorders, neurological factors, Genetic factors, and Brain injury. There exist several therapies which include Cognitive Rehabilitation, Deep brain stimulation, speech/Language Therapy, Occupational therapy, etc.

Occupational therapy for dysgraphia involves a comprehensive approach to address the challenges associated with handwriting difficulties. Through a variety of tailored strategies and activities, therapists work to enhance handwriting skills while minimizing the physical and emotional barriers that children with dysgraphia may face when expressing themselves on paper. These therapeutic interventions encompass a range of exercises and tasks, such as connect-the-dots puzzles, which help improve hand-eye coordination and spatial awareness. Writing in the air and using modeling clay assist in reinforcing letter formation and motor planning, while activities like navigating mazes and tracing letters with tactile materials strengthen fine motor skills. Moreover, occupational therapists may focus on specific muscle groups through wrist and hand exercises to improve dexterity and control. They also guide proper pencil grip techniques to reduce strain and fatigue during writing tasks. By addressing these fundamental aspects, occupational therapy aims to gradually build confidence and proficiency in handwriting, ensuring that the physical act of writing becomes less daunting and more enjoyable for children with dysgraphia.

In recent times, schools have been implementing accommodations and remediation strategies to support students with dysgraphia. Accommodations include offering oral exams and assignments instead of written ones, allowing the use of computers for typing, and providing shorter writing tasks or alternative audio/visual assignments, aiming to alleviate the challenges associated with handwriting. Remediation efforts led by special educators focus on improving handwriting and writing skills through techniques such as providing paper with raised lines, practicing writing letters in the air to enhance motor memory, teaching proper grip and posture for writing, and breaking assignments into smaller parts or providing step-by-step instruction sheets. By integrating these accommodations and remediation techniques, schools have created a supportive learning environment that enables students with dysgraphia to excel academically.

Augmented Reality (AR) will completely transform the educational experience by providing an immersive and interactive environment between the real and virtual worlds. AR transforms traditional classrooms into dynamic, interactive learning spaces, engaging students with immersive simulations and 3D visualizations. Educational institutions leverage AR to 10 enhance student understanding through hands-on experiences nowadays. Various machine learning techniques and deep learning techniques leverage the potential of image recognition, and personalized recommendations to optimize the efficiency and performance of the students.

Currently, many frameworks or library machine learning for hand gesture recognition have been built to make it easier to identify the gesture and give commands to the computer. The mediapipe framework is presented by Google for solving the problem using machine learning such as Face detection, Face mesh, Iris, hands, pose, holistic, object detection, instant motion tracking, etc. Cloud computing emphasizes on Central storage of programs and data which can be accessed from anywhere and anytime which has many advantages including access flexibility, ubiquity of data, and resilience. Also, cloud computing makes distant learning easier by providing access to information and data sharing.

Objectives of the invention
The main objective of "AR Scribe: Enhancing Learning for Children with Dysgraphia through Augmented Reality Air Writing" is to create an innovative and interactive learning tool tailored specifically for children with dysgraphia. By leveraging AR technology, the work aims to empower these children to improve their writing skills in a dynamic and engaging environment. The primary goal is to foster a positive attitude towards writing, encourage regular practice, and provide personalized support based on each child's unique needs.

Additionally, the project seeks to promote inclusivity in education by accommodating diverse writing styles and cognitive needs, ultimately unlocking the full potential of children with dysgraphia in their educational journey and beyond. Through the integration of AR. technology, machine learning, and personalized support, the project aims to create a positive 10 impact on the lives of dysgraphia children, educators, and parents, fostering a more inclusive and equitable educational landscape.

Brief description of prior art
Patent application no, 202421035288 details a cognitive Al-powcrcd software for screening Dyslexia and Dysgraphia. It analyzes audio and handwritten samples using machine learning, comparing them with typical data to detect these conditions, and offers personalized assessments and potential therapeutic guidance.

Patent application no. 201721002078 describes an assistive tool kit comprising an Android app and a specially designed stylus for children with Dysgraphia. The app analyzes handwriting to identify strengths and weaknesses, offers suggestions for improvement, and tracks progress, while also detecting Dysgraphia. The kit includes games to enhance the child's motor skills.

Patent application no. 202241033894 describes a screening system designed to identify Specific Learning Disabilities (SLD) like Dyslexia, Dysgraphia, and Dyscalculia in children aged 10-11. The system detects these issues early and automatically notifies parents and teachers, enabling timely educational . support to help children build self-esteem and reach their full academic potential.

Summary of the invention
In view of the present unavailability of a therapy application platform, AR Scribe acts as a teletherapy platform for enhancing the holistic learning process for children with Dysgraphia through Augmented reality(AR) air writing, AR is used to make the student more engaged and involved in the teaching-learning process. The system and method are implemented by means of Machine Learning Algorithms, Deep learning Techniques,
Mediapipe framework, and IoT-based Input Interface, The implementation is not just limited to a Machine Learning Algorithm and an IoT-based I/O. It can be implemented by means of using existing devices such as mobile devices for I/O operations other than PCs.

The main aspect of this invention is to provide a system, in which students interact with the virtual world through augmented reality and learn and practice air writing of different letters as big as possible, which is one of the best therapies followed by educationalists across the globe which will ultimately improve the understanding of characters and writing skills among r? the dysgraphic children while writing in the notebooks. Another aspect of the present invention is that the Machine learning model will predict the accuracy of improvement of the student in the period of the learning process and those models will learn and train themselves to adapt to several scenarios with the help of preexisting data to adapt and automate the question given to
different categories of students based on the time taken and difficulty in writing different characters. The ultimate goal of the invention is to enhance the writing and learning skills along with improvement in motor and cognitive skills of the students through augmented reality air writing
techniques.

Detailed description of the invention
[1] The following description provides specific details of certain aspects of the disclosure illustrated, in the drawings to provide a thorough understanding of those aspects. It should be recognized, however, that the present disclosure can be reflected in additional aspects and the disclosure may be practiced without some of the details in the following description.

The various aspects including the example aspects are now described more fully with reference to the accompanying drawings, in which the various aspects of the disclosure are shown. The disclosure may, however, be embodied in different forms and should not be construed as limited to the aspects set forth herein. Rather, these aspects are provided so that this disclosure is thorough and complete, and fully conveys the scope of the disclosure to those skilled in the art. In the drawings, the sizes of components may be exaggerated for clarity.

It is understood that when an element or layer is referred to as being "on," ."connected to," or "coupled to" another element or layer, it can be directly on, connected to, or coupled to the other element or layer or intervening elements or layers that may be present. As used herein, the term "and/or" includes any .and all combinations of one or more of the associated listed items.

The subject matter of example aspects, as disclosed herein, is described with specificity to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventor/inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different features or combinations of features similar to the ones described in this document, in conjunction with other technologies.

While the present invention has been described with reference to particular embodiments, it should be understood that the embodiments are illustrative and that the scope of the invention is not limited to these embodiments. Many variations, modifications, additions, and improvements to the embodiments described above are possible. It is contemplated that these variations, modifications, additions, and improvements fall within the scope of the invention.

[2] Referring now to the drawings, these are illustrated in FIG 1-8; the : present invention discloses an application for enhancing learning for Dysgraphic children through air writing therapy using Augmented Reality.
The application is deployed in the cloud for easier access from anywhere at any time.

[3] In accordance with another embodiment of the present invention, an intuitive user interface promotes an enhanced learning experience, which increases the effectiveness of the system. This platform's capacity to develop and adapt is strengthened by the smooth integration of user feedback and data-driven insights, which optimizes each dysgraphic child's learning experience. For dysgraphic kids, the learning recommendation system is essentially an innovative step toward inclusive and customized learning.

[4] In accordance with another embodiment of the present invention, Augmented Reality is integrated with the application which bridges the real world to the virtual world to capture the characters or shapes written made by the student in the air which will be analysed and interpreted by the application to obtain the necessary output for the further process and experience of writing as big in the real world can be visualized in AR 30 environment where the students can better understand and visualize the. formation and structure of characters or shapes which will altogether improve their motor and muscle memory.

[5] In accordance with another embodiment of the present invention, an image-capturing instrument or module is interfaced with an application that is configured to capture the characters or shapes air written by the dysgraphic student that will be processed by the application to find the correctness of the written structures and regulate and track on the improvement in the entire 10 process over a period of time.

[6] In accordance with another embodiment of the present invention. Machine Learning algorithms are interfaced, with an application that is confused to get its input from the image-capturing module in real-time and recognize characters/shapes written by children, time taken for correctly writing the shown characters/shapes during learning, practicing and testing are taken into consideration along with the amount of accuracy which was a considered over the duration of period to predict the difficulty level of the questions among the dysgraphic children. Later, the Application will adapt itself to operate at desired optimal conditions-based on the-student's speed and writing ability

[7] The invention described encompasses a system for enhancing the T> learning experience of children with dysgraphia. This innovative system aims to provide air writing therapy, among the best adaptable learning Methodologies/ therapies customized to each child's unique needs. The dysgraphic children in the age group of 6-10 years are allowed to use this AR Scribe application under the guidance of teachers and parents. Once a student undergoes a period using the application, their teachers and parents are automatically informed through mail for further assessments by the professionals.

[8] In accordance with another embodiment of the present invention, cloud computing is used along with an application to store data such as name, mail ID, score, accuracy, time taken, and different analyses made by models. By leveraging stored information we could infer the learning Progress, Preferences, and behavioral patterns of the Individual's Learning journey.

|9] In accordance with another embodiment of the present invention, the dysgraphic students are ready to learn, they can click on the learn tab. Once clicked, any shapes or characters i.e. Alphabets or Numbers will be 10 displayed in many forms. The students are insisted to draw the same within the provided region and write or draw the specific characters or shapes respectively. The time taken by the student and other writing parameters such as direction of strokes, steadiness of fingers, and letter spacing are recorded and stored on the cloud.

When they click on the practice tab, they can try the same characters or shapes as many times as they want to practice. Once enough time of learning and practice is being done, they are able to attend the test, and students can click the Test tab. The test tab will be enabled only after undergoing a learning and practice session. Here the students are supposed to write the characters or shapes to the provided questionnaire which are either voice- assisted, missing letters, flashcards, etc.

Each correct answer is provided by the students and the time taken by the children is calculated simultaneously. The final score and time duration are stored in the cloud and will be seen by the admin who can log in via admin login mode. The admin can be Teachers, Psychologists, and language pathologists.

[10] In-accordance with another embodiment of the present invention, AR Scribe, an online therapy application uses animated GIFs and videos of ways to write or draw the characters or shapes respectively to teach and guide dysgraphic children along with dotted characters or shapes in the virtual environment of augmented reality where the children write in the air as guided by the application. Using this application, students learn the right way of writing the alphabet during the learning and practice phase. During the test phase the results are only based on the overall structure of characters or shapes but not on the proper way of writing, i.e. there exist several ways of writing characters (letters, numbers) and shapes apart from standard ways. This is done so that the student's confidence level & self-esteem are improved by seeing the positive result in writing the proper structure of characters or shapes.

[11l The recommendations can be continually enhanced and refined due to an integrated feedback mechanism that lets users offer feedback on the efficacy of their work. The system's robust and scalable architecture allows it to adapt and change over time, taking into consideration user feedback and new methodologies in dysgraphia education to maximize each child's learning experience. With all aspects considered, the system is a pioneering step toward individualized and inclusive education, utilizing cutting-edge technology to build a platform that changes in accordance with every child's educational path.

[12]A System can be made commercialized subject to the institution or owning members following social ethics as it deals with sensitive data of dysgraphic student details. It can be commercialized to a large extent along with a dysgraphia screening facility which helps in identifying the dysgraphic student and giving them proper therapy for holistic improvement. The main users of the application could be dysgraphia students & institutions related to them. The Stakeholders include dysgraphic students, tutors, educators, Instructors, parents, and others associated with this field.

[13] A comprehensive experiment was undertaken, involving a group of ninety-four students diagnosed with dysgraphia, a condition impacting their holistic writing skills. The study focused on assessing the efficacy of a particular application designed to aid in overcoming writing challenges. After subjecting the participants to three complete cycles of utilizing this application, meticulous observation revealed a remarkable transformation in their overall writing performance.

[14] Generally, there is a statistical relationship based on time period, between reading time, understanding time, writing time, and drawing time for dysgraphic children who are given air-based writing therapy. Based on the timing relationship and score obtained over a period of time, the improvement could be observed among dysgraphic children, and results are analyzed for further development with the information, schools may frequently recommend educational alternatives or provide additional support to those children who need some more time practicing. Hence by Providing approximate practice and guidance. The children with dysgraphia can shine in academics and careers.

[15] A System has a few concerns related to the proper posture of the users, positioning of hands, lighting, and surrounding environment.

Description of an embodiment of the invention
Now the invention will be described with reference to the drawing accompanying the Invention wherein:
Figure 1: Functional process map of the proposed system.
Figure 2: Overall System flow diagram of AR Scribe.
Figure 3: Internal System How diagram for the Practice/Test phase of AR Scribe.
Figure 4: Flow diagram of admin mode activities.
Figure 5: Background framework and process flow for character recognition.
Figure 6: Diagram for identification of gesture and air writing initiation process.
Figure 7: Block diagram of different approaches used for letter and number dysgraphia.
Figure 8: Block diagram of layers in CNN algorithm.

5. Claims:

l/We claim,
1. AR Scribe is an augmented reality air writing therapy application that helps dysgraphic students improve motor and muscle memory, writing, and cognitive skills through interactive air writing exercises, while also offering dysgraphia screening.

2. The system uses augmented reality technology to overlay digital information onto the real-world environment, enhancing students' attention span by immersing them in an augmented environment, which fosters holistic development for dysgraphic students, accelerates their learning of characters and shapes, and reduces overall learning time.

3. A system as claimed in claim I wherein the system is used by psychologists, tutors, and special education trainers to load the activity cards, and questions remotely anywhere and different-algorithms allocate - different ranges of accuracy criteria across the period of learning, practicing, and test phases.

4. The system, as described in claim 1, focuses on dysgraphic students by providing an online therapy or teletherapy platform to improve their writing and cognitive skills, incorporating the drawing of displayed shapes and writing of displayed words as part of the overall enhancement process.

5. A system as claimed in claim 1 wherein the system is deployed in a cloud environment so the accuracy, status, improvement, and test results of the student are notified to the corresponding teachers and parents to know about the learning process and their child's development in the due
process.

6. The system, as described in claim 1, utilizes a convolutional neural network (CNN) model modified to extract specific features from children's air-written letters by performing grayscale image conversion, image preprocessing, feature selection, pooling, flattening, and training the model with a collected dataset, with the model tracking and calculating accuracy over time.

Signature of the applicant
Dr N Saravanakumar
Principal
PSG Institute of Technology and
Applied Research
EDniMb£tora a r
Principal, PSG Institute of Technology and Applied Research

7. A system as claimed in claim 2 wherein the air writing technique is used to write letters in the air and can be visualized through augmented Reality Technology.

8. The system, as outlined in claims I and 4, incorporates a feedback mechanism to enable reinforcement learning through engaging and repetitive methods, functioning in three phases: Learn, Practice, and Test, where students receive guided practice to correct errors in the first two phases, while the Test phase assesses performance using traditional therapy metrics such as time, accuracy, and proper letter formation.

9. A system as claimed in claim 8 wherein air writing is one among the best remedial education and the experience of writing as big in the real world can be seen written in the augmented virtual world where the students can better understand, analyze, visualize the formation and structure of characters/shapes which will ultimately improve their motor and muscle memory

10. A System as claimed in claim 1 wherein the system is scalable up to a large extent across the globe as it can be made personalized to each dysgraphic student. The application can deal with different written languages worldwide which are being used to improve their holistic writing skills through therapy.

Dated this 09 Day of October_2024

Signature of the applicant
Dr N Saravanakumar
Principal
PSG Institute of Technology and
Applied Research
Coimbatore, Pin Code- 641062
Principal,
PSG Institute of Technology and Applied Research

Documents