IOT-ENABLED DUAL-ARM ROBOTIC SYSTEM FOR ENHANCED PRECISION AND REAL-TIME MONITORING IN ENDOSCOPIC CARDIAC SURGERIES

Abstract

IOT-ENABLED DUAL-ARM ROBOTIC SYSTEM FOR ENHANCED PRECISION AND REAL-TIME MONITORING IN ENDOSCOPIC CARDIAC SURGERIES The method for the development of the suture pulling force is estimated using a neural network structure that combines Long Short-Term Memory (LSTM) networks with a modified Inception Resnet-V2. By altering the tool positions gathered from the master-slave robotic system, the interaction is recorded under two distinct artificial skin conditions and two distinct scenarios at 13 image viewing angles, confirming the viability of the suggested network using the generated database. Many commercial robots have been created for a variety of surgical procedures. We have examined the current commercially available robotic surgical systems in light of the recent advancements in surgical robotics and its substantial market potential. In addition to many other surgical specialties like gynecology, general surgery, otolaryngology, cardiothoracic surgery, and neurosurgery, RS has transformed the field of urology by empowering surgeons to carry out intricate procedures with increased accuracy and precision. Robotic-assisted surgery is becoming more and more common due to a number of advantages, including improved access to the surgical site, a three-dimensional image that enhances depth perception, smaller scars, improved range of motion that enables the surgeon to perform more complex surgical procedures, and fewer postoperative complications.

Specification

Description:IOT-ENABLED DUAL-ARM ROBOTIC SYSTEM FOR ENHANCED PRECISION AND REAL-TIME MONITORING IN ENDOSCOPIC CARDIAC SURGERIES

Technical Field
[0001] The embodiments herein generally relate to a method for IoT-enabled dual-arm robotic system for enhanced precision and real-time monitoring in endoscopic cardiac surgeries
[0002]
Description of the Related Art
[0003] The Minimally invasive surgery (MIS) only necessitates a small incision, whereas conventional open surgery demands a large one. MIS is well-liked due to its cosmetic advantages and reduction of recovery time and expense. Hand tremor and eye-hand coordination issues that arise during minimally invasive surgery (MIS) are resolved by robotics-assisted minimally invasive surgery (RAMIS). A surgical robot combines excellent vision quality with dexterous movements. However, because the interaction forces cannot be accurately measured, applying force feedback is still problematic. Unless it is an open surgery, it is frequently linked to minimally invasive surgery (MIS), which basically means that the patient will greatly benefit from the robotic surgical procedure's ability to be performed with a small number of incisions. When contrasted with traditional surgery, it may lessen post-operative trauma and recovery time. A development of minimally invasive surgery, robotic surgery (RS) blends engineering, robotics, and medical science. This advanced technique, also referred to as robot-assisted surgery, uses specialized robotic platforms to enhance the accuracy of surgeon movements in intricate procedures and small anatomical spaces. The trocar can be installed as an instrument entry port if the trauma is only about 1 cm in diameter. After that, the surgeon uses an endoscope to examine the intra-corporeal surgical site and the instrument configurations used to carry out the procedures. Meanwhile, the use of endoscope also allows various types of ENT surgery to be done directly using the natural entry port to position the surgical instruments.
[0004] Direct force sensing and sensor less force estimation are the two primary ways to get around the lack of force feedback. The force is measured at or close to the interface between the surgical instrument and the organs in direct force sensing. Using a piezoresistive sensor, Abiri et al. created a tool that can be fastened to the end of a surgical instrument. Furthermore, Reiley et al. measured the interaction force using a strain gauge that was fastened along the shaft of a surgical instrument. Although this offers the most straightforward solution, it has several drawbacks, including cost, biocompatibility, sterilization, and miniaturization. Tele surgical robots utilize the basic teleoperation mode, in which the end effector-in this case, an endoscope, orthopedic instrument, or any other surgical tool-is controlled by a slave robot that is controlled by a master console. The effectiveness of tele surgical robotic surgeries has increased due to advancements in haptic technologies, the removal of hand tremors, etc. However, time delay is an area that still needs a lot of research and development, especially in situations where there are great distances between the slave site and the master, such as in space or intercontinental applications. The primary surgeon is situated in the surgeon's master console, which offers a three-dimensional view of the surgical field via an endoscopic camera inside the patient's body. The surgeon can control the camera from the console to feel as though they are actually in the surgical field. The surgeon makes surgical movements with control devices like handles or joysticks, and the robotic arms above the patient translate those movements into real-time movements. For the surgeon to perform individual tasks with dexterity and intuition, the instrument design and the "fulcrum effect" of the instrument workspace upon the minimally invasive setup via the trocar are not ergonomic.
[0005] In order to create a sense of immersion in the surgical field, the primary surgeon is situated at the surgeon's master console, which offers a three-dimensional view of the surgical field via an endoscopic camera inside the patient's body. The surgeon uses control devices like joysticks or handles to make surgical movements, which the robotic arms above the patient translate into real-time movements. The fulcrum effect of the instrument workspace and its design make it difficult for the surgeon to perform individual tasks with dexterity and intuition due to the minimally invasive setup via the trocar. In the sense that surgical procedures can be carried out fully or partially autonomously without the assistance of a human operator, the imaging feedback feature essentially sets this method apart from telesurgery. Cooperative surgical robots eliminate a number of issues, including the loss of feedback caused by sensors seen in tele surgical systems, by allowing the surgeon to directly operate the hands-on robotic system. Rather, these devices are best characterized as master-slave manipulators that function as completely controlled remote extensions of the surgeon. The da Vinci Surgical System and the ZEUS system are the two primary master-slave systems that are currently in use and have received FDA approval. Although they have been used in industry and service for more than 50 years, robots have recently drawn a lot of interest in surgical applications during the past 30 years. The growing demand for medical and surgical workload, which is further influenced by a number of social factors such as the aging population and increased health awareness, particularly in developing nations, is the primary cause of this trend, in addition to the notable advancements in computerized robotic technology.
SUMMARY
[0001] In view of the foregoing, an embodiment herein provides a method for IoT-enabled dual-arm robotic system for enhanced precision and real-time monitoring in endoscopic cardiac surgeries. In some embodiments, wherein control-based methods estimate the interaction force using observers, surgical tool models, and information from motor devices, such as torque, current consumption, and angular position/velocity. From this angle, Zhao et al. created a technique to forecast the force by utilizing modeling data from the surgical instruments and the motor current. They then developed a force-feedback technique using their method for categorizing the variations in pig liver stiffness in order to show the system's accuracy. Wang et al. used a cable-tension disturbance observer for a surgical robot end-effector to measure the external force. Researchers used an acrylic box with a potato inside that was attached to an artificial penis to replicate the prostate tissue for the resection. This robot's cutting assembly was specially made. When the robot was moving forward, an axial cut was made, and on the return stroke, a diathermic cauterization was carried out. Notwithstanding certain drawbacks, including the size of the robotic arms and the possibility of arm collisions, the da Vinci system was adopted for use in clinical settings.
[0002] In some embodiments, wherein to estimate forces based on 2D or 3D deformation, several vision-based methods have been developed. Additionally, research is being done to measure and predict the shear force on an image. These methods initially calculate the amount of deformation brought on by specific applied forces beforehand. The mechanical properties are then used to calculate the relationship between deformation and force. The measurement of forces on a rubber membrane was investigated by Kennedy et al. The nodal displacements were tracked in order to calculate the deformation. The forces based on these deformations were calculated using a finite element method. The FDA approved Intuitive Surgical's da Vinci surgical system, the first FDA-approved surgical robot, after the company had been creating robotic surgical systems for a few years. Surgical robotics began with these technological advancements. Modern engineers are constructing smaller and more efficient surgical robots thanks to developments in medical imaging, haptics, 3D vision, and miniaturization.
[0003] In some embodiments, wherein this technique uses a deep learning model to simulate the prediction of tension by utilizing vision and proprioception. The suggested method's overall effectiveness is verified using a robotic surgery system that communicates with skin and sutures. The amount of target tensile force, camera views, and shifting soft objects were gathered into a database. The model's training and test sets were taken from the database. The development of medical robotics is being greatly aided by improvements in sensory technologies, which have offered a more comprehensive perspective on the surgical approach. The interaction between the robotic tool and the targeted organs is better understood thanks to these sensors. The proportion of total number of general robotic surgeries carried out in 2012 as a portion of total number of surgeries was as low as 1.8%. The arms and console form the foundation of the majority of the robots in use. The surgical tools are attached to one arm, which also has a camera, and the console provides the operator with a precise, high-quality, magnified view of the surgical site.
[0004] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.

BRIEF DESCRIPTION OF THE DRAWINGS
[0001] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:
[0002] FIG. 1 illustrates a method for IoT-enabled dual-arm robotic system for enhanced precision and real-time monitoring in endoscopic cardiac surgeries according to an embodiment herein; and
[0003] FIG. 2 illustrates a method for ion endoluminal system consisting of a cart and a controller according to an embodiment herein.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[0001] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
[0002] FIG. 1 illustrates a method for IoT-enabled dual-arm robotic system for enhanced precision and real-time monitoring in endoscopic cardiac surgeries according to an embodiment herein. In some embodiments, the interaction force between the tissue/organ and instruments for neurosurgery and robot-assisted surgery was calculated using the image deformation data and the organ's stiffness information. Naeini et al. suggested using Long Short-Term Memory (LSTM) networks to measure the contact force with a neuromorphic camera (dynamic vision sensor). Deep learning has been used in recent vision-based force estimation techniques, however, to forecast interaction forces from images using both spatial and temporal information. This eliminates the need for intricate algorithms to identify a deformation. Using a convolutional neural network and RGB lights and marker images, Yuan et al. and Donlon et al. calculated the contact force and shear force robotic arm, a camera stand, and a guidance module make up the basic system; the software module and any additional attachments, such as cutting tools, differ depending on the application.
[0003] A surgeon plans the surgical procedure using a preoperative image of the patient's anatomy of interest. The robotic arm uses orthopedic tools to operate on the patient while establishing a virtual boundary based on this preplanning. Urology has been transformed by RS, which allows surgeons to carry out intricate procedures more precisely and accurately. Robotic-assisted laparoscopic radical and partial prostatectomy for prostate cancer treatment is one of the main applications of RS in this field. Additionally, robotic-assisted laparoscopic radical and partial nephrectomy and lymphadenectomy are used to treat kidney cancer. It is possible for the terminal surgical instrument to move linearly along the insertion point. The surgical instrument's end can reach any position in the surgical area thanks to the cooperation of joints five and six. Electromagnetic clutches inside the slewing joint 5 and swing joint 6 enable both active and passive control techniques.
[0004] In some embodiments, the RCNN was made up of an LSTM network for temporal information and a VGG network for spatial information from the images, such as pictures of skin deformation. The occlusion, optical noise, and camera motion still result in an inaccurate force estimation from an image, even after a lot of research on replacing a force sensor with an image sensor. Recently, the electrical current of the robot, or the robot position, was added to an image as an input of a neural network for estimating the interaction force in order to overcome the lack of image information for this purpose. An army base and an electromechanical arm make up the other subsystem, TCAT. The controller, monitor, software, pedant control, and other components make up the base. The autonomous TCAT robotics system actively cuts bone while the surgeon watches over it, registering the anatomical position during the procedure. By the beginning of 2020, the Solution One system had been used to perform over 550 TKA procedures worldwide. Bladder cancer is treated with ileal conduit, extracorporeal or intracorporeal urinary diversion, and robotic-assisted minimally invasive radical cystectomy. Additionally, single-port robotic-assisted urinary tract reconstruction, ureteral reimplantation, and robotic-assisted biopsies are used to treat rectovesical and vesicovaginal fistulas. Additionally, advanced methods include robotic-assisted kidney transplantation, robotic-assisted pelvic floor reconstruction for females, robotic-assisted procedures for various pediatric conditions, endoscopic inguinal lymphadenectomy (R-VEIL) for penile cancer, retroperitoneal lymph node dissection, intracorporeal neobladder creation, and robotic-assisted procedures for a variety of pediatric kidney disorders, and bladder and urethral reconstructive techniques for congenital malformations.
[0005] In some embodiments, a training technique called data augmentation uses training data that consists of similar but distinct examples. By providing more training pairs, augmentation can simultaneously reduce overfitting and boost performance. Our images were augmented with random cropping, random brightness, random saturation, random hue, and random contrast. In order to limit the width to height ratio from 0.5 to 1.5, the bounding box's position and size are randomly assigned over the image at a range of 0.5 to 1.0. The monitoring factor of a random saturation has an upper bound of 1.5, a lower bound of 0.5, and a delta of 0.125 for a random brightness. In order to locate bone pins, the system automatically adjusts the surgeon's hand movements. After that, the mechanical cutting guides are attached to bone-mounted pins, enabling subsequent bone resection. The FDA-approved and CE-marked robotic system for TKR or TKA procedures, Omnibotics, was created by Corin. A workstation, an OMNI Bot cutting guide, and a robotic ligament balancer known as Balance Bot make up this system. Its particular use from the brain, spine, and peripheral nerves determines its design. There are four possible ways to achieve a particular posture from the end of the hand, according to the inverse solution derived from the inverse transform method. While all four of the scenarios resolved can satisfy the requirements, some are even impossible because of the limitations of the mechanical arm structure.
[0006] FIG. 2 illustrates a method for ion endoluminal system consisting of a cart and a controller according to an embodiment herein. In some embodiments, the interaction force during a suturing motion is estimated using a network architecture that combines the tool position and images. Spatial feature modeling and temporary feature modeling are the two phases of our approach. An encoding network that expresses the characteristics of the images is called spatial feature modeling. A network that uses temporal information, such as changes in the image features and the tool position, to predict interaction forces is known as serially connected temporal modeling. A portable surgical robotic tool for total knee arthroplasty (TKA) and unicompartmental knee arthroplasty (UKA) was created by Smith and Nephew. An optically tracked tool paints over the anatomy to create a real-time 3D image of the bone instead of requiring a preoperative image. For precise tracking, the anatomical site is held in place by optically tracked jigs. A surgeon can use Cori, an optically tracked burring tool, to remove bone from the target site after mapping the anatomical area. Procedures that are currently being performed include tonsillectomies, adenoidectomies, laryngectomies, and head and neck tumor removal.
[0007] In some embodiments, the outputs of 1056, 512, and 128 that are concatenated and obtained through FC layers are considered the Z-values. When the data is stacked, the Z-values alter to reflect the number of choices in the stack sequence data. In order to estimate the suture tension, the network passes the data through the FC layer after 32 data sequences with 32 and 16 nodes in the LSTM layer. By utilizing the changes in features over time, this temporal feature modeling aims to enhance the performance of tension estimation. DePuy Synthes, a Johnson & Johnson orthopedics company, created VELYS, a robotically assisted knee replacement procedure that has FDA approval. It is made up of a base station and a satellite station. A touch screen, a transfer mechanism, and a robotic assistance device make up the satellite station. The base station is made up of consoles that run different applications and control the robotic device, as well as a camera, touchscreen, and footswitch. Using optical markers, the camera monitors the positions of the instruments, robotic device, and bones. Atrial fibrillation (AF) surgery, intra-cardiac tumor removal, left ventricular lead placement, and patent ductus arteriosus (PDA) closure have all occurred, albeit less frequently, due to advancements in the field. However, traditional cardiac surgeries continue to be preferred due to a lack of data. Real-time control is diminished since the best option must be evaluated and chosen in real time, which necessitates numerous computations.
[0008] In some embodiments, the data collection, the figures' estimated forces were computed. A TITAN V GPU (NVIDIA, California, CA, USA) is used in this study to help train the suggested network on a PC. The test's calculation time of 12.13 ms for predicting the interaction force with an image completely demonstrates that the system's design satisfies real-time requirements. With the maximum pulling force in the test set being 3.114 N, the root mean square error (RMSE) and maximum absolute error (Max-AE) were used to quantify the errors between the estimated force from the suggested network model and the measured ground-truth force. A table-mounted arm, a surgeon's control panel, and a navigation system make up the guidance system. The surgeon creates a plan based on an O-arm scan during the procedure or the previous CT scan. The operating field is mapped by the navigation system. Additionally, each vertebral body is registered and a patient scan is conducted. The trajectory determined by the previous plan is followed by the robotic arm. The necessary procedure is carried out with the instruments once the position has been fixed. It may be possible to standardize surgical results by giving RS surgical systems more autonomy, regardless of the training, experience, and daily performance variations of surgeons. The results of the survival study showed that the developed robotic system could demonstrate a high degree of consistency while matching the performance of skilled surgeons in metrics like lumen patency and leak-free anastomosis. In order to obtain the relative pose information and some more sophisticated features of the end effector and target in the input image, the algorithm oversees the adjustment of the network weights and offsets.
, Claims:1. A method for IoT-enabled dual-arm robotic system for enhanced precision and real-time monitoring in endoscopic cardiac surgeries, wherein the method comprises;
providing unparalleled precision in endoscopic cardiac surgeries, minimizing human error and improving patient outcomes;
facilitating continuous real-time monitoring of surgical metrics, enabling immediate adjustments during critical procedures;
allowing for highly controlled and minimally invasive maneuvers, reducing tissue trauma and accelerating patient recovery times;
enabling remote oversight by expert surgeons, facilitating mentorship and support in complex cardiac surgeries;
collecting and analyzes real-time data, aiding surgeons in making informed decisions during procedures;
synchronizing through IoT, ensures seamless coordination and precise manipulation of instruments in restricted surgical fields;
identifying potential errors or deviations during surgery, enabling prompt corrective measures to enhance safety;
integrating high-definition imaging systems, offering surgeons an improved and dynamic view of the surgical site.
using preoperative data to customize robotic movements and strategies, improving the personalization of cardiac surgeries;
alleviating the physical and cognitive load on surgeons, improving focus and accuracy during lengthy procedures; and
operating within predefined safety parameters, meeting stringent medical standards and protocols.

Documents