SKIN DISEASE DIAGNOSIS, USING MULTISPECTRAL IMAGING AND DEEP LEARNING, WITHOUT NEEDING HUMAN INTERVENTION

Abstract

The present invention relates to a portable, non-invasive system and method for diagnosing skin diseases using multispectral imaging and deep learning. The system includes a multispectral camera capable of capturing high-resolution images of skin at multiple wavelengths (UV, visible, and NIR), a lighting system with adjustable LEDs and diffusers, and a central processing unit based on a Raspberry Pi 4. The system employs a convolutional neural network (CNN) trained on diverse datasets to analyze skin conditions and provide real-time diagnostic results. A touchscreen display allows for user interaction, image capture, and result visualization, while cloud integration enables secure data storage and continuous model updates. The system facilitates early detection and monitoring of various skin diseases, including acne, psoriasis, and skin cancers, offering enhanced diagnostic accuracy, personalized treatment recommendations, and seamless data management. This innovation provides a cost-effective, user-friendly solution for both clinicians and patients, with applications in remote healthcare settings.

Specification

Description:The present invention relates to a non-invasive, portable diagnostic system that classifies and identifies skin disorders by using multispectral imaging and deep learning technologies. It couples a multispectral camera to a Raspberry Pi 4 computing unit with LED lighting via diffusers and a touch screen for the real-time diagnosis of dermatological conditions to be accurately detected. It captures very detailed images of the skin under a range of wavelengths, which includes UV, visible, and near-infrared, all processed through machine learning algorithms to classify the degree of skin conditions, which include acne, eczema, psoriasis, rosacea, and even possible skin cancers. Using a type of deep learning known as Convolutional Neural Networks, the images are analyzed so that hidden skin diseases are diagnosed. The device has cloud-based services where it is incorporated to store patient's data safely in the cloud, while the models updated on real-time by machine learning for remote dermatological diagnosis. The combination of high-resolution imaging, deep learning algorithms, and cloud integration represents a significant advancement in dermatological diagnostics, offering both improved diagnostic accuracy and patient comfort.
In an embodiment, the multispectral camera is strategically positioned to capture images from the patient's face, leveraging wavelengths such as ultraviolet (UV), visible light, and near-infrared (NIR) light. These different spectrums provide enhanced visibility of various skin layers, allowing the device to identify early-stage conditions and underlying systemic issues that are invisible under standard lighting. The Raspberry Pi 4 is the CPU, which processes the images acquisition and preprocessing as well as performs model inference. It employs CNN-based machine learning models for image analysis. The system accesses and continuously updates a cloud-based dataset for improving the diagnostic performance. Diffusers are integrated into LED lighting to provide uniform illumination for captured images, preventing shadowing that otherwise affect the results of diagnosis. Such portable design allows the use of the system in clinics, remote health units, and even home-based settings by using the small form factor that allows the use of a Raspberry Pi 4. This invention provides dermatologists with a low-cost, portable, and highly accurate tool for the diagnosis of skin health with potential for early detection and prevention against skin-related diseases.
According to the various embodiments of the present invention, Figure 1 illustrates the outer view of the system comprising various components such as: Headrest (1) that provides support to stabilize the user's head during image capture, Chinrest (2) that supports the user's chin to maintain a steady posture during the scan. In conjunction with the headrest to align the user's face correctly with the camera, ensuring consistent positioning for accurate and repeatable scans, Multispectral camera (3) that captures images at multiple wavelengths (e.g., ultraviolet, visible, near-infrared) to analyze various skin layers and conditions, stand (4) for the device that holds and stabilizes the multispectral camera, headrest, chinrest, and other components in a fixed position, LED flash button (5) to turn on and off the LED lights (integrated with diffusers), USB connection (6) to camera and power supply for LEDs that provides a communication link between the multispectral camera and the Raspberry Pi which also supplies electrical power to the array of LEDs integrated around the camera, 10.1 inches touch screen Displays system (7) controls, captured images, diagnostic results, and allows for user interaction, protective housing with LED panels integrated with diffusers (8) that encloses the multispectral camera, LEDs, and other components, with integrated diffusers ensuring even light distribution. There is an array of LEDs (integrated with diffusers) for proper lighting and to avoid shadows and reflections, ensuring high-quality images for accurate analysis, which can be controlled by the LED flash button.
According to various embodiments of the present invention, Figure 2 illustrates the internal connections (behind the touch screen display) comprising various components such as: Power supply (9) which provides a stable source of power to the entire system, including the Raspberry Pi, touchscreen display, LEDs, cooling fans, and USB hub ensuring that all components operate effectively without power interruptions, HDMI Port (10) which serves as an interface for high-definition video and audio transmission from the Raspberry Pi to the touchscreen display, HDMI (11) connection to touchscreen facilitating the display of high-resolution output and touch-based interaction for controlling the multispectral camera and viewing results, Touchscreen circuit (12) that detects and processes touch inputs on the display surface. It converts these touches into electrical signals that are sent to the Raspberry Pi for interpretation. USB Hub (13) which enables peripheral connections by expanding the number of USB ports, USB C-Port (14) that acts as the primary power input for the Raspberry Pi, supplying it with sufficient power for all its functions, Micro SD (Secure Digital) card (15) which is a memory card used for storing the data for further processing and archiving. Here, the software application (ML model) runs locally. Raspberry Pi (16) that serves as the central processing unit for the system. It handles image acquisition, preprocessing, running the trained ML model for diagnosis, and displaying results on the touchscreen, Vents (17) that allow heat to dissipate and enhance cooling efficiency, leading to prolonged component life, cooling fans (18) that prevent overheating of Raspberry Pi, protective casing (19) which encloses the Raspberry Pi, touchscreen, and other components to protect them from dust, physical damage, and environmental factors.
The hardware setup (Figure 3) includes a multispectral camera capturing high-quality images under multiple light spectrums. The camera is connected to the Raspberry Pi 4 via a USB interface, and its operations are controlled through a touchscreen display. The system features LED lights around the camera, which illuminate the patient's face with consistent, shadow-free light. The Raspberry Pi 4 serves as the primary processing unit, and a MicroSD card is used to store the operating system, deep learning models, and other software required for image analysis. The device is designed to be compact and portable, enabling use in a variety of healthcare settings, including mobile health clinics, hospitals, and remote areas. The cloud integration ensures that data and diagnostic results are securely stored, allowing for remote consultations and access to updated diagnostic models (figure 4), further enhancing the system's scalability and adaptability to new skin diseases. The hardware parts and components embedded in the device are explained below:
• Multispectral Camera
The multispectral camera captures high-resolution images of the patient's skin under various wavelengths of light, including ultraviolet (UV), visible light, and near-infrared (NIR). These different spectral bands help detect hidden skin conditions that do not observed with the naked eye. As such, to ensure precision fidelity during accuracy in the processing by model deep learning, it followed that high definition was considered important in designing it. It sits on a rigid support base with a headrest and chin rest, so that the patient remains motionless during data acquisition to prevent distortion.
• Raspberry Pi 4
The Raspberry Pi 4 serves as the heart of the diagnostic system, providing computing power for processing images, running machine learning models, and managing system operations. The Raspberry Pi 4 is equipped with sufficient processing capacity to handle image preprocessing tasks (such as normalization, alignment, and augmentation) and to run the CNN-based deep learning models that power the diagnostic process. It has USB connectors connecting it to other peripherals and a camera and touchscreen display, HDMI. It is also Wi-Fi enabled so that communicate it with the cloud.
• LED Lighting System
The LED lighting system is all around the camera, and it produces uniform lighting when an image is captured. LEDs are diffused to avoid shadowing and illuminate the face uniformly. The consistency of lighting is required for the high-quality acquisition of images because any form of shadowing interfere with the system's ability to detect small skin abnormalities and subtle changes in texture and color.
• Touchscreen Display
The touchscreen displays acts as the primary user interface for the system. It allows healthcare providers and patients to interact with the diagnostic system in an intuitive way, controlling image capture, viewing real-time diagnostic results, and managing the device settings. The Raspberry Pi connects through USB and HDMI with the touchpad for data transfer and video output. The display is being able to provide instant feedback on the diagnostic result, which is required for a real-time consultation.
• MicroSD Card
The MicroSD card contains the operating system of the system, deep learning models, and diagnostic data. This have been enough to run machine learning models that make large amounts of image data, hence conduct local-running machine learning models analysis of images. It allows the system to work by itself, even when installed in remote locations not that reliant on cloud connectivity.
• Power Supply and Cooling System
The Raspberry Pi, camera, LED lights, and touchscreen are powered by a USB-C power supply, which gives the necessary electrical power. It also integrates a cooling fan with the Raspberry Pi setup to avoid overheating when used for long hours. The casing is vented to ensure maximum airflow that keeps the performance and longevity of the device. The hardware setup for the device as:
The multispectral camera is mounted on a stabilized platform along with the headrest and chinrest, keeping the patient's face steady for proper imaging. The camera is placed in front of the patient, and the LED light panels are placed around the camera to illuminate the face evenly without casting shadows. The camera is connected to the Raspberry Pi 4 through a USB cable, capturing real-time images and processing them. Raspberry Pi 4 mounted in protective vented casing with an attached cooling fan as well to prevent overheating of the device during operation. There is a touchscreen display placed adjacent to the camera and connected to Raspberry Pi via USB and HDMI. The touchscreen provides health providers and patients with access to the system, where they interact with real-time diagnostic results and control the operations of the camera.
The software parts and components embedded in the device along-with hardware are explained below:
The software stack of the diagnostic system consists of three main components: the machine learning application, the cloud integration for data storage and model updating, and the user interface. The machine learning application uses Convolutional Neural Networks (CNNs) to analyze the multispectral images captured by the camera. The models are trained using a labeled massive dataset of skin images to detect several skin conditions such as acne, psoriasis, eczema, and even skin cancer. The application has pre-processing and preprocessing capabilities through the normalization, alignment, and augmentation functionalities that clean the input images before processing by the model. After the image has been processed, the CNN model will give diagnostic results as they appear on the touchscreen interface. In addition, the software is provided with cloud-based storage where patient data, including diagnostic reports and images, are stored safely for later reference. This also enables the system to retrieve updated models and datasets from the cloud for the improvement of the diagnostic accuracy. The user interface needs to be intuitive, allowing non-experts to talk to the system. It provides a touch-based interface for camera control, diagnostic results, and management of patient data. The software setup for the device as (figure 6):
The software setup begins by installing the necessary libraries and dependencies on the Raspberry Pi 4. In addition, some setups involve library installation that deals with multispectral cameras like PiCamera and their setup. Then, in OpenCV, there involves image preprocessing normalization, alignment, and augmentation. The MicroSD card stores the CNN-deep learning model so that it works directly and without any connectivity to the cloud for the system's normal running. The labeled skin image datasets are stored in the cloud provider, such as AWS and Google Cloud. It allows continuous training and updating of the machine learning model. The user interface is being developed as a touch-sensitive application that allows the users to initiate image capture, view real-time results, and manage system settings.
The steps of working of the device (Figure 4) disclosed in the invention are as follows: User Positioning: The user is seated with a headrest and chinrest for stability in front of the multispectral camera. Lighting Setup: LEDs with diffusers are turned on to ensure consistent and adequate lighting on the user's face. Camera Operation: The multispectral camera is controlled via a touchscreen display, capturing images across various wavelengths. Image Processing: Captured images are transferred to the microSD card in the Raspberry Pi for preprocessing, including normalization, alignment, and augmentation. Model Inference: The pre-processed images are fed into the trained ML model stored on the microSD card to run inference. Data Analysis: Datasets stored in the cloud are utilized for further analysis. Result Display: The results of the analysis are displayed on the touchscreen display. Secure Storage: Diagnostic reports and data are securely stored in the cloud for future access and review by patients and dermatologists. Real-Time Updation: The ML model and datasets are continuously updated in real-time with new data from the cloud for improved accuracy.
Working Mechanism of the Device:
The working mechanism (Figure 5) of the diagnostic device is based on the synchronous operations between its hardware and software components. In this system, multispectral images of the patient's skin are captured using a multispectral camera, which capture images of various wavelengths, including UV, visible light, and near-infrared spectrums. These spectrums are used because they make the normally invisible features of the skin prominent to the human naked eye under normal lighting. The processing and analysis of the images received being reflected directly into diagnostic feedback.
Firstly, the user, who most often is a health worker or the patient, enters into the system through the use of the touchscreen display; he/ she proceeds with the selection of the process involved in image capture. When the system is on, LED lighting around the camera helps in uniformly illuminating the skin of a patient, keeping it uniformly lit and completely shadow-free during the actual process of image acquisition by the multispectral camera that takes images of the surface and subsurface layers in the skin at high-resolution resolution with various wavelengths of the light. The camera is positioned in front of the patient's face. A headrest and chinrest are used to keep the patient in a static position during the entire imaging process.
The images acquired are transmitted directly to the Raspberry Pi 4 for preprocessing. Normalization of the pixel values of the images are being done on all spectral images to reduce the lighting and environmental effects. The alignment of the images is carried out to get the multispectral images registered correctly with one another so that the effect of misalignment, which lead to misdiagnosis, is avoided. Naturally, augmentation methods on the images are used in rotation, flipping, and zooming to enhance the dataset's robustness by boosting the model's ability to spot various skin conditions.
After the images are pre-processed, they flow through the deep learning architecture that makes up the system. This CNN model has gone through a large number of labeled skin images for dermatological conditions recognition. The CNN model does this by picking important features within the image such as texture changes, color differences, abnormalities in skin pattern, diagnosis of diseases, including some of the worst like acnes, eczemas, psoriasis, rosaceae, and also cancer cases. This image is layered and checked one after the other, giving out and matching it against patterns observed during training as known ones. This diagnostic output either be a condition, for example, "Acne" and a referral for further analysis and evaluation by a dermatologist.
Once the CNN model is done analyzing it, then it projects it in real-time to the screen of the touchscreen device from where the healthcare provider/patient see what that diagnosis been along with any recommendation that is extra. The results are stored locally on the MicroSD card for direct access if necessary and uploaded to the cloud storage for future reference, as well as for further analysis and observation long-term. Cloud storage also ensures that all device data is safely backed up and available for remote consultations with a dermatologist, thus complementing the ability of this device in telemedicine applications. The cloud ensures that the system keeps updating machine learning models based on input data from the environment, thus diagnosing the device with increasing accuracy over time.
The whole diagnosing process runs efficiently and fast due to the computing might of the Raspberry Pi 4. The device handles image processing, model inference, and managing data for the system. It retains the connection via Wi-Fi and Bluetooth with cloud-based servers that keeps on updating the system for new models and patient information. The portability of the device, combined with user-friendliness of interface, makes it simple to use for healthcare providers in almost all kinds of different settings, from mobile health units up to hospitals, as well as even home-based care.
Functionalities of the Software Embedded within Device Hardware (Figure 6):
Image Capture and Preprocessing: The system utilizes the PiCamera library on the Raspberry Pi to control the multispectral camera. The software enables the camera to capture images under specific wavelengths (UV, visible, and NIR) by activating the LED light system in controlled conditions. Image preprocessing is an important first step after the image is captured. The software normalizes pixel intensity from the images captured from different light sources, ensuring that variability in lighting does not impact the diagnostic outcome of said images. This is critical as it enables accuracy in diagnostics based on images taken under different natural and artificial lights.
Image Registration and Alignment: After capturing images from different wavelengths, the software automatically registers and aligns the images to ensure they are in perfect spatial correspondence with one another. This image alignment ensures that the multispectral images are synchronized correctly, preventing errors that occurred if the images are slightly shifted and distorted. The system uses feature-based alignment techniques such as detecting key points and applying homography transformations to register the images properly, especially for situations where the patient moves slightly during the imaging process.
Deep Learning Model Processing (CNN Inference): Once the images are pre-processed and aligned, they are passed to the deep learning model, specifically a Convolutional Neural Network (CNN), for image classification. The software integrates the TensorFlow/ PyTorch deep learning frameworks to handle the model inference. This model extracts features from images and matches those features to known patterns of many dermatological conditions that it learned in the course of its training. The software then attaches a label, such as "Acne," "Psoriasis, " to the image based on the interpretation of the data given by the CNN. The model also prints out the confidence score, and probability of how likely the detected condition is to be skin disease. The value of the confidence score is the higher, the more confident the diagnosis.
Cloud Integration for Data Storage and Model Updates: The software allows cloud integration to store both patient data (including images and diagnostic results) and to periodically update the machine learning model. This integration ensures that the system adapt to new dermatological research and emerging skin conditions by retrieving new training datasets and model parameters from the cloud. The system automatically uploads images and diagnostic reports to a secure cloud storage service, ensuring that data is backed up for future reference and is available for remote consultations. Furthermore, periodically, the software checks whether the deep learning model used has any updates; downloading all improvements ensures better diagnostics with accuracy.
Real-Time Feedback and User Interface: The touchscreen interface provides real-time feedback to the user, displaying the diagnostic results as soon as the system has finished processing the images. The results are shown in an easy-to-read format, with clear visual indicators for different skin conditions and recommended actions. The user interacts with the touchscreen to perform a variety of actions, such as re-taking images, adjusting settings (such as camera focus or light intensity), and viewing historical diagnostic results. The interface also provides options for manual input, such as entering patient details and adjusting diagnostic thresholds if necessary.
Telemedicine Support via Remote Access: The software supports remote access to the diagnostic results, enabling healthcare providers to reach out from anywhere and access patient data and diagnostic reports through cloud-based portals. It is also helpful for telemedicine applications, whereby dermatologists remotely consult patients, review diagnostic images, and guide and treat them. Patients also access their own diagnostic results via a patient portal, where they track their skin condition over time, view past results, and receive updates from their healthcare provider.
System Maintenance and Updates: The software has an embedded update mechanism that ensures the system is always running the latest software version, including bug fixes, performance improvements, and model updates. The device regularly checks for updates from a central repository, and any new versions of the software and deep learning models are downloaded automatically, minimizing the need for manual intervention. The update system ensures that in case some vulnerabilities appear, they get fixed in the shortest possible time, so the patient data is always secure and the medical facility is in compliance with healthcare privacy regulations.
Local Storage and Data Management: The device also supports local storage on the MicroSD card where all patient images, reports, and diagnostic data been stored temporarily for upload to the cloud. It thus means that it is being possible for the system to be operable even when internet connectivity is limited and thereby ensuring continuation in some remote locations. In addition to this, the software uses data compression techniques to enable storage space management and allow the device to store bulk volumes of image data without running out of space.
User Authentication and Security: The system incorporates user authentication mechanisms, such as PIN codes and passwords, to ensure that only authorized personnel can access and interact with the device. This is critical for maintaining patient confidentiality and ensuring that diagnostic data is not accessed by unauthorized users. The software uses encryption protocols to secure sensitive data both during transmission (to the cloud) and while stored locally on the device, ensuring compliance with privacy laws such as HIPAA.
Through these functionalities, the software embedded in the device facilitates seamless integration of hardware components, enabling efficient and accurate skin disease detection while ensuring scalability, security, and ease of use.
, Claims:1. An automated portable skin disease diagnostic system comprising:
a multispectral camera configured to capture images of a patient's skin at multiple wavelengths, including ultraviolet (UV), visible light, and near-infrared (NIR), to allow for detailed analysis of skin conditions at various depths;
a lighting system integrated with the multispectral camera, comprising one or more LEDs with diffusers, configured to illuminate the patient's skin evenly and reduce shadowing during image capture;
a Raspberry Pi 4 microcomputer that serves as a central processing unit, performing image preprocessing, storage, and initial data processing, and executing machine learning (ML) algorithms for diagnosing skin diseases;
a convolutional neural network (CNN) based machine learning model stored in the Raspberry Pi/ a cloud server, capable of analyzing the captured multispectral images to identify and classify a variety of skin conditions;
a touchscreen displays for user interaction, configured to display diagnostic results, allow image capture initiation, and facilitate device control and settings adjustments;
a cloud integration module, enabling the transfer of captured images and diagnostic data to a cloud storage service, and enabling real-time model updates for improving diagnostic accuracy.
2. The skin disease diagnostic system of Claim 1, wherein the lighting system comprises a set of adjustable LEDs that provide customizable illumination intensity to optimize image capture under different environmental conditions.
3. The skin disease diagnostic system of Claim 1, wherein the Raspberry Pi 4 is connected to the cloud integration module via Wi-Fi, enabling real-time data synchronization and model updates.
4. The skin disease diagnostic system of Claim 1, further comprising a power supply unit configured to provide energy to the Raspberry Pi, the multispectral camera, the touchscreen displays, and the lighting system, with low power consumption for portable use.
5. The skin disease diagnostic system of Claim 1, wherein the convolutional neural network (CNN) model is trained using a diverse dataset of labeled skin images to enhance the model's ability to accurately identify a wide range of skin conditions, including acne, rosacea, psoriasis, eczema, and skin cancers.
6. The skin disease diagnostic system of Claim 1, wherein the multispectral camera captures images at a plurality of different spectral bands, and the captured images are analyzed to detect early-stage skin conditions that are not visible to the naked eye.
7. A method for diagnosing skin diseases using multispectral imaging and deep learning, comprising the steps of:
positioning a patient in front of a multispectral camera with a headrest and chinrest to stabilize the patient's face and ensure accurate image capture;
capturing images of the patient's skin using the multispectral camera at multiple wavelengths, including UV, visible light, and NIR, to acquire comprehensive data on the skin's condition;
preprocessing the captured images, including normalization, alignment, and augmentation to improve the quality and consistency of the images for analysis;
feeding the pre-processed images into a trained CNN-based machine learning model, either stored on a microSD card of the Raspberry Pi/ in a cloud server, to analyze and classify the skin condition;
displaying diagnostic results on a touchscreen display, including the identification of specific skin conditions and recommendations for further action;
transferring the diagnostic data to a cloud storage service for secure storage and remote access, facilitating future monitoring and follow-up.
8. The method of Claim 7, further comprising the step of storing diagnostic reports and patient data securely in cloud storage, with encryption and access controls to protect patient privacy and comply with healthcare data regulations.
9. The method of Claim 7, wherein the CNN-based machine learning model is continuously updated using new data uploaded to the cloud, enabling the system to improve diagnostic accuracy over time.
10. The method of Claim 7, wherein the diagnostic results provided on the touchscreen display include recommendations for further medical action, such as scheduling an in-person consultation with a dermatologist or initiating a treatment plan.

Documents