REAL-TIME SCENE VISIBILITY ENHANCEMENT IN UNDERWATER ENVIRONMENT

Abstract

This invention discloses a system for real-time underwater scene visibility enhancement using a 3-level deep multi-patch hierarchical neural network (100). The model is trained in two stages: the first stage minimizes standard image distortions, and the second stage optimizes temporal consistency, reducing flickering in real-time video applications. The system aims to assist underwater exploration by improving camera feed visibility.

Specification

DESCRIPTION:
[0001] The title of the invention is Real-time Scene Visibility Enhancement in Underwater Environment. The invention enhances an underwater video feed in real-time using a deep, multi-patch, hierarchical neural network.
PRIOR ART AND BACKGROUND:
[0002] CN110689490A: This patent describes a framework that uses texture color features to restore underwater images. Our invention extracts features from patches on multiple levels to restore underwater images.
[0003] CN115205166A: This patent describes a framework that comprises of a transmission diagram network and an ambient light network. Our invention has a single, end-to-end convolutional network. Our invention can also work on frames of any size.
[0004] CN115713469A: This patent describes a framework that fuses an adaptive channel attention module and a deformation convolution module to generate a confrontation network. Our invention uses multiple levels of encoders and decoders.
[0005] CN115035010A: This patent describes a framework that fuses multiple levels of enhancement using a depth dense residual error module but does not account for temporal artifacts. Our invention uses a second loss function for temporal coherence. Our invention also incorporates a temporal smoothing module.
[0006J CN115660980A: This patent describes a framework that comprises of a single parameter estimation network and an image enhancement network. Our network uses multiple levels of encoders and decoders, leading to multiple levels of parameter estimation and image enhancement.
[0007] CN114119383B: This patent describes a framework based on multi-feature fusion, solving color degradation problems using accurate background light and transmissivity. Our invention extracts temporal features along with other spatial features.
OBJECTIVE:
[0008] The primary objective is to develop a module that performs image visibility enhancement for underwater environments in a real-time system, such as a live camera feed. This module will assist in underwater exploration and can be combined with objectdetection models for more accurate results.
SUMMARY:
[0009] The invention proposes a 3-level deep multi-patch hierarchical neural network designed to enhance scene visibility in underwater environments in real time. The network is designed to handle images in multiple hierarchical patches.
The architecture consists of three levels, where each subsequent level handles an increasing number of patches (1, 4, 16). This architecture efficiently removes non-homogeneous artifacts from,the enhanced output image.
[0010] The training takes place in two stages, one training image-specific features and the other training temporal features.
DETAILED TECHNICAL DESCRIPTION:
[0011 ] The 3-level deep multi-patch hierarchical network is designed to perform visibility enhancement in underwater environments by progressively focusing on larger and smaller details through hierarchical patches.
[0012] Network Architecture:
. Level 1. Handles the entire image as a single patch. This layer captures global features, focusing on large-scale corrections such as overall color balancing and contrast enhancement.
Level 2. Divides the image into four non-overlapping patches with one vertical split and one horizontal split.
Level 3. Divides the image from the second level into four patches.
[0013] Training Strategy:
Stage 1. (Normal Loss Function): The model is initially trained to reduce standard underwater image distortions such as noise, low contrast, and blurriness. A combination of mean squared error (MSE) and perceptual losses are used in this stage.
Stage 2. (Temporal Consistency Loss Function): The second training phase adds a temporal consistency loss function to ensure smoothness between consecutive video frames. This step is crucial for real-time applications, ensuring that flickering and sudden visual changes are minimized, resulting in stable and coherent video outputs.
BRIEF DESCRIPTION OF THE DRAWING;
Fig 1:
A flow diagram (100) of the 3-level deep multi-patch hierarchical neural network.
Fig 2:
A flow diagram of the architecture and layer configurations of the encoder (102) and decoder (103).
LIST OF REFERENCE NUMERALS
100 - Multi-patch enhancement network
101 - Input node which accepts RGB images
102 - Encoder of the network that extracts features from the image
103 - Decoder of the network that reconstructs the image
104 - Output node which returns an RGB image
105 - Convolution layer with parameters of the format (in_channels, out_channels,. kernel_size, stride)
106 - Deconvolution (Conv transpose) layer with parameters of the format (in_channels, out_channels, kerneLsize, stride)
107 - ReLU activation layer
WE Claim.
1. A method for real-time scene visibility enhancement in underwater environments, comprising:
a. Implementing a 3-level deep multi-patch hierarchical neural network (100).
b. Training the network in two stages, wherein the first stage minimizes standard image distortions using a normal loss function, and the second stage optimizes temporal consistency using a temporal consistency loss function to reduce flickering between video frames.
2. The method of claim 1, wherein the network architecture comprises 3 levels, with the first level processes the image as 1 patch, the second as 4, and the third as 16 patches.
3. The method of claim 1, wherein each level consists of an encoder (102) and a decoder (103).
4. The method of claim 1, wherein a temporal consistency loss function that accounts for the difference between the current restored frame and the previous restored frame is used to minimize the changes between consecutive frames to reduce temporal artifacts such as flickering in real-time video feeds.

Documents