Deep Learning Framework for Video Frame Interpolation

Abstract

The present invention discloses a deep learning-based framework for video frame interpolation that generates high-quality intermediate frames between consecutive video frames, improving temporal smoothness and visual quality. By combining convolutional neural networks (CNNs), transformers, and generative adversarial networks (GANs), the framework captures both global motion and local pixel-level changes, enhancing the accuracy of interpolated frames while reducing common artifacts like motion blur and ghosting. Optimized for real-time performance, the system utilizes techniques such as model quantization and pruning to ensure efficient deployment on edge devices, making it suitable for applications in video streaming, gaming, virtual reality (VR), and augmented reality (AR).

Specification

Description:The present invention relates to the field of computer vision and video processing, specifically to a deep learning framework designed for video frame interpolation. The invention aims to enhance visual quality by generating intermediate frames between existing video frames, thereby improving temporal smoothness in various video applications such as streaming, gaming, and virtual reality (VR).
BACKGROUND OF THE INVENTION
The following description of related art is intended to provide background information pertaining to the field of the disclosure. This section may include certain aspects of the art that may be related to various features of the present disclosure. However, it should be appreciated that this section be used only to enhance the understanding of the reader with respect to the present disclosure, and not as admissions of prior art.

Video frame interpolation is a critical technique in video processing that involves generating intermediate frames between existing consecutive frames to increase th , Claims:1. A deep learning framework for video frame interpolation, comprising:
• An input module configured to receive two consecutive video frames;
• An encoder utilizing a convolutional neural network (CNN) to extract feature maps from the input frames;
• A transformer-based temporal modeling unit designed to capture spatial and temporal dependencies between the feature maps of the input frames;
• A decoder network configured to generate an interpolated video frame by fusing multi-scale features extracted from the encoder and transformer modules;
• A generative adversarial network (GAN) for refining the visual quality of the interpolated frame by minimizing artifacts;
• An output module to render the interpolated video frame as part of a continuous video sequence.
2. The framework of claim 1, wherein the encoder further comprises:
• A multi-scale feature extraction mechanism that captures global and local motion features at different resolutions.
3. The framework of claim 1, wherein the transformer-based temporal

Documents