A Proposed Video Super-Resolution Strategy using Wavelet Multi-Scale Convolutional Neural Networks

Elgohary, M. A.; E. Abd EL-Samie, Fathi; El-Shafai, Walid; Mohamed, M. A.; H. Abdelhay, E.

doi:10.21608/bfemu.2022.258300

A Proposed Video Super-Resolution Strategy using Wavelet Multi-Scale Convolutional Neural Networks

Document Type : Research Studies

Authors

¹ MSc Student, Department of Mechatronics Engineering, High Institute of Engineering and Technology, El-mahalla El-koubra, Egypt

² Professor, Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University Menouf, Egypt

³ Assistant Professor, Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University Menouf, Egypt

⁴ Editor in - chief, Dean of the College and Chairman of the Board of Directors, Faculty of Engineering, Mansoura University, professor Communications Engineering Dept., Faculty of Engineering, Mansoura University, Egypt

⁵ Assistant Professor, Department of Electronics and Communications Engineering, Faculty of Engineering, Mansoura University, Mansoura, Egypt

10.21608/bfemu.2022.258300

Abstract

High-resolution images are often required and desired for most applications, as they incorporate complementary information. However, the optimal utilization of sensor technology and visual technology to improve picture pixel density is often limited and prohibitively expensive. As a result, employing an image processing method to build a high-resolution image from a low-resolution one is a costly and comprehensive option. The goal of video super-resolution is to restore intricate points and reduce the sensory effects. This research builds on the multi-frame super-resolution approach by using wavelet analysis to train convolutional neural networks (CNNs). For that purpose, the approach begins by applying wavelet decomposition on video segments for multi-scale assessment. Then, several CNNs are trained independently to approximate wavelet multi-scale characterizations. The trained CNNs do inference by regressing wavelet multi-scale characterizations from LR frames, followed by wavelet reconstruction, which produces recovered HR frames. This research presents a learning-based method for preserving fine features in low-resolution multi-frame images captured with various camera zoom lenses. The experimental findings confirm the proposed strategy for restoring difficult frames.

Keywords