U.S. flag

An official website of the United States government, Department of Justice.

Recursive Multi-Scale Image Deraining With Sub-Pixel Convolution Based Feature Fusion and Context Aggregation

NCJ Number
Ieee Access Volume: 8 Dated: 2020 Pages: 177495-177505
Date Published
11 pages

This article proposes a deraining algorithm that can  boost the reconstruction/deraining quality without the problem of over- or under-deraining. 


Along with several other low-vision-based computer vision problems, single image deraining is also taken as a challenging one due to its ill-posedness. Several algorithms based on convolutional neural networks are devised that are either too simple to provide acceptable deraining results due to under-deraining or have complex architectures that may result in over-deraining. Along with the originally proposed network, two of its' light-weight versions with reduced computational costs are also devised. Basically, this project proposes a recursively trained architecture that has two major components: a front-end module and a refinement module. The front-end module is based on dense fusion of lower label features followed by sub-pixel convolutions (pixel shuffling based convolutions). To refine and generate the enhanced deraining results further, the authors cascade a refinement module to the front-end module using multi-scale Context Aggregation Network (CAN) which includes feature fusion and pixel shuffling based convolutions. The deraining results are presented in terms of Structural Similarity Index (SSIM) and Peak Signal to Noise Ratio (PSNR) on several benchmarks and compare with current state-of-the-art algorithms. With comprehensive experiments on both real-world and synthetic datasets and extensive ablation study, the authors demonstrate that the proposed approach produces better results compared to existing methods. (publisher abstract modified)

Date Published: January 1, 2020