
Journal of Northeastern University(Natural Science) ›› 2025, Vol. 46 ›› Issue (11): 48-57.DOI: 10.12068/j.issn.1005-3026.2025.20250072
• Information & Control • Previous Articles Next Articles
Tian-qi WANG1, Di ZHANG2(
), Xin-yu ZHANG2, Yi-ming ZHANG3
Received:2025-06-24
Online:2025-11-15
Published:2026-02-07
Contact:
Di ZHANG
CLC Number:
Tian-qi WANG, Di ZHANG, Xin-yu ZHANG, Yi-ming ZHANG. Dual-Stage Deep Learning-Based Image Inpainting Method[J]. Journal of Northeastern University(Natural Science), 2025, 46(11): 48-57.
| 类别 | 训练集 | 验证集 | 测试集 | 小计 |
|---|---|---|---|---|
| 合计 | 1 946 | 279 | 270 | 2 495 |
| 人物 | 963 | 148 | 141 | 1 252 |
| 建筑 | 983 | 131 | 129 | 1 243 |
Table 1 Image restoration dataset information
| 类别 | 训练集 | 验证集 | 测试集 | 小计 |
|---|---|---|---|---|
| 合计 | 1 946 | 279 | 270 | 2 495 |
| 人物 | 963 | 148 | 141 | 1 252 |
| 建筑 | 983 | 131 | 129 | 1 243 |
| 模型 | PSNR | LPIPS | FID | Colorfulness Score |
|---|---|---|---|---|
| CycleGAN | 18.1 | 0.36 | 62.3 | 31.9 |
| DeOldify | 18.9 | 0.29 | 56.1 | 36.3 |
| Restormer | 18.7 | 0.23 | 48.0 | 38.3 |
| DSS-HIRC | 19.8 | 0.21 | 43.5 | 39.9 |
Table 2 Performance of previous algorithms on
| 模型 | PSNR | LPIPS | FID | Colorfulness Score |
|---|---|---|---|---|
| CycleGAN | 18.1 | 0.36 | 62.3 | 31.9 |
| DeOldify | 18.9 | 0.29 | 56.1 | 36.3 |
| Restormer | 18.7 | 0.23 | 48.0 | 38.3 |
| DSS-HIRC | 19.8 | 0.21 | 43.5 | 39.9 |
| 模型 | 场景 | PSNR | LPIPS | FID | Colorfulness Score |
|---|---|---|---|---|---|
| CycleGAN | 人物 | 18.5 | 0.42 | 61.2 | 32.1 |
| 建筑 | 17.6 | 0.29 | 63.4 | 31.6 | |
| DeOldify | 人物 | 19.6 | 0.33 | 55.3 | 36.6 |
| 建筑 | 18.2 | 0.25 | 56.9 | 35.9 | |
| Restormer | 人物 | 20.3 | 0.17 | 42.9 | 41.1 |
| 建筑 | 17.1 | 0.29 | 53.1 | 35.4 | |
| DSS-HIRC | 人物 | 20.5 | 0.17 | 41.3 | 41.7 |
| 建筑 | 18.5 | 0.23 | 45.7 | 38.2 |
Table 3 Performance of previous algorithms on
| 模型 | 场景 | PSNR | LPIPS | FID | Colorfulness Score |
|---|---|---|---|---|---|
| CycleGAN | 人物 | 18.5 | 0.42 | 61.2 | 32.1 |
| 建筑 | 17.6 | 0.29 | 63.4 | 31.6 | |
| DeOldify | 人物 | 19.6 | 0.33 | 55.3 | 36.6 |
| 建筑 | 18.2 | 0.25 | 56.9 | 35.9 | |
| Restormer | 人物 | 20.3 | 0.17 | 42.9 | 41.1 |
| 建筑 | 17.1 | 0.29 | 53.1 | 35.4 | |
| DSS-HIRC | 人物 | 20.5 | 0.17 | 41.3 | 41.7 |
| 建筑 | 18.5 | 0.23 | 45.7 | 38.2 |
| 方法 | PSNR | LPIPS | FID | 训练时长/h |
|---|---|---|---|---|
| 无自监督退化 | 17.3 | 0.36 | 58.8 | 18.3 |
| 使用ViT特征 | 18.3 | 0.25 | 45.2 | 19.1 |
| 使用ResNet-50特征 | 17.9 | 0.22 | 47.3 | 18.6 |
| 采用独立训练 | 18.9 | 0.22 | 45.4 | 25.2 |
| DSS-HIRC | 19.8 | 0.21 | 43.5 | 20.0 |
Table 4 Comparison of model performance for different model settings
| 方法 | PSNR | LPIPS | FID | 训练时长/h |
|---|---|---|---|---|
| 无自监督退化 | 17.3 | 0.36 | 58.8 | 18.3 |
| 使用ViT特征 | 18.3 | 0.25 | 45.2 | 19.1 |
| 使用ResNet-50特征 | 17.9 | 0.22 | 47.3 | 18.6 |
| 采用独立训练 | 18.9 | 0.22 | 45.4 | 25.2 |
| DSS-HIRC | 19.8 | 0.21 | 43.5 | 20.0 |
| 批大小 | 嵌入维度 | 卷积核 | PSNR | LPIPS | FID | 训练时长/h |
|---|---|---|---|---|---|---|
| 8×8 | 256 | 3×3 | 18.9 | 0.24 | 46.7 | 19.3 |
| 16×16 | 256 | 3×3 | 19.3 | 0.23 | 45.1 | 19.8 |
| 16×16 | 256 | 5×5 | 19.6 | 0.22 | 44.3 | 20.2 |
| 16×16 | 512 | 3×3 | 19.7 | 0.21 | 43.9 | 20.5 |
| 16×16 | 512 | 5×5 | 19.8 | 0.21 | 45.4 | 20.0 |
Table 5 Performance under different network-related hyperparameter settings
| 批大小 | 嵌入维度 | 卷积核 | PSNR | LPIPS | FID | 训练时长/h |
|---|---|---|---|---|---|---|
| 8×8 | 256 | 3×3 | 18.9 | 0.24 | 46.7 | 19.3 |
| 16×16 | 256 | 3×3 | 19.3 | 0.23 | 45.1 | 19.8 |
| 16×16 | 256 | 5×5 | 19.6 | 0.22 | 44.3 | 20.2 |
| 16×16 | 512 | 3×3 | 19.7 | 0.21 | 43.9 | 20.5 |
| 16×16 | 512 | 5×5 | 19.8 | 0.21 | 45.4 | 20.0 |
| 参数设置 | PSNR | LPIPS | FID | 训练时长/h |
|---|---|---|---|---|
| 学习率=5e-4 | 18.5 | 0.28 | 50.6 | 17.8 |
| 学习率=5e-5 | 19.8 | 0.21 | 43.5 | 20.0 |
| 批大小=8×8 | 19.0 | 0.22 | 47.3 | 20.4 |
| 批大小=16×16 | 19.6 | 0.21 | 43.6 | 20.1 |
| 批大小=32×32 | 18.8 | 0.22 | 44.0 | 19.2 |
| 权重衰减=0 | 19.1 | 0.24 | 47.9 | 19.6 |
| 权重衰减=1e-4 | 19.8 | 0.21 | 43.5 | 20.0 |
Table 6 Comparison of model performance under different hyperparameter settings
| 参数设置 | PSNR | LPIPS | FID | 训练时长/h |
|---|---|---|---|---|
| 学习率=5e-4 | 18.5 | 0.28 | 50.6 | 17.8 |
| 学习率=5e-5 | 19.8 | 0.21 | 43.5 | 20.0 |
| 批大小=8×8 | 19.0 | 0.22 | 47.3 | 20.4 |
| 批大小=16×16 | 19.6 | 0.21 | 43.6 | 20.1 |
| 批大小=32×32 | 18.8 | 0.22 | 44.0 | 19.2 |
| 权重衰减=0 | 19.1 | 0.24 | 47.9 | 19.6 |
| 权重衰减=1e-4 | 19.8 | 0.21 | 43.5 | 20.0 |
| [1] | 陈文祥,田启川,廉露,等.基于深度学习的图像修复方法研究进展[J].计算机工程与应用,2024, 60(22):58-73. |
| Chen Wen-xiang, Tian Qi-chuan, Lian Lu, et al. Research progress of image inpainting methods based on deep learning[J]. Computer Engineering and Applications, 2024, 60(22):58-73. | |
| [2] | Anwar S, Tahir M, Li C Y, et al. Image colorization: a survey and dataset[J]. Information Fusion, 2025, 114: 102720. |
| [3] | Chua L O. CNN: a vision of complexity[J]. International Journal of Bifurcation and Chaos, 1997, 7(10): 2219-2425. |
| [4] | Isola P, Zhu J Y, Zhou T H, et al. Image-to-image translation with conditional adversarial networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, 2017: 5967-5976. |
| [5] | Parmar N, Vaswani A, Uszkoreit J, et al. Image Transformer[C]//Proceedings of the 35th International Conference on Machine Learning (ICML). Stockholm, 2018: 4055-4064. |
| [6] | 魏赟,王璐璐,辛子昊,等.由粗到细的内容一致性图像修复方法[J].华中科技大学学报(自然科学版), 2025, 53(5): 178-184. |
| Wei Yun, Wang Lu-lu, Xin Zi-hao, et al. Coarse to fine approach to content-consistent image inpainting[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2025, 53(5): 178-184. | |
| [7] | Tai Y, Yang J, Liu X M, et al. MemNet: a persistent memory network for image restoration[C]//2017 IEEE International Conference on Computer Vision (ICCV). Venice, 2017: 4549-4557. |
| [8] | 周啟雪, 余映, 胡家绿. 利用Involution级联注意力机制的古代壁画图像修复网络[J]. 计算机科学, 2025, 52(12): 158-165. |
| Zhou Qi-xue, Yu Ying, Hu Jia-lyu). Ancient mural image restoration network using Involution cascade attention mechanism[J]. Computer Science, 2025, 52(12: 158-165. | |
| [9] | Wang Z D, Cun X D, Bao J M, et al. Uformer: a general U-shaped Transformer for image restoration[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, 2022: 17662-17672. |
| [10] | Liang J Y, Cao J Z, Sun G L, et al. SwinIR: image restoration using swin transformer[C]//2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). Montreal, 2021: 1833-1844. |
| [11] | Karras T, Laine S, Aittala M, et al. Analyzing and improving the image quality of StyleGAN[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, 2020: 8107-8116. |
| [12] | Vitoria P, Raad L, Ballester C. ChromaGAN: adversarial picture colorization with semantic class distribution[C]//2020 IEEE Winter Conference on Applications of Computer Vision (WACV). Snowmass Village, 2020: 2434-2443. |
| [13] | Guo Y, Gao Y, Lu Y X, et al. OneRestore: a universal restoration framework forComposite degradation[C]//Computer Vision-ECCV 2024. Cham: Springer, 2025: 255-272. |
| [14] | Gaa D, Chizhov V, Peter P, et al. Connecting image inpainting with denoising in the homogeneous diffusion setting[J]. Advances in Continuous and Discrete Models, 2025, 2025(1): 74. |
| [15] | Kim G, Kang K, Kim S, et al. BigColor: colorization using a generative color prior for natural images[M]//Computer Vision-ECCV 2022. Cham: Springer Nature Switzerland, 2022: 350-366. |
| [16] | Kang X Y, Yang T, Ouyang W Q, et al. DDColor: towards photo-realistic image colorization via dual decoders[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). Paris, 2024: 328-338. |
| [17] | Salmona A, Bouza L, Delon J. DeOldify: a review and implementation of an automatic colorization method[J]. Image Processing on Line, 2022, 12: 347-368. |
| [18] | Cui M Z, Jiang H, Li C Z. Progressive-augmented-based DeepFill for high-resolution image inpainting[J]. Information, 2023, 14(9): 512. |
| [19] | Zhang H M, Wang M C, Zhang Y X, et al. TDA-Net: a novel transfer deep attention network for rapid response to building damage discovery[J]. Remote Sensing, 2022, 14(15): 3687. |
| [20] | Yang F, Ning B, Li H Q. An overview of multimodal fusion learning[C]//Mobile Multimedia Communications. Cham: Springer, 2022: 259-268. |
| [21] | Chen Y T, Xia R L, Yang K, et al. Dual degradation image inpainting method via adaptive feature fusion and U-Net network[J]. Applied Soft Computing, 2025, 174: 113010. |
| [22] | Rong W B, Li Z J, Zhang W, et al. An improved Canny edge detection algorithm[C]//2014 IEEE International Conference on Mechatronics and Automation. Tianjin, 2014: 577-582. |
| [23] | 杨晓雨, 王爱侠, 杨钢, 等. 基于生成对抗网络的人脸年龄渐进合成算法[J]. 东北大学学报(自然科学版), 2024, 45(7): 944-952. |
| Yang Xiao-yu, Wang Ai-xia, Yang Gang, et al. Progressive face age synthesis algorithm based on generative adversarial network[J]. Journal of Northeastern University (Natural Science), 2024, 45(7): 944-952. | |
| [24] | Almahairi A, Rajeswar S, Sordoni A, et al. Augmented CycleGAN: learning many-to-many mappings from unpaired data[C]//Proceedings of the 35th International Conference on Machine Learning (ICML). Stockholm, 2018: 195-204. |
| [25] | Zamir S W, Arora A, Khan S, et al. Restormer: efficient transformer for high-resolution image restoration[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, 2022: 5718-5729. |
| [1] | Zhen-long HOU, Jin-rong SHEN, Ji-kang WEI, Wen-tian ZHAO. Intelligent Identification of Minerals Polished Section Under Microscope Based on SSD and Image Transformation [J]. Journal of Northeastern University(Natural Science), 2025, 46(6): 131-137. |
| [2] | Bin YU, Hong-chun SUN, Da-yong YE. Intelligent Fault Diagnosis Under Variable Loads Based on the Embedded Gear Fault Mechanism [J]. Journal of Northeastern University(Natural Science), 2025, 46(4): 61-70. |
| [3] | Hui ZOU, Li-huang SHE, Ye-han CHEN, Yi YUE. 3D Gesture Estimation Algorithm Based on Geometric Attention Mechanism [J]. Journal of Northeastern University(Natural Science), 2025, 46(10): 44-50. |
| [4] | Ji-hong LIU, Xi-xiong HUANG. Face Inpainting Model Based on Denoising Diffusion Probability Models [J]. Journal of Northeastern University(Natural Science), 2024, 45(9): 1227-1234. |
| [5] | Yan LIU, Qi-jie BU, Hong-chen ZHAO, Xin GUO. Operating Performance Assessment of Flotation Process Based on Multi-source Heterogeneous Information [J]. Journal of Northeastern University(Natural Science), 2024, 45(9): 1217-1226. |
| [6] | Hong-li LI, Hao-yu LIU, Rong-hua ZHANG, Yi CHENG. Emotional Classification Based on Multidimensional Feature Matrix and Improved Dense Connection Network [J]. Journal of Northeastern University(Natural Science), 2024, 45(7): 928-935. |
| [7] | Ji-hong LIU, Lü-heng ZHANG, Hai-xu YANG. A Saturation Artifact Inpainting Algorithm for Cell Fluorescence Microscopic Images [J]. Journal of Northeastern University(Natural Science), 2024, 45(7): 921-927. |
| [8] | Dong-hong HAN, Yan-ru KONG, Yi-meng ZHAN, Yuan LIU. Research on Emotion Recognition Method of Music Multimodal Data [J]. Journal of Northeastern University(Natural Science), 2024, 45(6): 776-785. |
| [9] | Wei-qi ZHANG, Hui-ming WANG. Interpretable Deep Learning Prediction Model for Compressive Strength of Concrete [J]. Journal of Northeastern University(Natural Science), 2024, 45(5): 738-744. |
| [10] | Ai-ping YANG, Si-jie FANG, Ming-fu SHAO, Teng-fei ZHANG. Transformer-based Multi-scale Underwater Image Enhancement Network [J]. Journal of Northeastern University(Natural Science), 2024, 45(12): 1696-1705. |
| [11] | Peng SHAN, Lin ZHANG, Hong-ming XIAO, Yu-liang ZHAO. CT Diagnosis Method for Coronavirus Pneumonia with Integrated Multi-scale Attention Mechanism [J]. Journal of Northeastern University(Natural Science), 2024, 45(12): 1673-1679. |
| [12] | WEI Jian-yi, WU Jing-jing. Resource Allocation Algorithm in Industrial Internet of Things Based on Edge Computing [J]. Journal of Northeastern University(Natural Science), 2023, 44(8): 1072-1078. |
| [13] | LI Hai-yan, XIONG Li-chang, GUO Lei, LI Hai-jiang. Two-Stage Inpainting Algorithm Based on U-net Edge Generation and Hypergraphs Convolution [J]. Journal of Northeastern University(Natural Science), 2023, 44(3): 331-339. |
| [14] | YANG Xuan, HE Zhan-qi. Improved Two-layer BiLSTM Electrocardiosignal Segmentation Method [J]. Journal of Northeastern University(Natural Science), 2023, 44(12): 1705-1711. |
| [15] | CHEN Cheng, SHI Pei-xin, WANG Zhan-sheng, JIA Peng-jiao. Shield Load Prediction Method Based on Deep Learning with Multiattention Mechanism [J]. Journal of Northeastern University(Natural Science), 2023, 44(11): 1631-1638. |
| Viewed | ||||||
|
Full text |
|
|||||
|
Abstract |
|
|||||