一种基于深度网络的视图重建方法

doi:10.12068/j.issn.1005-3026.2020.08.001

东北大学学报:自然科学版 ›› 2020, Vol. 41 ›› Issue (8): 1065-1069.DOI: 10.12068/j.issn.1005-3026.2020.08.001

• 信息与控制 • 下一篇

一种基于深度网络的视图重建方法

张之敏，乔建忠，林树宽，王品贺

(东北大学计算机科学与工程学院，辽宁沈阳110169)

收稿日期:2020-01-18 修回日期:2020-01-18 出版日期:2020-08-15 发布日期:2020-08-28
通讯作者: 张之敏
作者简介:张之敏(1988-)，男，山东德州人，东北大学博士研究生; 乔建忠(1954-)，男，辽宁葫芦岛人，东北大学教授，博士生导师.
基金资助:
国家自然科学基金青年基金资助项目(61902261).

A View Reconstruction Method Based on Deep Network

ZHANG Zhi-min， QIAO Jian-zhong， LIN Shu-kuan， WANG Pin-he

School of Computer Science & Engineering， Northeastern University， Shenyang 110169， China.

Received:2020-01-18 Revised:2020-01-18 Online:2020-08-15 Published:2020-08-28
Contact: QIAO Jian-zhong
About author:-
Supported by:
-

摘要/Abstract

摘要： 为了解决在仅有单目视图的环境下实现立体匹配的问题，在现有视图重构网络模型Deep3D的基础上，提出了基于加权局部对比归一化约束的全卷积重构模型.该模型采用改进的全卷积神经网络架构作为模型的特征提取模块，以期减少训练参数，降低训练时间，增加模型的非线性.为了进一步提高重构精度，设计了新的基于加权局部对比归一化的约束条件，并采用结构相似性成本(SSIM)与L1成本相结合的损失优化函数对模型进行优化.在KITTI 2015数据集上展开实验，并与Deep3D模型及其后续的改进方法进行比较.实验结果表明，在只使用左视图作为训练数据的情况下，生成的右视图在SSIM和峰值信噪比两个指标上有很大提升，能够满足立体匹配方法中右视图的精度要求.

关键词: 视图重构, 卷积神经网路, 立体匹配, 全卷积网络, 加权局部对比归一化

Abstract: To deal with stereo matching in the environment of only a single view， a full convolution reconstruction model with weighted local contrast normalization constraint is proposed on the basis of the existing view reconstruction network model Deep3D. This model adopts the improved full convolutional neural network architecture as the feature extraction module of the model to reduce the training parameters and training time， and to increase the nonlinearity of the model. In order to further improve the accuracy of reconstruction， a new constraint condition based on weighted local comparison normalization is designed， and a loss optimization function combining structural similarity (SSIM) cost and L1 cost is used to optimize the model. Experiments were carried out on the KITTI 2015 dataset， and compared with the Deep3D model and subsequent improvements. The experimental results show that the generated right view has a great improvement in SSIM and peak signal to noise ratio when only the left view is used as the training data， which can meet the accuracy requirements of the right view in the stereo matching method.

Key words: view reconstruction, convolutional neural network, stereo matching, fully convolutional network, weighted local contrast normalization

中图分类号:

TP18

张之敏，乔建忠，林树宽，王品贺. 一种基于深度网络的视图重建方法[J]. 东北大学学报:自然科学版, 2020, 41(8): 1065-1069.

ZHANG Zhi-min， QIAO Jian-zhong， LIN Shu-kuan， WANG Pin-he. A View Reconstruction Method Based on Deep Network[J]. Journal of Northeastern University Natural Science, 2020, 41(8): 1065-1069.

参考文献

[1]Eigen D，Puhrsch C，Fergus R.Depth map prediction from a single image using a multi-scale deep network［C/OL］//Advances in Neural Information Processing Systems 27 (NIPS2014).Kuching，Malaysia，2014［2019-12-15］.https://www.researchgate.net/publication/262974294_Depth_Map_Prediction_from_a_Single_Image_using_a_Multi-Scale_Deep_Network.
[2]Saxena A，Sun M，Ng A Y.Learning 3-D scene structure from a single still image［C］//2007 IEEE 11th International Conference on Computer Vision.New York:IEEE，2007:1-8.
[3]Garg R，Kumar V，Carneiro G，et al.Unsupervised CNN for single view depth estimation:geometry to the rescue［C］//European Conference on Computer Vision.Amsterdam，2016:740-756.
[4]Godard C，Mac Aodha O，Brostow G J.Unsupervised monocular depth estimation with left-right consistency［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE，2017:270-279.
[5]Luo Y，Ren J，Lin M，et al.Single view stereo matching［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE，2018:155-163.
[6]Flynn J，Neulander I，Philbin J，et al.DeepStereo:learning to predict new views from the world’s imagery［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE，2016:5515-5524.
[7]Zhou T，Tulsiani S，Sun W，et al.View synthesis by appearance flow［C］//European Conference on Computer Vision.Amsterdam，2016:286-301.
[8]Xie J，Girshick R，Farhadi A.Deep3D:fully automatic 2D-to-3D video conversion with deep convolutional neural networks［C］//European Conference on Computer Vision.Amsterdam，2016:842-857.
[9]Long J，Shelhamer E，Darrell T.Fully convolutional networks for semantic segmentation［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE，2015:3431-3440.
[10]Springenberg J T，Dosovitskiy A，Brox T，et al.Striving for simplicity:the all convolutional net［C/OL］//International Conference on Learning Representations.San Diego，2015［2019-12-15］.http://arxiv.org/pdf/1412.6806.pdf.
[11]Zhang Y，Khamis S，Rhemann C，et al.Activestereonet:end-to-end self-supervised learning for active stereo systems［C］//Proceedings of the European Conference on Computer Vision (ECCV).Munich，2018:784-801.
[12]Zhao H，Gallo O，Frosio I，et al.Loss functions for image restoration with neural networks［J］.IEEE Transactions on Computational Imaging，2016，3(1):47-57.
[15]关守平，房少纯.一种新型的区间-粒子群优化算法［J］.东北大学学报(自然科学版)，2012，33(10):1381-1384.(Guan Shou-ping，Fang Shao-chun.A new interval particle swarm optimization algorithm［J］.Journal of Northeastern University(Natural Science)，2012，33(10):1381-1384.)

一种基于深度网络的视图重建方法

A View Reconstruction Method Based on Deep Network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

编辑推荐

Metrics

本文评价

[1]	庞彦伟，苏畅，龙涛. 自适应构造与聚合多尺度代价体的双目立体匹配[J]. 东北大学学报（自然科学版）, 2023, 44(4): 457-468.
[2]	魏颖，林子涵，齐林，李伯群. 基于空间自注意力机制和深度特征重建的脑MR图像分割方法[J]. 东北大学学报（自然科学版）, 2023, 44(2): 177-185.
[3]	原培新，蔡炟，曹文伟，陈超. 基于双目立体视觉的列车目标识别和测距技术[J]. 东北大学学报（自然科学版）, 2022, 43(3): 335-343.
[4]	徐礼胜，张书琪，牛潇，徐阳. 基于全卷积网络的左心室射血分数自动检测[J]. 东北大学学报:自然科学版, 2018, 39(11): 1572-1576.
[5]	齐林，吕旭阳，杨本强，徐礼胜. 基于全卷积网络迁移学习的左心室内膜分割[J]. 东北大学学报:自然科学版, 2018, 39(11): 1577-1582.
[6]	李晶皎，马利，王爱侠，马帅. 基于改进Patchmatch及切片采样粒子置信度传播的立体匹配算法[J]. 东北大学学报:自然科学版, 2016, 37(5): 609-613.