Three-Dimensional Reconstruction Method of Monocular Video Image Sequences

doi:10.12068/j.issn.1005-3026.2024.12.002

Abstract

Abstract:

For 3D reconstruction in uncalibrated monocular scene， due to the existence of motion blur and small baseline image， there are still problems such as blurred image， low feature matching， and reconstruction accuracy. A 3D reconstruction method of monocular vision system is proposed. Firstly， the fuzzy image in the original data is detected and removed by wavelet transform algorithm according to the edge information of the image， and the wide baseline image is screened by geometric robustness criterion to obtain clear images for 3D reconstruction. Then， an algorithm for eliminating mismatched feature points based on region division is proposed to eliminate repeated matching and mismatched feature points. Finally， three different regional growth methods are proposed to obtain more feature points for three‑dimensional point calculation. The results show that the proposed method can effectively remove mismatched feature points， improve the accuracy of feature point matching， obtain more point clouds， and improve the integrity and accuracy of the reconstructed model.

Key words: 3D reconstruction, key frame screening, feature extraction, sparse and dense, regional growth

CLC Number:

TP 391.4

Xiao-peng SHA, Jia-qi CAO, Wen-jing LI, Ye QIN. Three-Dimensional Reconstruction Method of Monocular Video Image Sequences[J]. Journal of Northeastern University(Natural Science), 2024, 45(12): 1680-1687.

Figures/Tables 14

References 14

1	Barrile V， Bernardo E， Bilotta G.An experimental HBIM processing：innovative tool for 3D model reconstruction of morpho‑typological phases for the cultural heritage［J］.Remote Sensing，2022，14：1288.
2	胡正乙，谭庆昌，孙秋成.基于RGB-D的室内场景实时三维重建算法［J］.东北大学学报（自然科学版），2017，38（12）：1764-1768.
	Hu Zheng‑yi， Tan Qing‑chang， Sun Qiu‑cheng.RGB-D based indoor scene real‑time 3D reconstruction algorithm［J］.Journal of Northeastern University （Natural Science），2017，38（12）：1764-1768.
3	Tian F G， Gao Y B， Fang Z J，et al.3D reconstruction with auto‑selected keyframes based on depth completion correction and pose fusion［J］.Visual Communication Image Represent，2021，79：103199.
4	Zhang Y Z， Zhang J Y， Liu R X，et al.Key frame extraction based on quaternion Fourier transform with multiple features fusion［J］.Expert Systems with Applications，2023，216：11119467.
5	何鑫睿，李秀梅，孙军梅，等.基于改进Pix2Vox的单图像三维重建网络［J］.计算机辅助设计与图形学学报，2022，34（3）：364-372.
	He Xin‑rui， Li Xiu‑mei， Sun Jun‑mei，et al.Improved Pix2Vox based 3D reconstruction network from single image［J］.Journal of Computer-Aided Design & Computer Graphics，2022，34（3）：364-372.
6	Won J， Park J W， Song M H，et al.Robust vision‑based displacement measurement and acceleration estimation using PANSAC and Kalman filter［J］.Earthquake Engineering and Engineering Vibration，2023，22（2）：347-358.
7	Nielsen M S， Nikolov I， Kruse E K，et al.Quantifying the influence of surface texture and shape on structure from motion 3D reconstruction［J］.Sensors，2023，23（1）：178-200.
8	Ito K， Ito T， Aoki T.PM-MVS：PatchMatch multi‑view stereo［J］.Machine Vision and Applications，2023，34：32-48.
9	Li Z X， Zuo W M， Wang Z Q，et al.Robust 3D reconstruction from uncalibrated small motion clips［J］.The Visual Computer，2022，38（5）：1589-1605.
10	Tong H H， Li M J， Zhang H J，et al.Blur detection for digital images using wavelet transform［C］//IEEE International Conference on Multimedia & Expo.Taipei，2004：17-20.
11	Torr P H S.Geometric motion segmentation and model selection［J］.Philosophical Transactions of the Royal Society of London.Series A：Mathematical，Physical and Engineering Sciences，1998，356：1321-1340.
12	Olsson C， Enqvist O.Stable structure from motion for unordered image collections［C］// Proceedings of the Scandinavian Conference on Image Analysis.Berlin：Springer，2011：524-535.
13	Enqvist O， Kahl F， Olsson C.Non-sequential structure from motion［C］//2011 IEEE International Conference on Computer Vision Workshops.Barcelona，2011：264-271.
14	Furukawa Y， Ponce J.Accurate，dense，and robust multiview stereopsis［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，2010，32（8）：1362-1376.

筛选方法	总图像	模糊图像	清晰图像	非关键帧	关键帧
Harr+对应约束	278	66	212	74	138
Harr+SSIM	278	66	212	53	159
本文方法	278	66	212	37	175

筛选方法	总图像	模糊图像	清晰图像	非关键帧	关键帧
Harr+对应约束	278	66	212	74	138
Harr+SSIM	278	66	212	53	159
本文方法	278	66	212	37	175

图像数据	方法	初始匹配/个	去重后/个	正确匹配点/个	正确率/%	时间/s
LUsphinx^[12]	RANSAC	1 311	1 311	602	46	7
LUsphinx^[12]	本文方法	1 311	856	832	63	4
Golden_statue^[13]	RANSAC	5 386	5 386	2 950	55	23
Golden_statue^[13]	本文方法	5 386	5 184	4 281	79	15
本文图像数据	RANSAC	2 133	2 133	405	19	10
本文图像数据	本文方法	2 133	1 800	1 057	50	6

图像数据	方法	初始匹配/个	去重后/个	正确匹配点/个	正确率/%	时间/s
LUsphinx^[12]	RANSAC	1 311	1 311	602	46	7
LUsphinx^[12]	本文方法	1 311	856	832	63	4
Golden_statue^[13]	RANSAC	5 386	5 386	2 950	55	23
Golden_statue^[13]	本文方法	5 386	5 184	4 281	79	15
本文图像数据	RANSAC	2 133	2 133	405	19	10
本文图像数据	本文方法	2 133	1 800	1 057	50	6

方法	LUsphinx稀疏点云	Golden_statue稀疏点云
文献[12-13]	32 668	39 950
SFM方法	67 488	15 347
本文方法（RG4）	114 254	27 712
本文方法（RG8）	113 381	27 483
本文方法（RG16）	137 854	32 886