Improved Binocular Stereo Matching Algorithm Based on AD-Census

doi:10.12068/j.issn.1005-3026.2024.11.013

Abstract

Abstract:

A binocular stereo matching algorithm that integrates large?scale window information and Manhattan distance is proposed to address the low matching accuracy of traditional methods， such as AD-Census， in areas with low or repeated textures. The algorithm first uses an improved SAD cost and multi?gray threshold Census cost to calculate the fusion cost， and assigns weights based on the Manhattan distance between neighboring pixels and the center point to reduce the influence of edge pixels on the cost. The algorithm also screens and filters the difference information extracted from large scale windows to improve the accuracy in areas with repeated textures and low gray similarities. Compared to traditional AD-Census algorithm， the proposed algorithm reduces the false matching rate by approximately 18%. Furthermore， the algorithm has been transplanted to the GPU， allowing it to run 1~2 orders of magnitude faster on images with different scale resolutions， thus meeting the demands of quick and accurate binocular stereo matching.

Key words: binocular stereo vision, stereo matching, AD-Census, SAD （sum of absolute differences）

CLC Number:

P 23

De-fu CHE, Xiang-xiang SHANG, Duo WANG, Yan-en SUN. Improved Binocular Stereo Matching Algorithm Based on AD-Census[J]. Journal of Northeastern University(Natural Science), 2024, 45(11): 1621-1628.

Figures/Tables 18

Fig.1 Flow chat of algorithm

Fig.2 Incorrect matching of the same name points in the image

Fig.3 Comparison of real parallax and AD-Census calculated parallax

Fig.4 Position of matching points in the left and right views

Fig.5 Comparison of matching errors between points with the same name and mismatched points

Fig.6 Selected point distribution

Fig.7 Comparison of matching errors between points with different names and mismatched points

Fig.8 Large?scale window image matching effect

Fig.9 Quality comparison of AD cost and improved SAD cost

Table 1 Comparison of AD cost and improved SAD cost error

代价	误差
代价	Adi	Art	Mot
改进SAD代价	41.1	67.1	31.0
AD代价	71.3	70.5	48.2

Fig.10 Census code map

Fig.11 Improved multi-gray threshold Census coding map

Fig.12 Quality comparison of census cost and improved Census cost

Table 2 Comparison of Census cost and improved Census cost error

代价	误差
代价	Adi	Art	Mot
改进Census代价	26.6	24.4	16.1
Census代价	32.3	32.6	26.3

Table 3 Experimental parameters setting

$λ e$	$λ c$	$λ s$	$α$	$β$	$γ$	$λ 1$	$σ 1$
10	30	10	3	11	52	1.0	11

Table 3 Experimental parameters setting

$λ e$	$λ c$	$λ s$	$α$	$β$	$γ$	$λ 1$	$σ 1$
10	30	10	3	11	52	1.0	11

Fig.13 Parallax comparison diagram

Table 4 Mismatching rates of different algorithms in the unshaded region of Middlebury data set

算法	平均误差	误差
算法	平均误差	Adi	Art	Jad	Mot	MoE	Pia	Pil	Pip	Pla	Plt	PlP	Rec	She	Ted	Vin
本文改进算法	7.51	1.98	4.76	10.6	2.49	3.12	5.03	14.4	8.30	5.31	12.5	5.01	3.12	20.8	1.86	13.5
AD-Census^[2]	9.21	8.81	5.27	21.3	4.10	5.58	6.57	31.4	8.50	7.44	24.8	5.48	3.32	24.8	3.11	14.0
MCSC^[13]	7.71	2.54	5.73	9.73	3.12	2.94	12.2	19.0	5.03	8.60	7.74	6.22	4.97	26.4	2.01	22.2
SRM^[14]	7.75	1.89	5.94	11.5	2.58	2.71	10.6	17.5	4.03	10.5	6.30	5.77	5.89	30.5	2.99	21.2
INTS^[15]	9.31	4.31	4.88	12.3	3.47	3.20	11.2	20.2	4.92	10.9	15.5	9.25	5.58	34.1	4.14	25.7
TMAP^[16]	10.7	4.62	7.24	15.8	3.93	3.73	11.6	21.0	6.04	12.9	26.0	9.18	6.44	35.7	4.22	26.2
LPSM^[17]	11.1	4.09	10.1	17.7	6.89	6.14	11.1	24.8	9.88	13.2	20.8	9.83	6.78	22.7	4.56	22.4
SGM-RVC^[18]	12.2	6.13	6.13	15.3	5.13	4.30	13.1	25.4	8.22	14.1	39.8	10.1	6.64	40.5	2.67	28.6
ADSG^[19]	13.9	7.13	13.2	16.1	6.83	5.91	17.9	26.7	10.1	17.6	27.3	12.5	9.12	38.6	6.22	27.5

Table 5 Comparison of running speed under

分辨率/像素	t_GPU/ms	t_CPU/ms	加速倍率
450×375	30	4 012	133
718×496	89	5 660	71
800×552	124	7 270	59
900×621	175	9 710	55

References 19

1	Binh P， Nguyen Q.A review of stereo‑photogrammetry method for 3-D reconstruction in computer vision［C］//Proceedings of the 19th International Symposium on Communications and Information Technologies.Piscataway：IEEE，2019：138-143.
2	Mei X， Sun X， Zhou M C，et al.On building an accurate stereo matching system on graphics hardware［C］//2011 IEEE International Conference on Computer Vision Workshops.Barcelona，2011：467-474.
3	Huang C H， Yang J F.Improved quadruple sparse census transform and adaptive multi‑shape aggregation algorithms for precise stereo matching［J］.IET Computer Vision，2021，2（16）：159-179.
4	Zhang L， Zhang K， Chang T S，et al.Real‑time high‑definition stereo matching on FPGA［C］//ACM/SIGDA International Symposium on Field Programmable Gate Arrays.Monterey，2011：55-64.
5	Cheng X J， Zhao Y， Yang W B，et al.LESC：superpixel cut‑based local expansion for accurate stereo matching［J］.IET Image Processing，2022，16（2）：470-484.
6	Haq Q M U， Lin C H， Ruan S J，et al.An edge‑aware based adaptive multi‑feature set extraction for stereo matching of binocular images［J］.Journal of Ambient Intelligence and Humanized Computing，2021，13（4）：1-15.
7	张浩东，宋嘉菲，张广慧.边缘引导特征融合和代价聚合的立体匹配算法［J］.计算机工程与应用，2022，58（21）：182-188.
	Zhang Hao‑dong， Song Jia‑fei， Zhang Guang‑hui.Stereo matching network based on edge‑guided feature fusion and cost aggregation［J］.Computer Engineering and Applications，2022，58（21）：182-188.
8	Scharstein D， Szeliski R.A taxonomy and evaluation of dense two‑frame stereo correspondence algorithms［J］.International Journal of Computer Vision，2002，47（1）：7-42.
9	Scharstein D， Szeliski R.High‑accuracy stereo depth maps using structured light［C］//2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Madison，2003：195-202.
10	Scharstein D， Pal C.Learning conditional random fields for stereo［C］//2007 IEEE Conference on Computer Vision and Pattern Recognition.Minneapolis，2007：1-8.
11	Hirschmuller H， Scharstein D.Evaluation of cost functions for stereo matching［C］//2007 IEEE Conference on Computer Vision and Pattern Recognition.Minneapolis，2007：1-8.
12	Scharstein D， Hirschmüller H， Kitajima Y，et al.High‑resolution stereo datasets with subpixel‑accurate ground truth［C］//German Conference on Pattern Recognition.Cham：Springer International Publishing，2014：31-42.
13	Yang M L， Lyu X B.Learning both matching cost and smoothness constraint for stereo matching［J］.Neurocomputing，2018，314：234-241.
14	Okae J， Du J， Hu Y M.Robust statistical approach to stereo disparity maps denoising and refinement［J］.Control Theory and Technology，2020，18（4）：348-361.
15	Huang X， Zhang Y J， Yue Z X.Image‑guided non‑local dense matching with three‑steps optimization［J］.ISPRS Annals of Photogrammetry，Remote Sensing and Spatial Information Sciences，2016，3（3）：67-74.
16	Psota E T， Kowalczuk J， Mittek M，et al.MAP disparity estimation using hidden Markov trees［C］//2015 IEEE International Conference on Computer Vision（ICCV）.Santiago，2015：2219-2227.
17	Xu C L， Wu C D， Qu D K，et al.Accurate and efficient stereo matching by log‑angle and pyramid‑tree［J］.IEEE Transactions on Circuits and Systems for Video Technology，2020，31（10）：4007-4019.
18	Hirschmuller H.Stereo processing by semiglobal matching and mutual information［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，2007，30（2）：328-341.
19	Liu H， Zhang H L， Nie X X，et al.Stereo matching algorithm based on two‑phase adaptive optimization of AD-census and gradient fusion［C］//2021 IEEE International Conference on Real-Time Computing and Robotics（RCAR）.Xining，2021：726-731.

[1]	PANG Yan-wei， SU Chang， LONG Tao. Adaptive Multi-scale Cost Volume Construction and Aggregation for Stereo Matching [J]. Journal of Northeastern University(Natural Science), 2023, 44(4): 457-468.
[2]	ZHANG Zhi-min， QIAO Jian-zhong， LIN Shu-kuan， WANG Pin-he. A View Reconstruction Method Based on Deep Network [J]. Journal of Northeastern University Natural Science, 2020, 41(8): 1065-1069.
[3]	LI Jing-jiao， MA Li， WANG Ai-xia， MA Shuai. Stereo Matching Algorithm Based on Improved Patchmatch and Slice Sampling Particle Belief Propagation [J]. Journal of Northeastern University Natural Science, 2016, 37(5): 609-613.