基于AD-Census的双目立体匹配改进算法

doi:10.12068/j.issn.1005-3026.2024.11.013

摘要/Abstract

摘要：

针对绝对值之差（absolute difference，AD）-Census等传统双目立体匹配算法在低纹理区域、重复纹理区域匹配精度低的问题，提出一种融合大尺度窗口信息与曼哈顿距离的双目立体匹配算法.使用改进的绝对误差和（sum of absolute differences，SAD）代价与多灰度阈值Census代价计算得到融合代价，根据邻域像素与中心点的曼哈顿距离赋予权重，减少边缘像素对代价的影响.通过大尺度的窗口提取到的差异信息对融合代价进行筛选过滤，改善了算法在重复纹理区域、灰度相似区域精度较低的问题.与传统的AD-Census算法相比，该算法误匹配率减少约18%，对算法进行图形处理器（graphic processing unit，GPU）移植，使得算法在不同尺度分辨率的图像上运行速度提升1~2个数量级，满足双目立体匹配算法快速准确的需求.

关键词: 双目立体视觉, 立体匹配, AD-Census, 绝对误差和

Abstract:

A binocular stereo matching algorithm that integrates large?scale window information and Manhattan distance is proposed to address the low matching accuracy of traditional methods， such as AD-Census， in areas with low or repeated textures. The algorithm first uses an improved SAD cost and multi?gray threshold Census cost to calculate the fusion cost， and assigns weights based on the Manhattan distance between neighboring pixels and the center point to reduce the influence of edge pixels on the cost. The algorithm also screens and filters the difference information extracted from large scale windows to improve the accuracy in areas with repeated textures and low gray similarities. Compared to traditional AD-Census algorithm， the proposed algorithm reduces the false matching rate by approximately 18%. Furthermore， the algorithm has been transplanted to the GPU， allowing it to run 1~2 orders of magnitude faster on images with different scale resolutions， thus meeting the demands of quick and accurate binocular stereo matching.

Key words: binocular stereo vision, stereo matching, AD-Census, SAD （sum of absolute differences）

中图分类号:

P 23

车德福, 尚祥祥, 王夺, 孙彦恩. 基于AD-Census的双目立体匹配改进算法[J]. 东北大学学报（自然科学版）, 2024, 45(11): 1621-1628.

De-fu CHE, Xiang-xiang SHANG, Duo WANG, Yan-en SUN. Improved Binocular Stereo Matching Algorithm Based on AD-Census[J]. Journal of Northeastern University(Natural Science), 2024, 45(11): 1621-1628.

图/表 18

图1 算法流程

Fig.1 Flow chat of algorithm

图2 图像同名点错误匹配

Fig.2 Incorrect matching of the same name points in the image

图3 真实视差与AD-Census计算视差对比（a）—真实视差；（b）—AD-Census计算视差.

Fig.3 Comparison of real parallax and AD-Census calculated parallax

图4 左右视图匹配点位置情况

Fig.4 Position of matching points in the left and right views

图5 同名点与误匹配点匹配误差对比

Fig.5 Comparison of matching errors between points with the same name and mismatched points

图6 选取点位分布

Fig.6 Selected point distribution

图7 不同同名点与误匹配点匹配误差对比（a）—点A；（b）—点C；（d）—点E；（d）—点B；（e）—点D；（f）—点F.

Fig.7 Comparison of matching errors between points with different names and mismatched points

图8 大尺度窗口图像匹配效果

Fig.8 Large?scale window image matching effect

图9 AD代价与改进SAD代价质量对比（a）—AD代价；（b）—改进SAD代价.

Fig.9 Quality comparison of AD cost and improved SAD cost

表1 AD代价与改进SAD代价误差对比 (%)

Table 1 Comparison of AD cost and improved SAD cost error

代价	误差
代价	Adi	Art	Mot
改进SAD代价	41.1	67.1	31.0
AD代价	71.3	70.5	48.2

图10 Census编码图

Fig.10 Census code map

图11 改进的多灰度阈值Census编码图

Fig.11 Improved multi-gray threshold Census coding map

图12 Census代价与改进Census代价质量对比（a）—Census代价；（b）—改进Census代价.

Fig.12 Quality comparison of census cost and improved Census cost

表2 Census代价与改进Census代价误差对比 (%)

Table 2 Comparison of Census cost and improved Census cost error

代价	误差
代价	Adi	Art	Mot
改进Census代价	26.6	24.4	16.1
Census代价	32.3	32.6	26.3

表3 实验参数设置

Table 3 Experimental parameters setting

$λ e$	$λ c$	$λ s$	$α$	$β$	$γ$	$λ 1$	$σ 1$
10	30	10	3	11	52	1.0	11

表3 实验参数设置

Table 3 Experimental parameters setting

$λ e$	$λ c$	$λ s$	$α$	$β$	$γ$	$λ 1$	$σ 1$
10	30	10	3	11	52	1.0	11

图13 视差对比图（a）—真实视差；（b）—AD-Census；（c）—改进算法.

Fig.13 Parallax comparison diagram

表4 不同算法在Middlebury数据集中非遮挡区域误匹配率 (%)

Table 4 Mismatching rates of different algorithms in the unshaded region of Middlebury data set

算法	平均误差	误差
算法	平均误差	Adi	Art	Jad	Mot	MoE	Pia	Pil	Pip	Pla	Plt	PlP	Rec	She	Ted	Vin
本文改进算法	7.51	1.98	4.76	10.6	2.49	3.12	5.03	14.4	8.30	5.31	12.5	5.01	3.12	20.8	1.86	13.5
AD-Census^[2]	9.21	8.81	5.27	21.3	4.10	5.58	6.57	31.4	8.50	7.44	24.8	5.48	3.32	24.8	3.11	14.0
MCSC^[13]	7.71	2.54	5.73	9.73	3.12	2.94	12.2	19.0	5.03	8.60	7.74	6.22	4.97	26.4	2.01	22.2
SRM^[14]	7.75	1.89	5.94	11.5	2.58	2.71	10.6	17.5	4.03	10.5	6.30	5.77	5.89	30.5	2.99	21.2
INTS^[15]	9.31	4.31	4.88	12.3	3.47	3.20	11.2	20.2	4.92	10.9	15.5	9.25	5.58	34.1	4.14	25.7
TMAP^[16]	10.7	4.62	7.24	15.8	3.93	3.73	11.6	21.0	6.04	12.9	26.0	9.18	6.44	35.7	4.22	26.2
LPSM^[17]	11.1	4.09	10.1	17.7	6.89	6.14	11.1	24.8	9.88	13.2	20.8	9.83	6.78	22.7	4.56	22.4
SGM-RVC^[18]	12.2	6.13	6.13	15.3	5.13	4.30	13.1	25.4	8.22	14.1	39.8	10.1	6.64	40.5	2.67	28.6
ADSG^[19]	13.9	7.13	13.2	16.1	6.83	5.91	17.9	26.7	10.1	17.6	27.3	12.5	9.12	38.6	6.22	27.5

表5 不同分辨率下运行速度对比 (different resolutions)

Table 5 Comparison of running speed under

分辨率/像素	t_GPU/ms	t_CPU/ms	加速倍率
450×375	30	4 012	133
718×496	89	5 660	71
800×552	124	7 270	59
900×621	175	9 710	55

参考文献 19

1	Binh P， Nguyen Q.A review of stereo‑photogrammetry method for 3-D reconstruction in computer vision［C］//Proceedings of the 19th International Symposium on Communications and Information Technologies.Piscataway：IEEE，2019：138-143.
2	Mei X， Sun X， Zhou M C，et al.On building an accurate stereo matching system on graphics hardware［C］//2011 IEEE International Conference on Computer Vision Workshops.Barcelona，2011：467-474.
3	Huang C H， Yang J F.Improved quadruple sparse census transform and adaptive multi‑shape aggregation algorithms for precise stereo matching［J］.IET Computer Vision，2021，2（16）：159-179.
4	Zhang L， Zhang K， Chang T S，et al.Real‑time high‑definition stereo matching on FPGA［C］//ACM/SIGDA International Symposium on Field Programmable Gate Arrays.Monterey，2011：55-64.
5	Cheng X J， Zhao Y， Yang W B，et al.LESC：superpixel cut‑based local expansion for accurate stereo matching［J］.IET Image Processing，2022，16（2）：470-484.
6	Haq Q M U， Lin C H， Ruan S J，et al.An edge‑aware based adaptive multi‑feature set extraction for stereo matching of binocular images［J］.Journal of Ambient Intelligence and Humanized Computing，2021，13（4）：1-15.
7	张浩东，宋嘉菲，张广慧.边缘引导特征融合和代价聚合的立体匹配算法［J］.计算机工程与应用，2022，58（21）：182-188.
	Zhang Hao‑dong， Song Jia‑fei， Zhang Guang‑hui.Stereo matching network based on edge‑guided feature fusion and cost aggregation［J］.Computer Engineering and Applications，2022，58（21）：182-188.
8	Scharstein D， Szeliski R.A taxonomy and evaluation of dense two‑frame stereo correspondence algorithms［J］.International Journal of Computer Vision，2002，47（1）：7-42.
9	Scharstein D， Szeliski R.High‑accuracy stereo depth maps using structured light［C］//2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Madison，2003：195-202.
10	Scharstein D， Pal C.Learning conditional random fields for stereo［C］//2007 IEEE Conference on Computer Vision and Pattern Recognition.Minneapolis，2007：1-8.
11	Hirschmuller H， Scharstein D.Evaluation of cost functions for stereo matching［C］//2007 IEEE Conference on Computer Vision and Pattern Recognition.Minneapolis，2007：1-8.
12	Scharstein D， Hirschmüller H， Kitajima Y，et al.High‑resolution stereo datasets with subpixel‑accurate ground truth［C］//German Conference on Pattern Recognition.Cham：Springer International Publishing，2014：31-42.
13	Yang M L， Lyu X B.Learning both matching cost and smoothness constraint for stereo matching［J］.Neurocomputing，2018，314：234-241.
14	Okae J， Du J， Hu Y M.Robust statistical approach to stereo disparity maps denoising and refinement［J］.Control Theory and Technology，2020，18（4）：348-361.
15	Huang X， Zhang Y J， Yue Z X.Image‑guided non‑local dense matching with three‑steps optimization［J］.ISPRS Annals of Photogrammetry，Remote Sensing and Spatial Information Sciences，2016，3（3）：67-74.
16	Psota E T， Kowalczuk J， Mittek M，et al.MAP disparity estimation using hidden Markov trees［C］//2015 IEEE International Conference on Computer Vision（ICCV）.Santiago，2015：2219-2227.
17	Xu C L， Wu C D， Qu D K，et al.Accurate and efficient stereo matching by log‑angle and pyramid‑tree［J］.IEEE Transactions on Circuits and Systems for Video Technology，2020，31（10）：4007-4019.
18	Hirschmuller H.Stereo processing by semiglobal matching and mutual information［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，2007，30（2）：328-341.
19	Liu H， Zhang H L， Nie X X，et al.Stereo matching algorithm based on two‑phase adaptive optimization of AD-census and gradient fusion［C］//2021 IEEE International Conference on Real-Time Computing and Robotics（RCAR）.Xining，2021：726-731.

[1]	庞彦伟，苏畅，龙涛. 自适应构造与聚合多尺度代价体的双目立体匹配[J]. 东北大学学报（自然科学版）, 2023, 44(4): 457-468.
[2]	原培新，蔡炟，曹文伟，陈超. 基于双目立体视觉的列车目标识别和测距技术[J]. 东北大学学报（自然科学版）, 2022, 43(3): 335-343.
[3]	张之敏，乔建忠，林树宽，王品贺. 一种基于深度网络的视图重建方法[J]. 东北大学学报:自然科学版, 2020, 41(8): 1065-1069.
[4]	李晶皎，马利，王爱侠，马帅. 基于改进Patchmatch及切片采样粒子置信度传播的立体匹配算法[J]. 东北大学学报:自然科学版, 2016, 37(5): 609-613.