Risk-Oriented Crowd Navigation Strategy Based on Deep Reinforcement Learning

doi:10.12068/j.issn.1005-3026.2025.12.20240094

Abstract

Abstract:

To improve robot freezing and suboptimal performance of traditional navigation methods in the presence of dynamic obstacles， a navigation method based on deep reinforcement learning was proposed. The core of this method lies in its risk perception module and path selection module. The risk perception module calculated the collision probability between the robot and nearby dynamic obstacles in real time， allowing the robot to prioritize avoiding more hazardous obstacles. Concurrently， the path selection module evaluated the “passing ability” of the robot in surrounding areas in real time， guiding the robot to choose safer paths. In comparison experiments with a deep reinforcement learning method that lacks these modules， the proposed method achieved the highest navigation success rate in all simulation test environments， with an improvement rate of up to 11%.

Key words: deep reinforcement learning, crowd navigation, dynamic obstacle avoidance, robot freezing, risk perception, path selection

CLC Number:

TP 181

Yang JIANG, Tian-xiang ZHAO, Ruo-huai SUN, Lei WANG. Risk-Oriented Crowd Navigation Strategy Based on Deep Reinforcement Learning[J]. Journal of Northeastern University(Natural Science), 2025, 46(12): 1-8.

Figures/Tables 16

Fig.1 System structure diagram

Fig.2 TD3 network architecture

Fig.3 Time-based collision probability

Fig.4 Distance-based collision probability

Fig.5 Intersection over union

Fig.6 Algorithm training environment

Fig.7 Obstacle-free training process

Fig.8 Training process with static obstacles

Fig.9 Training process with dynamic obstacles

Fig.10 Testing environment with dynamic obstacles

Fig.11 Average time to arrival comparison in low-speed environments

Fig.12 Collision rate comparison in low-speed environments

Fig.13 Obstacle avoidance success rate comparison in low-speed environments

Fig.14 Average time to arrival comparison in high-speed environments

Fig.15 Average collision rate comparison in high-speed environments

Fig.16 Obstacle avoidance success rate comparison in high-speed environments

References 15

[1]	谢洪乐，陈卫东，范亚娴，等. 月面特征稀疏环境下的视觉惯性SLAM方法［J］. 航空学报， 2021， 42（1）： 524169.
	Xie Hong-le， Chen Wei-dong， Fan Ya-xian， et al. Visual-inertial SLAM in featureless environments on lunar surface［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（1）： 524169.
[2]	Xing Z W， Zhu X R， Dong D C. DE-SLAM： SLAM for highly dynamic environment［J］. Journal of Field Robotics， 2022， 39（5）： 528-542.
[3]	Zhou Z Q， Zhu P M， Zeng Z W， et al. Robot navigation in a crowd by integrating deep reinforcement learning and online planning［J］. Applied Intelligence， 2022， 52（13）： 15600-15616.
[4]	Everett M， Chen Y F， How J P. Motion planning among dynamic， decision-making agents with deep reinforcement learning［C］//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）. Madrid， 2019： 3052-3059.
[5]	Tai L， Paolo G， Liu M. Virtual-to-real deep reinforcement learning： continuous control of mobile robots for mapless navigation［C］//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）. Vancouver， 2017： 31-36.
[6]	孙喜亮，关宏灿，苏艳军，等. 面向高精度城市测绘的激光紧耦合SLAM 方法［J］. 测绘学报， 2021， 50（11）：1585-1593.
	Sun Xi-liang， Guan Hong-can， Su Yan-jun， et al. A tightly coupled SLAM method for precise urban mapping［J］. Acta Geodaetica et Cartographica Sinica， 2021， 50（11）：1585-1593.
[7]	Tian Y， Chen H W， Wang F S， et al. Overview of SLAM algorithms for mobile robots［J］. Computer Science， 2021， 48（9）：223-234.
[8]	张宇桐，宋建梅，丁艳，等. 基于鱼眼相机和RGBD相机的异构协同SLAM［J］. 航空学报， 2023， 44（10）： 327621.
	Zhang Yu-tong， Song Jian-mei， Ding Yan， et al. Heterogeneous collaborative SLAM based on fisheye and RGBD cameras［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（10）： 327621.
[9]	陶交，范馨月，周非. 点线特征融合的双目视觉SLAM算法［J］. 小型微型计算机系统， 2022， 43（6）： 1191-1196.
	Tao Jiao， Fan Xin-yue， Zhou Fei. Point-line feature fusion in stereo visual SLAM algorithm［J］. Journal of Chinese Computer Systems， 2022， 43（6）： 1191-1196.
[10]	Chen W Q， Wang Y， Chen H Y， et al. EIL-SLAM： depth-enhanced edge-based infrared-LiDAR SLAM［J］. Journal of Field Robotics， 2022， 39（2）： 117-130.
[11]	蒋林，刘林锐，周安娜，等. 基于运动预测的改进ORB-SLAM算法［J］. 浙江大学学报（工学版）， 2023， 57（1）： 170-177.
	Jiang Lin， Liu Lin-rui， Zhou An-na， et al. Improved ORB-SLAM algorithm based on motion prediction［J］. Journal of Zhejiang University （Engineering Science）， 2023， 57（1）： 170-177.
[12]	李荣华，祁宇峰，谢辉，等. 面向未知环境的紧耦合激光SLAM方法［J］. 红外与激光工程， 2023， 52（9）： 135-144.
	Li Rong-hua， Qi Yu-feng， Xie Hui， et al. Tightly coupled LiDAR SLAM method for unknown environment［J］. Infrared and Laser Engineering， 2023， 52（9）： 135-144.
[13]	Li J Q， Pei L， Zou D P， et al. Attention-SLAM： a visual monocular SLAM learning from human gaze［J］. IEEE Sensors Journal， 2021， 21（5）： 6408-6420.
[14]	沈晔湖，陈嘉皓，李星，等. 基于几何-语义联合约束的动态环境视觉SLAM算法［J］. 数据采集与处理，2022， 37（3）： 597-608.
	Shen Ye-hu， Chen Jia-hao， Li Xing， et al. Dynamic visual SLAM based on unified geometric-semantic constraints［J］. Journal of Data Acquisition and Processing， 2022， 37（3）： 597-608.
[15]	刘少哲，刘作军，胡超芳，等. 点线特征自适应融合室内SLAM算法［J］. 小型微型计算机系统， 2023， 44（5）： 1015-1022.
	Liu Shao-zhe， Liu Zuo-jun， Hu Chao-fang， et al. Point-line feature adaptive fusion indoor SLAM algorithm［J］. Journal of Chinese Computer Systems， 2023， 44（5）： 1015-1022.