Pedestrian Trajectory Prediction Algorithm Based on Graph Convolution and Convolution

doi:10.12068/j.issn.1005-3026.2024.11.002

Abstract

Abstract:

Significant progress has been made in pedestrian trajectory prediction， but most of the existing methods are constrained by the limited on?board computing resources. How to achieve efficient pedestrian trajectory prediction in autonomous vehicles is still insufficient. To solve this problem， a lightweight pedestrian trajectory prediction algorithm is proposed， which uses convolutional neural network （CNN） and graph convolutional neural network （GCN） to process and integrate multimodal information. Firstly， a multi?scale feature processing module is designed based on CNN. Multiple convolution modules are used to capture the features of pedestrian tracks and scene information at different time and spatial scales. Then， a feature integration module is constructed based on GCN， which is used to efficiently integrate the spatial?temporal relationship between trajectory and scene features and obtain multiple prediction representations. Finally， multiple prediction representations are integrated to obtain pedestrian trajectory prediction results. Experiments on PIE and JAAD datasets show that the proposed method achieves competitive and optimal prediction performance with the least network parameters， respectively， which verifies the effectiveness of the proposed method. Compared with the previous lightest method， the parameters are optimized by 73%.

Key words: autonomous driving, pedestrian trajectory prediction, graph convolution, multi?scale features, lightweight model

CLC Number:

TP 391

Ang FENG, Jun GONG, Nian WANG, Jing-long WANG. Pedestrian Trajectory Prediction Algorithm Based on Graph Convolution and Convolution[J]. Journal of Northeastern University(Natural Science), 2024, 45(11): 1529-1536.

Figures/Tables 13

Fig.1 Overall architecture of the algorithm

Fig.2 Multi?scale trajectory feature extraction module

Fig.3 Multi?scale scene feature extraction module

Fig.4 Multi?scale feature integration module

Fig.5 Visualization of multi?scale feature integration

Fig.6 Forecast node relationship graph

Table 1 Comparison with other methods on PIE and JAAD

算法	参数×10^-6	PIE			JAAD
算法	参数×10^-6	MSE	$C M S E$	$C F M S E$	MSE	$C M S E$	$C F M S E$
DTP-MOF	11.3	665	566	2 373	1 158	1 014	4 143
$P I E f u l l$	3.07	551	512	2 232	—	—	—
STED	13.94	461	415	1 871	1 044	960	4 031
MTN	0.134	444	414	1 627	1 005	951	4 011
SGnet	7.622	442	413	1 761	1 049	996	4 076
本文	0.036	458	425	1 805	994	943	3 821

Table 1 Comparison with other methods on PIE and JAAD

算法	参数×10^-6	PIE			JAAD
算法	参数×10^-6	MSE	$C M S E$	$C F M S E$	MSE	$C M S E$	$C F M S E$
DTP-MOF	11.3	665	566	2 373	1 158	1 014	4 143
$P I E f u l l$	3.07	551	512	2 232	—	—	—
STED	13.94	461	415	1 871	1 044	960	4 031
MTN	0.134	444	414	1 627	1 005	951	4 011
SGnet	7.622	442	413	1 761	1 049	996	4 076
本文	0.036	458	425	1 805	994	943	3 821

Fig.7 Visualization of successful pedestrian trajectory prediction

Fig.8 Comparison of inference time in PIE

Fig.9 Comparison of different time steps in PIE

Fig.10 Comparison of different time steps in JAAD

Table 2 Comparison of ablation experiment results

轨迹特征	场景特征	MSE	$C M S E$	$C F M S E$
$F t 1$ ， $F t 1$ ， $F t 1$	$? 1$ ， $? 2$ ， $? 3$	793	751	2 872
$F t 2$ ， $F t 2$ ， $F t 2$	$? 1$ ， $? 2$ ， $? 3$	509	476	1 964
$F t 3$ ， $F t 3$ ， $F t 3$	$? 1$ ， $? 2$ ， $? 3$	499	467	1 944
$F t 1$ ， $F t 2$ ， $F t 3$	$? 1$ ， $? 2$ ， $? 3$	520	486	2 006
$F t 1$ ， $F t 2$ ， $F t 3$	$? 1$ ， $? 2$ ， $? 3$	509	475	1 997
$F t 1$ ， $F t 2$ ， $F t 3$	$? 1$ ， $? 2$ ， $? 3$	579	546	2 151
$F t 1$ ， $F t 2$ ， $F t 3$	$? 1$ ， $? 2$ ， $? 3$	458	425	1 805

Table 2 Comparison of ablation experiment results

轨迹特征	场景特征	MSE	$C M S E$	$C F M S E$
$F t 1$ ， $F t 1$ ， $F t 1$	$? 1$ ， $? 2$ ， $? 3$	793	751	2 872
$F t 2$ ， $F t 2$ ， $F t 2$	$? 1$ ， $? 2$ ， $? 3$	509	476	1 964
$F t 3$ ， $F t 3$ ， $F t 3$	$? 1$ ， $? 2$ ， $? 3$	499	467	1 944
$F t 1$ ， $F t 2$ ， $F t 3$	$? 1$ ， $? 2$ ， $? 3$	520	486	2 006
$F t 1$ ， $F t 2$ ， $F t 3$	$? 1$ ， $? 2$ ， $? 3$	509	475	1 997
$F t 1$ ， $F t 2$ ， $F t 3$	$? 1$ ， $? 2$ ， $? 3$	579	546	2 151
$F t 1$ ， $F t 2$ ， $F t 3$	$? 1$ ， $? 2$ ， $? 3$	458	425	1 805

Fig.11 Visualization of prediction failure

References 23

1	Xie G T， Gao H B， Qian L J，et al.Vehicle trajectoryprediction by integrating physics‑and maneuver‑based approaches using interactive multiple models［J］.IEEE Transactions on Industrial Electronics，2017，65（7）：5999-6008.
2	Gao H B， Cheng B， Wang J Q，et al.Object classification using CNN-based fusion of vision and LIDAR in autonomous vehicle environment［J］.IEEE Transactions on Industrial Informatics，2018，14（9）：4224-4231.
3	Gao H B， Lyu C， Zhang T，et al.A structure constraint matrix factorization framework for human behavior segmentation［J］.IEEE Transactions on Cybernetics，2021，52（12）：12978-12988.
4	Liang J W， Jiang L， Niebles J C，et al.Peeking into the future：predicting future person activities and locations in videos［C］//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.Long Beach，2019：5718-5727.
5	Kosaraju V， Sadeghian A， Martin R，et al.Social-BiGAT：multimodal trajectory forecasting using bicycle-GAN and graph attention networks［J］.ArXiv e-Prints，2019：1907.03395.
6	Sadeghian A， Kosaraju V， Sadeghian A，et al.SoPhie：an attentive GAN for predicting paths compliant to social and physical constraints［C］//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）.Long Beach，2019：1349-1358.
7	姚冲，周晖.基于时空图的行人多模态轨迹预测方法［J］.计算机工程与设计，2022，43（10）：2918-2925.
	Yao Chong， Zhou Hui.Pedestrian multi‑modal traectory prediction method based on spatial‑temporal graph［J］.Computer Engineering and Design，2022，43（10）：2918-2925.
8	Rasouli A， Kotseruba I， Tsotsos J K.Are they going to cross？a benchmark dataset and baseline for pedestrian crosswalk behavior［C］// Proceedings of the IEEE International Conference on Computer Vision Workshops.Venice，2017：206-213.
9	Rasouli A， Kotseruba I， Kunic T，et al.PIE：a large‑scale dataset and models for pedestrian intention estimation and trajectory prediction［C］//Proceedings of the IEEE/CVF International Conference on Computer Vision.Seoul，2019：6262-6271.
10	Girase H， Gang H M， Malla S，et al.LOKI：long term and key intentions for trajectory prediction［C］//2021 IEEE/CVF International Conference on Computer Vision （ICCV）.Montreal，2021：9783-9792.
11	Gao H B， Su H， Cai Y F，et al.Trajectory prediction of cyclist based on dynamic Bayesian network and long short‑term memory model at unsignalized intersections［J］.Science China Information Sciences，2021，64（7）：172207.
12	Park S K， Chung J H， Pae D S，et al.Binary dense SIFT flow based position‑information added two‑stream CNN for pedestrian action recognition［J］.Applied Sciences，2022，12（20）：10445.
13	Styles O， Ross A， Sanchez V.Forecasting pedestrian trajectory with machine‑annotated training data［C］//2019 IEEE Intelligent Vehicles Symposium （IV）.Paris，2019：716-721.
14	翟靖宇，陈金立.基于LSTM-Attention的毫米波雷达行人轨迹预测方法［J］.中国电子科学研究院学报，2022，17（6）：534-541.
	Zhai Jing‑yu， Chen Jin‑li.Pedestrian trajectory prediction method of millimeter wave radar based on LSTM-attention［J］.Journal of China Academy of Electronics and Information Technology，2022，17（6）：534-541.
15	Rasouli A， Rohani M， Luo J.Bifold and semantic reasoning for pedestrian behavior prediction［C］//Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal，2021：15600-15610.
16	裴炤，邱文涛，王淼，等.基于Transformer动态场景信息生成对抗网络的行人轨迹预测方法［J］.电子学报，2022，50（7）：1537-1547.
	Pei Zhao， Qiu Wen‑tao， Wang Miao，et al.Pedestrian trajectory prediction method using dynamic scene infor？ mation based transformer generative adversarial network［J］.Acta Electronica Sinica，2022，50（7）：1537-1547.
17	Lorenzo J， Alonso I P， Izquierdo R，et al.CAP former：pedestrian crossing action prediction using transformer［J］.Sensors，2021，21（17）：5694.
18	Yin Z Y， Liu R J， Xiong Z L，et al.Multimodal transformer networks for pedestrian trajectory prediction［C］//Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence.Montreal，2021：1259-1265.
19	Yang C L， Pei Z.Long‑short term spatio‑temporal aggregation for trajectory prediction［J］.IEEE Transactions on Intelligent Transportation Systems，2023，24（4）：4114-4126.
20	Mohamed A， Qian K， Elhoseiny M，et al.Social-STGCNN：a social spatio‑temporal graph convolutional neural network for human trajectory prediction［C］//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.Seattle，2020：14412-14420.
21	Cadena P R G， Qian Y Q， Wang C X，et al.Pedestrian graph：a fast pedestrian crossing prediction model based on graph convolutional networks［J］.IEEE Transactions on Intelligent Transportation Systems，2022，23（11）：21050-21061.
22	Styles O， Guha T， Sanchez V.Multiple object forecasting：predicting future object locations in diverse environments［C］//2020 IEEE Winter Conference on Applications of Computer Vision （WACV）.Snowmass，2020：679-688.
23	Wang C H， Wang Y C， Xu M Z，et al.Stepwise goal‑driven networks for trajectory prediction［J］.IEEE Robotics and Automation Letters，2022，7（2）：2716-2723.

[1]	Yuan MA, Li-huang SHE, Jia-wei LI, Xi-rong BAO. Adaptive Graph Convolutional 3D Point Cloud Recognition Algorithm Based on Attention Mechanism [J]. Journal of Northeastern University(Natural Science), 2024, 45(6): 786-792.
[2]	Chuan-yin TANG, Ji-feng XIA, Ming-li ZHANG, Long-jie WU. Research on Event-Triggered Control of Heterogeneous Cooperative Vehicle Platoons Considering Time Delay [J]. Journal of Northeastern University(Natural Science), 2024, 45(4): 540-547.
[3]	LIU Qi-ran， LIAN Jing， CHEN Shi， FAN Rong. Motion Planning Algorithm of Autonomous Driving Considering Interactive Trajectory Prediction [J]. Journal of Northeastern University(Natural Science), 2022, 43(7): 930-936.
[4]	LIU Fang， QIAO Jian-zhong， DAI Qin， SHI Xiang-bin. Skeleton-based Action Recognition Method with Two-Stream Multi-relational GCNs [J]. Journal of Northeastern University(Natural Science), 2021, 42(6): 768-774.