A Graph Reinforcement-Based Approach to Task Offloading and Resource Allocation in Partially Observable Environment

doi:10.12068/j.issn.1005-3026.2025.20230216

Abstract

Abstract:

To address the issue of global information loss due to ineffective communication among edge servers in partially observable environment， an inter‑edge server communication mechanism based on a graph attention mechanism is constructed， where the mobile edge computing （MEC） system is represented as a graph structure， allowing message passing between edge servers through the edges in the graph to indirectly obtain the global state information of the MEC system. The dual attention mechanism is introduced to enable agents to focus more on communication messages that are more useful for policy optimization， thereby accelerating the convergence speed of the model and improving algorithm performance. Simulation experimental results demonstrate that compared to baseline algorithms， the proposed algorithm effectively reduces task completion delay and energy consumption while exhibiting faster convergence speed.

Key words: mobile edge computing （MEC）, deep reinforcement learning, task offloading, resource allocation, message communication

CLC Number:

TP 393

Yu DAI, Zong-ming JING, Lei YANG, Zhen GAO. A Graph Reinforcement-Based Approach to Task Offloading and Resource Allocation in Partially Observable Environment[J]. Journal of Northeastern University(Natural Science), 2025, 46(1): 9-17.

Figures/Tables 9

Fig.1 Mobile edge network scenario

Fig.2 Network structure of MAGA algorithm

Fig.3 Impact of different learning rates on system rewards

Fig.4 Impact of different discount factors on system rewards

Fig.5 System rewards of different algorithms

Fig.6 Average task completion delay of different algorithms

Fig.7 Average energy consumption of different algorithms

Fig.8 Convergence of the three algorithms

Fig.9 Impact of the number of mobile devices on algorithm delay

References 25

1	Chen C， Chen L L， Liu L，et al.Delay‑optimized V2V‑based computation offloading in urban vehicular edge computing and networks［J］.IEEE Access，2020，8：18863-18873.
2	Zhao L， Zhang E C， Wan S H，et al.MESON：a mobility‑aware dependent task offloading scheme for urban vehicular edge computing［J］.IEEE Transactions on Mobile Computing，2023，23（5）：4259-4272.
3	Pu L J， Chen X， Mao G Q，et al.Chimera：an energy‑efficient and deadline‑aware hybrid edge computing framework for vehicular crowdsensing applications［J］.IEEE Internet of Things Journal，2019，6（1）：84-99.
4	Feng L， Zhou Y， Liu T，et al.Energy‑efficient offloading for mission‑critical IoT services using EVT‑embedded intelligent learning［J］.IEEE Transactions on Green Communications and Networking，2021，5（3）：1179-1190.
5	Qian Y C， Wang R， Wu J，et al.Reinforcement learning‑based optimal computing and caching in mobile edge network［J］.IEEE Journal on Selected Areas in Communications，2020，38（10）：2343-2355.
6	Shi J M， Du J， Shen Y，et al.DRL‑based V2V computation offloading for blockchain‑enabled vehicular networks［J］.IEEE Transactions on Mobile Computing，2022，22（7）：3882-3897.
7	Chen J， Xing H L， Xiao Z W，et al.A DRL agent for jointly optimizing computation offloading and resource allocation in MEC［J］.IEEE Internet of Things Journal，2021，8（24）：17508-17524.
8	Zhao N， Ye Z Y， Pei Y Y，et al.Multi‑agent deep reinforcement learning for task offloading in UAV‑assisted mobile edge computing［J］.IEEE Transactions on Wireless Communications，2022，21（9）：6949-6960.
9	Tang L， Du Y C， Liu Q H，et al.Digital‑twin‑assisted resource allocation for network slicing in industry 4.0 and beyond using distributed deep reinforcement learning［J］.IEEE Internet of Things Journal，2023，10（19）：16989-17006.
10	Ye Z H， Wang K， Chen Y N，et al.Multi‑UAV navigation for partially observable communication coverage by graph reinforcement learning［J］.IEEE Transactions on Mobile Computing，2022，22（7）：4056-4069.
11	Tang M， Wong V W S.Deep reinforcement learning for task offloading in mobile edge computing systems［J］.IEEE Transactions on Mobile Computing，2020，21（6）：1985-1997.
12	Gao Z， Yang L， Dai Y.Fast adaptive task offloading and resource allocation in large‑scale MEC systems via multiagent graph reinforcement learning［J］.IEEE Internet of Things Journal，2023，11（1）：758-776.
13	Yan J， Bi S Z， Zhang Y J A.Offloading and resource allocation with general task graph in mobile edge computing：a deep reinforcement learning approach［J］.IEEE Transactions on Wireless Communications，2020，19（8）：5404-5419.
14	Zhu X Y， Luo Y Y， Liu A F，et al.Multiagent deep reinforcement learning for vehicular computation offloading in IoT［J］.IEEE Internet of Things Journal，2021，8（12）：9763-9773.
15	Shannon C E.A mathematical theory of communication［J］.The Bell System Technical Journal，1948，27（3）：379-423.
16	Gao Z， Yang L， Dai Y.Large‑scale cooperative task offloading and resource allocation in heterogeneous mec systems via multi‑agent reinforcement learning［J］.IEEE Internet of Things Journal，2023，11（2）：2303-2321.
17	Zhai Z W， Wu Q， Yu S，et al.FedLEO：an offloading‑assisted decentralized federated learning framework for low earth orbit satellite networks［J］.IEEE Transactions on Mobile Computing，2023，23（5）：5260-5279.
18	Chen Q J， Yang C， Lan S L，et al.Two‑stage evolutionary search for efficient task offloading in edge computing power networks［J］.IEEE Internet of Things Journal，2024，11（19）：30787-30799.
19	Knuth D E.Two notes on notation［J］.American Mathematical Monthly，1992，99（5）：403-422.
20	Tang Z， Wan B Y， Yang L.Word‑character graph convolution network for chinese named entity recognition［J］.IEEE/ACM Transactions on Audio，Speech，and Language Processing，2020，28：1520-1532.
21	Gündüz D， Qin Z J， Inaki E A，et al.Beyond transmitting bits：context，semantics，and task‑oriented communications［J］.IEEE Journal on Selected Areas in Communications，2022，41（1）：5-41.
22	Dai B， Niu J W， Ren T，et al.Towards energy‑efficient scheduling of UAV and base station hybrid enabled mobile edge computing［J］.IEEE Transactions on Vehicular Technology，2021，71（1）：915-930.
23	Qiu X Y， Zhang W K， Chen W H，et al.Distributed and collective deep reinforcement learning for computation offloading：a practical perspective［J］.IEEE Transactions on Parallel and Distributed Systems，2021，32（5）：1085-1101.
24	Song F H， Xing H L， Wang X H，et al.Evolutionary multi‑objective reinforcement learning based trajectory control and task offloading in UAV‑assisted mobile edge computing［J］.IEEE Transactions on Mobile Computing，2022，22（12）：7387-7405.
25	Song F H， Xing H L， Wang X H，et al.Offloading dependent tasks in multi‑access edge computing：a multi‑objective reinforcement learning approach［J］.Future Generation Computer Systems，2022，128：333-348.

[1]	WEI Jian-yi， WU Jing-jing. Resource Allocation Algorithm in Industrial Internet of Things Based on Edge Computing [J]. Journal of Northeastern University(Natural Science), 2023, 44(8): 1072-1078.
[2]	ZHAO Zhao， YUAN Pei-xin， TANG Jun-wen， CHEN Jin-lin. Agent Path Planning Algorithm Based on Improved SNN-HRL [J]. Journal of Northeastern University(Natural Science), 2023, 44(11): 1548-1555.
[3]	LIU Jun， DAI Fu-cheng， XIN Ning. Virtual Machine Placement Strategy Based on Multi-objective Optimization [J]. Journal of Northeastern University(Natural Science), 2022, 43(5): 609-617.
[4]	YU Xiao， LEI Wei-min， XIE Bing， ZHAO Jin-feng. A Cross Layer Resource Allocation Algorithm for Constellation Communication System Based on QoS [J]. Journal of Northeastern University Natural Science, 2018, 39(3): 320-324.
[5]	GONG Jun， JIA Qian-qian， ZHANG Jia-sheng， YU Miao. Robust Program Model of Human Resource Management for Staffing Call Centers [J]. Journal of Northeastern University Natural Science, 2018, 39(1): 20-25.
[6]	ZHANG Bin， ZHU Meng-xiao， ZHAO Xiu-tao， ZHANG Chang-sheng. Cost Oriented Virtualized Resource Optimization Allocation for SBS [J]. Journal of Northeastern University Natural Science, 2015, 36(7): 929-933.
[7]	LIU Fulai， QIU Huizhen， ZHOU Qingping， DU Ruiyan. An Effective Proportional Fairness Algorithm [J]. Journal of Northeastern University Natural Science, 2014, 35(11): 1535-1538.