, 2026
MULTI-UAV COORDINATED PATH PLANNING USING A MULTI-AGENT SOFT ACTOR-CRITIC ALGORITHM
This work considers the issue of multi-agent coordination path planning for multiple UAVs with different properties working in a dynamic and uncertain environment. In order to resolve the stated problem, an algorithm based on multi-agent deep reinforcement learning called Multi-Agent Soft Actor-Critic (MASAC) is suggested. An original simulation environment is developed to simulate a scenario of UAVs navigation towards certain goals, taking into account their kinematics constraints, obstacle avoidance and UAVs heterogeneity. As a consequence, the task is modeled as partially observable Markov decision process (POMDP). Performance assessment includes such metrics as task completion ratio, formation, flight efficiency and energy costs. The suggested framework for MASAC involves centralized training but decentralized execution which allows agents to coordinate actions and solve POMDP by means of local perception during testing time. As a result of experimental evaluation, an efficient way to resolve the curse of dimensionality, improve obstacle avoidance and cooperative formation control of UAVs was found. Such an approach shows great prospects of practical application in such domains as military operations, search and rescue missions, transport automation and disaster management (pp.28-38).
- Dewangan, R. K., Shukla, A., & Godfrey, W. W. (2019). Three-dimensional path planning using grey wolf optimizer for UAVs. Applied Intelligence, 49, 2201–2217. https://doi.org/10.1007/s10489-018-1384
- Enders, T., Harrison, J., Pavone, M., et al. (2023). Hybrid multi-agent deep reinforcement learning for autonomous mobility on demand systems. In Proceedings of the Learning for Dynamics and Control Conference. PMLR.
https://proceedings.mlr.press/v211/enders23a.html - Hadi, B., Khosravi, A., & Sarhadi, P. (2022). Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle. Applied Ocean Research, 129, Article 103326. https://doi.org/10.1016/j.apor.2022.103326
- Haarnoja, T., Tang, H., Abbeel, P., & Levine, S. (2017). Reinforcement learning with deep energy-based policies. In Proceedings of the 34th International Conference on Machine Learning. PMLR. https://doi.org/10.48550/arXiv.1702.08165
- Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the 35th International Conference on Machine Learning. PMLR. https://doi.org/10.48550/arXiv.1801.01290
- Han, Z., Chen, M., Shao, S., et al. (2023). Path planning of unmanned autonomous helicopter based on hybrid satisficing decision-enhanced swarm intelligence algorithm. IEEE Transactions on Cognitive and Developmental Systems, 15(3), 1371–1385. https://doi.org/10.1109/TCDS.2022.3212062
- Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR). https://doi.org/10.48550/arXiv.1412.6980
- Li, J., Xiong, Y., & She, J. (2023). UAV path planning for target coverage task in dynamic environment. IEEE Internet of Things Journal, 10(20), 17734–17745. Lillicrap, T. P., et al. (2015). Continuous control with deep reinforcement learning. In International Conference on Learning Representations (ICLR). https://doi.org/10.48550/arXiv.1509.02971
- Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., & Mordatch, I. (2017). Multi-agent actor-critic for mixed cooperative-competitive environments. In Advances in Neural Information Processing Systems (Vol. 30). Curran Associates, Inc. https://doi.org/10.48550/arXiv.1706.02275
- Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533. https://doi.org/10.1038/nature14236
- Qiao, Z., Li, S. L., Wang, J. Z., et al. (2022). UAV path planning based on PER-PDDPG. Unmanned Systems Technology, (5), 12–23.
- Shen, C., Li, L., Wu, Y., et al. (2018). Research on the capability of U.S. manned/unmanned autonomous collaborative operations. Tactical Missile Technology, (6), 16–21.
- Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press. http://incompleteideas.net/book/the-book-2nd.html
