Generalizable and Energy Efficient Deep Reinforcement Learning for Urban Delivery Robot Navigation
DOI:
https://doi.org/10.61978/digitus.v3i2.954Keywords:
Reinforcement Learning, Autonomous Delivery, Urban Navigation, Sim to Real Transfer, Multi Objective Learning, Domain Adaptation, Energy EfficiencyAbstract
The increasing demand for contactless urban logistics has driven the integration of autonomous delivery robots into real world operations. This study investigates the application of Deep Reinforcement Learning (DRL) to enhance robot navigation in complex urban environments, focusing on three advanced models: MODSRL, SOAR RL, and NavDP. MODSRL employs a multi objective framework to balance safety, efficiency, and success rate. SOAR RL is designed to handle high obstacle densities using anticipatory decision making. NavDP addresses the sim to real gap through domain adaptation and few shot learning. The models were trained and evaluated in simulation environments (CARLA, nuScenes, Argoverse) and validated using real world deployment data. Evaluation metrics included success rate, collision frequency, and energy efficiency. MODSRL achieved a 91.3% success rate with only 4.2% collision, outperforming baseline methods. SOAR RL showed robust performance in obstacle rich scenarios but highlighted a safety efficiency trade off. NavDP improved real world success rates from 50% to 80% with minimal adaptation data, demonstrating the feasibility of sim to real transfer. The results confirm the effectiveness of DRL in advancing autonomous delivery navigation. Integrating domain generalization, hybrid learning, and real time adaptation strategies will be essential to support large scale urban deployment. Future research should prioritize explainability, continual learning, and user centric navigation policies.
References
Bonardi, A., James, S., & Davison, A. J. (2020). Learning One-Shot Imitation From Humans Without Humans. Ieee Robotics and Automation Letters, 5(2), 3533–3539. https://doi.org/10.1109/lra.2020.2977835 DOI: https://doi.org/10.1109/LRA.2020.2977835
Bridgelall, R. (2024). Locating Electrified Aircraft Service to Reduce Urban Congestion. Information, 15(4), 186. https://doi.org/10.3390/info15040186 DOI: https://doi.org/10.3390/info15040186
Daranda, A., & Dzemyda, G. (2022). Reinforcement Learning Strategies for Vessel Navigation. Integrated Computer-Aided Engineering, 30(1), 53–66. https://doi.org/10.3233/ica-220688 DOI: https://doi.org/10.3233/ICA-220688
Elsken, T., Staffler, B., Metzen, J. H., & Hutter, F. (2019). Meta-Learning of Neural Architectures for Few-Shot Learning. https://doi.org/10.48550/arxiv.1911.11090 DOI: https://doi.org/10.1109/CVPR42600.2020.01238
Gao, Y., Wu, J., Yang, X., & Ji, Z. (2024). Efficient Hierarchical Reinforcement Learning for Mapless Navigation With Predictive Neighbouring Space Scoring. Ieee Transactions on Automation Science and Engineering, 21(4), 5457–5472. https://doi.org/10.1109/tase.2023.3312237 DOI: https://doi.org/10.1109/TASE.2023.3312237
Hou, Y., Lai, Y., Chen, C., Che, W., & Liu, T. (2021). Learning to Bridge Metric Spaces: Few-Shot Joint Learning of Intent Detection and Slot Filling. 3190–3200. https://doi.org/10.18653/v1/2021.findings-acl.282 DOI: https://doi.org/10.18653/v1/2021.findings-acl.282
Jiang, Y., Zhang, T., Ho, D. E., Bai, Y., Liu, C. K., Levine, S., & Tan, J. (2021). SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning. 2884–2890. https://doi.org/10.1109/icra48506.2021.9561731 DOI: https://doi.org/10.1109/ICRA48506.2021.9561731
Karnan, H., Nair, A., Xiao, X., Warnell, G., Pirk, S., Toshev, A., Hart, J. L., Biswas, J., & Stone, P. (2022). Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation. https://doi.org/10.48550/arxiv.2203.15041 DOI: https://doi.org/10.1109/LRA.2022.3184025
Lee, J., Bjelonic, M., Reske, A., Wellhausen, L., Miki, T., & Hutter, M. (2024). Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots. Science Robotics, 9(89). https://doi.org/10.1126/scirobotics.adi9641 DOI: https://doi.org/10.1126/scirobotics.adi9641
Scheikl, P. M., Tagliabue, E., Gyenes, B., Wagner, M., Dall’Alba, D., Fiorini, P., & Mathis-Ullrich, F. (2023). Sim-to-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgery. Ieee Robotics and Automation Letters, 8(2), 560–567. https://doi.org/10.1109/lra.2022.3227873 DOI: https://doi.org/10.1109/LRA.2022.3227873
Tonioni, A., Rahnama, O., Joy, T., Stefano, L. D., Ajanthan, T., & Torr, P. H. S. (2019). Learning to Adapt for Stereo. https://doi.org/10.1109/cvpr.2019.00989 DOI: https://doi.org/10.1109/CVPR.2019.00989
Vieira, M., Vieira, M. A., Galvão, G., Louro, P., Fantoni, A., Vieira, P., & Véstias, M. (2025). Enhancing Airport Traffic Flow: Intelligent System Based on VLC, Rerouting Techniques, and Adaptive Reward Learning. Sensors, 25(9), 2842. https://doi.org/10.3390/s25092842 DOI: https://doi.org/10.3390/s25092842
Wang, C., Chen, J., & Yu, X. (2024). Research on Benefit Allocation Based on Multi-Weight H-Shapley Value: A Case Study of Express Logistics Sharing. Plos One, 19(7), e0305656. https://doi.org/10.1371/journal.pone.0305656 DOI: https://doi.org/10.1371/journal.pone.0305656
Whittlestone, J., Arulkumaran, K., & Crosby, M. (2021). The Societal Implications of Deep Reinforcement Learning. Journal of Artificial Intelligence Research, 70. https://doi.org/10.1613/jair.1.12360 DOI: https://doi.org/10.1613/jair.1.12360
Xia, Y., & Mei, C. (2024). Integrating UAS for 3D Terrain Mapping and Autonomous Navigation: A Review of Multi-Camera and Reinforcement Learning. https://doi.org/10.31219/osf.io/z7mdv DOI: https://doi.org/10.31219/osf.io/z7mdv
Xu, S., Hao, J., Xue-mei, C., & Hu, Y. (2023). Navigating Autonomous Vehicles in Uncertain Environments With Distributional Reinforcement Learning. Proceedings of the Institution of Mechanical Engineers Part D Journal of Automobile Engineering, 238(12), 3653–3663. https://doi.org/10.1177/09544070231186841 DOI: https://doi.org/10.1177/09544070231186841
Xu, Z., Wu, K., Che, Z., Tang, J., & Ye, J. (2020). Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Control. https://doi.org/10.48550/arxiv.2010.07494
Yang, Y., Bevan, M. A., & Li, B. (2019). Efficient Navigation of Colloidal Robots in an Unknown Environment via Deep Reinforcement Learning. Advanced Intelligent Systems, 2(1). https://doi.org/10.1002/aisy.201900106 DOI: https://doi.org/10.1002/aisy.201900106
Yingjie, Z., Hasan, W. Z. W., Ramli, H. R., Norsahperi, N. M. H., Kassim, M. S. M., & Yao, Y. (2025). Deep Reinforcement Learning of Mobile Robot Navigation in Dynamic Environment: A Review. Sensors, 25(11), 3394. https://doi.org/10.3390/s25113394 DOI: https://doi.org/10.3390/s25113394
Zhang, H., Song, M., & He, H. (2020). Achieving the Success of Sustainability Development Projects Through Big Data Analytics and Artificial Intelligence Capability. Sustainability, 12(3), 949. https://doi.org/10.3390/su12030949 DOI: https://doi.org/10.3390/su12030949
Zhao, P., Ram, P., Lu, S., Yao, Y., Bouneffouf, D., Lin, X., & Liu, S. (2020). Learning to Generate Image Source-Agnostic Universal Adversarial Perturbations. https://doi.org/10.48550/arxiv.2009.13714
