Integrating Symbolic Reasoning into Deep Reinforcement Learning for Autonomous Driving Safety
Keywords:
Neuro-Symbolic Learning, Autonomous Vehicles, Reinforcement Learning, Traffic Rule Compliance, Explainable AI, Safety-Critical SystemsAbstract
Autonomous vehicle (AV) safety depends on both adaptive behavior and strict adherence to traffic regulations. This study proposes a neuro-symbolic reinforcement learning (NSRL) framework that combines deep Q-networks (DQN) with symbolic traffic rules to enhance decision-making transparency and safety performance. The NSRL model was trained using 1000 episodes in a simulated urban driving environment and evaluated on 50 test episodes. The symbolic module implemented rules such as "Red Light → Must Stop" and "Pedestrian Detected → Must Yield." Evaluation metrics included reward scores for specific violations, collision frequency, and reward trajectory over training. Results show marked improvements in rule adherence: red light violation scores improved from 80 to 95, pedestrian yield from 75 to 90, and overall reduction in collision frequencies with over 80% of test episodes resulting in zero or one collision. Additionally, the model exhibited a steadily rising reward curve during training, indicating stable learning behavior. The integration of symbolic reasoning not only improved safety outcomes but also provided interpretable justifications for AV actions, thereby enhancing transparency and regulatory acceptability. This approach shows promise for real-world deployment and could be adapted to other safety-critical domains.
References
Alizadeh, A., Moghadam, M., Bicer, Y., Üre, N. K., Yavaş, U., & Kurtulus, C. (2019). Automated Lane Change Decision Making Using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environment (pp. 1399–1404). https://doi.org/10.1109/itsc.2019.8917192
Bueff, A., & Belle, V. (2023). Deep Inductive Logic Programming Meets Reinforcement Learning. Electronic Proceedings in Theoretical Computer Science, 385, 339–352. https://doi.org/10.4204/eptcs.385.37
Confalonieri, R., Çoba, L., Wagner, B. J., & Besold, T. R. (2020). A Historical Perspective of Explainable Artificial Intelligence. Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery, 11(1). https://doi.org/10.1002/widm.1391
Dabney, W., Rowland, M., Bellemare, M. G., & Munos, R. (2018). Distributional Reinforcement Learning With Quantile Regression. Proceedings of the Aaai Conference on Artificial Intelligence, 32(1). https://doi.org/10.1609/aaai.v32i1.11791
Dong, J., Chen, S., Miralinaghi, M., Chen, T., & Labi, S. (2022). Development and Testing of an Image Transformer for Explainable Autonomous Driving Systems. Journal of Intelligent and Connected Vehicles, 5(3), 235–249. https://doi.org/10.1108/jicv-06-2022-0021
Grigorescu, S., Trăsnea, B., Cocias, T., & Măceşanu, G. (2019). A Survey of Deep Learning Techniques for Autonomous Driving. Journal of Field Robotics, 37(3), 362–386. https://doi.org/10.1002/rob.21918
Hamilton, J. L., Torous, J., Szlyk, H. S., Biernesser, C., Kruzan, K. P., Jensen, M., Reyes‐Portillo, J. A., Primack, B. A., Zelazny, J., & Weigle, P. E. (2024). Leveraging Digital Media to Promote Youth Mental Health: Flipping the Script on Social Media-Related Risk. Current Treatment Options in Psychiatry, 11(2), 67–75. https://doi.org/10.1007/s40501-024-00315-y
Hayworth, K. J., & Marblestone, A. (2024). How Thalamic Relays Might Orchestrate Supervised Deep Training and Symbolic Computation in the Brain. https://doi.org/10.1101/304980
Jayaraman, S. K., Creech, C., Robert, L., Tilbury, D. M., Yang, X. J., Pradhan, A. K., & Tsui, K. M. (2018). Trust in AV (pp. 133–134). https://doi.org/10.1145/3173386.3177073
Kassner, N., Krojer, B., & Schütze, H. (2020). Are Pretrained Language Models Symbolic Reasoners Over Knowledge? https://doi.org/10.48550/arxiv.2006.10413
Kim, S. K., Kirchner, E. A., Stefes, A., & Kirchner, F. (2017). Intrinsic Interactive Reinforcement Learning – Using Error-Related Potentials for Real World Human-Robot Interaction. Scientific Reports, 7(1). https://doi.org/10.1038/s41598-017-17682-7
Klein-Flügge, M. C., Wittmann, M. K., Shpektor, A., Jensen, D. E. A., & Rushworth, M. F. S. (2019). Multiple Associative Structures Created by Reinforcement and Incidental Statistical Learning Mechanisms. Nature Communications, 10(1). https://doi.org/10.1038/s41467-019-12557-z
Manjunath, N. K., Shiri, A., Hosseini, M., Prakash, B., Waytowich, N. R., & Mohsenin, T. (2021). An Energy Efficient EdgeAI Autoencoder Accelerator for Reinforcement Learning. Ieee Open Journal of Circuits and Systems, 2, 182–195. https://doi.org/10.1109/ojcas.2020.3043737
Martires, P. Z. D., Kumar, N. D., Persson, A., Loutfi, A., & Raedt, L. D. (2020). Symbolic Learning and Reasoning With Noisy Data for Probabilistic Anchoring. Frontiers in Robotics and Ai. https://doi.org/10.3389/frobt.2020.00100
Minervini, P., Riedel, S., Stenetorp, P., Grefenstette, E., & Rocktäschel, T. (2021). Chapter 12. Learning Reasoning Strategies in End-to-End Differentiable Proving. https://doi.org/10.3233/faia210359
Omeiza, D. (2021). Explanations in Autonomous Driving: A Survey. https://doi.org/10.48550/arxiv.2103.05154
Ozturk, A., Gunel, M. B., Dagdanov, R., Vural, M. E., Yurdakul, F., Dal, M., & Üre, N. K. (2021). Investigating Value of Curriculum Reinforcement Learning in Autonomous Driving Under Diverse Road and Weather Conditions. https://doi.org/10.48550/arxiv.2103.07903
Paxton, C., Raman, V., Hager, G. D., & Kobilarov, M. (2017). Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments (pp. 6059–6066). https://doi.org/10.1109/iros.2017.8206505
Piot, B., Geist, M., & Pietquin, O. (2016). Difference of Convex Functions Programming Applied to Control With Expert Data. https://doi.org/10.48550/arxiv.1606.01128
Ramos, I. F. F., Gianini, G., & Damiani, E. (2022). Neuro-Symbolic AI for Sensor-Based Human Performance Prediction. System Architectures and Applications, 3210–3217. https://doi.org/10.3850/978-981-18-5183-4_s33-01-310-cd
Rivas, A., Icarte, R. T., Klassen, T. Q., Valenzano, R., & McIlraith, S. A. (2019). LTL and Beyond: Formal Languages for Reward Function Specification in Reinforcement Learning (pp. 6065–6073). https://doi.org/10.24963/ijcai.2019/840
Rovira, E., McLaughlin, A. C., Pak, R., & High, L. (2019). Looking for Age Differences in Self-Driving Vehicles: Examining the Effects of Automation Reliability. Driving Risk, and Physical Impairment on Trust. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2019.00800
Rudenko, A., Palmieri, L., Herman, M., Kitani, K., Gavrila, D. M., & Arras, K. O. (2020). Human Motion Trajectory Prediction: A Survey. The International Journal of Robotics Research, 39(8), 895–935. https://doi.org/10.1177/0278364920917446
Setiawan, M. A., Setiadi, R. I. M., Astuti, E. Z., Sutojo, T., & Setiyanto, N. A. (2024). Exploring Deep Q-Network for Autonomous Driving Simulation Across Different Driving Modes. J. Fut. Artif. Intell. Tech, 1(3), 217–227. https://doi.org/10.62411/faith.3048-3719-31
Waghmare, A. A., Ganesan, S., & Chen, J. (2024). Role of Artificial Intelligence in Autonomous Vehicles. https://doi.org/10.20944/preprints202408.0974.v1
Wen, S. (2023). Dynamic Path Planning in Autonomous Driving. Journal of Physics Conference Series, 2649(1). https://doi.org/10.1088/1742-6596/2649/1/012048
Xiong, X., & Zheng, M. (2024). Integrating Deep Learning With Symbolic Reasoning in TinyLlama for Accurate Information Retrieval. https://doi.org/10.21203/rs.3.rs-3883562/v1
Zhang, S., Zhang, Y., Liu, Q., Li, H. L., Liang, Z., & Wu, H. (2022). Dynamical Driving Interactions Between Human and Mentalizing-Designed Autonomous Vehicle. https://doi.org/10.31234/osf.io/tn5xp
