Explainable Transformer-Based Object Detection for Autonomous Systems under Adversarial and Low-Light Conditions
DOI:
https://doi.org/10.56313/jictas.v4i2.444Keywords:
Object Detection, Vision Transformer (ViT), Explainable AI, Grad-CAM, autonomous systemsAbstract
Recent advancements in object detection have demonstrated remarkable performance in autonomous systems; however, most deep learning models still suffer significant accuracy degradation under low-light or adversarial conditions. This study proposes an Explainable Transformer-Based Object Detection (ETOD) framework that integrates Vision Transformer (ViT) architecture with Explainable Artificial Intelligence (XAI) mechanisms to achieve robust and interpretable object detection in adverse environments. The proposed ETOD model employs a dual-branch structure: (i) a low-light enhancement module that uses contrastive illumination normalization to recover critical features, and (ii) a transformer-based detection head optimized for global contextual reasoning. To ensure explainability, Grad-CAM and attention visualization maps are incorporated to highlight the model’s focus regions, providing interpretive insights for human operators and safety auditors. Experimental evaluation was conducted using benchmark datasets (ExDark, BDD100K-Night, and COCO-Adversarial) with simulated adversarial perturbations and low-illumination conditions. The proposed ETOD achieved a 12.8% improvement in mAP over standard DETR and 17.5% higher robustness against adversarial attacks while maintaining real- time inference on edge GPUs. Qualitative analysis demonstrates that the explainability module provides clear visual cues that correlate strongly with detected object boundaries. The findings suggest that integrating transformer- based detection with explainable reasoning mechanisms offers a promising pathway for trustworthy and safety-critical perception systems in autonomous vehicles and drones
References
H. Hu et al., “Thermal-sensing actuator based on conductive polymer ionogel for autonomous human-machine interaction,” Sensors Actuators B Chem., vol. 398, 2024, doi: 10.1016/j.snb.2023.134756.
S. Mishra and P. Palanisamy, “Autonomous Advanced Aerial Mobility - An End-to-End Autonomy Framework for UAVs and Beyond,” IEEE Access, vol. 11, 2023, doi: 10.1109/ACCESS.2023.3339631.
S. Roy, T. Vo, S. Hernandez, A. Lehrmann, A. Ali, and S. Kalafatis, “IoT Security and Computation Management on a Multi-Robot System for Rescue Operations Based on a Cloud Framework,” Sensors, vol. 22, no. 15, 2022, doi: 10.3390/s22155569.
N. Kapetanovi? et al., “Heterogeneous Autonomous Robotic System in Viticulture and Mariculture: Vehicles Development and Systems Integration,” Sensors, vol. 22, no. 8, 2022, doi: 10.3390/s22082961.
F. Corradi and F. Fioranelli, “Radar Perception for Autonomous Unmanned Aerial Vehicles: A Survey,” 2022. doi: 10.1145/3522784.3522787.
J. Zhu, J. Hu, M. Zhang, Y. Chen, and S. Bi, “A fog computing model for implementing motion guide to visually impaired,” Simul. Model. Pract. Theory, vol. 101, 2020, doi: 10.1016/j.simpat.2019.102015.
S. Khattak, C. Papachristos, and K. Alexis, “Visual-Thermal Landmarks and Inertial Fusion for Navigation in Degraded Visual Environments,” in IEEE Aerospace Conference Proceedings, 2019, vol. 2019-March. doi: 10.1109/AERO.2019.8741787.
B. Yanto, B. -, J. -, and B. H. Hayadi, “Indentifikasi Pola Aksara Arab Melayu Dengan Jaringan Syaraf Tiruan Convolutional Neural Network (Cnn),” JSAI (Journal Sci. Appl. Informatics), vol. 3, no. 3, pp. 106–114, 2020, doi: 10.36085/jsai.v3i3.1151.
B. Yanto, B. -, J. -, and B. H. Hayadi, “INDENTIFIKASI POLA AKSARA ARAB MELAYU DENGAN JARINGAN SYARAF TIRUAN CONVOLUTIONAL NEURAL NETWORK (CNN),” JSAI (Journal Sci.
Appl. Informatics), vol. 3, no. 3, 2020, doi: 10.36085/jsai.v3i3.1151.
B. Yanto, L. Fimawahib, A. Supriyanto, B. H. Hayadi, and R. R. Pratama, “Klasifikasi Tekstur Kematangan Buah Jeruk Manis Berdasarkan Tingkat Kecerahan Warna dengan Metode Deep Learning Convolutional Neural Network,” INOVTEK Polbeng - Seri Inform., vol. 6, no. 2, 2021, doi: 10.35314/isi.v6i2.2104.
C. van Zyl, X. Ye, and R. Naidoo, “Harnessing eXplainable artificial intelligence for feature selection in time series energy forecasting: A comparative analysis of Grad-CAM and SHAP,” Appl. Energy, vol. 353, 2024, doi: 10.1016/j.apenergy.2023.122079.
F. G. Rebitschek, “Boosting Consumers: Algorithm-Supported Decision-Making under Uncertainty to (Learn to) Navigate Algorithm-Based Decision Environments,” in Knowledge and Space, vol. 19, 2024. doi: 10.1007/978- 3-031-39101-9_4.
K. Champion, P. Zheng, A. Y. Aravkin, S. L. Brunton, and J. N. Kutz, “A unified sparse optimization framework to learn parsimonious physics-informed models from data,” IEEE Access, vol. 8, 2020, doi: 10.1109/ACCESS.2020.3023625.
B. Yanto, E. Rouza, L. Fimawahib, B. H. Hayadi, and R. R. Pratama, “Penerapan Algoritma Deep Learning Convolutional Neural Network Dalam Menentukan Kematangan Buah Jeruk Manis Berdasarkan Citra Red Green Blue (RGB),” J. Teknol. Inf. dan Ilmu Komput., vol. 10, no. 1, 2023, doi: 10.25126/jtiik.20231015695.
B. Citra, R. E. D. Green, and B. Rgb, “PENERAPAN ALGORITMA DEEP LEARNING CONVOLUTIONAL NEURAL NETWORK DALAM MENENTUKAN KEMATANGAN BUAH JERUK MANIS APPLICATION OF THE DEEP LEARNING CONVOLUTIONAL NEURAL NETWORK ALGORITHM IN DETERMINING THE MURABILITY OF SWEET ORANGE FRUIT BASED ON IMAGES RED GRE,” vol. 10, no. 1, pp. 59–
, 2023, doi: 10.25126/jtiik.2023105695.
B. Yanto, J. Jufri, A. Lubis, B. H. Hayadi, and E. Armita, NST, “Klarifikasi Kematangan Buah Nanas Dengan Ruang Warna Hue Saturation Intensity (Hsi),” INOVTEK Polbeng - Seri Inform., vol. 6, no. 1, p. 135, 2021, doi: 10.35314/isi.v6i1.1882.
B. Yanto`, Maria Angela Kartawidjaja, Ronald Sukwadi, and Marsellinus Bachtiar, “Implementation of Hue Saturation Intensity (Hsi) Color Space Transformation Algorithm With Red, Green, Blue (Rgb) Color Brightness in Assessing Tomato Fruit Maturity,” RJOCS (Riau J. Comput. Sci., vol. 9, no. 2, pp. 167–178, 2023, doi: 10.30606/rjocs.v9i2.2428.
H. Z. Yuan, K. H. Ghazali, A. Lubis, S. Sunardi, and B. Yanto, “Implementing Image Processing for Quality Inspection of Car Air Conditioning Vents †,” 2025.
X. Fu et al., “Crop pest image recognition based on the improved ViT method,” Inf. Process. Agric., vol. 11, no. 2, 2024, doi: 10.1016/j.inpa.2023.02.007.
X. Li, M. Yu, D. Xu, S. Zhao, H. Tan, and X. Liu, “Non-Contact Measurement of Pregnant Sows’ Backfat Thickness Based on a Hybrid CNN-ViT Model,” Agric., vol. 13, no. 7, 2023, doi: 10.3390/agriculture13071395.
J. Tagnamas, H. Ramadan, A. Yahyaouy, and H. Tairi, “Correction to: Multi-task approach based on combined CNN-transformer for efficient segmentation and classification of breast tumors in ultrasound images (Visual Computing for Industry, Biomedicine, and Art, (2024), 7, 1, (2), 10.1186/s42492-024-00155-w),” Visual Computing for Industry, Biomedicine, and Art, vol. 7, no. 1. 2024. doi: 10.1186/s42492-024-00156-9.
F. Li, H. Zhang, S. Liu, J. Guo, L. M. Ni, and L. Zhang, “DN-DETR: Accelerate DETR Training by Introducing Query DeNoising,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 4, 2024, doi: 10.1109/TPAMI.2023.3335410.
F. Vaquerizo-Villar et al., “An explainable deep-learning model to stage sleep states in children and propose novel EEG-related patterns in sleep apnea,” Comput. Biol. Med., vol. 165, 2023, doi: 10.1016/j.compbiomed.2023.107419.
L. S. Chow, G. S. Tang, M. I. Solihin, N. M. Gowdh, N. Ramli, and K. Rahmat, “Quantitative and Qualitative Analysis of 18 Deep Convolutional Neural Network (CNN) Models with Transfer Learning to Diagnose COVID-19 on Chest X-Ray (CXR) Images,” SN Comput. Sci., vol. 4, no. 2, 2023, doi: 10.1007/s42979-022- 01545-8.
Published
How to Cite
Issue
Section
Copyright (c) 2025 Journal of ICT Aplications and System
This work is licensed under a Creative Commons Attribution 4.0 International License.
