Detecting Driver’s Fatigue, Distraction and Activity Using a Non-Intrusive Ai-Based Monitoring System

Open access


The lack of attention during the driving task is considered as a major risk factor for fatal road accidents around the world. Despite the ever-growing trend for autonomous driving which promises to bring greater road-safety benefits, the fact is today’s vehicles still only feature partial and conditional automation, demanding frequent driver action. Moreover, the monotony of such a scenario may induce fatigue or distraction, reducing driver awareness and impairing the regain of the vehicle’s control. To address this challenge, we introduce a non-intrusive system to monitor the driver in terms of fatigue, distraction, and activity. The proposed system explores state-of-the-art sensors, as well as machine learning algorithms for data extraction and modeling. In the domain of fatigue supervision, we propose a feature set that considers the vehicle’s automation level. In terms of distraction assessment, the contributions concern (i) a holistic system that covers the full range of driver distraction types and (ii) a monitoring unit that predicts the driver activity causing the faulty behavior. By comparing the performance of Support Vector Machines against Decision Trees, conducted experiments indicated that our system can predict the driver’s state with an accuracy ranging from 89% to 93%.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1] World Health Organization Global status report on road safety 2015 World Health Organization Tech. Rep. 2015.

  • [2] S. Singh Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey National Highway Traffic Safety Administration Washington DC Tech. Rep. 2015.

  • [3] C. Craye A. Rashwan M. S. Kamel and F. Karray A Multi-Modal Driver Fatigue and Distraction Assessment System International Journal of Intelligent Transportation Systems Research vol. 14 no. 3 pp. 173–194 Sept. 2016.

  • [4] G. Turan and S. Gupta Road Accidents Prevention system using Driver’s Drowsiness Detection International Journal of Advanced Research in Computer Engineering & Technology vol. 2 no. 11 Nov. 2013.

  • [5] C. Braunagel E. Kasneci W. Stolzmann and W. Rosenstiel Driver-Activity Recognition in the Context of Conditionally Autonomous Driving in 2015 IEEE 18th International Conference on Intelligent Transportation Systems Sept. 2015 pp. 1652–1657.

  • [6] SAE International Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems SAE International Tech. Rep. 2016.

  • [7] M. Cunningham and M. Regan Autonomous Vehicles: Human Factors Issues and Future Research in Proceedings of the 2015 Australasian Road Safety Conference Gold Coast 2015.

  • [8] N. Merat A. H. Jamson F. C. H. Lai and O. Carsten Highly Automated Driving Secondary Task Performance and Driver State Human Factors vol. 54 no. 5 pp. 762–771 2012.

  • [9] J. Radlmayr C. Gold L. Lorenz M. Farid and K. Bengler How Traffic Situations and Non-Driving Related Tasks Affect the Take-Over Quality in Highly Automated Driving Proceedings of the Human Factors and Ergonomics Society Annual Meeting vol. 58 no. 1 pp. 2063–2067 Oct. 2014.

  • [10] N. Schömig V. Hargutt A. Neukum I. Petermann-Stock and I. Othersen The Interaction Between Highly Automated Driving and the Development of Drowsiness Procedia Manufacturing vol. 3 pp. 6652–6659 2015.

  • [11] H. Rahman S. Begum and M. U. Ahmed Driver monitoring in the context of autonomous vehicle in Frontiers in Artificial Intelligence and Applications vol. 278. Mälardalen University Embedded Systems 2015 pp. 108–117.

  • [12] A. Amditis L. Andreone K. Pagle G. Markkula E. Deregibus M. Romera Rue F. Bellotti A. Engelsberg R. Brouwer B. Peters and A. De Gloria Towards the Automotive HMI of the Future: Overview of the AIDE-Integrated Project Results IEEE Transactions on Intelligent Transportation Systems vol. 11 no. 3 pp. 567–578 Sept. 2010.

  • [13] Q. He W. Li X. Fan and Z. Fei Evaluation of driver fatigue with multi-indicators based on artificial neural network IET Intelligent Transport Systems vol. 10 no. 8 pp. 555–561 2016.

  • [14] J. Jo S. J. Lee H. G. Jung K. R. Park and J. Kim Vision-based method for detecting driver drowsiness and distraction in driver monitoring system Optical Engineering vol. 50 no. 12 pp. 127 202–1–127 202–24 2011.

  • [15] T.-H. Chang and Y.-R. Chen Driver fatigue surveillance via eye detection in 17th International IEEE Conference on Intelligent Transportation Systems (ITSC) Oct. 2014 pp. 366–371.

  • [16] O. Gusikhin N. Rychtyckyj and D. Filev Intelligent systems in the automotive industry: applications and trends Knowledge and Information Systems vol. 12 no. 2 pp. 147–168 July 2007.

  • [17] G. Lugano Virtual assistants and self-driving cars in 2017 15th International Conference on ITS Telecommunications (ITST) May 2017 pp. 1–5.

  • [18] S. Maralappanavar R. Behera and U. Mudenagudi Driver’s distraction detection based on gaze estimation in 2016 International Conference on Advances in Computing Communications and Informatics (ICACCI) Sept. 2016 pp. 2489–2494.

  • [19] Erie Insurance Erie Insurance releases police data on top 10 driving distractions involved in fatal car crashes Erie Insurance Group Tech. Rep. Apr. 2013.

  • [20] J. F. May and C. L. Baldwin Driver fatigue: The importance of identifying causal factors of fatigue when considering detection and countermeasure technologies Transportation Research Part F: Traffic Psychology and Behaviour vol. 12 no. 3 pp. 218–224 2009.

  • [21] M. Körber A. Cingel M. Zimmermann and K. Bengler Vigilance Decrement and Passive Fatigue Caused by Monotony in Automated Driving Procedia Manufacturing vol. 3 pp. 2403–2409 2015.

  • [22] B. Pfleging M. Rang and N. Broy Investigating User Needs for Non-driving-related Activities During Automated Driving in Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia ser. MUM ’16. Rovaniemi Finland: ACM 2016 pp. 91–99.

  • [23] R. Spies M. Ablaßmeier H. Bubb and W. Hamberger Augmented Interaction and Visualization in the Automotive Domain in Human-Computer Interaction. Ambient Ubiquitous and Intelligent Interaction J. A. Jacko Ed. Berlin Heidelberg: Springer Berlin Heidelberg 2009 pp. 211–220.

  • [24] B. Pfleging and A. Schmidt (Non-) Driving-Related Activities in the Car: Defining Driver Activities for Manual and Automated Driving in Workshop on Experiencing Autonomous Vehicles: Crossing the Boundaries between a Drive and a Ride at CHI ’15 2015.

  • [25] A. Bulling J. A. Ward H. Gellersen and G. Troster Eye Movement Analysis for Activity Recognition Using Electrooculography IEEE Transactions on Pattern Analysis and Machine Intelligence vol. 33 no. 4 pp. 741–753 2011.

  • [26] S. Chen and J. Epps Automatic classification of eye activity for cognitive load measurement with emotion interference Computer Methods and Programs in Biomedicine vol. 110 no. 2 pp. 111–124 2013.

  • [27] C. Helmchen and H. Rambold The Eyelid and Its Contribution to Eye Movements Developments in Ophthalmology vol. 40 pp. 110–131 2007.

  • [28] Smart Eye AB. SE PRO — Smart Eye. (2018 Sept. 29). [Online]. Available:

  • [29] Flat Earth Inc. Ancho Radar Development Kit - LTSA mini 5-11 GHz – Flat Earth Online Store. (2018 Oct. 9). [Online]. Available:

  • [30] Ergoneers GmbH Sim Lab driving simulator — ERGONEERS (2018 Oct. 29). [Online]. Available:

  • [31] K. Kircher and C. Ahlström Issues related to the driver distraction detection algorithm AttenD in 1st International Conference on Driver Distraction and Inattention (DDI 2009) Gothenburg Sweden Sept. 2009.

  • [32] Daimler AG Driver Assistance Systems — Technology Guide Daimler AG Stuttgart Germany Tech. Rep. 2013.

  • [33] D. Stawarczyk and A. D’Argembeau Conjoint Influence of Mind-Wandering and Sleepiness on Task Performance Journal of Experimental Psychology: Human Perception and Performance vol. 42 no. 10 pp. 1587–1600 2016.

  • [34] R. Nowosielski and L. M. Trick How Common In-Car Distractions Affect Driving Performance in Simple and Complex Road Environments in Proceedings of the 9th International Driving Symposium on Human Factors in Driver Assessment Training and Vehicle Design. University of Iowa Nov. 2017 pp. 249–255.

  • [35] R. Chen X. Wang L. Zhang W. Yi Y. Ke H. Qi F. He X. Zhao X. Wang D. Ming and P. Zhou Research on multi-dimensional N-back task induced EEG variations 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) pp. 5163–5166 2015.

  • [36] R. Griffin C. Huisingh and G. McGwin Prevalence of and factors associated with distraction among public transit bus drivers Traffic injury prevention vol. 15 no. 7 pp. 720–725 2014.

  • [37] G. Lemaître F. Nogueira and C. K. Aridas Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning Journal of Machine Learning Research vol. 18 no. 1 pp. 559–563 2017.

  • [38] C. Huertas and R. Juárez-Ramirez Filter feature selection performance comparison in high-dimensional data: A theoretical and empirical analysis of most popular algorithms in 17th International Conference on Information Fusion (FUSION) 2014 pp. 1–8.

  • [39] M. A. Hall Feature Selection for Discrete and Numeric Class Machine Learning 1999.

  • [40] J. Strickland Data Analytics Using Open-Source Tools. 2016.

  • [41] P. Kashyap Machine Learning for Decision Makers: Cognitive Computing Fundamentals for Better Decision Making. Apress 2018.

  • [42] F. Pedregosa G. Varoquaux A. Gramfort V. Michel B. Thirion O. Grisel M. Blondel P. Prettenhofer R. Weiss V. Dubourg J. Vanderplas A. Passos D. Cournapeau M. Brucher M. Perrot and E. Duchesnay “Scikit-learn: Machine learning in Python” Journal of Machine Learning Research vol. 12 pp. 2825–2830 2011.

  • [43] M. Sacco and R. A. Farrugia Driver fatigue monitoring system using Support Vector Machines in 2012 5th International Symposium on Communications Control and Signal Processing May 2012 pp. 1–5.

  • [44] W. Zhang B. Cheng and Y. Lin Driver drowsiness recognition based on computer vision technology Tsinghua Science and Technology vol. 17 no. 3 pp. 354–362 June 2012.

  • [45] IEE. Hands Off Detection - IEE - a sense for innovation. (2018 Oct. 17). [Online]. Available:

  • [46] M. Miyaji H. Kawanaka and K. Oguri Driver’s cognitive distraction detection using physiological features by the adaboost in 2009 12th International IEEE Conference on Intelligent Transportation Systems Oct. 2009 pp. 90–95.

  • [47] Y. Liao S. E. Li G. Li W. Wang B. Cheng and F. Chen Detection of driver cognitive distraction: an SVM based real-time algorithm and its comparison study in typical driving scenarios in 2016 IEEE Intelligent Vehicles Symposium (IV) 2016 pp. 394–399.

  • [48] J. Engström E. Johansson and J.Östlund Effects of visual and cognitive load in real and simulated motorway driving Transportation Research Part F: Traffic Psychology and Behaviour vol. 8 no. 2 pp. 97–120 2005.

Journal information
Impact Factor

CiteScore 2018: 4.70

SCImago Journal Rank (SJR) 2018: 0.351
Source Normalized Impact per Paper (SNIP) 2018: 4.066

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 447 447 52
PDF Downloads 399 399 72