Contact-Free Pulse Signal Extraction from Human Face Videos: A Review and New Optimized Filtering Approach

In this chapter, we review methods for video-based heart monitoring, from classical signal processing approaches to modern deep learning methods. In addition, we propose a new method for learning an optimal filter that can overcome many of the problems that can affect classical approaches, such as light reflection and subject’s movements, at a fraction of the training cost of deep learning approaches. Following the usual procedures for region of interest extraction and tracking, robust skin color estimation and signal pre-processing, we introduce a least-squares error optimal filter, learnt using an established training dataset to estimate the photoplethysmographic (PPG) signal more accurately from the measured color changes over time. This method not only improves the accuracy of heart rate measurement but also resulted in the extraction of a cleaner pulse signal, which could be integrated into many other useful applications such as human biometric recognition or recognition of emotional state. The method was tested on the DEAP dataset and showed improved performance over the best previous classical method on that dataset. The results obtained show that our proposed contact-free heart rate measurement method has significantly improved on existing methods.