Emovideo : automatic prediction of emotional responses to videos using its cinematic features

Abstract

Movies can evoke intense emotions in their audiences and are often used in the field of psychology to study emotion. Predicting how video content affects viewer’s emotional response has become a popular area of research over the past years due to its extensive applications. In order to provoke an emotional response, film makers use a variety of techniques while of filming and editing a movie. However, there are not many studies on how these techniques affect viewer’s emotional responses and if they can be used solely to predict these responses. With this work we developed a solution that predicts emotional responses to videos using cinematic features. To accomplish this goal, we started by studying cinematic techniques and their effects on viewer’s emotions. With this information, we decided to focus on shot length, key lighting, shot type and camera movement in this work. Then, we experimented with different methods to extract these features from videos. First, we used a trained model to segment videos into shots, and then segmented these shots into frames, key frames, dense flow frames and frames with the subject and background isolated. The key lighting was calculated from the contrast of each key frame. We trained and tested models to classify shot type and camera movement, with different Convolutional Neural Networks, parameters, types of frames and labels. The best results achieved were 81% for shot type and 89% for camera movement. Lastly, we created models to predict valence and arousal values with classification and regression algorithms using all the extracted cinematic features. Overall, our method had results close to previous works, especially with error metrics, with only cinematic features. This shows they affect viewers’ valence and arousal and can be a tool in predicting exact values, while providing interpretability.