This post is to share a side project on extracting ‘reaction faces’ from YouTube videos.

Example Output

Output from video: ‘PLOTCON 2016: Hadley Wickham, New open viz in R’.

Neutral

Scared

Happy

Usage

The code requires opencv & keras (model trained with TensorFlow backend). To use the reaction face finder:

1. Clone the repo at AdamSpannbauer/youtube_reaction_face
2. Launch a terminal window and cd into the newly cloned repo’s directory
3. Use the below command format to run the program
4. Wait for the output to appear in the specified output directory

#command format
python youtube_react_face.py -y YOUTUBEURL -o OUTPUT [-m MAXFRAMES]

#example command
python youtube_react_face.py --youtubeURL https://www.youtube.com/watch?v=tVb0g0-JCfk --output output

Arguments:

#required
-y YOUTUBEURL, --youtubeURL YOUTUBEURL
                 url for youtube video to find reaction faces in
-o OUTPUT, --output OUTPUT
             dir to output reactions to
                        
#optional
-m MAXFRAMES, --maxFrames MAXFRAMES
                stop processing video frames after this many (default 5000)

Disclaimer

Before classifying the emotion of a face the program has to find the face. This project currently uses a Haar cascade face detector. This style of detection is great for speed, but is prone to false positives. So every now and then the output will show a scared tie or an angry shirt.

Sources

Huge thanks to Adrian at PyImageSearch for his book, Deep Learning for Computer Vision with Python. Adrian and the book gave me all the tools needed for building the emotion model. Since I followed along with the book so closely for creating the model, I don’t feel that it would be appropriate to share the code used.