Facial Expression Recognition For Affective Video Games

Affective video games are still a relatively new field of research and entertainment. Even
so, being a form of entertainment media, emotion plays a large role in video games as a whole.
This project seeks to gain an understanding of what emotions are most prominent during game
play. From there, a system will be created wherein the game will record the player’s facial
expressions and interpret those expressions as emotions, allowing the game to adjust its difficulty
to create a more tailored experience.
The first portion of this project, understanding the relationship between emotions and
games, was done by recording myself as I played three different games of different genres for
thirty minutes each. The same system that would be used in the later game I created to evaluate
emotions was used to evaluate these recordings.
After the data was interpreted, I created three different versions of the same game, based
on a template created by Stan’s Assets, which was a version of the arcade game Stacker. The
three versions of the game included one where no changes were made to the gameplay
experience, it simply recorded the player’s face and extrapolated emotions from that recording,
one where the speed increased in an attempt to maintain a certain level of positive emotions, and
a third where, in addition to increasing the speed of the game, it also decreased the speed in an
attempt to minimize negative emotions.
These tests, together, show that the emotional experience of a player is heavily dependent
on how tailored the game is towards that particular emotion. Additionally, in creating a system
meant to interact with these emotions, it is easier to create a one-dimensional system that focuses
on one emotion (or range of emotions) as opposed to a more complex system, as the system
begins to become unstable, and can lead to undesirable gameplay effects.