Pages

Tuesday, October 4, 2016

Experience-Driven Content Generation: A Brief Overview

Exploring and implementing methods for measuring experience, understanding and quantifying emotions and personalising users' experience have been the focus of my research for quite sometime. In this post, I will try to summarise some of my knowledge in this area.

The Big Picture

My theory is that: if I can accurately
predict user’s affective states at any point during her interaction with a
digital system, I can ultimately implement affect-aware methods that can automatically personalise the content leading to an improved and deeper user experience. I'm particularly interested in
studying these aspects within the computer games domain as I believe games are a rich medium for expressing emotions, an interesting platform for collecting, recognising and modelling experience and an easy to controllable environment for personalisation of content.

These ideas have gathered interest from numerous researchers working in interdisciplinary areas trying to solve parts of the puzzles. There is for instance a whole field of research trying to measure and quantify emotions; a relatively new, but very fast growing, field on automatic generation of content in games (there is a new book about this subject here); and a growing interest in linking ideas from these two areas so that we can build content generators that are centred around users' experience as a core component in the content creation process.

None of the above is easy and implementing a complete working system where all modules work together effectively is a big challenge. In my own work, I'm interested in realising the affective loop in games (see the figure below). I have a working implementation of what I believe a simple, yet easily extendable prototype of how the whole framework works in the game Super Mario Bros. (speed forward to the end of the post if you are eager to play!). My work revolves around revising and improving the different parts of the system so that the framework becomes general enough to be applied to predicting users' affects and personalising experience in any game (or more broadly, any digital interface).

The components of the experience-driven content generation framework.

Ultimately, we want the
system to be active (accurately choosing what information is important to learn from), adaptive (continuously
learning and improving), reactive (acting in real
time), multimodal (utilising information about the user from different
sources) and generic (working well in various applications). I have made a
progress along a number of these lines and I will be sharing them in individual posts that will follow. For now, I want to share insights about some of the main considerations towards realising the framework.

Features for Measuring Player Experience

If you survey the literature you will find numerous
methods for gathering information of users' experience or emotion. Here are the three main dominant types:

Subjective measures: The most obvious
measure is to ask players' about their experience. This method is the
easiest to implement, and hence the widely used, but it comes with a number of limitations (including
subjectivity, cognitive load and interruption of the experiment) as well as other
concerns related to the nature of the experimental design protocol. So, to
compensate for the drawbacks, other complementary or alternative methods are
usually employed to gather information from other modalities.

Objective measures: These are usually harder to control by the subject and therefore more reliable. Most of them also universal making them highly scalable. Your heart
rate, for instance, can reveal information about your excitement and your brain
activity can tell whether you are surprised, under cognitive load, or relaxed.
Your facial expressions can tell if you are happy or angry and your head pose
can tell whether you are engaged, attentive or bored. Information gathered by such measures is more reliable than the subjective ones but obviously harder to collect, annotate and analyse. Moreover, some of the equipments used for collection are quite intrusive that they can’t be
used in real-life interaction settings. Therefore, most of the widely used methods rely on accessible and unintrusive mediums such as web cameras and Kinect devices to analyse facial expression, extract gaze information and capture body gesture to infer emotion.

Interaction data: The interaction between the users and the digital interface also holds patterns that can help us understand users' experience. Gameplay data for instance, is a rich, easy to collect,
and reliable source of information for profiling players. By relying only on this modality, methods have been developed for predicting retention, progression analysis, discovering design flaws and clustering players for target
segment marketing and content customisation.

When it comes to modelling players experience in games, I believe a multimodal approach that combines and align information from multiple sources is the most effective. Gameplay data is the main source of information about experience that is usually analysed by most studies in academia and the industry. I believe information coming from relatively cheap sources such as the camera (which is already available in most gaming platforms) will soon become another standard for analysing emotion and improving the prediction of the experience, especially with the recent advancement in accurate real-time prediction of emotion from videos of faces. I believe in the not-too-distant future, there will be no need to ask users about their experience as other reliable modalities will provide accurate, less intrusive sources.

Facial reaction of players playing Super Mario Bros. when losing, winning and faced with a challenging encounter.

Methods for Feature Processing

The above types of features come with different forms: some are discrete numbers while others are sequences of temporal or spacial relationship. This means that different methods should be employed to handle each type and special care should be taken when combining different sources of different nature. For instance, gameplay data can be collected as discrete statistics about different actions taken or as continuous sequences of actions taken at each timestamp. Different methods can be applied in each case leading to various insights.

Discrete features are the most common and can be directly processed by most machine learning methods (they should of course be cleaned and normalised in most cases).

Features of continuous nature, such as objective measures of experience can be processed with methods that are sensitive to time-series data such as recurrent neural networks and regression analysis. Sequences can also be processed to extract meta data such as frequently occurring patterns. This can be done using frequent pattern mining methods.

Combining features from multiple modalities can be tricky especially if they are of different nature. The signals should be aligned and either transformed into the same space or handled on multiple resolutions. Take for instance a system receiving a continuous signal from a facial-emotion recogniser and discrete statistics about the keyboard buttons pressed. To combine such information, one option would be to process the continuous signal and transform it to discrete values of emotions calculated within specific intervals. Another option would be to handle each signal by an appropriate method and combine the results in a later stage. A third option would be to sync the features so that we can extract the facial reactions around each keyboard press action.

Methods for Modelling User Experience

You can imaging user experience models as magic black-boxes where you feed them with information about your users and the interface they interact with and in-return, you get useful categorisations or profiling that you can use for decision making. The input information can be one or a combination of the features presented above. The output can be an estimation of how fit the content is for a particular user, what profile best match the user (is she a buyer in amazon, a fighter in a first-person shooter, a puzzle-solver in an online course) or a recommendation of the best adjustment of the interface that will increase user's engagement.

Now any machine learning methods that can accurately estimate the mapping between your input and output can be implemented. The most widely used methods for profiling for instance are supervised or unsupervised clustering and classification methods such as support vector machines, self-organizing maps or regression models. Non-linear regression models are more powerful when attempting to predict affective states based on behavioural information. One can use neural networks, multivariate regression spline models or Random Forest to reduce the size of the search space while optimising the mapping functions. When trying to come up with recommendations or personalised content, efficient search and filtering methods can be applied such as collaborative filtering, genetic algorithms or reinforcement learning. There are many interesting applications for each approach and the choice of the appropriate method depends on the type of the data you have, the type of the problem you are trying to solve and the characterisation of the insights you are interested in.

Adaptive User Modelling

The experience models I talked about so
far are average models, meaning: they apply equally to all players and they are
not tied to a certain individual. They serve a very good purpose if we want to
ship them with the system and if we are looking for methods that work well with the majority of users. But
we can do better.

The accuracy of the models is very much
confined by the data used for training; your data need to be divers enough to include representatives of the majority of your users. Event when your data distribution is wide enough, it is very likely that the method will not recognise every individual. People are different and each of us has her unique preferences and ways of interactions. To accommodate for different personalities, one could implement adaptive systems that keep learning, improving, and personalising as the user interacts with the system. The models learned offline forms a good starting point for an initial rich experience and for learning more powerful personalised versions. Model improvement can be achieved through a brach of methods called active learning. Active learners attempt to improve their performance by sampling the instances from the space that lead to the fastest improvement. This means that they try to learn as much as possible about the user in the quickest way possible so that they become more accurate predictors of experience. Doing so, they also become more personalised for a specific user.

The Future

We live in a time where more and more data about the users is becoming available and where people from academia and the industry are eager to understand the users and make better decisions. We are also equipped with powerful methods that facilitate realisation of such goals. There are indeed a number of interesting research directions that can improve our understanding of users, emotions, behaviour and how emotion is manifistated through behaviour. Moreover, data-driven content personalisation is also a hot and interesting topic where lots of improvement could be done. I'm confident however that a lot can be achieved already with what we currently have in terms of data and methods.

Now just for fun, I will leave you with an example where many of the ideas presented are implemented to improve the experience of the interaction with the system.

Example: Content Personalisation in Super Mario Bros.

You will play an initial level of Super Mario Bros. and the system will collect information about how well you did. This information, along with your choice of the type of experience you would like to explore, are used by the system, using machine learning methods, to explore the space of possible content you would prefer. The best piece of content is then chosen and presented to you.

Let the fun begins! Try it here (PS. you will need a browser with a support for java, otherwise, you can download the demo jar file from here (option 4)).

References

If you would like to read more about the subject, there is a nice paper by Georgios Yannakakis and Julian Togelius that you can find here. The demo above is described in the paper here.