On how autonomous agents and flocking behavior became integral for an artistic interactive installation.

When you walk into Microsoft’s headquarters in Denmark, one of the first things you will see is this large, colorful, interactive wall. Squares seem to randomly float around, flock together, and even chase each other - their movements are constant and mesmerising. Step closer to it, and a Kinect device will detect your presence, and your outline will be mapped onto the screen. The squares become excited by your presence and make this wall a fun interactive experience for all to enjoy.

At Unity Studios we use the Unity® game engine for a variety of business-2-business solutions. We have found it works excellently across many platforms and currently we are spending a lot of time applying it to new and exciting technology such as the HTC Vive, Oculus Rift, and Microsoft HoloLens. In this case, we designed an interactive installation using a Kinect device for Microsoft Denmark.

Developing the Conceptual Design

The challenge was to create something that was interactive, but still doubled as art, and also reflected the building it was situated in – the building was designed by Danish Henning Larsen’s Architects and based its color-scheme on the four seasons of the year.

Variety was key for the installation, as we wanted it to be ever changing and always offer slight variations of what you see. Creating flocking Boids proved key to creating a mesmerizing experience for visitors.

Controlling Autonomous Agents

Now, if you have worked with autonomous agents that display flocking behavior before, then you are aware of how fiddly the process can be, especially when having to make sure to limit the processing power it will require to run. If you haven’t worked with flocking before, we recommend chapter 6 from the book "The nature of Code" available online for free here.

Autonomous agents exhibit self-directed behavior, so our job was to create certain boundaries for these behaviors and ensure the flocking looked organized, not chaotic. The organization is what partly ensures the installation becomes captivating to spectate. However, it was hard to predict the outcome of the boundaries we set, as each Boid received its own set of parameters (to ensure they don’t all behave identically). As such, their behavior when interacting with each other could be unpredictable.

So yes, chaos would sometimes occur and order had to be restored. If you like tinkering back and forth, this kind of project would be something for you. It really was a trial and error process.

The Autonomous Agents as a Mesmerizing Centerpiece

In order to improve the flocking behavior experience further, we decided to introduce predators. In short, after a set amount of time without human input, some of the Boids would transform into predators and in effect hunt the other Boids. This created some really interesting imagery and a far more satisfying result when viewing the installation for longer periods of time.

Moreover we wanted to make the Boids’ interactions with people interesting, and therefore made the flocking behavior react to people’s hands. The Boids were programmed to be interested, yet wary, of people's hands. Keep them still, and they will flock around them, wave your hands about and you will see the boids flee in giddy excitement. It lets you connect and interact with the Boids in meaningful ways.

As mentioned, the Boids reacted to a series of different inputs. For instance, when it is cold outside, the Boids will be larger and move slowly, and when it is becomes gradually warmer outside, they will begin to shrink in size and move significantly faster. Their movements will also be affected by the amount of people that are mapped - more people means faster movement. Time of day also affects our little friends. So in the morning, and when the sun begins to set, they become more sleepy, and therefore their movement slows down as a result. Art, in many ways, can be seen as expressions of feelings, and that is exactly what these Boids are all about. They express and reflect a mood, or state of mind, that is prevalent in people - depending on time of day, season, and company - establishing that connection between us, and the installation.

Art made fun

To put things in a slightly different way, the backgrounds and use of colors functioned as the foundation of the project's artistic style through its reflection of real world time, days, and seasons. The Boids added some life to our canvas. Their presence ensures that the installation can take a whole variety of shapes and expressions, and stay interesting for long periods of time; whether it is left alone for a while and begins to send predators out to hunt the flocks, or it detects a person and interacts, flocks around them, or even takes the shape of a game. That last item is meant quite literally by the way – you can play Pong on the wall.

Creating autonomous agents with flocking behavior really became essential to the development of the interactive wall - they became the linking element between the artistic installation and human interaction. The results are interesting and fun to play with. It can be a challenge initially to work with, but it's also rewarding. Here at Unity Studios we had a lot of fun messing around with this, and we think you might enjoy it too!