Surround Music in video games – we consider the viability and aesthetics9 min read

Considering the Aesthetics of Surround Music in Games- By Rob Bridgett

Surround Music: The Emperor’s New Clothes?

There has been much talk recently, predominantly among composers, dedicated to the virtues and benefits of surround music in video games. Certainly with the increased memory capacity of next-generation platforms, and a greatly increased install-base of surround sound systems, the prospect of surround score and licensed music seems more feasible than it was on previous generation consoles. While this exciting expansion in the dimensions of the video-games medium offers some fantastic opportunities and new horizons for music in video games, there are also pitfalls which should be carefully considered when designing a game’s soundtrack with surround music in mind. Leaving aside any technical aspects of surround music for games (these have been discussed in depth and frequently elsewhere), there are some pertinent questions that need to be asked of when surround music is useful and when it is a needless distraction.

Diegetic and Non-Diegetic Boundaries

For certain types of games, the notion of surround sound music offers some very particular spatial and aesthetic challenges. In 3D first or third person perspective games (mostly those games which aim to imitate a ‘cinematic’ sound model), spatialization of music can break the ‘non-diegetic contract’ between the space occupied by score and sound effects. Non-diegetic music, or score, is not a part of the game world (diegesis). The score is not expected, for example, to get effected by environmental reverbs and it’s volume or position is not attenuated according to where the player/listener is positioned. So the spatialization of various instruments or textures within a piece of score, particularly to the rear of the surround-field, can be mistakenly read as a sound effect that is positional in the game-world by the listener. This is due in part to the tonal similarity of much ambient sound design to certain elements of an orchestral score. The same would be the case if the game were set on board a space-ship and the score employed positional ambient electronic bleeps in the rears, there would be a high likelihood that the listener would be unable to determine what was a diegetic sound effect and what was a non-diegetic positional part of the score. If the style of the music and the style of sound effects are too similar, then this spatial confusion can easily arise.

Distraction From Action and Immersion

Surround sound has a unique function in 3D games as it allows for navigation and essential off-screen action and space to be communicated to the player. Sound effects such as gunshots, dialogue and footsteps coming from positional points actually play a huge role in helping the gamer to play the game and navigate the 3D space effectively. In cinema a very different set of rules apply, anything too prominent in the surrounds is considered a distraction from the screen, essentially having your audience looking behind them towards the exit signs in the theatre to figure out what that sound was. The use of surrounds in film, often tends towards ambience to softly envelop the listener within the diegesis, essentially using sounds that are non distracting. This is a clear, tried-and-tested formula of maintaining immersion in cinema. Similarly, within these specific types of games, anything in the musical score that can be misread by the listener as a sound effect that is physically located behind them, can distract the player and draw their attention to the music.

Not only would music that is making prominent use of surround channels be confusing to the player who is attempting to use the surrounds for navigation and for knowledge of enemy positions etc, it also has great implications for the final mix of all the elements of the game’s soundtrack, particularly sound effects and dialogue. With music potentially taking up a wide band of frequency in the rear-field as well as the front, this introduces more clutter into the overall mix of sound effects, dialogue and music and places further demands upon interactive mixing (an area that still needs to make huge leaps technically and artistically in order to catch up with the quality of cinema sound) to compensate and allow the listener to clearly hear dialogue and sound effects at key moments without being overwhelmed by the score.

Where Surround Music Will Work Well…

This is by no means a definitive list, but there are of course many exciting areas where surround music can be exploited in ways that are unique to interactive games and that look away from the cinematic ‘film-sound’ model. In 3D games, a score that remains as ‘underscore’, and remains in the ‘ambient realm’ could work well in a 3D environment. Provided there is nothing to draw the player’s attention to the rear field, nothing sudden and unexpected in the score that reads as a sound effect. It is therefore essential that communication between the composer and the sound director occurs to figure out exactly how much activity is required in the rear-field in terms of the score. Again, anything too prominent, and unexpected, runs a high risk of being read as a positional sound effect by the player.

Positional sound effects are useful to the player, a cowbell positioned in the rear field of the score is not. Having said this, there are also occasions where the diegetic and non-diegetic ‘contract’ can be exploited. Having surround elements in the music deliberately ‘read’ as positional could be used in particularly scary or quiet moments in a game, such as a survival horror, to create some very tangible aspects of tension and confusion. “Was that a sound behind me? Ah no it was just the music! phew… oh damn” etc. It all depends upon the requirements of the game that are placed upon the sound, music and dialogue.

Racing games in particular also offer significant opportunities for use of surround music in games as so much of the action in a racing title is focused on what is directly in front of the player. There is also a direct real-life example and sonic analogy that can be drawn between the player and the listener as though they were sitting in a real car and listening to a surround sound piece of music (from speakers actually placed in the front and in the rear of the vehicle). The only sound effects that the player is also likely to be interested in coming from the rears are the sounds of other cars, not in themselves entirely subtle or hard to mistake for elements of the surround music. Not only this, but the style of music expected in a racing game, that of high-octane, electronic adrenalin-pumping beats, is completely different to a ‘score’ and can be read and understood more clearly by a listener as ‘positional electronic music’ as opposed to ‘positional sound effects’. It again comes down to a clear difference between style of music (electronic) and style of sound effects (engines).

Another good example is in 2D environments where there is no suggested diegetic rear space for sounds to occupy. One such example is in front-end menus, there is a good opportunity to use surround music there to fill the rear field space. In 2D games there is also a great opportunity to explore the space created with surround music, because, again, there are no implications for confusing 3D sounds with any spatialized music that may be heard.

A third example is diegetic, positional music, i.e. music that is emanating directly from the game world, for example a live band playing in a nightclub. The sounds of the separate instruments could be positioned in 3D at their exact points of origin on the stage, thereby when the player walks around, the balance of the music will change accordingly. The player could even get up on stage and among the musicians to hear the vocalist in front, the drummer coming from the rears, guitar and bass from left and right respectively. There could even be specific audience reactions at a particular moments in the performance positionally coming from the audience space. An 8-channel interleaved stream, with each individual channel mapped to a particular position would cater well for this implementation.

Planning Ahead for Surround Music

Deciding on exactly where each element of the soundtrack is to be placed and what belongs to the scene (source sounds / music) and what comes from ‘beyond the scene (non-diegetic) is a critical decision to be made up-front by the audio director in conjunction with the game designers before the composer begins their work. Also determining the ‘style’ and timbre of sound effects and music will help to separate any confusion as to what is music and what is a sound effect. Having a composer write a surround score with specific uses in mind from the beginning will have a great many benefits from suddenly deciding that the music should be up-mixed into surround format when it is all complete. Finally, the type of game will greatly inform many of these decisions, but also thinking about the specifically designed functions of the music within a particular type of game will help to define further creative strategies for the use of surround music in games, and eventually make games a more immersive and interactive experience for gamers. Any decision whether or not to commission a surround score, or surround licensed music content , must first consider all the elements of the finished games’ soundtrack together as a single final entity and the effects that this will have on the player.

About the author: Rob Bridgett (of www.sounddesign.org.uk)
was one of the first to complete the Master’s degree in Sound Design
for the Moving Image at Bournemouth University and is currently Senior
Sound Director at Radical Entertainment in Vancouver. Work for games
includes… 50 Cent: Blood on the Sand (2009) (sound director) ,
Prototype (2009) (sound mixer & cut-scene sound designer) , Crash:
Mind Over Mutant (2008) (sound mixer) , TimeShift (2007) (cut-scene
sound designer) , Crash of the Titans (2007) (additional sound design) ,
World in Conflict (2007) (additional sound design) , Scarface: The
World Is Yours (2006) (sound director) , Sudeki (2004) (VG) (additional
sound designer) , Serious Sam: Next Encounter (2004) (sound designer
& composer) , Vanishing Point (2000) (sound designer).