Stefan Rutherford

The game is released and I’m very happy with the result considering the time we had. If you get chance to play it look out for the audio in the hub (The open world representation of Middle Earth) as the audio for this was one of my main responsibilities on the project. I did contribute in several other places too though such as the music for IGN’s ‘item of the year’:

Thanks go to Luke Hatton for helping me out with this (that’s his mad geetar skillz you’re hearing!).

Thanks for reading. I’m starting my first ‘blog series’ soon which I’m excited about so expect to hear from me soon.

Recently my girlfriend and I visited the lake district and found an incredible walk leading up to a place called High Dam (pictured above, taken on a phone and does not do it justice!). After finishing work on my first title Lego LotR, it was just the break I needed. On the small hike up we followed a river which I of course proceeded to record.

The excerpt below includes just under half of the sounds included in the download. I’ve saved my favorite recordings for the download ;) (located at the bottom).

You can hear an excerpt of these recordings here:

The recordings are downloadable from here:

http://stefanrutherford.com/RiverLib_StefanRutherford111712.zip

You are free to use these in projects as you please. Do not re-distribute them as sound effects.

First of all, I’ve been incredi-lame with regards to contributing to the online game audio community recently. I am sorry. This is largely due to me experiencing my first ‘crunch’ … it has melted my brain slightly. Fortunately its re-congealing nicely.

I’ll post more details about the work I’ve been doing shortly, but for now check this out:

http://goo.gl/N1C0o

It is an extension of an earlier blog post I did and is intended to give sound designers and programmers a good background in the procedural methods that may be available to them. A springboard of sorts. Enjoy!

You can access the earlier post from here: http://goo.gl/8Gqrg
(includes some interesting discussion in the comments section from Lostlab and Nikunj Raghuvanshi, at least I presume it’s him he only signed it ‘Nikunj’)

Recently I did an interview with Creating Sound about how education helped me get into games. Check it out here: http://creatingsound.com/2012/08/sonic-backgrounds-stefan-rutherford/ The interview is part of a series which features some other great sound designers so be sure to check those out too! Special thanks to Bryan Ploof for sorting it all out :)

Okay, so it isn’t a real military uniform but the idea was to make it sound as much like a soldier fully clad with gear. I did this by borrowing a jacket (from my girlfriend) that appeared to be similar material to the cloth that the military use (used google image search as a reference). I then strapped two belts, which had nice clunky buckles, to myself. I attached a couple of leather pouches to the belts and I also wore a backpack loosely packed with some ‘bits and bobs’ inside.

It was recorded in stereo using two shotgun microphones (in an A-B pattern) pointed at my shoulders pointing at me from just above.

In some ways it is a shame I didn’t get a photo of myself so you can see what the recorded gear looked like… However, it is probably a good thing I didn’t, according to my girlfriend I looked like a total twerp, or at least she indicated this was the case by laughing at me from the control booth! Thanks to her for helping me record though :)

Download the files free to us in your projects here:

stefanrutherford.com/MilitaryUniformFoleyRecording_StefanRutherford_06172012.zip

There are plenty more variations of the sounds than can be heard in the example included in the download.

I’m very pleased to announce that tomorrow I shall be starting at Traveler’s Tales as a Junior Sound Designer!! I cannot wait :D

Okay so I’ve been at it again, smashing things up. Last week I was moving out and my flatmate came downstairs and said “This printer/scanner is broken, do you want it” … I already have a printer, I don’t need a scanner and besides I wouldn’t know how to fix it anyway. There was only one thing for it. IT MUST BE DESTROYED.

I also got hold of a big oil container drum thingy from the back of a takeaway. Can you believe that the takeaway owner was going to throw it away!!! Doesn’t he know the sounds that it would be able to make…

Here is a preview of the sounds:

Download them for free from: stefanrutherford.com/PrinterScannerOilDrum_29062012_StefanRutherford.zip

Use them however you like as long as you don’t sell them as SFX.

It’s been a while, for this I’m very sorry, but alas I have found time to blog something I’ve been up to recently.

I’ve been editing some of the material from Anton’s Wildeye trip for the game audio podcast and it has made me super excited about wildlife recording again. So, full of inspiration I ventured to Eccup reservoir with my girlfriend for a nice walk and opportunity to record some sound… I forgot the blasted batteries… I wasn’t going to let that deter me so I found a local convenience store and bought 8 AA ‘s. Ready!

I was really happy with what I managed to get and look forward to going back to get more material with some better mic’s. The star’s of the show were a bunch of baby blue tits (I think that is what they are!?) nested in a wall right next to the road. Got in close enough to get good signal to noise ratio without disturbing them.

After going to Eccup I went to a party with some friends. I stumbled in at 4AM and noticed that the dawn chorus had started, Great! got my recorder out and captured it from my window. I really like the reverberant tail of the main birds voice.

All sounds were recorded with a Tascam DR-100 (onboard uni-pair) at 48Khz 24bit.

Here are the results:

Download from here:

http://stefanrutherford.com/StefanRutherford_EccupResevoir_And_LeedsCity_NatureRecording.zip

Free to use just don’t sell them as SFX.

I’ll certainly be posting more stuff like this it has been great fun!

Introduction

Using the practical examples and referring to literature this post will deconstruct how sound design might be used to characterize a piece of film.

Background
Before discussing how characterization can be used in film it should first be understood why sound can be used to characterize events and actions in film.

When a person sees an action in the world and a sound accompanies the sound both in timing, direction, and distance a person naturally deduces the events to be intrinsically related.

This phenomenon is so strong that it translates to the medium of film. A viewer will simply accept this because sound is not naturally put under as much scrutiny by viewers as the visual aspects of film.

“While all viewers can tell apart the various objects in a picture - an actor, a table the walls of a room, listeners barely ever perceive sound so analytically.” [3]

Chion discusses the aforementioned naming it as synchresis:

“The forging of an immediate and necessary relationship between something one sees and something one hears at the same time (from synchronism and synthesis). The psychological phenomenon of synchresis is what makes dubbing and much other postproduction sound mixing possible”[1]

Because this relationship between visual and auditory events can be established in postproduction, sounds differing from the original source can be used to characterize the sounds. The sound can then be used as a tool to communicate information about an event it is connected with. According to the book Aesthetics of Film this can be three different things:

“The sound constituents may, according to the instance, reinforce the image, contradict it, or simply maintain parallel discourse” [2]

Using this knowledge it is possible to consciously inform a viewer of things they would not otherwise have known.

The following two sections will tackle sound for reinforcing events and for contradictory/opposing events. This report will not be studying the parallel discourse as it is not a device used for characterizing specific actions or events on screen at the moment of occurrence.

Reinforcing Events

In order to reinforce the information contained in a moving image the approach is simple, select appropriate sounds that do not interfere with the core meaning of what the visual action is communicating. When undertaking this methodology no extra information should be communicated.

The impact the aforementioned has on characterization is that it serves to simply underpin and/or strengthen the already present character of an on-screen event. In the case of sound driving a narrative event the sound used should only support the intended event as directed by the script or other such pre-confirmed direction. “Dialog and narration tell the story and narrative sound effects can be used in such capacity too, for example to draw the attention of the characters for an off screen event.” [3]

­Contradicting Events

In order for sound to express additional information than is already present in the film the sound must at some level oppose the visual event in some way. This disparity between event and sound should provide the viewer with two sets of information which then connect in a viewers mind creating one complete view.

Video example1. Note as the prisoner in the jail cell (Lord Blackwood) comes up behind the detective (Sherlock) the feeling suddenly turns very tense and threatening. When listening more carefully the sound that accompanies Lord Blackwood’s sudden appearance is a lion’s roar. The sound of a lion roaring calls upon an instinctual reaction of fear in the audience. Walter Murch discusses this idea further:

“The mental effort of fusing image and sound in a film produces a “dimensionality” that the mind projects back onto the image as if it had come from the image in the first place.”[1]

In other words the sound of the lion is not seen as a separate event, rather it is perceived all as part of one event in the viewers mind. The  viewers learned characteristics of a lion are then projected onto Lord Blackwood as a result.

Practical Study

To explore the idea of characterization using sound in film further practical examples have been constructed. The knowledge gained in the first half of this report will be applied to the examples thus further delineating what has been learned. Each attempts to manipulate the footage in different ways through the use of synchresis as a tool.

Video Demonstration 1

In the first video demonstration it was decided that the sounds should reinforce the events on screen and provide further emotional context only at key moments in the video. The sounds selected for the first half can be considered emotionally neutral. The punch sounds are weighty but carry no more information than the actions already taking place on screen.

At 0:07 seconds the video slows down as the boxer’s fist travels through the air to land the final blow. Here a subtle use of lion and elephant roars have been used as the fist is travelling through the air. This helps describe the present and impending danger. The sounds are only just recognizable as animal sounds working subliminally on the viewer. At 0:17 seconds when the winning boxer throws his arm in the air in celebration. Another lions roar can be heard here much clearer. This confirms the dominance of the fearsome victor likening him to a lion.

Video Demonstration 2

The second video demonstration took a very different approach. Through the use of synchresis it was decided that the sound should give the viewer completely new information that they would not have known otherwise. The theme ‘fighting robots’ was chosen.

Throughout the scene there is a lot of movements from each character. In order to create a convincing effect the most important action at any given moment was prioritized and suitable sounds were synchronized with the movements on-screen.

Impacts were replaced with ‘metallic clunks’, movements used the sounds of mechanical servos and for the slow motion punch the sounds of jet fighter planes were added. The sounds were pitch and volume automated to match the motions on-screen.

Bibliography

[1] Michel Chion (1976). Audio-Vision Sound on Screen. USA: Columbia University Press. p224.

[2] Jacques Aumont, Alain Bergala, Michel Marie, Marc Vernet (1983). Aesthetics of Film. USA: University of Texas Press. p65.

[3] Tomlinson Holman (1997). Introduction to Sound for Film and TV. USA: Academic Press.

This is the result of a mini-survey I did recently to figure out what game dev’s want from sound designers.

To gain an idea of the skills required by industry a selection of appropriate jobs listings were looked at. The required/preferred skills on the job listings were fitted into categories and presented as percentages.

Hope this helped as a ‘skills hitlist’ for people like me looking to get work as a sound designer for games.

It should be noted that many of the jobs may require/prefer skills not explicitly stated in the job listings.

Introduction

This post is a collection of ideas and conclusions that I have come to regarding the topic of procedural audio. The report refers to practitioners of procedural audio and attempts to propose possible next steps in the procedural audio movement for interactive games. I decided to make write this post after reading a recent interview with Andrew Farnell which can be accessed here. I thought to myself ‘Having these tools would be awesome!!’ But a question remained ambiguous. ‘How can we get them?’ So this is my stab at trying to define, as a sound designer, what we might need to do to get these tools.

If you are familiar with procedural content generation for audio skip to the section titled ‘Top Down’.

Background Context
This section  of the report will define procedural audio and then outline two possible approaches to procedural audio content generation (the ‘top down’ and ‘bottom up’ approaches).

­Procedural Audio

Procedural content generation can be defined using the following definition:

“Procedural refers to the process that computes a particular function, procedural content generation [is the process of] generating content by computing functions.” (Nicolas Fournel, 2010). Procedural audio can therefore be seen as the process of producing audio using the aforementioned method.

Currently most audio in interactive games requires some form of sample playback. This method requires audio data to be stored on disk and any pieces of audio required to be played back at any time (ie needs to happen quicker than can be streamed from disk) such as footsteps or gunshots must be stored in RAM. The Game Audio Tutorial makes the following point:

” Some of your sounds will play off disk, where they still have to compete for space with all the graphics assets, but many will need to be loaded up into RAM to be able to play instantaneously when needed. Ram costs money. Ram is limited.” (R. Stevens, D. Raybould, 2011).

If the audio was produced procedurally there would be little, or no requirement for audio data to be stored In RAM.

The other problem with ‘data driven’ approach to audio is the fact that the sounds can only be as reactive as the processing/layering allowed by the audio data produced by a sound designer.  Procedural audio however can be generated according to the exact conditions within a game.

Nicolas Fournel summarizes when procedural content generation is useful:
“Procedural content generation is used due to memory constraints or other technological limitations.” He continues and states it can also be “used when there is too much content to create, when we need variations of the same asset and when the asset changes depending on the game context.” (Nicolas Fournel, 2010).

Top Down (analysis  and re-synthesis)

The ‘top down’ approach involves analysis of sounds and then various methods of synthesis to either partially or fully reconstruct that sound. This method could also be considered as ‘data driven procedural audio’.

Nicolas Fournel describes this method:
“Top down, you analyze [the] example of the sound you want to create and you find the adequate synthesis system to emulate them.” (Nicolas Fournel. 2010).

This method allows a sound designer to create an asset such as a gunshot sound in the normal way. The sound is then synthesized using appropriate methods of synthesis. The advantage of this is that artistic control over a sound is maintained whilst many of the benefits of procedural content generation are still received.

Bottom Up (Physical Modeling)

The ‘bottom up’ approach uses physical modeling to synthesize a sound from scratch below is a discussion of the advantages and disadvantages of this approach.

“Sound is, in the end a physical phenomena. So you could go and simulate everything on a machine. This gives you lots of automation.” (Nikunj Raghuvanshi. 2011). Essentially the sound is created according to the physical model in the game, this allows sound to react according to the exact conditions in game at any given moment. Nikunj continues to state the disadvantages of this approach:
“It takes a lot of computing, you could not [currently] do this at run-time.” he also states another disadvantage “There is no space for any artistic control over sounds that are produced or propagate.” (Nikunj Raghuvanshi. 2011).

If we refer back to Nicolas Fournel’s definition of Procedural content generation uses stated earlier.  This approach best serves the advantage: “asset changes depending on the game context.”

Andy Farnell provides another reason as to why this method could be very useful. The “problem is how to provide the colossal amounts of content required to populate virtual worlds for modern video games.” (Andy Farnell. 2007). Farnell also discusses another advantage of this ‘bottom up’ approach:

Another advantage of this  stated by Farnell is the following: “One of the great advantages is that it gives 90% of your assets for free. You just put your objects in the world and you get default sounds” (Andy Farnell. 2012).

Comparison and Conclusion

In this section each method will be comparatively discussed attempting to define how each can be used and what the possible ‘next steps’ for procedural audio in interactive games may be.

As stated earlier Nikunj Raghuvanshi states that the bottom up approach requires loss of “artistic control.” (Nikunj Raghuvanshi. 2011). However Andy Farnell argues that this does not have to be the case with a physical modelling based method.  “Every Procedural Audio team would need a good sound designer.  I wouldn’t leave it to the programmers, I want somebody who has a great set of ears and I would actually put them in a higher position and get them to direct the programmers and say, ‘No its more like this, listen to these examples. I want to get this emotion across’, and they can direct it aesthetically.” (Andy Farnell. 2012).

As mentioned in the ‘Bottom Up’ section of this report Farnell believes that you could get a lot of sounds without having to create the content from scratch. The sound designers would then be liberated and able to fine tune the sounds, thus maintaining creative control. 

The implication of this method is it requires a complete re-structuring of how audio is created for games. In other words the actions performed by a sound designer to achieve their roles as designers of sounds will be completely changed.

The top down approach can provide at least a temporary solution to this quandary acting as a step in which current sound design methods can be met with procedural techniques. This allows sound designers to create the hyper-real, artistically controlled sounds game developers are used to with some of the benefits of procedural audio. This, I believe, is an important step towards getting game developers to accept these approaches. As Farnell states: “it dawned on me that there weren’t any fundamental obstacles to radical technical progress. We could do this. The obstacles were structural and political. How do you introduce a new technology?” (Andy Farnell. 2012). Currently the game industry has a highly developed process of audio content creation if the bottom up approach to procedural audio is introduced hastily sound designers responsible for the quality of the audio will not be able to reach the potential and perceived quality of audio currently in games. This is due to a lack in education regarding procedural audio. Therefore the education must be a gradual process and the top down approach could feasibly begin providing sound designers with this education.

Before physical model based procedural audio can be practiced in games sound designers must be able to reach and surpass the perceived quality of game audio using those techniques. As the Game Audio Tutorial states: “You will have to constantly convince people of the importance of investing in sound and music.” (R. Stevens, D. Raybould. 2011). In order to convince them it must at least sound as perceptually good as current sound design, with the additional benefits of procedural content generation being the selling point. If procedural techniques are proposed to developers before they are up to scratch it could damage future efforts when trying to introduce such systems.

To allow movement towards the physical model based approach programmers with an intimate knowledge of sound design and the physical creation of sound must begin contributing synthesis models to the community allowing sound designers to tweak and get the best from such systems.

Summary

·         Top down approach fits in well with current sound design practice.

·         Procedural approaches should only be proposed (as commercially viable solutions) when they sound as perceptually good as the current data driven approach with the added benefits of procedural content generation.

·         Synthesis models should be shared allowing sound designers and audio professionals to learn and make the most out of them.

Thank you for taking the time to read this collection of thoughts and information. I’d love to discuss things with people about this. You can email me (s.rutherford8566@student.leedsmet.ac.uk) or use the comment box on this post (linked top left of post).

For more information on procedural audio Fournel has put together the wonderful:
http://www.procedural-audio.com/

Bibliography

Andy Farnell. (2007). Synthetic game audio with Puredata.

Andy Farnell. (2012). Procedural Audio: Interview with Andy Farnell. Available: http://designingsound.org/2012/01/procedural-audio-interview-with-andy-farnell/. Last accessed 21/1/2012.

Nicolas Fournel. (2010). What is Procedural Audio?. Available: http://www.gdcvault.com/play/1012704/Procedural-Audio-for-Video-Games. Last accessed 20/1/2012.

Nikunj Raghuvanshi. (2011). Sound Synthesis in CRACKDOWN 2 and Wave Acoustics for Games. Available: http://www.gdcvault.com/play/1014416/Sound-Synthesis-in-CRACKDOWN-2. Last accessed 19/1/2012.

R. Stevens, D. Raybould (2011). The Game Audio Tutorial. UK: Focal Press. p33, pxvii.