Site index page
Simon N Goodwin's Codemasters softography, 2003..2015, updated January 2017
This page collates background information about my work in the Central Technology department at Codemasters since 2003.
Codemasters Central Tech
Brian Lara Cricket
Colin McRae DiRT
Colin McRae DiRT2
Formula One 2010
Formula One 2011
OFP and Bodycount
DiRT3 & DiRT Showdown
F1 Race Stars
Colin McRae Rally mobile
Colin McRae Rally 2005, PS2/Xbox/PC, Codemasters 2004
Brian Lara Cricket 2005, PS2/Xbox/PC, Codemasters 2005
For more than a decade I worked primarily on sound and streaming data systems for
leading UK publishers Codemasters. I made early contributions to several of their
football titles, where my voice codec enabled the 150,000 speech samples
used in the commentary system to fit the DVD distribution without the
quality and efficiency problems caused by standard Windows formats,
and on the 2005 incarnation of their Colin McRae Rally franchise,
where I worked on the cross-platform game audio and wrote a custom
nine-stream DVD audio player on the PlayStation 2 for stereo weather
effects, Dolby Pro-Logic 2 ambient sounds, and co-driver commentaries.
Rally did well in the UK console charts, despite the contention for the
Christmas season, reaching number 6 on PS2 and Xbox soon after launch and
remaining in the top 20 three months later.
I've also picked up credits for audio and streaming optimisation work on several games published by Codemasters, including Worms Mayhem and
Brian Lara International Cricket, both developed for PS2, Xbox and PC, and published in 2005. Brian Lara Cricket spent four weeks at number 1 in the UK all-formats game charts in the summer.
Since then I've been working on platform-specific audio systems for the new Brian Lara Cricket and Colin McRae Rally games, performance and reliability improvements for external publishing deals, and several internal projects. My most recent technical credits were for work on a couple of rather visceral titles: the Clive Barker game Jericho, published in 2007, and 2009's Operation Flashpoint: Dragon Rising, where I led audio aspects of the pre-publication production engineering, as part of the Central Tech optimisation effort.
Oddly enough the head honcho at Codemasters is now Rod Cousens, who ran
Quicksilva twenty years ago when I designed the Magic Micro Mission game for
them, and David Gower (who appeared, like me, on the associated TV series)
provides commentary for our cricket games. And the
artist who painted several package covers for my programs in the 1980s now
runs a company developing a radically new type of game based on my proof
of concept and optimised Digital Signal Processing systems (see later). So
while it's a much bigger industry, it seems the company names change much
more often than those of my associates on the UK game development scene.
One of the first things I was asked to do when I joined Codemasters was investigate a proposal
for a PS2 rhythm-action game that worked with the player's choice of music, from CD - a bit like a
cross between Vib Ribbon and Dance Dance Revolution. Within ten weeks I'd coded a proof-of-concept
that could read and replay CD audio using a bunch of threads on the IOP (the IO Processor which
masterminds disc access, memory cards, controllers, and PS1 emulation) and pass data blocks up
to the EE - the PS2's main 128-bit multiprocessor Emotion Engine - for filtering and analysis.
This prototype was able to extract beats and interpolate between them on a select few CDs,
scrolling various views of the track across the screen with beat times overlaid. I demonstrated
this to company founder Richard Darling and Chris Southall, our CTO at the time, and was given
the thumbs up for further development, as a collaboration between Codemasters Audio, Graphics,
QA and Central Tech departments, with one of the third-party firms that are the mainstay of
UK game development. But who could do it? After casting around for developers with relevant
experience, we hit upon Broadsword Interactive (run by Dave Rowe, the artist mentioned above).
Their lead programmer Jim Finnis asked several intelligent questions, and got the commission.
More than two years and many experiments later, Dance Factory was released in Europe, Australia
(where it was still in the PS2 top ten six months after launch) and North America.
I've got two credits - for the Proof of Concept and Signal Processing - and been involved,
along with Jim and his sidekick Steve Rose, who coded the dance generator, in most aspects of the
game design, development, and related patent applications. I ought also to laud the sustained
efforts of producer Dave Brickley, without whom such an original game would surely never have
reached the shops.
I could probably write a book about the development of Dance Factory, but I'm not sure
whether many people would read it - the game is more approachable! However there are lots
of interesting things to say, which reveal special things about the way this game was made
as well as insights into the 21st Century development process. I've deliberately skipped
details about the way the game works, because those are proprietary to Codemasters and
subject to patent protection. Contact our corporate lawyer Julian Ward if you want to
know more, especially if your company is interested in licencing the unique technology, e.g: United States Patent 7528315, which was granted in May 2009.
With its open-ended support for CDs the developers never heard - including ones unreleased at
the time we wrote the game - Dance Factory required an exceptional QA and testing effort. Each
time a new build of the analysis and dance generation code was available, Dan Flannigan's dedicated
QA team had to dance their way back through our test suite, consisting of more than a thousand songs,
ranging over 40 years and many genres, to check and rate the expected improvements and make sure
those had not introduced problems on any tracks that previously worked well. However big the
test suite, there's always going to be lots we've never tried, so our challenge was to ensure
sufficient variety of coverage to make sure that Dance Factory has a good crack at anything
players throw at it - with the possible exception of "Let's speak Spanish", a language course
one magazine playfully sumbmitted for dance generation - if that's really what you want to dance
to, you'll get what you deserve.
The challenge of testing and refinement explains why Dance Factory spent three years from
concept to release. Within a year or so we had it working reliably on typical modern dance
music with a strong, regular, beat, but we decided this did not live up to the potential of
the concept and went back into a fresh research phase, initially with no limits on the
RAM or CPU time for analysis. To get the dance quality we wanted on classic tracks,
as well as sequenced ones, we deployed all the resources of a Sony T1000 devkit -
originally marketed as a PS2-based super-computer workstation, with four times the
main RAM and IOP (32 bit Input/Output Processor) memory of the stock PS2 - and let
the multiple processors spend twenty minutes processing each song, digging out all
they could about not just the the beats but the structure, patterns and tempo changes
in the track.
I then production-engineered that 'optimised C++' code, rewriting it in machine
code to run across multiple CPU pipelines, shoehorning it into a third the space
and speeding it up to do the same job - and better - about FORTY times faster.
This exploited the parallelism of the PS2 processors - where nine-tenths of the
processing power is in the vector units, rather than the 128 bit MIPS R5900 CPU
that runs generic code ported from other platforms - to the point where Dance
Factory can generate dances at about seven or eight times real-time. So a
typical CD track can be analysed in around half a minute - less than the
loading time for many PS2 game levels - while at the same time Dance Factory
runs the Cubric minigame, with 24 channel music and sound effects. So
players don't have to wait around, even if they want to generate dances -
including optional Eyetoy moves - for a whole compilation album at one go.
Once this analysis has been performed there's no need to do it again; a map
of the album is stored on the memory card, labelled to identify the CD and
recognise it - or another copy of the same disc, parhaps on a friend's PS2
- when you want to dance to the same disc later. So the deep analysis only
has to be done once; after that the memory-card data can be used to generate
a range of synchronised dances with varying difficulty levels and features
like fitness (calorie counter) and creature (dance avatar) modes, within a
few seconds of re-inserting the disc. Even if you don't have the original
memory card, the dances for a given setting are consistent and repeatable
- a requirement established from the start by Richard Darling - so anyone
else with the same CD, a PS2 and Dance Factory can learn the same dances
and compare scores with you later.
The official line is that Dance Factory works only with mass-produced CDs,
but many players have found that it copes just as well with their private
compilations, burned onto CD-R media. Codemasters can't guarantee this,
as they don't control the quality of the media or the recorder, but it can
be a great way to enhance the 'endurance mode' where you dance through a
whole custom album - potentially a seque of dozens of songs, never dropping
a beat. Unlike arcade-based rhythm action games, with their simplified,
cut-down remixes, Dance Factory does not limit the length of a dance. It
copes with whatever you throw at it.
Dance Factory is paradoxically one of the smallest games ever on PS2 - it all has
to fit in memory, which explains the simple background graphics, though those
build up as you play and respond to the beat, unlike simpler light-synthesisers,
and had to be chosen to provide variety without overwhelming the display of
arrows, scores and key information needed to play the game. Graphics also have to
compete for memory with the dance database - the library of varying-length dance
step sequences, customised for each tempo and time signature - and the CD analysis.
Dance Factory includes 29 graphic themes, some of which you see from the start
and others which are unlocked through play. You never need to re-insert the game
disc; we decided that would make the game too fiddly to play. Yet in another
sense Dance Factory is by far the biggest game ever released on PS2 -
with terabytes of 'add on' content available, on tens of thousands of
commercial CDs, not to mention home-made ones. So the unusual features
extend beyond the USP, pervading the game design.
The music player, used in the front end and during the cubric mini-game, runs
entirely on the IOP, leaving all the main memory and 'Emotion Engine' processing
power for analysis and dance generation. It's a supercharged version of the sort
of 'tracker' module player popularised in the 16 bit home-computer days, using
similar techniques to build up tunes from short samples - but this time with
24 voices each positionable in stereo or ProLogic surround, rather than just
four hard-panned left and right, and room for more samples and high sample
data rates than an Amiga could manage even if all its 'chip RAM' were to be
allocated to music. And to make the best of this revived technology, Codies
gave Amiga veterans Allister Brimble and Tim Bartlett the job of producing
the music and sound-effects; the module player is an updated version of
code Sony sound guru Jason Page wrote for the original PlayStation.
Sample Rate conversion
Another key decision involved the handling of audio in the game. Apart from
the optional combo and level change sounds, chosen to fit in with all sorts
of CD track and with their own volume control, it's just your favorite music
that you hear as you're playing the game. So the CD audio replay quality is
crucial. Whereas other games typically hack and compress custom tracks to fit
them onto the game disc, we were determined to deliver the original 16 bit CD
stereo without compromise. In Dance Factory it's the music you choose and love.
CD Audio runs at 44,100 Hertz - 88,200 samples per second, one each for left
and right speakers - yet the PS2 hardware - unlike that of the original
CD-based PlayStation - works at a slightly higher rate, 48 KHz.
Without special processing, the original music would play a semitone or
so sharp, and faster than expected - much as film soundtracks do when
broadcast to a PAL television - on the PS2. Some people might not notice,
but we didn't want to mess with the pitch or the timing of your
The maths gets rather hairy; for every 147 samples that come from the
CD, the PS2 outputs need 160 that follow exactly the same curve but at
a higher rate. The spacing between CD samples and PS2 ones wanders
back and forth 300 times a second, and the ear is very sensitive to
any 'jitter' that results from this variation, muddying the sound.
The PS2 console's own CD player uses the main processor to
resample audio, but it doesn't have to run a game at the same time -
in fact the built-in player animates little more than a single cube. So
another key bit of code I contributed to the project oversamples the
CD audio up to short-wave radio frequencies - over 7 MHz - generating
a precise stream of 32 bit floating-point data containing all the samples
at both the CD rate and the higher (DAT-based) PS2 rate, mapping exactly
between the two formats, for both left and right channels.
Unlike the analysis and dance generation, this is fairly standard
signal processing theory - though the choice of approach to this
quickly sorts out sheep from the goats, among audio programmers
- the challenge is to do it reliably, with high fidelity yet
negligible load on the console, which needs to spend most of its
time rendering and playing the game, not fiddling with samples.
Another challenge was to give players an easy way to enter song and
CD names, since there's no text recorded on a standard music CD and
network look-ups don't work for typical PS2 users or any custom CDs.
In theory you can plug a USB keyboard into a PS2, but few players
have one and they require custom driver code and support for dozens
of different key layouts in the world-wide PS2 market. Besides, we
wanted to focus on dancing and gameplay, not peripheral shuffling.
The initial approach to text entry worked like a 'high score' table,
with the arrows on the dance mat or PlayStation pad moving a cursor
over a grid of characters and another press or step confirming each
entry. This is OK for short names, but tortuous for a whole album of
song titles, especially after allowing for punctuation characters,
regional accents, and other glyphs likely to appear in real song lists.
After careful analysis of several tech entry schemes, including the
many variants used on phone dials and keypads, I came up with an
approach that was easy to learn, three or four times faster than
the obvious alphabet-grid, and easy to customise for foreign accents
and punctuation variants.
The steps for some letters match those typically assigned to digits 6,
7, 8 and 9 on a phone keypad, but using the dance mat rather than a
phone pad. Since there's no button in the position where '5' appears
on a phone, the other letters are mapped to the four other dance step
directions, in the same groups - which are simply alphabetic, with three
or four letters to make the whole Latin alphabet fit on 8 buttons - used
in the text-messaging ubiquitous among the target audience. So familiar
sequences of taps - or steps - can be used to enter CD details, as if
they were part of a telephone text message, with the layout adjustment
necessary to suit feet rather than fingers.
Even then, special techniques were needed. The start and select
buttons - the only others on all dance mats - were allocated to
delete and enter respectively. To keep the average number of steps
down - and hence the speed of entry up - pairs of characters were
allocated the same position, when this made sense. In the English
layout you start with an open-bracket symbol; once you've entered
one of those (and are writing text in brackets) the same gesture
generates the matching closing bracket, without using up another
grid position. This compromise precludes entry of song names
with (nested (brackets)) or starting with a closing parenthesis
- ) we decided this was acceptable for songs, though it might not
be for programming!
This is a minor benefit when using the simple unaccented English
alphabet, but very handy for other languages, where the characters
normally used for quoting text are different before and after the
quote, or in Spanish where questions and exclamations are similarly
bracketed. We ruled out the idea of making one grid with all the
characters for every language on it - that would just be lazy -
so the initial choice of language selects a custom set of extra
characters. French players don't have to skip over umlauts to
find the accents they want, and tildes, circumflex accents and
other national variants appear only when needed. Accented
characters are allocated by vowel and diacritical mark to
specific rows and columns in a logical way that makes it
easy to learn (whether you notice the organisation or not).
Other characters are in the same place for all languages.
Again, this involved some compromises - relatively obscure symbols,
such as the Welsh accented 'w' (sorry, Broadsword!) are absent,
CAPITALS and lower-case letters are equivalent (throughout the
game) and if you want to enter a name with characters not native
to your locale you may need to switch language setings to do so
- but I think we've acheived the aim of having a consistent way
to enter all the text most likely to be needed, in a familiar way
that's easy to learn, with no special tricks or extra steps (sic).
Once you've entered the names for a disc you can share the
game save file with your friends, along with the analysis, so
they can get straight on with the dancing!
These details help to make the game viable and fun, but could detract
from the main point - a game that makes fun dances from your own CDs.
Luckily most of the reviewers - apart from a few DDR diehards for
whom any changes from the arcade formula would be unwelcome - picked
up on that, so Dance Factory has had some great reviews:
BBC National Radio 1 (Sarah Cox show, 11:53 am, Tuesday 2006-08-15)
"Dance Factory on PlayStation 2, Now this is brilliant! Dancing games - these ones you
buy the dance mats for - Dancing Stage has been the key series for years. And what you've
never been able to do on it is use your own music.
And this time, for the first time, you can take any CD in your collection, put it in the PS2,
and it generates dance steps for you on the fly, or you can make your own. Now, I've been
trying this out at home, and ... it works really, really well..."
[Cox, giggling]: "Tried Radiohead or something?" [Minkley]:"...I tried it with Paranoid
Android and it threw it a bit, but then I actually went in and I worked out my own dance
steps along to Paranoid Android and it's brilliant! ... You can do your own steps, it will
record it, then you play it back and you're just bouncing around to all the guitar bits
and stuff and it's just superb fun.
So if you've ever liked this type of game before, but you've always felt restricted by the
songs on there, I mean, this opens up your entire record collection ... it's just brilliant,
this is really really good fun."
Veteran games journalist and editor of Eurogamer TV,Johnny Minkley
Full review (MP3 audio format, 350K)
Game Informer (Leading US magazine total paid circulation 1,934,859 in 2005)
I'm going to go out on a limb here and offend a lot of DDR purists and tell you
that this is the dance/rhythm action game you should play this year....
So what's the deal? Dance Factory can take any CD you put into the PS2 and
transfer all or any of the songs to danceable tracks - and I'm not talking
cheap, non-rhythmic versions. The newly created dance tracks firmly match up
with the song rhythms, especially if you find music you'd actually dance to
in real life. The feature works great, and often creates some stellar fun
from music that (gasp!) you'd actually want to listen to! Score: 8/10
The Times (UK national newspaper, September 16, 2006)
Although they are guaranteed to deliver a few laughs and lots of energetic posturing,
the failing of most dance mat games is the dire selection of twangy hits licensed for the
experience. Even Dance Factory comes with just five saccharine offerings, but then turns
the genre on its head quite magnificently by allowing players to import their own CD
tracks for fancy footwork treatment.
It is not an especially fast process, with a four-minute track taking about a minute to
convert for dance mat use, but the options that it brings are limited only by the tracks
in your CD collection....
The package also boasts some nifty secondary ideas to help to extend the title's
replayability, such as a Fitness mode for calorie counters. Meanwhile, the multiplayer
games include a knock-out tournament that can accommodate 16 players in rotation. An
Tim Wapshot Score: ***** (5 stars, the maximum)
Gaming Age (online magazine)
No matter where your loyalties lie in the DDR world, there is no denying that Dance
Factory is taking the right steps (no pun intended) in designing new ideas that the
group of thinkers at Konami haven't come up with yet.... What Dance Factory brings to the table
is new features that improve the genre, along with recreating already popular and proven
modes for fans of the series.
What has gamers coming back is the music and the basic fundamentals of addicting game
play. Dance Factory will not let you down as this game is every bit as fun and enjoyable
as DDR, but it is the features that will make you go hmm...
Yes, Dance Factory has a solo mode, multiplayer mode with elimination and tournament
rounds, and even a cardio calorie-burning mode for the vain and/or overly sensitive. What
sets this game apart is the track listing. You have five songs to choose from including
"I Like It, I Love It" from Tim McGraw, "Get Down On It" from Kool and the Gang, "I Like
the Way" from Bodyrockers, "Pon De Replay" from Rihanna, and "Don't Cha" from the
Pussycat Dolls. With a track listing like that, how long do you think it will take before
you are sick of these songs? Well fear not true believers, as the developers are not that
lazy or stupid, in fact they are geniuses as they have implemented an option yet seen in
a dancing title until now.
With Dance Factory, you can use ANY music CD as your track listing. Think of it as
Monster Rancher for the Dancing Genre. Your dance beats, backgrounds, and routines
are generated to fit the song you have loaded into the PS2...
If it has a beat, you can dance to it. This allows for the most eclectic soundtrack
known to man, as the possibilies are limitless. Sure, we all know some of these bands
are not really made for dancing, but at least no one can tell you that it CAN'T be done
Brian Peterson Grade: B+ Great
GameSpot.co.uk online review
Finally a dance game where you can dance to your own favorite music.
Dance Factory features a CD recording mode that captures the beat of the song and creates dance
patterns to it. You can use original CDs or your own burnt compilation CDs of your favorite
tracks. It takes about 10 minutes for the software to create patterns out of a whole CD...
It can generate 1/1, 1/2, 1/4, 1/8 and double steps; no 1/16 steps, but you can create 1/16
steps by recording your own dance pattern. However if your chosen
music has a distinctive beat, the created patterns will probably follow it quite strictly....
You can also record your own steps... This is actually quite fun and it works pretty well, you
just let the music play and you jump on the dance mat, making your own patterns... The coolest
part is that you can finally use the dance mat with your own music. Good game, worth a try.
Difficulty: Just Right
Learning Curve: 30 to 60 Minutes
Time Spent: 10 to 20 Hours
Other comments from players: "Highly addictive"; "massive replay value"; "THIS GAME IS AWESOME!!!! GET IT!"
Full Review on GameSpot
GamesTM (Imagine Publishing, UK news-stand game magazine, Oct 2006, p.115)
Codemasters delivers the ultimate dance mat package
Dance Factory's rather ingenious USP is that it allows you to put any
CD into your PS2 and it will convert it into arrows for your dancing pleasure.
Simple, but absolutely superb.
And it works. After experimenting with tunes from Rage Against The Machine, Air,
Metallica, Sway and even a bit of classic old-school house, it would seem Dance
Factory cannot be beaten.
Using the same technology that fueled Vib Ribbon, the game picks out beats
immaculately, creating combo opportunities at every turn.
Obviously, not every tune you pick will lead to a perfect 'dance', but half the
fun of Vib Ribbon was finding which tunes worked best, and the same can be said
of Dance Factory.
This is a game that's all about innovation, and this is the only way that
Konami's dancing games could be bettered. Dance Factory is a superb product
that deserves to be noticed, and as such Codemasters should be commended.
8/10, Raises the bar for Dance Mat games
Imagine Games Network (IGN) online review
Codemasters' entry in the dance mat rhythm action genre and a game which does away with the
pointless faff of licensing half a dozen songs so infuriating they make you want to stove your brain
in with the sharp end of your DualShock. Instead, its single, and utterly genius, gimmick is that
you can slap in your own CDs and dance around like nobody's business.
In fact, if you've played and enjoyed any type of dance mat game before, that's probably all you
need to know about Dance Factory - oh, and that it works brilliantly.
CD collection withstanding, Dance Factory is just about the best dance mat game you're
going to find on PS2 - thanks entirely to its impressive track conversion capabilities...
with its potentially limitless song list, you may never need to buy another dance mat game again.
Best of all, it's an incredibly simple process - slot your CD into your PS2 while in the game,
select the track (or tracks) you want to convert and - shazam - Dance Factory's off like the
clappers. It's all very clever and, by and large, works brilliantly. During our rigourous tests, we
threw a veritable smorgasbord of stuff its way. Impressively, Daft Punk, Pulp, Mint Royale, Muse and
Kenickie (okay, we haven't really bought that many CDs since the 90s, but we'll get onto that in a
moment) all magically transformed into perfectly playable dance-offs.
Overall score 8.5/10, Great
Full Review on IGN
SPOnG VideoGames database review
SPOnG was lucky enough to be the first bunch of journos in the world to actually have a bit of a gawp
and a bit of a jiggle around with mind-blowing dance mat game, Dance Factory.
Why mind-blowing you ask? Well, simply because you can insert your favourite CD and the game will
generate dance moves in time to your favourite tracks. Pure genius.
...When we first heard about the Dance Factory concept, "it can't work, surely?" and "nice gimmick!"
and other such phrases were being bandied around the office when the game was first announced...
Well here's the good news - our ingrained northern cynicism was totally unfounded. It actually
works! ... it's the best dance mat game ever developed"
Full Review on SPOnG
BBC Focus magazine, November 2006
"Without hyperbole, this is the best thing to happen in all the universe, throughout history,
since we emerged from the primordial ooze... if you've never danced to Morrisey while having
your PS tell you you're a big fat sod, then you've missed out on one of the definitive human
These 2007 releases use my CMStream game audio library for surround sound and DSP effects,
on all four platforms. This is the first outing for cross-platform system programming work
I've been doing since completing CMR5 in 2004. I gave a talk about this at the Games
Developer Conference in San Francisco in March 2007. Follow
this link for an interview about the PC version of the Rally game, or click here for
details of the advanced surround sound on PS3 or if you'd like to hear about
it from the horse's mouth follow
this and associated links to online videos of what I had to say at the
conference (parts three through seven of the 'Cross-platform OpenAL' presentation).
Brian Lara Cricket 2007 combined the PS2 audio and streaming work I contributed
to the previous hit release with new code, built upon OpenAL, for Microsoft game
platforms. It was the first version of Lara with surround sound, using layers of
reactive crowd ambience, as well as positional sounds from the pitch.
A few months later rally driving game DiRT added support for Sony's latest console,
including 7.1 channel sound with
Ambisonic panning and full surround reflections,
and went to number 1 in the UK PS3 chart. DiRT got great reviews and attracted
awards for the game in general and the audio in particular.
Over 12,000 readers of German news-stand magazines GamePro and GameStar voted
Colin McRae DiRT 'Best PC sports/racing game' of 2007, beating off the latest
versions of two other long-running franchises: Koanami's Pro Evolution
Soccer 2008 and EA's Need for Speed: Pro Street, and the Spike TV
awards voted DiRT 'Best Driving Game'. Specifically in audio and regardless
of genre DiRT was pipped into second place (alongside Marty O'Donnell's Halo 3)
in EDGE magazine's annual 'Best Audio Design' category, losing out to Mario
Galaxy on the Wii - which indeed has stunning audio, though very different
in style. I'm proud to have worked with Stafford Bawler and Adam Sawkins,
lead audio designer and game audio programmer respectively, on that game.
This is the latest outing for CMStream and scales the tech developed for DiRT up from
intense single-car racing to a full grid of 20 cars, involving up to a dozen human players
at once. GRID won the 2008 BAFTA for best Sports game at the British Academy of Film and
Television Awards in March 2009.
I'd contributed codec, streaming and DSP tech to the audio systems on GRID's previous-gen
parent, TOCA RaceDriver 3, but was not much involved in the game programming. GRID builds
on the code I wrote for DiRT and Cricket, substantially extending it to include streaming
features which were in the original CMStream for PS2 and Xbox1 (developed with Jon
Mitchell) but had not previously been necessary on later consoles with far more RAM
and more effective codecs, as well as the CMR and TOCA legacies of DiRT.
Since random access to spiral-recorded optical discs is not much faster on PS3
and Xbox360 than it was on the previous generation - or older CD consoles, for that
matter - it's harder to justify nowadays, especially when games like GRID also stream
high-resolution graphics from the disc as they run. But with eight-layer interactive
music and a vast speech database, any phrase of which could be triggered at any time,
GRID could not fit all the main game audio into RAM even on a half-gig console, and
so CMStream introduced reams of extra code to bring streaming up to scratch on new
game systems. This time I was assisted by two other experienced audio programmers,
recruited from companies spun off from the former EMI Central R&D labs - Aristotel
Digenis and Pete Goodwin (no relation).
So Codemasters Central Tech Audio is now a team of three, rather than a one-man
band, and we are working on new DSP, tools and related systems for Codemasters
EGO game systems, the latest incarnation of the Neon cross-platform engine and
the tech behind several games now in development.
Meanwhile, here's some in-depth information about GRID audio
GRID takes the advanced audio systems used for single-car rally racing
in DiRT and extends those to model 20 vehicles racing at once - mixing
760 sound sources for the cars alone, PLUS music, collisions, damage,
crowds and many other ambient sounds.
GRID plays several banks of interactive music, eight layers deep,
positioned in surround around the player, adapting to their performance
and position in each race. It also tailors the crowd sounds on the fly
to cheer or boo depending upon what you and other players have recently
done (whereas the 'interactive crowds' in Colin McRae 2005 only varied
from one stage to the next, depending upon championship performance).
As well as more than an hour of sounds instantly-accessible in memory
and mixed on the fly - extending the intricate system used in DiRT - GRID
seamlessly streams context-sensitive speech and music, as well as graphic
data, from disc as you play the game, squeezing yet more varied audio
from the console or PC hardware.
GRID uses well over 100 simultaneous audio filter and reverberation
effects to further tailor each sound to its context and player activity.
This ensures that the game sounds vary as you play and the differences
can make a difference to how you play, by telling you more about what's
going on in the race, especially and most importantly off-screen.
GRID audio systems use sub-bass Low Frequency Effects and controller
shocks together - what we call 5.1.1 sound - for crushing impacts and
rumble strip feedback that you can really feel as well as hear!
GRID now uses Ambisonic techniques for immersive surround on Xbox360 as
well as PS3. Ambisonics were developed by Oxford mathematician Michael
Gerzon in the 1970s and have since become the focus of 3D surround sound
research worldwide; Gerzon's original work was funded by the UK NRDC
(National Research and Development Council) and the trade mark and
patents (now lapsed) were bought by classical music recording company
Nimbus. But like Alan Blumlein's pioneering work on stereo at EMI in
the 1930s, it has taken decades for hardware makers to catch up, and
games - rather than cinema - appear to be the 'killer application'
where precise surround - rather than vague ambience - is crucial.
In brief, Ambisonics means all the speakers work together to create
realistic soundfields, rather than just picking the nearest one or two
for each direction - a difference that really has to be heard to be
appreciated - once you've heard Ambisonic surround, typical cinema and
game panning just sounds fake (which it is).
Almost all the sounds in GRID on PC are in uncompressed PCM format - so
at last players can hear the sounds at the full quality of our original
recordings (especially with X-Fi advanced mixing and effects). Even so,
close collaboration between Codemasters and Creative Labs, developers of
the most often-used OpenAL drivers, mean the sound systems in GRID are
far more efficient than those in DiRT and the full audio design runs
even on minimum-spec PCs without significant impact on the frame
Yet PC specs march on, and GRID is one of the few games that takes
full advantage of quad-core systems and the latest multiple GPU
configurations. So if you've got all the latest hardware - necessarily
including an X-Fi with the EMU20K DSP chip and good headphones with
CMSS 3D or 7.1 speakers - the PC version of GRID can now outperform
the current generation of consoles. In practice they're all more often
limited by the listening environment and speaker systems than by the
audio tech in the game - but if you really care about sound we've done
the work that'll make your system sing - and you can be the conductor!
The sequel to DiRT combines technology from GRID and the original, plus new granular synthesis and Ambisonic soundfield technology, drawing together the biggest audio team yet for any Codemasters game, and especially noteworthy contributions from my colleagues Aristotel Digenis and Rob Pattenden. The game is particularly impressive in surround - the more speakers, the better! The console versions make pioneering use of the concept of volumetric emitters, in which almost every sound in the game has an appropriate size, rather than being rendered simplistically as a collection of point sources - still the norm for games, though not the real world.
DiRT2 is built upon NeSound, the latest incarnation of my CMStream cross-platform audio system, now fully integrated with Codemasters' EGO engine. As well as support for hardware-accelerated audio on all versions of Windows from XP through Vista to Windows 7, for the first time it includes genuinely periphonic 3D loudspeaker audio options, via HDMI on the PS3 and the Rapture3D advanced OpenAL driver, from Blue Ripple Sound, which is ideally suited to modern multi-core PCs.
This '3D7.1' option is the first commercial fruit of talks and demonstrations I gave at the International Conference of the Audio Engineering Society in London in February 2009, illustrating how practical 3D sound could be delivered via existing 7.1 channel audio hardware, while retaining compatibility with existing stereo and Cinema 5.1 media (reported in Develop magazine), earlier discussions with Sony, Creative Labs and the Interactive Audio Special Interest Group (IA-SIG) at the 2007 Games Developer Conference in San Francisco, and the European SpACE-NET spatial audio academic network, where I met Ambisonics expert Richard Furse of Blue Ripple and introduced him to OpenAL as an ideal source of high-def 3D audio for his uniquely powerful and scalable higher-order soundfield tech, since used to test and demonstrate GRID, DiRT and other games, in 14.1 channel 3D surround sound.
DiRT2 brings practical 3D loudspeaker audio capabilities out of the lab and into the homes of over a million gamers, but this is just the start - impressive though DiRT2 audio is, you ain't heard nothing yet! Codemasters is collaborating with the biggest names in the games industry, and other respected organisations like the AES and BBC, to develop game audio from a junior partner of the music and movie industries to a leader and exemplar.
With games now turning over more cash at retail and online than either music or home cinema, and arguably the primary source of HD audio and video worldwide, we have reached a tipping point where innovation in games can lead the way to truly immersive audio. Surround sound in cinema is a luxury, often erratically used. Gamers deserve better - for them it can be a matter of life and death!
In 2009 Codemasters signed up the rights to develop games based on the Formula One world championship, and our Birmingham Studio was assigned the task of developing this game for the current lead platforms, building on the tech used in DiRT and GRID. The main technical advances in F1 2010 are the use of granular synthesis for non-player car sounds as well as the player's engine, and an elaborate reflection system prototyped by Rob Pattenden and production-engineered by Hugh Lowry, which models the intricate echos and pass-by sounds as cars and listeners move around the circuit. Ambisonic soundfields are again extensively used in the game and in the paddock, ensuring that the game makes best use of 7.1 surround sound on PC and PS3 - in two or three dimensions - without compromise to the mono, stereo and 5.1 channel surround mixes. Custom HRTF options in the Rapture3D OpenAL driver optimise the surround-sound experience for PC players who prefer to listen on headphones.
All the sound design and much of the audio programming for F1 was done at our Southam HQ, by the team responsible for DiRT2 - the Central Tech Audio posse of myself, Aristotel Digenis and Pete Goodwin, plus game audio programmer Adam Sawkins - assisted by new hires Lars Hammer and Andy Morris at the former Rage/Swordfish studio in Birmingham. Sound design drew on the skills of Andrew Grier, Mark Knight, Jon Newman, Stephen Root, Dave Sullivan, Ed Walker, Pete Ward and Claire Woodcock, making this the biggest audio project yet to emerge from Codemasters.
The game went straight to number one in the UK all platform games charts in the week of release! It subsequently earned our team a second BAFTA for best sports game of the year
Follow-up F1 2011 built on the same NeSound custom audio tech with more input from the Birmingham posse, including greater use of stereo sounds in the front end and full Ambisonic soundfields for the paddock and garage ambiences, terminal damage effects, loading and other transitions including scenes after qualifying and each race. These fourteen custom soundfields are rendered in mono, stereo, 2D or 3D on PS3 (via HDMI and 3D7.1 speakers) and in that layout and many others, and on headphones with HRTF modelling, on PC through Blur Ripple's Rapture3D OpenAL driver which was again bundled with the game.
As well as these premade soundfields all the platforms use Ambisonic mixing for around 130 individually-positioned voices, hybrid third-order on PS3, first-order on Xbox and up to fourth order on PC depending upon hardware spec. The mandatory switch from XAudio to XAudio2 required some minor trimming on XBox360 but additional parametric equalisation - based on code I'd written for Silicon Studio decades before - helped improve the quality of the engine sounds on all platforms.
The 3D 7.1 mix on PS3 was tweaked slightly, applying the 'tilt' suggested following Audio Engineering Society reviews of the scheme, to make the front and rear triangles parallel with walls rather than floor and ceiling. The difference is subtle and within the variability of playing postures, but improves compatibility with 5.1 cinema surround mixes. It's the layout shown in figure 6 of my AES paper..
Shooters! Bodycount was the first Codemasters combat game with audio in 3D on PS3, using my tech. There was no PC version. The last two Operation Flashpoint projects were derived from the externally-developed PC hit. These used bought-in FMOD middleware so they lacked the Ambisonic 3D features of our racing games, though I did spend six months tuning up OFP Dragon Rising to make best use of the mixing and streaming resources available - but the Guildford Studio project Bodycount showed what NeSound could add to a 3D shooter.
Bodycount also pushed the X360 hard thanks to further optimisations and extensions for NeSound running via XAudio2, programmed by Andy Mucho. The PS3 version running on a single SPU co-processor was cranked to to 256 voices, each running through a fast ATRAC3 decoder and a 1024 element FFT/DFT for tonal tweaking, with six directional reverbs, but even this work and 3rd order Ambisonic panning didn't stretch Sony's extraordinary co-processor, which apparently 'played whatever we threw at it' with no need for further tweaks.
I did learn one important new thing from this game, which is that in a full 3D game like this, with enemies crawling around in buildings above, below and behind the player, the overhead reverb needs tweaking depending upon the location - the sky is rather boomier than intended when you're out in the open!
Sadly this was the only project completed by the Guildford Studio and its subsequent closure resulted in my long-time collaborator Pete Goodwin briefly leaving the games industry. I'm glad to say he's now working with Jason Page at Sony Europe in London, where the unrivaled PS3 Multistream audio system was born. Many of the audible advantages of our games on PS3 stem from Multistream - combined with Ambisonic extensions and the cross-platform capabilities of NeSound and its predecessor CMStream which I wrote with Jon Mitchell. In particular Sony's ATRAC3 codec enabled us to get more audio into less RAM, at high quality and with excellent surround and streaming compatibility.
The main audio extensions in the third and fourth incarnations of the DiRT driving series related to the in-built YouTube capture, which includes an adaptive custom game audio remix rather than the easy option of padding muzak. Rapture3D remained a built-in feature on PC, with incremental updates and improved HRTFs for 3D headphone listeners.
At the end of 2011 I was redeployed within Codemasters into a Central Technology group working to extend our games to new mobile and console platforms. As you might expect, most of this has involved new audio systems. Not all the fruits of this work have surfaced at the time of writing, but a couple have:
Race Stars marks a return to Nintendo development for me - the last such game I worked on was Drome Racers at ATD more than a decade before, and the audio tech was immediately recognisable from my GameCube experience, with the advantage of HDMI 5.1 output and the ability to play sounds via the WiiU's distinctive tray peripheral as well as Wii controllers.
On the established platforms Race Stars uses a combination of mature NeSound platform runtimes and new design tools closely integrated with the latest update of the EGO game engine, supervised by Aristotel Digenis who had by then risen to the rank of Senior Programmer. The WiiU version was released later, in 2013 with extra features and content, initially in Japan only.
A modified version of this game, adapted for free-play on mobile devices, has since been released on Apple's iOS. It can be downloaded at no cost from the App Store.
This small project was especially fun for me because it involved leading all aspects of the audio work, from design through game programming as well as runtime tech, while still able to draw on the expertise of the large sound design team at Codies. The resultant game uses fewer voices and DSP effects than the PS1 original, and deliberately includes many of the same samples, yet sounds a lot richer as these are no longer compressed, taking advantage of the extra RAM available nowadays, and many hifi assets from DiRT, GRID and even a few from the F1 series have found their way into the mobile remix. All four wheels have independent physics, skid, scrub and surface sounds, most apparent if you listen on headphones, and dynamic mix adaptations mean the co-driver pace-notes are clear even if you choose to play the game with your own music from iTunes or similar apps alongside.
The pace notes are exactly as they were in the 30 featured rally stages from massive hit Colin McRae Rally 2, voiced by Colin McRae's co-driver Nicky Grist; in this case many of them are as remixed for Dreamcast and amusement arcade versions of Colin McRae Rally which did not see release at the time but cut through very well on modern mobile platforms.
The engine replay system is based on the approach used in DiRT and GRID (pioneered in Race Driver 3) rather than old-school crossfaded loops, and all mixing is performed at 48 KHz rather than the usual Unity Engine/FMOD mobile default of 24 KHz (except on the iPhone 4 version, which is dynamically trimmed to make best use of that relatively slow platform). Crowd and other ambiences owe more to CMR5, as I'd worked on that game to improve those aspects; hundreds of them were carefully placed in-game by a freelance designer Tom McCaren who subsequently and deservedly landed a full time job at Codies.
The producer of Colin McRae Rally for iOS was Pete Harrison, who also took resonsibility for the very successful reworking of Race Driver GRID playing on Sega cabinets in an amusement arcade near you. Pete has a background in audio, having being European Tech Evangelist at Creative Labs a decade before, instrumental in the support for 7.1 surround and advanced CMSS3D headphone support in Colin McRae rally games from CMR4 onward, and the advantages of having an audiophile producer cannot be understated, from my point of view. Listen to the game - it's a couple of quid on the app store - and hear what I mean!
Quick ports of the iOS product I worked on have since been released by various publishers for Android, Blackberry, Apple OS X and Steam PCs.
My final Codemasters credit was for work on the Mobile versions of MicroMachines, licensed from Hasbro and published by EA subsidiary Chillingo. The Ego 2.5 engine tech was credited to me and my colleague Adrian Smith in Southam, last men standing in the once 30-strong Central Tech department, with gameplay from a small group in the Birmingham studio which mainly makes Formula 1 games for consoles and PCs.
The latest F1 mobile game was outsourced and Ade works on the Colin McRae Rally team now, as CT closed a few days before I left. So I was both the first programmer in Central Tech, and the last. 12 years is not a bad run, especially given the multi-million selling number ones we shipped along the way.
Dean Bilotti deserves strong recognition for the Ego engine mobile rendering systems, though the only credit that hinted at this was for Toybox Turbos, which came out on obsolescent consoles but never surfaced on iOS or Android though we had playable versions of that and several other unreleased titles in-house before he left. It even featured on the retail packaging of the Nvidia Shield console, though sadly not inside (see photo right). Dean reinvigorated CT and played a major role in almost all titles made during the 'dash for mobile' as the department otherwise dwindled. He works for Apple now.
R.I.P. Codemasters Central Tech, 2003-2015
As in any console studio, where high dev budgets are still dwarfed by the product release costs, I worked on many other games lost to licence and investment gambles, console and tech transitions, including: Colin McRae Redline, with awesome audio made with my erstwhile sidekicks Staff Bawler and Jon Michell; Operation Flashpoint Middle East Crisis, the first proposed followup to Bohemia Interactive's PC-only OFP and an early experiment in Ambisonic soundfields with ace designer Ben McCullough, who I'd love to have worked with more; and another transitional rally title for PC and Xbox360, one of three internally labelled 'CMR7' games, prototyped then canned before Neon Engine, Ego and PS3 changed the landscape decisively to make Colin McRae DiRT and RaceDriver Grid possible.
But overall the work that shipped more than vindicated Chris Southall's decision to create a new Central Tech department over the ashes of the old Core team (leaders of whom founded Freestyle Games) and to appoint me, an audio specialist, as the first programmer, joining 'white hat hacker' and security specialist Murray Rigluth, who ended up running Sony's DADC plant in Austria. When Chris left for Sega his successor as CTO, Bryan Marshall, now at Nominet, grew the department and made the case to make the new in-house game engines - Neon Engine, then Ego - central to Codies' successes on Playstation 3 and Xbox.
I should also say thank you to the dozens of sound designers and handful of game audio programmers I've worked with at Codemasters. In my role as Principal Programmer (Audio) I was directly answerable to both the Audio Director and the CTO, and indeed hired to bridge those departments and the game team programmers, which had previously sometimes communicated more by dead letter drops than active collaboration. Not only did we get get some great audio tech that way, we also got more technically-adventurous sound designers and greater audio-awareness elsewhere in the company.
I'm especially proud of Dance Factory, which emerged to generate four patents in my name as well as millions of pounds of profits after three years of progressive refinement. Working with Adam Sawkins on that, as well as DiRT and Grid, was exciting and insightful. The Ambisonic mixing tech used in those and subsequent BAFTA-winning titles is arguably my greatest contribution to modern gaming - through 12 million games shipped, grossing half a billion pounds, and demonstrations for the Audio Engineering Society and firms like Audio Kinetic which spread the idea far beyond Codemasters.
Ambisonics is now recognised as crucial to VR and advanced surround in games and other interactive systems, as I gambled it would be at the turn of the millennium when it seemed to be just another failed NRDC project. It was just decades ahead of its time.
No tale of Ambisonics, especially in the UK, should end without a tip of the hat to the Three Doctors of 21st century surround sound, Dave Malham, Richard Furse and Bruce Wiggins; the sonic visionary mathematician and explainer, Michael Gerzon; and Alan Blumlein without whose 1930s inventions we might still be listening to mono discs made of crushed beetles. Like Newton, and Noel Gallagher, I have stood on the shoulders of giants - thank you all.
© Simon N Goodwin, Warwick 2005..2017, and reviewers
Link to the top of this document
Link to the article index