Game Audio Production with Wwise is an exploration of the world of video games using Audiokinetic’s Wwise, a powerful and accessible middleware tool for integrating audio into a game. This course utilizes two actual video games ( Limbo and AngryBots ) that you will fill with your own music, voice, and sound. The ability to integrate Wwise into your production pipeline means that your audio creators have more artistic control. Wwise features an open API and integrates easily into in-house game engines, the world's top commercial game engines, and any tool or application. Created By: Game Audio Resource Ltd Date: Version: 2.00 In this guide, we will script material types to texture tiles, so the game engine can inform Wwise which footstep material type is being moved on by any entity in game.
Sound Swapping and the Wwise Format
What you need before sound swapping:
- Wwise (Optional)--IMPORTANT: Newer versions of Wwise may not work! The .wem format seems to have changed in v2016.1.1. Download v2015.1.9 to be safe!
- A method to get your modified soundbank back into the game (either the QAR Tool or SnakeBite)
Wwise Tutorial
This tutorial is specifically for voice swapping. It can be used for some other audio files as well, but may not work for all audio files.
First thing’s first. Update your GzsTool’s qar_dictionary.txt with the updated one you downloaded. Now you’re ready to unpack the chunk.dat files (with GzsTool) and find the soundbanks you want.
If you’re looking to swap a character’s voice, Snake’s soundbank is in chunk0, and all of the playable staff members’ are in chunk1. Inside either folder, go to “Assetstppsoundasset” and you’ll find the respective soundbanks. All soundbanks have the .sbp extension.
Now, that you’ve found your soundbank, we can start getting sounds out of it! If you are planning on modifying this soundbank, there’s an extra step first. Drag the .sbp file onto GzsTool. GzsTool will unpack it and inside will be a .bnk, .sab and .stp file. The .bnk is a wwise audio container. The other two file formats seem to be related to 3D animation, but the .stp also contains wwise audio files.
The file we care about right now is the .bnk. Run Ravioli’s RScanner. Click “New Scan…”. Navigate to the folder that GzsTool unpacked the contents of the .sbp into and select the .bnk file. The contents of the .bnk will show up in RScanner’s window. Scroll to the bottom and look for the last file listed. There may be a file called Unkn####.dat after this file, but we can ignore it. We only care about the last file named File####.wwise_x (note that “x” can be any letter). In example, the last file in Snake’s voice’s .bnk is File0075.wwise_v. We need to remember the number of the file, this is important later. Following my example, I would need to remember 75.
Now, we can close RScanner and delete the folder and xml file that GzsTool extracted from the .sbp file. We don’t need the files inside the folder or the xml file anymore. All we needed was the number we found on the last file.
Next, we need to get all of the audio files in an audible format. Create a new folder that will hold the audible versions of your files. Change the extension on the .sbp file you’ve been working with to “.bnk”. Now, start RExtractor. Under “Input file(s)” select the .sbp file you just changed the extension on. Under “Output directory” select the new folder you created. Under options, select “Convert sounds to: Wave” and “Allow scanning of unknown files”. After that, click “Start” and allow it to finish. It may say some files failed to convert. Ignore and delete these files. They are likely not voice files anyway.
Hooray! You now have the files in an audible format! You can now close RExtractor. Now, we need to get the files in their original format. Create another new folder, which will hold the files in their original format. Change the extension of the .sbp you changed back to “.sbp”. Start RScanner again. Once again, click “New scan…”. Choose your .sbp file. This time, click “Extract All…” and select the newly created folder. Now we have the files both in an audible format and their original format!
Now, you’ll need to repeat the whole process up to this point on a second .sbp to get the files you want to swap in. However, you can skip the part where we opened the .sbp with GzsTool and opened the .bnk file with RScanner to see what the last file’s number was. We do not need this number since we aren’t editing this file.
Once you’ve got the files you want to swap in, we can get started on the fun part (by fun I mean very long and tedious). The first thing to do is to listen to all of the audio you extracted, so you know exactly what each file contains. It’s a good idea to note what every file is in a text editor and then reference your notes later. Once you’ve got your notes down, we can begin.
Remember the number we got earlier? This is where it’s going to come in handy. That number tells you which files came from the .bnk and which files came from the .stp. Using my Snake example, the number was 75. This means that File0001 - File0075 came from the .bnk. All of the others came from the .stp. Why is it important to know this? It’s much easier to swap over files from the .stp! You can just rename the file you want to swap in and replace the old one! This is NOT the case with .bnk files though. They require some manual hex editing to work correctly. When swapping in files, be weary of their file size. The size of the sound you’re swapping in must be smaller than or equal to the file size of the original. If a file is slightly too big to fit in, we can fix it with Audacity and Wwise (I’ll cover this later). If it’s much too big, it will not work. Find another file.
So now that you know the limitations of sound swapping, you can begin! Start by replacing all of the wwise files that came from the .stp. Again, you can do this by renaming the file you want to swap in to the original file’s name and then overwriting it.
Once you’re done that, we can talk about how to replace files from the .bnk. For each of these files, you’re going to need to edit them in HxD. (Note: size restrictions still apply with hex editing files.)
Don’t understand what the hex says? No problem! All of the work has been done for you already:
- 0x00 - 0x03: (32) 'RIFF' name
- 0x04 - 0x07: (32) Riff size
- 0x08 - 0x0A: (32) 'WAVE' name
- 0x0B - 0x0F: (32) Chunk type: 'fmt '
- 0x10 - 0x13: (32) Chunk size
- 0x14 - 0x15: (16) 0xFFFF (audio format) 0xFFFF means experimental
- 0x16 - 0x17: (16) Num of channels
- 0x18 - 0x1B: (32) Sample rate in Hz (stored backwards) ex: 44 AC 00 00 = 44100Hz
- 0x1C - 0x1F: (32) Avg bytes per second
- 0x20 - 0x21: (16) 0x0000 (block align)
- 0x22 - 0x23: (16) 0x0000 bits per sample (expected 0bps)
- 0x24 - 0x25: (16) Chunk size
- 0x26 - 0x27: (16) Extra fmt? (1 = yes)
- 0x28 - 0x2B: (32) Subtype (4 = 1 channel, no seek table)
- 0x2C - 0x2F: (32) Num of samples (num of bit per sample)
- 0x30 - 0x33: (32) Mod signal
- 0x34 - 0x35: (16) Data size (go to 0x5A - 0x5D)
- 0x36 - 0x39: (32) Setup packet offset
- 0x3A - 0x3D: (32) First audio packet offset
- 0x3E - 0x3F: Unknown
- 0x40 - 0x43: (32) Mod signal
- 0x44 - 0x47: Unknown
- 0x48 - 0x4D: Unknown relation to subtype
- 0x4E - 0x4F: (16) Padding(?)
- 0x50 - 0x53: (32) Uid
- 0x54: _blocksize_0_pow
- 0x55: _blocksize_1_pow
This is the layout for the wwise format’s header. It’s not necessary to understand what most of it means for what we’re doing. But if you’re getting into more advanced sound modding, this information is important.
Open up both the file you’re replacing and the file you want to replace it with in HxD. You’ll want to overwrite the following sections on the file you’re replacing by copying the same section from the file you’re replacing it with:
- 0x16 - 0x17 (very rarely necessary, but if it does differ, this needs to be overwritten)
- 0x18-0x1B
- 0x28-0x2B (again, very rarely necessary, but if it does differ, this needs to be overwritten)
- 0x48-0x4D (also rarely necessary, but if it differs, overwrite it)
- 0x50-0x53
That covers the header. Now to get the actual data in. On the file you’re replacing the original with, look for a string of text that says “data”. In most of MGSV’s wwise files there are two of these strings. We want the second one. (The first one is usually followed by a bunch of periods and then LIST. If you see this one, ignore it. If you find a wwise file that only has one data string, like custom built ones, follow the same process, but use the single data string.) After the four bytes following the string is the actual data. You’re going to want to highlight from this point all the way to the bottom of the file and copy it. You’re going to want to highlight this same section in the original file and paste over it.
Example: If I see “dataø..É” I want to start highlighting at the É since it is the fifth byte after the data string.
Once you’ve done that, you’re done editing the file! HxD will have created a .bak file of the file you edited. Delete the .bak file. We do not need it. Repeat this process for every file that came from the .bnk.
Done editing all of your files? Good! We can put the .sbp back together now. Start RScanner. Click “New scan…” and choose the .sbp. This time, click “Combine…”. Now choose the folder that contains the edited/replaced wwise files. It will probably tell you that it needs to add padding to the files to make them the correct size. Just keep clicking ok (or hold enter) until it’s done. Once it’s finished, you can either save the edited soundbank in a new place, or just overwright the old one. With that, you’re done! Just put the edited soundbank in the game and you should hear your changes.
If game audio is where you want to be, you'll need to look at one of the audio middleware solutions that are available. The one I particularly like is Audiokinetic's Wwise. You may be wondering what is middleware? In its most basic explanation, audio middleware is a tool that manages your audio assets that will be used in a game and integrates them with the game build.
What I like about Wwise is that it's a feature-rich audio middleware solution. At first glance, the interface looks quite daunting as there are many menus, submenus, and folder structures. But once you get familiar where all these locations are, you'll realize how powerful Wwise is. Let's look a little closer at Wwise and clear up the mystery around audio middleware and how it's used.
The Wwise Interface
How it works with Wwise is that you'll import audio assets for your game into Wwise. You'll then create events for these sounds so that the game knows how these sounds will react in the game. For example, maybe you want a specific win sound to play whenever a treasure is found, or maybe as you move into a space like a room, you want the fan noise to play as you walk under it. These events are all created within Wwise.
Wise Audio Jefferson City
Why this is so beneficial is that as a sound designer, you will have a better understanding of how you want the sounds to work in the game. So you can use a tool like Wwise to program in how these sounds work in the game, and then pass this onto the developer. This alleviates extra work that the game developer needs to do with the sound and helps with the game build workflow. So that's Wwise in a nutshell. Let's see what other nifty things Wwise can do.
Using Wwise
First off, Wwise is free to use for evaluation and non-commercial purposes such as creating game prototypes, developing freeware and for academic projects. The limitation here is that you can only use up to 200 audio assets in a project. If you're an indie game developer with less than 200 audio assets, you can use Wwise for free for Windows, Mac OS X, Linux, iOS, Android and Windows Phone 8 platforms. If you need more audio assets and are working on other platforms, and for other purposes than mentioned above, then you'll need to look at the licensing options on the Audiokinetic website.
Audio Assets in Wwise
In Wwise you add your audio assets to the Audio folder, and then you can create events for these audio assets and assign them to the event. This way the game will know when to play these sounds.
Wwise gives you flexibility when working with your audio assets. You can make volume adjustments and even add in pitch and filtering to the sounds. These will be nondestructive and are only applied through the audio engine. Your original sounds are still retained.
Using Random Containers
A great feature on Wwise is the random container. What you can do with this tool is choose a few sounds, and then randomize them with Wwise's randomizing functions.
With the random container, you can choose settings so that it shuffles through the sounds, plus that it doesn't repeat the same sound as much. This adds interest and realism to your game audio, and the player won't recognize the same audio samples being played over and over, making it less likely that they'll get bored by the same sounds.
You can even randomize the pitch of each sound, adding even more variation to them. Oraciones cientificas pdf.
Plus you can even add extra effects that are bundled with Wwise to further enhance your sounds.
RTPC Game Parameters
In Wwise, you can create game parameters that change sounds depending on different data that the game engine receives. One of Wwise's most handy game parameters is the Real Time Parameter Control (RTPC) feature. As you can tell from the name, this parameter will change the sounds in real time in the game. So for example, you could add a volume fade out to sound so that when you walk away from the object the sound fades out. This is a very simple example, but you can set up way more complex RTPC's depending on what is happening in the game.
In this example, the player's health (the sound of a heartbeat) drops in volume as the health decrease. A low-pass filter also cuts out higher frequencies as the health drops. So as you can see, you can set up multiple RTPC parameters that change in different ways depending on the game state, giving you much more flexibility with your sounds and how they adapt over real-time parameters.
Cross-platform
What's great about Wwise is that you can install it on Windows or Mac. And when it comes to game build time it supports various platforms such as Windows, Apple, Xbox, Playstation, etc. For an extensive list on all the supported game platforms take a look here: https://www.audiokinetic.com/products/supported-platforms/
Summing Bus
One of the more important aspects of audio middleware is the audio signal flow and the mixing of these sounds. With Wwise, you can apply audio summing to your sounds, so when 10s or even hundreds of sounds play at the same time, there isn't any audio clipping and distortion.
Even better is that you can create different buses, and control them with a MIDI controller. When you use the Soundscaper tool included in Wwise, you can test out how audio assets sound against each other when they are triggered at the same time.
MIDI in Audioware
Wwise has introduced the use of MIDI in their application. You can use their bundled sampler to load up audio samples and then program in MIDI to trigger these audio samples playing different keys.
Or you can use the synth to generate waveforms, which can then play MIDI sequences. With the introduction of MIDI, you can cut down on audio assets and resources. For example with mobile games there are limitations to the amount of sounds you can use, so with MIDI you can reduce the amount of sounds used by using the MIDI features to generate sounds.
Audio File Format Conversions
Wwise gives you great control over your audio and the file size of your audio assets. It allows you to quickly and easily reduce the size of your audio by saving the assets out as different formats, plus reducing the sample rates. As you can imagine with all the new mobile media, you have many file size restrictions. With Wwise, you can see how many resources you are going to use with your audio, and if you need to reduce it by changing the file format, it'll recalculate how much resources you've saved. I find this a real timesaver instead of having to jump between Wwise and an audio editor to convert the audio formats.
Conclusion
So that's really just touching the surface of Wwise, but hopefully these features entice you to look further into Wwise and explore how it can benefit you as a music composer and sound designer for games. Give Wwise a try and see how it can help improve your game audio workflow.