Game music in C++ with raylib audio

Intro

0:00 Hello and welcome back to episode thirty-two, where we're going to have the game play music while you're in the world, with raylib's audio, and give the player a way to control it. Let's begin with a quick look at where we're going to be by the end of the episode.

0:15 While alice runs through the Meadows, the music plays, from a track that streams as it goes. The settings window has the volume, the music and a mute, with sliders that follow the way we hear loudness, and the client keeps them for next time.

0:33 On the website of the series you can find more information about the project. The code for the project is open source on GitHub, tagged at each step of each episode so that you can follow along.

Last time

0:45 Now, before we get to today's work, let's quickly go over what we did last time. The camera got a tilt with the right mouse button, which stops between five and eighty degrees, after we sent it over the top. The cube over our player went, and our name is in gold instead. Four skills went into the content, with a level and a cooldown each. The server decides each one, and a monster that's hurt goes for whoever hurt it. The client got a hotbar on the number keys, with the cooldowns counting down, once Parts kept them. And the characters play three new clips from Mixamo, with a green glow for second wind.

Today

1:19 So far, our game has made no sound at all, and a world with music feels a lot more alive. Today, we're going to make it play music, which fades in and out and rests between plays. raylib has an audio module, on top of a library called miniaudio, which opens the system's sound device for us. A song is long, so raylib streams it from its file a little at a time, instead of decoding all of it up front. The music fades in as the player enters the world, rests between plays, so that an hour of one track doesn't wear thin, and fades out as they leave. The settings window gets a volume and a mute, with sliders that follow the way we hear loudness, and the client keeps them between runs, in a file that a crash halfway through writing leaves whole. The track is called Background Music, by Pavel Bekirov, from Pixabay, which I've downloaded for us. All right, let's get started, with the audio device.

2:12 raylib's audio module plays through miniaudio, by David Reid, an audio playback and capture library in a single file of C. It talks to each system's own audio, such as Core Audio on macOS and PulseAudio or ALSA on Linux. When raylib opens the device, miniaudio tries them in order, PulseAudio before ALSA on Linux, and keeps the first that works, so our code doesn't change with the one it gets.

The audio device

2:36 Pixabay's licence lets us use it in the game, but not hand out the file on its own. So like the Mixamo files, it stays out of the repo, and the README in assets slash music says where to get it. Git ignores the MP3s in that folder, and CMake builds audio dot c c into the client, and copies the folder next to it, with the others. Our audio goes in a new header, audio dot h, which includes raylib, for its music type, memory and string, and absl's status or, like the window's header. Music folder is where the client finds the track, next to its program. The audio device is raylib's, which plays our sound through the system's, and its comment says how it goes with the window: there's one, opened after the window and closed before it. Like the window, it comes from a create function, which gives back the device or why there's none, such as in a container with no sound card. The guide doesn't use exceptions, so that's how a constructor's failure gets back to us. The destructor closes the device, and it can't be copied, as two copies would close it twice. A track is a piece of music, which raylib streams from its file as it plays, a little at a time, as the comment says, so that the whole song doesn't sit decoded in memory. Our track is a minute and a half of stereo at forty-four point one kilohertz, and raylib decodes MP3 into 32-bit floats, so decoded, it would take about thirty megabytes. Loading needs the device to be open, so load takes one, even though it doesn't use it, which the comment says. That way, the compiler won't let us load a track before the device is open. Then play, from the start, stop, and whether it's playing. The track keeps raylib's music, which is a struct of handles into raylib's audio. Audio dot c c includes raylib, memory and string, and from absl, its memory header for wrap unique, status for the errors, and str cat to put the path in the error. Music folder is the folder called music in raylib's application directory, like the characters' folder. Create opens the device through miniaudio, on the system's default output, and checks that it's ready, as raylib's init function returns nothing. Then wrap unique hands back the device, as its constructor is private. The destructor closes it, so that the device goes with the object. Load opens the file as a music stream, which reads its header and gets a decoder ready. Is music valid tells us whether that worked, with not found as the error when it didn't, which names the path. The track's destructor unloads it, and play, stop and playing are raylib's functions for the stream. In main dot c c, the client includes audio dot h, and opens the audio device after the window. The comment says what happens without one: as in a container, the game plays on in silence. The device and the track are unique pointers, empty when there's none, as unloading the track needs the device still open. C++ destroys locals in the reverse order, so the track goes first, then the device, and then the window. The if with an initialiser keeps audio or inside the if, where we take the device out of it. Then we load background dot m p three from the music folder. Either failure is printed with its status, and the game goes on without music. A game without sound is still playable, so unlike a missing window, a missing device doesn't stop the client. At the end of each frame, after drawing, we compare whether we're in the world with whether the track plays, and when the two disagree, we start or stop it. The comment over it says what we want: the music plays while we're in the world. And the README gets a section on the music, with the credit.

6:14 Let's build it and log in, and alice is in the world, where we should hear the music. We hear nothing at all, in the Meadows. raylib's is music stream playing says that the track plays, and yet no sound comes out.

6:28 raylib's example for streaming music shows what we missed. In its loop, it calls update music stream, each frame, which fills the stream's buffer with the next part of the file. We don't call it anywhere, so the buffer stays empty, and the device plays silence from it.

6:45 So the track gets an update, which the client calls each frame. Its comment says what it does: it refills the buffer the track plays from with what comes next in its file. raylib needs this each frame, or the buffer runs dry, and the track goes quiet. The buffer has two halves, of about a quarter of a second each. While the device plays one, update decodes the next part of the file into the other, so the decoding happens on our thread, and the device's thread mixes what we've decoded into its output. In audio dot c c, update calls raylib's update music stream, and in main dot c c, the client updates the music each frame, before it decides whether it plays. At sixty frames a second, that's about sixteen updates for each half of the buffer, so a frame would have to take a quarter of a second before the music skipped. And the README says how the stream's buffer works.

7:37 Now the music starts as alice comes into the world, at full volume. raylib loops a music stream by default, so it would play the track over and over, for as long as she stays.

A jukebox

7:50 Music that starts at full volume and loops forever is a bit much. Minecraft is known for its long silences between pieces. Many games fade their music in, play it, and leave a quiet stretch before the next track, so that it doesn't wear thin, and that's what the jukebox is going to do. Its comment says it: when the music plays, and how loud, kept apart from raylib so that it can be tested without a sound card. The music fades in as the player enters the world, plays the track through, rests for a while in silence and plays it again, and it fades out as they leave. The times are constants, in seconds: three to fade in, a second and a half to fade out, and forty-five to rest. Each frame, it gives back a cue, for what the track should do: nothing, start or stop. Update moves it on by the frame's time, from whether the player is in the world, and whether the track is playing. And volume is from nothing to the track's full volume, for the fades. It keeps the volume, how long it has left to rest, and whether it started the track, as a track that isn't playing could have ended, or not have started yet. Jukebox dot c c includes algorithm, for min and max, and update starts with a player who isn't in the world. As the comment says, a player who comes back hears the track from the start, straight away, so the rest is cleared. While the track plays, the volume falls by the frame's share of the fade-out, and once it reaches nothing, the cue is to stop. In the world, while the track plays, the volume rises by the frame's share of the fade-in, up to full. A track that isn't playing, after we started it, has played through, so the jukebox rests before it starts again, which the comment says. The rest counts down by the frame's time, and while some is left, the cue is nothing. A player entering the world for the first time has no rest to wait for, as it starts at zero. Once the rest is over, the jukebox notes that it started the track, and the cue is to start it, from a volume of nothing, so that it fades in again. The test plays the jukebox through, with the frame times we give it, so it runs in a moment. It includes jukebox dot h and GoogleTest, and an alias makes the cues shorter to write. The first update, with the player in the world and nothing playing, starts the track. A second and a half later it's halfway in, and ten seconds on, at full volume. Then, as the comment says, the track ends, and the music rests before it plays again. A second into the rest, the cue is nothing, and it's still nothing a second before the rest is over, and on the rest's last second, it starts the track again. And leaving the world fades it out, and then stops it, as the comment says. After ten seconds, which fade the track back in to full, three quarters of a second out of the world take it halfway down. The cue is still nothing, and three quarters more bring it to nothing, and the cue to stop. CMake builds the jukebox into the client, and its test into the client's tests. The track gets a set volume, from nothing to the file's own loudness, Set volume is raylib's set music volume. And as the comment says, raylib loops music by default, and the jukebox decides when it plays again, so load sets the music's looping to false. In main dot c c, the jukebox sits next to the track, and each frame it gets raylib's frame time, whether we have a connection, and whether the track plays. The switch names all three cues with no default, so that a fourth one would get a warning from the compiler, which has all its warnings on. Then the track takes the jukebox's volume, and the update refills it. And the README says how the music plays now.

11:32 This time, the music fades in as alice enters the world, over three seconds, as she starts to run across the Meadows. And when she logs out, it fades out, over a second and a half.

11:44 The jukebox's test runs in Docker, without a sound card, and passes, as the jukebox leaves raylib's audio to the track.

Volume and mute

11:53 Each player wants the music at their own volume, or not at all, so the settings window gets a section for the sound, and a new header holds what it sets. The sound settings are how loud the game is, as the player sets it. They keep where the sliders are, from nothing to full: the volume, which covers everything the game plays, the music, and mute. The music starts at seventy percent, which is twelve decibels down, as background music shouldn't be the loudest thing in a game. Gain turns a slider's position into the gain raylib takes, and its comment says why the gain isn't the slider's position. We hear loudness on a logarithmic scale, so the slider goes through forty decibels, and its bottom is silence. A slider that moved the amplitude in a straight line would do nearly all of its work at its bottom end, and its whole top half would change the loudness by no more than six decibels. Sound dot c c includes c math, for the power function, and the range is forty decibels. At the bottom, the gain is nothing, as decibels get closer and closer to zero amplitude without reaching it. Anywhere else, the slider's distance from the top is the decibels below full, and the comment says how those turn into a gain: twenty decibels are ten times the amplitude. The test pins the top, at full, the middle, at a tenth, which is twenty decibels down, and the bottom, at nothing. CMake builds the sound into the client, and its test with the others. Menu dot h includes the sound's header, and build takes the sound settings, before the atmosphere. Its comment says the settings window edits them now, as well as the atmosphere, where Reset puts back the land's own. In menu dot c c, c math comes in for rounding. Slider percent shows a setting from nothing to full as a percentage, as its comment says, which reads better than a fraction. It rounds the setting to a whole percent for ImGui's slider of whole numbers, with l round, and the slider's format prints it with a percent sign, which takes two in a format, as one alone starts a number. The slider gets the always clamp flag, and the comment over it says why. Control and a click lets a player type a number, which ImGui takes past the slider's ends unless it's told to clamp it, and five hundred percent would be far too loud. When the value changes, by a drag or a typed number, ImGui gives back true, and we put back the percent as a share. Build takes the sound settings, by reference, so that the window edits the client's own. The settings window opens with the sound, under a heading with Font Awesome's speaker, with the volume and the music, a checkbox to mute it, and the track's credit, greyed out under them, with a music note. The sky's settings get a heading of their own. In main dot c c, the client keeps the sound settings, and hands them to the menu. The music's volume is its slider's gain, times the jukebox's fade, and the volume goes to raylib's master volume, which applies to each sound the game makes, so far the music, at nothing while it's muted. It's set when there's a device, as raylib's master volume belongs to the open device. And the README has the new settings, with why the sliders follow the decibels.

15:00 View, settings opens the window, with the sound at the top. Dragging the music down to half puts it twenty decibels below full, and eight below where it was, which sounds a little over half as loud, and dragging it back brings it up again. Mute silences it, and unmuting brings it back at the volume it had.

Kept between runs

15:19 A player who turns the music down expects it to stay down next time, so the client keeps the settings in a file, in protobuf's text format, like the content. Its schema is settings dot proto, in the client, in a package of its own, nanommo dot settings, so that its names stay apart from the protocol's. Sound has the volume, the music and mute, and each field is optional, and the comment says why: so that one the file doesn't have keeps its default, where protobuf would give a plain field zero, which is no sound. An optional field records whether it was set, so we can tell a missing volume from one at zero. Settings holds the sound, so that other settings can join it later, and its comment says it's what the player sets, which the client keeps between runs. Settings dot h includes string, absl's status and status or, and the sound's header, and it has three functions, for the file's path, for loading and for saving. Settings path is where the file goes, and its comment says where that is: in Application Support on macOS, and in the XDG config folder elsewhere, which is dot config in the home folder, unless XDG config home names another. Each system has its own place for an application's settings, so that's where a game puts them. Load settings reads the file, with the defaults for what it doesn't have, and with no file, as on a first run, they're all the defaults. Save settings writes the file, and its comment says how: next to the path first, and then it renames it over the old file, as a rename replaces a file in one step. If the game crashed halfway through writing, the old file is still whole, where writing over it would leave half a file. Settings dot c c includes raylib, for its file functions, c errno for the error when a rename fails, c math to tell a number from one that isn't, and c std io for the rename. Then c std lib for the environment's variables, and c string to turn that error into words. Algorithm comes in for std clamp, f stream to write the file, and string. Then str cat for the messages, the code protoc generates from our schema, content's text dot h, for read file and parse text, and protobuf's text format. Environment sits in an unnamed namespace, as nothing outside the file needs it. It reads a variable, or gives back what we say otherwise, when it's missing or empty. Settings path takes the home folder, or the current one without it, as for a service. The hash if tells the preprocessor to build one of the two paths, on macOS or elsewhere, where XDG config home wins when it's set. Load settings starts from the defaults, and gives them back when there's no file. raylib has file exists, and Google's guide sets aside the standard library's file system header, for its thin support for testing and its security holes, so raylib's file functions do that job here. Content's read file gives back the text, or why it couldn't. Then it parses the text, like the content, and an error names the file. Each field the file has replaces its default. A player can write anything into the file, and the comment says why that matters for the volumes: one past full would play many times louder, so each stays between nothing and full, with std clamp. The text format reads nan, not a number, as a float too, and as the comment says, std clamp lets it through, as nan is neither below nor above anything. So a volume that isn't a number keeps its default, with std is nan. Mute is a bool, so there's nothing to clamp. Like load, it takes the path, so that the tests can give it one of their own. Save settings prints the whole file into a string first, so nothing on disk changes until the text is ready. It sets each field of a settings message from the sound settings, all three of them, so that the file holds everything the player set. Then protobuf's text format prints the message as text, a field to a line, which a player could open and change by hand. Printing returns whether it worked, and when it didn't, save gives back an internal error that says so. The folder may not be there yet, and as the comment says, raylib makes the folders on the way too. It writes the text to a file next to the settings, with dot new on the end of its name. It goes in the settings' folder, as a rename is one step on one disk, and across disks it would be a copy. Trunc empties a dot new file that a crash left behind, so that the new text doesn't land on top of an old one. The stream writes the text, and flush pushes it out of the stream's buffer to the file, so that an error writing shows up here. Otherwise it would show up as the stream closes at the end of the block, where we couldn't see it. Then the rename replaces the old file, and on failure, we say why, with the system's error. The tests include raylib, for its folders, f stream to write a file by hand, and string. Test path puts each test's file in GoogleTest's folder for temporary files, in a settings folder of its own. The first test saves settings that aren't the defaults, a half, a quarter and muted, and loads them back, so each one comes through the file. Assert true stops the test when saving fails, as loading after it would add nothing but a second failure. The loaded settings have to equal the saved ones, with the equals operator we're about to give them. The expected settings go in parentheses, as the macro would split its arguments at the commas inside the braces. Then a missing file gives the defaults, as on a first run. The second test makes its folder too, as a filter can run it on its own. A file we write by hand, with the music alone, loads with its music, and keeps the default volume, as the file doesn't set the volume. The std ofstream of the path is a stream that lasts for that line, so the file is closed and whole before we load it. And that file with the field misspelt is refused, next to the spelling that loads, so that we know the refusal is for the typo. And a last test writes a volume of five and music at minus one, which come back as full and as nothing. Five is past the top and minus one is below the bottom, so each volume hits its own end of the clamp. Then a file with music that isn't a number loads with the music's default. The sound settings get the equals operator, defaulted, so that the client can tell when they've changed, and the tests can compare them. In the client's CMake file, settings dot proto joins the library's sources, and protobuf generate makes its C++ into the build folder, like the content's. The generated file's warnings are turned off, as they're protoc's, and the library's include folders get the build folder, where the generated header is, so that settings dot c c can include it. In main dot c c, ImGui's header comes in, for is any item active, and the client loads the settings from their path as it starts, and prints the error when it can't, keeping the defaults. Saved keeps what's in the file, so that the client can tell when the settings have changed since. After the menu has drawn, the client compares the two each frame, which costs next to nothing, as they're three numbers. As the comment says, it writes them once a slider lets go, so that a drag across one writes once. Is any item active is true while a slider is held, so the file waits for the release. And the README says where the settings go.

22:45 The settings' tests pass, with the sliders' and the jukebox's, all without a sound card. And here's the file the first test kept, a field to a line, with the volume at a half, the music at a quarter, and muted.

Wrap-up

23:03 So, today the client opened raylib's audio device, which plays through miniaudio. The music streams from its file, and plays once we refill its buffer each frame, which we found out when it made no sound. The jukebox fades it in as the player enters, rests between plays, and fades it out as they leave, with a test that needs no sound card. The settings window has a volume, the music and a mute, with sliders that follow the decibels, and the client keeps them in a file, written next to the old one and renamed over it. And the track is Pavel Bekirov's, from Pixabay, with its credit in the settings window.

Playing it

24:43 Let me know in the comments what you'd like to see next. And if you want to follow along with the series, subscribe to the channel. I hope you find this useful, and thanks for watching.