Mixamo animations in raylib with glTF and C++

Intro

0:00 Hello and welcome back to episode seven, where our cubes are going to become characters that walk and run, with animations from Mixamo. Let's begin with a quick look at where we're going to be by the end of the episode.

0:14 Thirty bots walk and run around the board, each of them an animated character, in its player's colour. I'm holding Shift to run up the board with mine, and then I walk to the right. The server decides who walks and who runs, and each client animates what the server sends.

0:31 On the website of the series you can find more information about the project. The code for the project is open source on GitHub, tagged at each step of each episode so that you can follow along.

Last time

0:43 Now, before we get to today's work, let's quickly go over what we did in the last two episodes. Our players got accounts, kept in MySQL, with the schema in the repo. Passwords are hashed with Argon2id, through libsodium, so the database doesn't hold any password in plain text. A login server checks them, away from the game server's tick, and it hands each login over to the game server as a token in Redis, which works once. The game server saves where each player is, so you come back where you left off, with your name over your cube. And the bots log in too, with accounts of their own.

Today

1:19 Today, our players get bodies, which takes a few pieces, all the way from Mixamo to the GPU. The characters and their animations come from Mixamo, Adobe's free library of characters and clips. They come as FBX files, so FBX2glTF converts them to glTF, which raylib can load. Then a small tool of ours puts each character and its clips into one file, in a shape that raylib can work with, as the converted files don't load properly as they come. The server starts telling the clients which way each player faces, and whether it stands, walks or runs. The client animates them, with the skinning done on the GPU. And the bots run some of the time too, like players would. All right, let's get started with where the characters come from.

Mixamo

2:09 Mixamo's characters are rigged, which means each one has a skeleton inside that moves its mesh. Mixamo also has a few thousand animations, recorded with motion capture, which it fits to whichever character you pick. It's free with an Adobe account, and we can use the characters and the animations royalty free, in commercial projects and video games too. However, we can't redistribute the files, and our game's repo is public, under the MIT licence. In Mixamo's Characters tab, a search for bot finds X Bot and Y Bot, the mannequins that Mixamo shows its animations on. Download opens the settings, where the format is FBX binary, and the pose is a T-pose, with the arms out. A T-pose is the pose the skeleton is bound in, and Y Bot comes down with those settings too. In the Animations tab, a search for walking finds a lot of them, and we take plain Walking, which is thirty-two frames long. In Place would take the walk's forward motion out for us, but we leave it off, as our merge tool does that, and measures the walk's speed as it does so. In its download settings, skin goes to without skin, as the mesh comes from the character. Then FBX binary again, at thirty frames a second, with no keyframe reduction, and the idle and the slow run come down like that too.

3:31 So the Mixamo files stay out of git, in assets slash mixamo, which git ignores. A README in that folder lists what to download, and with which settings. What we make from them stays out of git too, as it's still Mixamo's characters, in another format. And a client built without them draws the players as cubes, like before, so the project still builds and runs for anyone who clones it, and in CI.

FBX and glTF

3:59 FBX is Autodesk's binary format, which most 3D tools can export, and the way to read it properly is Autodesk's FBX SDK, which is closed source. raylib reads glTF instead, an open standard from the Khronos Group, who are also behind OpenGL and Vulkan. Khronos designed glTF so that engines can load 3D models quickly, with little processing. A glTF file describes the scene in JSON: the meshes, the materials, the skeleton and the animations. The raw numbers, such as the vertices and the keyframes, sit in binary buffers next to it. A dot g l b file packs both into one binary file, which is what we're going to use. raylib loads it with cgltf, a small C library for reading glTF.

4:47 The specification lives on GitHub, in a repo under the Khronos Group. glTF two point oh came out in 2017, and most engines and 3D tools read and write it by now, from Blender to Godot and Unreal.

5:00 To convert, we're going to use FBX2glTF, which started at Facebook, in the fork that the Godot engine's team maintains. It's built on Autodesk's FBX SDK, so it reads FBX files the way Autodesk's own tools do. Its releases come as ready-built binaries for Linux, macOS and Windows, with the macOS one built for Intel Macs, and the Linux one for x86-64.

5:26 I first tried converting the files, and loading the character and the walk with raylib, and the ways it failed show what raylib expects. Each Mixamo character has two meshes, the body and the joints, and FBX2glTF gives each of them a skin of its own. A skin is the list of joints a mesh is bound to, and each vertex names its bones by their place in that list. And the lists don't agree: on Y Bot, the body's skin has fifty-two joints, the clips have sixty-five, and they come in different orders. raylib takes the bones from the first skin in the file, and it finds a clip's bones by their place in its list, not by their names. So raylib refused Y Bot's walk outright, and X Bot walked with one of its joint spheres floating around its feet. And the walk carries the hips forward as it plays, which is called root motion. So the character walked away from where our server says it is.

6:20 raylib says so in its own source, in a list of limitations above its glTF loader's animation code. The first one is a single armature per file, which is one skeleton. A few lines down, it takes the bones from skins zero, the first skin, and it logs a warning when the file has more than one.

The merge tool

6:39 To fix the files, we're going to use glTF-Transform, by Don McCurdy. It's a JavaScript library and a command line tool for reading, editing and writing glTF files. It comes with functions for common jobs, such as welding vertices, simplifying meshes and pruning unused data, and we're going to use a few of them.

7:01 This is our merge tool, merge dot m j s, written in JavaScript and run with Node, as glTF-Transform is a JavaScript library. I've written it beforehand, as it's tooling around the game and not the game's code, so instead of typing it, let's go through what it does, and why. It takes the character, the file to write, and the clips on the command line, and reads them all. It also keeps a map of the character's bones by their names, as names are what we match on from here on. First, one skin for the whole character, with its joints in the order of the first clip's skin. We take the clips' order, as Mixamo's clips all list the bones in one order, and the character's meshes don't. For each name, we find the character's bone, and stop with an error if there isn't one. So a clip made for another skeleton can't load and play on the wrong bones. Each joint gets an inverse bind matrix, which undoes where the joint stands in the T-pose, and the joints all go into a new skin. The vertices still name their bones by their place in their mesh's old skin. So we build a table from each old place to the new one, by name, and rewrite each vertex's joints through it. Then the old skins go, leaving a single skin for raylib to pick. Then the materials, where the bodies become white, and the joints dark grey. raylib multiplies a model's colour by the tint we draw it with. So a white body comes out in the player's colour, and the joints in a darker shade of it. Next, the clips, where we first remove the character's own animations, as they're empty takes. Each clip is named after its file, so walk dot g l b becomes the clip called walk, which the client looks for. In each clip, a channel is the motion of one bone. We copy each channel onto the character's bone with that name, whatever order each file lists the bones in. And when the channel moves the hips, which is the first joint, it goes through in place first. In place takes the hips' travel out of the keyframes as an even drift from the first frame to the last. So the clip ends where it started, while the bob and the sway stay. And it prints how fast the hips travelled, which is the speed that keeps the feet from sliding. The walk's comes to one point eight metres a second, and the run's to three point one. Last, fewer triangles, as Mixamo's bots have about fifty thousand each, and a character a couple of hundred pixels tall on our board doesn't need anywhere near that many. Weld joins the vertices that sit in one place, as simplify works on the edges between them. Then simplify, with meshoptimizer, collapses edges until a tenth of the triangles are left, or stops sooner if the shape would move by more than a hundredth of its size. A tenth comes to about five thousand triangles a character, which matters once there are dozens of players on screen. Prune and dedup then drop what nothing uses any more, and anything stored twice, before the file is written.

9:55 Now, the pieces that run FBX2glTF, and then our tool. The tool lives in assets slash tools, with a package dot json, which is how a Node project says what it is and what it needs. Private keeps npm from publishing it by accident, and type module lets our JavaScript files load each other with import. Then the four dependencies, pinned: glTF-Transform's core, which reads and writes the files, and its functions, such as weld and simplify. Then gl-matrix, for the inverse bind matrices, and meshoptimizer, which simplify calls into. npm writes package lock dot json next to it, with the versions of everything underneath, and we commit that too. Build dot s h is a plain shell script that runs both tools, in order. Set dash e u stops the script at the first command that fails, or at a variable that isn't set. So a missing download stops the build, instead of making half a character. Convert is a small shell function, which runs FBX2glTF on a Mixamo file. Dash dash binary writes a g l b, the single-file form. Skinning weights four keeps the four bones that pull hardest on each vertex, as four is what raylib reads, and what our shader is going to take. And anim framerate bake thirty samples the clips thirty times a second, which is what we downloaded them at. Make dir dash p makes glb, for the converted files, and out, for the merged ones. Then we convert the two bots and the three clips, under short names without spaces, as idle, walk and run are the names the client looks the clips up by. Last, a loop runs merge dot m j s on each of the two characters, with the three clips after it. We run both tools in Docker, so that nobody needs Node or the converter installed on their own machine. A new stage, characters build, starts from Node's slim image on Debian bookworm. Dash dash platform linux slash amd64 asks for the image built for sixty-four-bit Intel and AMD processors, whichever machine we're on. The comment over it reads: FBX2glTF comes built for x86-64 Linux, with no Arm build, so on an Arm Mac it's emulated. And emulation is slower, but the conversion still takes a few seconds. We install unzip, like the packages in our other stages. ADD fetches a file into the image, and its checksum fails the build if the file isn't the one we checked. So a release that changes behind its URL doesn't get into our build. Here, the file is FBX2glTF's zero point thirteen point one release for Linux, as a zip. Unzip dash p prints the binary out of the zip, into slash usr slash local slash bin, so that it's on the path. Then change mode makes it executable, as a file printed out of a zip doesn't keep its permissions. The work directory is slash assets, and package dot json and the lock file are copied in on their own, first. So npm ci, which installs from the lock file, stays cached by Docker until the dependencies change. Then we copy the tools, then the Mixamo files, and run the build script. A last stage, characters, starts from scratch and copies in what the build made, so that docker build dash dash output writes those files into a folder, like the docs. Next, dot gitignore gets three lines: the FBX files from Mixamo, the g l b files we make from them, and node modules, which is where npm installs the packages. And dot dockerignore leaves node modules out of the build's context. So copying the tools folder doesn't bring my Mac's packages over the ones npm ci installed in the image. The README in assets slash mixamo lists the five files and the settings to download each one with, and the docker build that makes the characters. And the one in assets slash characters says where its files come from, and points to the Mixamo one.

13:55 Docker build dash dash target characters builds that stage, and dash dash output writes what's in it into assets slash characters. Progress plain prints each step's output as it goes, so that we can see what the tools print. FBX2glTF writes each file, and the animation it found in it. Then merge prints the speed of each clip, and for each bot, its sixty-five bones, its triangles, and its three clips. Our two characters come out at around three quarters of a megabyte each.

Facing and motion

14:27 Now, the server, as the clients are going to need two things they don't get today: which way each player faces, and whether it stands, walks or runs. And the server needs to know when a player wants to run. So we start with the contract, as both sides build on it. In nanommo dot proto, Input gets run, a bool, as field three, which the client sets while the player holds Shift. Then a new enum, Motion, for what a player is doing. Protobuf puts an enum's values in the package's namespace, and not inside the enum. So, by convention, each value starts with the enum's name, so that motion idle can't clash with another enum's idle. Motion idle comes first, as zero, since zero is what a field holds when it isn't sent, so a player standing still costs no bytes on the wire. Then motion walk is one, and motion run is two. Player gets yaw, a float, for the direction it faces, as field five, and motion, as field six. Like we saw in episode six, a client built before this change skips the two fields it doesn't know, so the contract grows without breaking older clients. In the world's header, a Motion of our own, an enum class with idle, walk and run, like the window's visibility in episode two. The world is plain game logic, which we test on its own, so it doesn't include the generated code. And the server converts one type into the other when it sends the world out. The player gets a yaw as well, and over it a comment: in radians, around the vertical, and zero faces along z, towards the camera. Radians are what the C++ maths functions take and give back. Then its motion, which starts out idle, and input run, which is whether its client asked to run, next to the direction it asked for. The old speed goes, with its comment, as four tiles a second was fine for a cube, but a character's legs move at the speed of their clip. So k walk speed is one point eight, with a comment: in tiles a second, where a tile is a metre, and that's the walk clip's speed. K run speed is three point one, and both are the speeds our merge tool printed. Set input takes run as well, as a fourth argument. In the world's source file, set input takes run, a bool, and stores it with the direction. We don't need to check it, as it's true or false, unlike the floats, which can be not a number. In step, the old lines that moved the player go, as we need to know how far it went. Speed is the run speed if the player asked to run, and the walk speed otherwise. Then we work out where the player ends up, x and z, clamped to the board like before. They go into locals first, as we still need the old position for a moment. D x is how far the player went along x, which its motion and its yaw are going to come from. Over it goes a comment: motion comes from how far it actually went, so that a player pushing against the edge stands still, instead of walking on the spot. D z is how far it went along z, towards the camera or away from it. If it didn't move at all, it's idle, and we leave its yaw alone, so a player who stops keeps facing the way it went. Otherwise, it walks or runs, as its input says, and its yaw comes from a tan two of d x and d z. A tan two, from cmath, gives the angle of a direction from its two sides, the right way round in all four quarters. A plain arc tangent of d x over d z can't tell a direction from its opposite, and it divides by zero going sideways. The arguments go x first, so that zero faces along z. Last, we store the new position in the player, now that we're done with the old one.

18:07 Then the server has to send all of this out to the clients. In server dot c c, at the top, in an unnamed namespace, we write to protocol, which takes our Motion and returns the matching value of the protocol's. We could cast one to the other, as the values line up today. However, the day someone reorders either enum, a cast would quietly send the wrong motion. So a switch maps it case by case, whatever order the values are in. Walk and run return the protocol's values, while idle breaks out of the switch, to the return at the end. We leave out a default case, so that the compiler warns us when Motion gets a value the switch doesn't handle. The return after the switch is still needed, as an enum can hold a number outside its list, and the compiler wants a value on each path. In on message, where a client's input arrives, set input passes run along with the direction. And in broadcast, each player we send gets its yaw, and its motion, through to protocol. In the world's tests, first comes the numbers header, from C++20's standard library, for pi. The test that checked four tiles a second becomes walks at the walk speed. Set input takes false for run, and after a second of that, we expect x to have moved by k walk speed. We compare with the constant, so that tuning it later doesn't break the test. Then a second expectation, which is the new part: its motion is walk. Diagonals are no faster passes false too, and it still checks the distance covered along the diagonal, which is now the walk speed. Then a new test, faces the way it moves, as the clients take the yaw from the server and can't work it out on their own. The player steps once to the right, along x, for a single tick, k tick, and its yaw should be a quarter turn, pi over two. Pi v of float is the numbers header's pi, as a float, instead of typing out the digits. Then the player steps away from the camera, along minus z, and its yaw should be half a turn, pi. Two directions are enough to catch x and z swapped, or a sign the wrong way round. Keeps players on the ground now runs into the edge for ten seconds, with true for run, and the player should be at the edge, and idle. This pins the case from the comment in step, where the input still says run, but the player isn't going anywhere. And ignores input that is not a number passes false as well. Last, the protocol's round trip test sets a yaw and a motion on the player, and expects them back once the world has been serialised and parsed. The motion is run, and not idle, as idle is zero, which protobuf doesn't send, so the test couldn't tell a lost field from a zero.

20:46 Once it's built, we run the server's tests, with a filter for the world's. All six pass, with faces the way it moves and walks at the walk speed among them.

Animation in the client

20:56 The server sends the world twenty times a second, and the client draws sixty frames a second. With cubes, we drew each player where the latest world put it, so each cube moved in small steps, three frames at a time. The steps are hard to notice on a cube, but a character whose legs move smoothly while its body jumps forward stands out. So the client is going to follow the server's positions, instead of jumping to them, with each body closing part of the gap each frame. That puts it a few frames behind the server, and moving smoothly. When a player goes from walking to running, the run blends in over a quarter of a second, instead of cutting in. And when a player turns, the body turns at two turns a second, the short way round. All of this is presentation, so it stays in the client, while the server keeps deciding where everyone is and what they're doing.

21:46 All of that lives in a class of its own, which takes the worlds from the server and works out what to draw each frame. Its new header starts with its header guard, then raylib, for Vector3, its type for a point in 3D. Then, from the standard library, cstdint, for the fixed-size integer of a player's id. Map, for the figures we keep from one frame to the next, by id. Numbers, for pi, as we're going to turn players by angles. String, for the player's name, which the figure carries for its nameplate. And vector, for the list of figures we hand back each frame. Last, protocol slash nanommo dot p b dot h, our generated code, as the animator reads the server's worlds. Then a struct, Figure, which is a player as we're going to draw it this frame. It has the player's id, its name, and you, which is whether it's our own player, for the marker over it. Then its position, as a Vector3, and its yaw, which are where we draw the figure, smoothed from the server's. Then its pose, a blend of two clips, as a figure can be on its way from one clip to the next. From is the clip it's leaving, and from time is how far into that clip it is, in seconds. To is the clip it's going into, with its own time. And blend is how far the pose has got from the first clip to the second, from zero to one. It starts at one, fully blended, so that a figure that isn't changing clips plays its to clip, and nothing else. Then the class, Animator, where k blend time is how long a change of clip takes: a quarter of a second. A quarter of a second is long enough to hide the cut, and short enough that a player who starts running is seen to run right away. K follow rate is how quickly a figure catches up with the server's position, fifteen a second. Its comment says what the number means: the share of the gap still left after a second is e to the minus this. E to the minus fifteen is next to nothing, and at sixty frames a second, a single frame covers about a fifth of the gap. K turn rate is how fast a figure turns: four pi radians a second, which is two whole turns. Two turns a second is quick, while a U-turn still takes a visible quarter of a second. Update is called once a frame, with the latest world and the seconds since the last frame, and it returns the figures to draw. It takes the seconds as an argument, instead of asking raylib for the frame time, so that a test can step it by any amount of time without a window. It isn't const, as it changes the figures the animator keeps. And privately, the figures we keep, in a map by player id, so that each frame carries on from the last one. We find them by id, and not by their place in the world's list, as a player's place moves whenever someone ahead of them joins or leaves. In animator dot c c, our own header comes first, then raymath, which is raylib's vector maths, for moving a point part of the way towards another. Then algorithm, for clamp and min, and cmath, for the remainder and the exponential. Map, for the figures we keep, and numbers, for pi again. Utility, for move, and vector, for the figures we return, as each file includes the standard headers it uses, even when our header brings them in. First, in an unnamed namespace, a helper called turn, which turns from one angle towards another, by no more than step. Angles wrap around, so there's a short way round and a long way. Remainder, from cmath, divides the gap by a whole turn, two pi, and gives back what's left, between minus pi and pi, which is the short way. Then clamp keeps the turn within the step, so that a figure turns at its rate, instead of snapping round in a frame. Then update, where next is the map of figures for this frame, and figures is the list we return, in the order the server sent the players. For each player in this world, target is where the server says it is, on the ground, and we look for the figure we already have for that player. If there's one, we carry on from a copy of it. Otherwise, it's a new player, whose figure starts at the target. So a player who logs in appears where they are, instead of sliding in from the middle of the board. It also takes the server's yaw, and the server's motion as both of its clips, so there's nothing to blend, in a designated initialiser like we used for the cubes. Then we fill in what we take from the server as it is, each frame: the id, the name, and whether it's us. Then both clip times move on by the frame's time, so the clip we're leaving keeps playing while it fades out, instead of freezing mid-step. If the server's motion has changed, the clip we were going into becomes the one we're leaving, with its time. The new one starts from zero, and so does the blend. Then the blend grows by the frame's time over the blend time, and min stops it at one. Then the position, where follow is how much of the gap to close this frame: one minus e to the minus the rate times the time. Closing a fixed fraction each frame would look right at sixty frames a second, and twice as slow at thirty. Worked out from the time, it catches up at the rate we set, whatever the frame rate. Vector3 lerp, short for linear interpolation, moves the position that share of the way to the target. And the yaw turns towards the server's, by up to the turn rate times the time. We keep the figure in next, and move it into the list. Last, next replaces our map, and as a player who left isn't in the world any more, it doesn't make it into next, so we need no code to remove players. The tests don't need a window, as the animator is plain maths and doesn't draw, which is part of why it's a class of its own. We include our header, then numbers, for pi, and then GoogleTest. In an unnamed namespace, a helper, world with, takes a position on the board, as x and z, then a yaw, and a motion. It builds a world as the server would send it, with one player in it: player one, called alice, who is us, at the position, the yaw and the motion we passed in. Each test then feeds the animator a few of those worlds, frame by frame, the way the client would. Blends into the next clip is the first, where the player walks for half a second, and then the server says it runs. A tenth of a second later, the clip the figure is leaving should be the walk, and the one it's going into should be the run. The walk's time should have carried on, to point six, instead of starting over. And the blend should be at point four, as a tenth over a quarter of a second is four tenths. Then two tenths of a second later, the run is two tenths in, and fully blended. If the blend jumped, or the walk's time reset, we'd see the legs pop, and this test would fail. Catches up with the server over a few frames is the second test. It makes an animator, and starts the player in the middle of the board, with a frame that takes no time, so the figure starts there too. Then the server moves the player one tile along x, a bigger step than a tick would make, so the smoothing is easy to see in numbers. We run one frame, at sixty frames a second, and read the figure's x, which should be part of the way there. Part of the way means more than a tenth of a tile, so that it moves, and less than half, so that it doesn't jump. Then a loop runs fourteen more frames, which makes fifteen, a quarter of a second. Expect near compares two floats within a tolerance, as the exponential isn't going to land on a round number. By now, the figure should be within three hundredths of a tile of the server's position. Turns the short way round is the last test, which keeps pi in a short local, as it uses it three times. The player starts facing just under half a turn, pi minus a tenth. Then the server says it faces minus pi plus a tenth, which is a fifth of a radian away, across pi. The figure should take that small step, and end up at pi plus a tenth, and not turn most of the way back round, through zero. Remainder is there for this case, as without it, players would spin the long way round whenever they cross that line. Last, in the client's CMakeLists, animator dot c c joins the core library, and its tests join the client's test executable, next to the renderer's tests.

30:01 Before we load the characters, let's look at how a skeleton moves a mesh. Each vertex is bound to up to four bones, with a weight for each, and the weights add up to one. To pose a vertex, we move it with each of its bones, and blend the results by the weights. Each bone's matrix comes in two steps, multiplied together into one matrix. The inverse bind matrix takes the vertex from where it sits in the T-pose into the bone's own space. Then the pose matrix takes it from there to where the bone is, in this frame of the clip. raylib does this on the CPU by default, for each vertex in each frame, and then uploads the moved vertices to the GPU. With thousands of vertices a character and dozens of players, that's a lot of work for the CPU. So we're going to have the GPU do it, in our vertex shader, where the vertices already are. And the CPU sends no more than a matrix for each bone.

30:54 Now we can load the characters into the client, starting with the build. In the client's CMakeLists, characters dot c c joins the core library. The characters have to be next to the client when it runs, so we add a custom target, nanommo client characters. A custom target is a step of the build that runs a command, instead of compiling something. ALL makes it part of each build, and the command is CMake, with dash E, which gives us file commands that work on any platform. Copy directory if different copies the folder from assets slash characters, in the source tree, to a characters folder next to the binaries, and copies nothing when nothing changed. Add dependencies makes the client wait for that target, so the characters are in place before it runs. The tests get a third file, characters test, and they wait for the copy too. We copy at build time, and not once when CMake configures, so that characters you make later reach the next build without configuring again. Then a new header, for the class that holds them, starting with raylib, for its Model, Shader and Color types. Array comes next, for a list with a size fixed at compile time. Then cstddef, for size t, and string, for the folder. And vector, for the characters we load, and our animator, for Figure, which is what we're going to draw. Characters folder returns the folder the build copies the characters into, so that the client and its tests look in one place. Then the class, Characters, which holds the animated characters on the GPU. Its constructor takes the folder, and the shader to draw them with, as the renderer owns the shaders. A folder without characters is fine, and leaves the renderer drawing cubes. The destructor unloads them from the GPU, and copying is deleted, like our other classes that own something on the GPU. Size says how many we loaded, with a lowercase name, as the guide allows for a plain accessor. Draw draws a figure as one of the characters, tinted with a colour, and it's const, as drawing doesn't change which characters we hold. Privately, a struct, Character, for each one we load, with the model, which is the meshes, the materials and the skeleton. Then the clips, which raylib hands us as a pointer to the first one, and a clip count. And clip for says where each motion's clip is in that list, as a file doesn't have to list its clips in our order. Last comes the list of characters we managed to load. In the source file, after our header, cmath, for f mod, and c string, for comparing the clips' names, which raylib gives us as C strings. And string, for the paths we build out of the folder's name. In an unnamed namespace, k files are the two files the asset pipeline makes, x bot and y bot. It's a constexpr std array, so the list is fixed at compile time. K clips are the three clips a character needs, by name, in the protocol's order of motions: idle, walk and run. That order matters, as a motion's number is going to be its place in this list. Then frame at turns a time into a clip into a frame number, as raylib poses a model by frame. Its comment reads: raylib samples glTF clips sixty times a second, and the last frame is the first one again, so a clip loops after one frame fewer. So frames is the keyframe count less one, and a clip with a single frame keeps one, so that we don't hand f mod a zero. F mod, the remainder for floats, wraps the time around the clip so that it loops, and we keep the fraction, as raylib blends between two neighbouring frames. Characters folder is get application directory, the folder the executable is in, plus characters, so it doesn't depend on which folder you start the client from. The constructor goes through the files, and skips one that isn't there, which is the case on a machine that hasn't made the characters. Otherwise, load model reads its meshes, its materials and its skeleton, and load model animations reads its clips, and tells us how many there are. Then we look for the clips we need, and complete starts out true. We go through the motions, in the order of k clips. And for each one, found starts at minus one, for not found yet, as we go through the clips the file has. A clip is the one we want when its name matches, which string compare checks, as equals on two C strings would compare their addresses. And is model animation valid has to agree, as it checks that the clip has as many bones as the model. So raylib doesn't pose the wrong bones, or read past the end of the list. Complete stays true while each motion finds its clip, and clip for keeps where each one is. If something's missing, we log a warning with raylib's trace log, unload the clips and the model, and skip that character. So the game still runs, with the other one, or with cubes. Otherwise, each of the model's materials gets our shader, and the character goes into our list, with push back. The destructor goes through the characters we kept, and unloads the clips, then the models, which frees their buffers on the GPU too. Then draw picks the character by the figure's id, modulo how many there are. So each player keeps its character, and all the clients agree on it without the server having to say. Then a lambda, clip, finds the clip for a motion, capturing by reference to reach the character we picked. Its arrow says it returns a reference to the clip, so that we don't copy a clip and all its frames. A motion this client doesn't know, say from a newer server, plays idle, instead of reading past the end of our list, as protobuf keeps enum values it doesn't know. Then the lambda returns that motion's clip, wherever clip for found it in the file. From and to are the two clips the figure is between. And update model animation E X poses the model's bones between the two, each at its own frame, by the figure's blend. Above it, we write why: players share the model's bones, so we pose them right before each draw. Say twenty players are X Bots, then that's one model drawn twenty times, each time with its own pose. Last, draw model E X draws it at the figure's position, turned around the vertical axis by its yaw, which RAD two DEG converts from radians to degrees for raylib. The scale is one, and it's tinted in the player's colour.

37:22 Next, we're going to move the skinning onto the GPU, into our own vertex shader. In the top-level CMakeLists, right after fetching raylib, we give the raylib target a compile definition: support GPU skinning, set to one. It's a switch in raylib's config, read when raylib's sources are compiled, so it goes on raylib's target, as private, since nothing else needs to see it. With it on, raylib uploads each vertex's bone indices and weights to the GPU, as two more vertex buffers. Then it hands our shader the bone matrices, instead of moving the vertices with them. raylib leaves it off, as some GPUs can't take more than eight vertex buffers. Our comment says that by default, raylib skins on the CPU, and this leaves it to our vertex shader. In the renderer's header, we include the animator's header and the characters'. The renderer gets the figures, the players to draw this frame, next to the cubes, which are the ground now. Like the background and the cubes, the list is private, with accessors. Figures, in lower case, hands the list back by const reference, for the nameplates to read. Set figures takes a new list by value, and moves it in, so the list the animator hands back each frame isn't copied. Privately, the figures go after the two members the class already had. Then the shader becomes two: the cube shader, and the character shader, as the characters' vertices need skinning, and the cubes' don't. And the renderer owns the characters, declared after the shaders, as members are built in the order they're declared, and the characters need their shader to exist first. In the renderer's source, we include the scene header, for the players' colours and the marker's height. Then the skinned vertex shader, next to the cubes' one, as another raw string. It's GLSL version three thirty, and its inputs start like the cubes', with each vertex's position and normal. Then two more: vertex bone indices says which bones pull on this vertex, and vertex bone weights says how hard each one pulls. Four of each, in a vec4, which is why our build script kept four weights for each vertex. Those names are the ones raylib looks for, so it binds its buffers to them without us asking. The M V P matrix and the normal matrix uniforms are there like before. Then bone matrices, an array of up to a hundred and twenty-eight matrices, one for each bone, which raylib fills in from the pose. And the array's size leaves room over our sixty-five bones. Out float light goes to the fragment shader, like the cubes'. In main, skin blends the four bones' matrices by their weights, over four lines. Each line takes one of its four bones, looks up that bone's matrix, by the index it came with, and scales it by that bone's weight. The four of them add up to one matrix for this vertex, which moves it the way its bones pull. The normal turns with the skin as well as with the model, so that an arm that bends gets its light from the way it faces now. The sun and the light are worked out like the cubes', and the position is the vertex moved by the skin, then by the M V P, onto the screen. The fragment shader stays as it was, and both shaders use it, as a character is lit and coloured like a cube. In the constructor, the old shader becomes the cube shader, loaded from the cubes' vertex shader as before. Then the character shader, from the skinned vertex shader and our fragment shader. Then the cube, and the characters, from the characters folder, with the character shader. The cube's material gets the cube shader, and the destructor unloads the cube's model, and then both shaders. In draw, after the cubes of the ground, we go through the figures. Each one gets its player's colour, from its id, and if we loaded any characters, it's drawn as one of them, in that colour. Otherwise, as on a fresh clone, we fall back to what we drew before. At is a short name for the figure's position, and the cube goes half a unit up, so that it sits on the ground, tinted in the player's colour. Then, if the figure is us, we draw a small cube over it, at the marker height, a quarter of the size, and in white. So you can find your own character in a crowd. In the scene's header, raylib comes in, for Color, and cstdint, for the id. The generated protocol header goes, as the scene doesn't read worlds any more, and so does the scene function, as the players are figures now, drawn by the renderer. K marker height is how high the marker floats: two point oh five, over a character's head, which is about one point eight metres up. And player color gives a player's colour from its id, for the renderer to colour a character or a cube with. In the scene's source file, the marker's colour goes, as the marker is white now, and so does the old scene function. Player color picks from the list of colours by the id, like the scene function did. And as ids start at one, we take one off before the modulo, so player one stays orange. Back in the shader, these four lines are the sum from the formula, run by the GPU for each vertex. The bones' matrices come from raylib, which uploads them with each draw, after we've posed the model.

42:41 The nameplates still follow the server's positions, so they'd run ahead of the smoothed bodies, three frames at a time. So they take the figures instead, and in the nameplates' header, the generated protocol header goes. Vector comes in, for the list of figures, and so does our animator's header, and draw nameplates takes the figures, instead of the world. In the nameplates' source file, vector comes in here as well, as this file names the list of figures too. The function takes the figures, like in the header, and the loop goes over them, where it went over the world's players. Position is a short name for where the figure is drawn, the smoothed spot. Each name now goes at two point three five metres, where it was at one point nine over a cube. So it sits just over the marker, above the character's head. And the name text comes from the figure, which copied it from the server. Next, the client needs a way to ask for a run. In the input header, running says whether the player wants to run, next to movement, which gives the direction. In the source file, it's true while either Shift key is down, left or right, through is key down, like movement uses for W, A, S and D. In main, we include the animator, and once the renderer is up, set cubes gives it the ground, a single time, as the ground doesn't change. Before, we rebuilt the cubes from each world, as the players were cubes too. Then we make the animator, so that it lives as long as the game loop. In the loop, run is whether Shift is down, and we send our input when the direction or run changes. So holding Shift sends one message, and not one each frame. And the input we send carries run, next to the direction. Then, each frame, set figures hands the renderer what the animator's update returns, with the latest world and get frame time, the seconds the last frame took. And the nameplates are drawn from those figures, so the names stay over the bodies.

44:42 Let's build it and see our characters, before we write their tests. I log in as alice, and where the cube used to be, a character stands, playing its idle clip. W walks the character up the board, facing the way it goes, and with Shift held, it runs to the right. When I let go, the animator blends back into the idle, over a quarter of a second.

45:06 Now the characters' tests, which include our characters header first. Then memory and Abseil's status or, as our window comes in a unique pointer, inside a status or. Then, from our client, the camera, the offscreen target, the renderer and the window, as one test is going to draw a frame, and GoogleTest last. Have characters looks for x bot dot g l b in the folder the build copies the characters into. The characters aren't in git, so on a machine that hasn't made them, such as CI, there's nothing to test against. So both tests start with GTEST SKIP, like the database tests in episode five, with a message that says where to look. A skipped test shows up as skipped in the results, so it doesn't pass quietly, and a missing file doesn't fail the build for someone who just cloned the repo. Loads both bots with their clips creates a window, sixty-four pixels square, and hidden, as raylib needs an OpenGL context to load models onto the GPU. After assert true, we keep the status or around, as it owns the window, which has to stay open until the test ends. Load shader from memory, with no code of our own, gives raylib's default shader, which is enough for loading. Then it loads the characters from the folder, and expects two, as a character whose clips don't fit its bones would have been skipped, with a warning. Draws a player standing in the middle checks that a character makes it onto the screen, and it's skipped without the characters too. Then, like the first test, it creates a hidden window and asserts that it's OK. Then an offscreen target of that size, which is a texture we draw into and read back, like in the renderer's tests. Next, a renderer, which loads the characters with its own shaders, and set background makes it a bright blue. And set figures gives it player one, with nothing more than its id, so it stands at the origin, in the middle of the board. We render a frame, and then get world to screen E X turns a point at chest height, one point four metres up, into a pixel of our sixty-four by sixty-four picture. The comment over it says: chest height, higher than a cube reaches, is orange: player one's colour. As a cube reaches up to one metre, it can't pass this test by accident. We read that pixel, and orange means more red than green, and more green than blue, which the blue background isn't.

47:25 Then we run the client's tests, where the renderer's three pass like before, and so do the animator's three. With the characters in place, the characters' two tests run instead of skipping, and pass too, so all eight of the client's tests pass.

Bots that run

47:41 Now, the bots should run some of the time, like players would, which keeps our load tests in step with what the game does. In wander dot c c, random direction, which picks the keys a bot holds, now holds Shift as well, some of the time. A Bernoulli distribution is the standard library's coin toss, with a chance we choose, here one in three, drawn from the generator we already pass in. We call it shift, after the key it stands for, and what it draws sets run on the input. A third keeps most of the bots walking, with a few of them running at any time. And a new test, runs about a third of the time, seeds the generator with one, draws a thousand directions, and counts the runs. We expect more than two hundred and eighty, and fewer than three hundred and ninety. The seed makes the count repeatable from run to run on one machine. However, C++ doesn't say how a distribution turns the generator's numbers into values, so the standard library on the Mac and the one on Linux can give different counts. A range passes on both, and still fails if the chance were a half, or a tenth.

48:47 Let's play it together, with two clients and a crowd of bots. The login server and the game server are at the bottom, and two clients log in, as alice and as maria. Alice walks up the board, and both windows show her walking, facing the way she goes. Now she holds Shift and runs to the right, and in Maria's window she runs too, as the server sends both clients her motion. Maria walks down and to the left, and stops, and her idle starts again. Next to the servers, our bot program starts twenty bots, over six seconds. They come in walking, running and standing around, each in its player's colour, and the two windows match. Each client plays the clips at its own pace, while the positions and the motions come from the server.

Docs

49:31 Last, the README, where each player is an animated character now, or a cube until you've made the characters, and Shift makes yours run. A new section says how to make them: what to download from Mixamo and where to put it, as those files can't be in our repo, then the docker build that converts and merges them. Without them, the game still runs, with cubes, and the client skips its character tests.

Wrap-up

49:53 So, today we took two characters and three clips from Mixamo, and kept the files out of our public repo, as Mixamo's terms ask. FBX2glTF converts them to glTF, in Docker, and our merge tool gives raylib what it expects: one skin for the bones, the clips matched by name, the hips in place, and a tenth of the triangles. The server decides which way each player faces, and whether it walks or runs, at the speeds of the clips. The client follows the server smoothly between ticks, and blends from one clip into the next, with the skinning done on the GPU.

Playing it

51:22 Let me know in the comments what you'd like to see next, and if you want to follow along, subscribe. I hope you find this useful, and thanks for watching.