TCP vs UDP for a game server, under packet loss

Intro

0:00 Hello and welcome back to episode nine, where we're going to revisit a choice we made in episode three. We picked TCP for everything the server sends, and we're going to measure it against UDP, on a network that loses packets. Let's begin with a quick look at where we're going to be by the end of the episode.

0:19 Two clients play in one world, on a network that drops five percent of the packets. On the left, the world comes over TCP, and the players stop and catch up in jumps. On the right, it comes over UDP, and they keep moving.

0:33 On the website of the series you can find more information about the project. The code for the project is open source on GitHub, tagged at each step of each episode so that you can follow along.

Last time

0:45 Now, before we get to today's work, let's quickly go over what we did last time. We wrote a benchmark that adds bots in steps, and reads the server's own metrics from Prometheus at each step. On my machine, the server fell behind past a thousand players. perf and flame graphs showed us where its time went, and Tracy showed us each tick on a timeline. Both pointed at the broadcast, which serialised the whole world for each client. Now it serialises the players once a tick, and all the sessions share the bytes, so the server keeps up with more than twice the players. All right, let's get started, as this time it's the network that we're going to make worse.

Today

1:25 In episode three, I said that we'd come back to UDP once we could measure the difference, and now we can. We have bots that play like people, and Prometheus to measure them. We're still missing a bad network, and a number for what a player sees. So first, we're going to measure how old the world on a player's screen is. Then we make the network worse, with netem, which comes with Linux, and measure TCP as more and more packets get lost. Then we send the world over UDP, and run the benchmark again. And then we let the numbers decide which transport the world goes over. I have a guess, like you probably do, and we measure so that we don't have to go by it.

The question

2:06 Let's quickly go back over what TCP does when a packet gets lost, from episode three. TCP gives us a stream of bytes that arrives reliably and in order. So when a packet gets lost, the kernel at our end holds on to everything after it, even what has arrived already, until the lost packet has been sent again. The holding on is called head-of-line blocking, and for positions, it means waiting for an old world while a newer one is sitting there. UDP has no stream at all, and each send goes out as one message, called a datagram, which arrives whole or not at all, and isn't sent again. So a lost world is gone, and the next one, with newer positions, is handed over as soon as it arrives. So we're going to measure how much that waiting costs a player, on a network that loses some of its packets.

What a player sees

2:55 So the number we want is about what a player sees, which is the world on their screen. We're going to measure how old that world is, which is how long ago the server decided where everyone was. So the server is going to put the time into each world it sends, in microseconds since the start of 1970, which is how Unix counts time. Each bot keeps the newest world it's had, as that's the one a client would draw. And about sixty times a second, like a client drawing its frames, the bots look at how old their newest world is. And that age is the time now, minus the time the server sent it. With worlds fifty milliseconds apart, that's somewhere between zero and fifty, plus however long the world took to arrive. When worlds get held up on the way, the age grows, as the bots keep looking at an old one. The ages go into a histogram, like the tick durations in episode four, and Prometheus scrapes it from the bots. Comparing the server's clock with the bots' works here, as they run on one machine and share a clock. Two machines' clocks can be milliseconds apart, so a real client would have to estimate the difference.

4:03 We start in the protocol, in nanommo dot proto, where World gets a third field, sent, for when the server sent it. It holds microseconds since 1970, so it's a sixty-four-bit int, as thirty-two bits would hold about an hour of microseconds. A client from before today doesn't know field three, and protobuf skips a field it doesn't know, so older clients keep working. The server and the bots both need the time in that form, so it gets a header of its own in the protocol folder, clock dot h, with its header guard. Then chrono for the clocks, like in episode one, and cstdint for int64 t. In our namespace, now micros is inline, like before, as its body lives in this header. Above it, we write why: the system clock, not the steady one, so that a client can compare it with its own, as long as their clocks agree. Our ticks use the steady clock, which can't jump when the time of day gets corrected. However, it counts from whenever the machine started, so we can't compare it with another machine's, while the system clock is the time of day, which both ends have. So we return the system clock's now, as the time since the epoch, and duration cast converts that into microseconds. Then count turns it into a plain number for the message. In server dot c c, we include clock dot h, and in broadcast, once all the players are in the message, we set sent to now micros. That comes before the players are serialised, so the time goes into the bytes that all the clients share. So the age we measure covers serialising, the kernel, the network, and the wait until the next frame at the other end.

5:42 Next, the bots measure the age, and for that they need metrics of their own, like the server got in episode four. So a new header goes into the bot's folder, bot metrics dot h, with its header guard. The standard library's chrono, memory and string come first, for the age's duration, a shared pointer and the transport's name. Then prometheus-cpp's histogram and registry, as the ages go into a histogram, like the server's tick durations. The class is BotMetrics, and its constructor takes the transport, which goes into a label. So later today, the ages over TCP and over UDP can sit side by side, under one name, and the constructor is explicit, like before. Observe age records one age, and it takes it as microseconds, a chrono duration, so that a caller can't hand it milliseconds by mistake. Registry is a const function that the exposer reads our metrics through, like the server's. The members are the registry, in a shared pointer, and age seconds, a reference to the histogram that the registry owns. So observing an age is a single call on the histogram, without looking it up by its name and label each time. In bot metrics dot c c, we include its own header first, and then string. The constructor makes the registry and the histogram like the server's metrics, with the registry declared first, so it's there by the time the histogram registers in it. The histogram is nanommo bot world age seconds, which ends in its unit, like the tick's, and its help text says how old each bot's newest world is. Add makes its series, with transport as the label, and then come the bucket boundaries. The boundaries have to be narrow where the values land, as a percentile from a histogram is an estimate, spread across the bucket it falls in. A world is up to fifty milliseconds old by the time the next tick replaces it, plus the time it took to arrive. So the buckets go up in tens of milliseconds, to a fifth of a second, where most of the ages are going to be. After that, they get wider, up to five seconds, so that a bad network still lands in a bucket. Observe age turns the microseconds into seconds, as a double, with a chrono duration of double, and observes that. Now the bot has to read the worlds, which until today it threw away. In bot dot h, we include string view, to pass a message around without a copy. And framing dot h, for the frame reader from episode three, which turns the stream of bytes from TCP back into whole messages. Newest is a new public function, which returns when the server sent the newest world we've had, in microseconds, or zero before the first one arrives. Then a private function, receive, which takes a message's bytes, and two members: the frame reader, and newest underscore, which starts at zero. In bot dot c c, we include algorithm, for max. Over read, the comment from episode four said that the bot ignores the world, which isn't true any more. Now it says that reading each world keeps the server's writes to us from backing up. If a bot stopped reading, its socket's buffers would fill, and the server's writes to it would pile up in its session. Read now needs the size of what arrived, as the bytes go into the reader with append, and while next hands us a whole message, receive gets it. After the loop, a reader whose status isn't OK, say after a message over our sixty-four kilobyte limit, means a broken stream. So we treat it like an error on the socket, and the bot disconnects, cancels its timer and stops reading. Then receive parses a World from the bytes, and if that works, keeps the larger of newest and the world's sent. Over TCP, that's simply the latest, while over UDP, later today, worlds can arrive out of order. Then the bot's main dot c c, where the swarm does the measuring. We include our bot metrics, then prometheus exposer, like the server's in episode four, and clock dot h, for now micros. K metrics port is nine four six five, next to the server's nine four six four, so that both can run on one machine. Then k frame, sixteen milliseconds, as a client draws about sixty frames a second, and each frame shows the newest world it has. Its comment says why sixteen: sixty frames a second, near enough, and sixteen doesn't divide the tick's fifty, so the frames land all over the tick, like a client's would. Sixteen and two thirds would be three frames to a tick, landing at three points of the tick and staying there, so the ages would depend on which three. With sixteen, each tick's frames land two milliseconds earlier than the last, so over a second or so, they cover all of it. The swarm's constructor takes the metrics by reference, as main owns them, and a third timer, frame, joins the two for adding bots and reporting. At the end of the constructor, the frame timer waits a frame, and then calls look. Look reads now micros once, at the start, so that all the bots are measured against one moment. Then, for each bot that's connected and has had a world, it observes now minus that bot's newest, in microseconds. A bot without a world yet would report an age of over fifty years, so look skips it. Then expires at takes the last expiry plus a frame, like the server's tick, so that the frames don't drift later when look takes a while. If the wait is cancelled, as when the program stops, we stop looking. Down in the members go the metrics and the frame timer, and in main, metrics is a BotMetrics labelled tcp, as that's how the worlds come for now. The exposer listens on the metrics port, on all the container's addresses, so that Prometheus can reach it from its own container. Register collectable hands it the registry, and last, the swarm gets the metrics.

11:29 In the bot's CMakeLists, the core library gets bot metrics dot c c and prometheus-cpp's core, and the executable links pull, for the exposer in main, like the server does. Then Prometheus has to find the bots, and it can't do that like it does for the server, which is a single container with a fixed name, listed as a static target. The bots come and go, though, and with docker compose run, there can be several containers of them at once, each with an address of its own. So in prometheus dot yml, a second job, nanommo bots, uses DNS service discovery. Prometheus asks Docker's DNS for the name bots, with type A, for addresses, and Docker answers with one for each container that goes by that name. Prometheus scrapes port nine four six five on each of them. And refresh interval asks again after five seconds, instead of the default thirty, so that a new swarm shows up on the dashboard quickly. And the dashboard gets a panel along the bottom, world age at the bots, which is JSON that Grafana writes for us, so I'm pasting it in. It draws the p50 and the p99 of the age, from the histogram, with a line for each transport.

A worse network

12:36 To make the network worse, we're going to use netem, which is part of Linux. Its command comes in iproute2, the package with the ip and tc commands. netem adds delay, packet loss, duplicates and more to the packets going out of a network interface. tc, for traffic control, sets it up as a queueing discipline, a qdisc, which is what Linux calls the code that decides when each packet goes out. Limit is how many packets it holds while they wait. And delay holds each packet back for a while, which is how we're going to pretend that our bots are some way from the server. And loss random drops a share of the packets, each with its own chance, independent of the others.

13:15 Here's where netem goes, starting with our server's container, which has a network interface of its own, eth zero. netem works on the packets going out of an interface, so we put it on the server's, where the worlds leave. Each packet waits fifty milliseconds on its way out, while the bots' packets to the server aren't held up. So a round trip takes fifty milliseconds, which is a server on your continent, some distance away. And we drop none of the packets, then one percent, two, five and ten. A few percent is what a bad Wi-Fi signal or a crowded mobile network can do. The kernel hands the network card a large write in one piece, up to sixty-four kilobytes, for the card to cut into packets. Leaving the cutting to the card is called segmentation offload, and it saves the CPU a lot of work. However, netem sees that piece before it's cut up, so it would drop forty-odd packets at once. So we turn the offload off, with ethtool, and netem drops packets one at a time, like a real network would. netem's limit would add losses of its own, as it's a thousand packets by default, and it drops any more. With two hundred bots, our server sends around fifteen thousand packets a second, so fifty milliseconds' worth comes close to a thousand, and a tick's burst goes over. So we give it room for a hundred thousand packets, far more than it's going to hold. To change a container's network, tc needs a Linux capability, NET ADMIN, so we give it to a small container of its own, which shares the server's network.

14:43 A new stage, netem, goes into the Dockerfile, after the bot's, and it starts from plain Ubuntu, as it doesn't need anything we build. Apt get installs iproute2, for tc, and ethtool, the way our other stages install their packages. Then we copy netem dot s h in and make it the entry point, so that running the image runs the script with the arguments we give it. Unlike our other images, this one stays root, as changing the network takes that. The script goes into bench, as netem dot s h, a bash script with set dash e u o pipefail, like load dot s h. Delay is the first argument, in milliseconds, fifty if we don't give one, and loss is the second, in percent, none by default. Then ethtool dash capital K, which switches an interface's offloads on and off. On eth zero, it turns off t s o, TCP segmentation offload, and g s o, the kernel's own version of it for cards that can't. Over it, we write why: without this, netem sees big writes before they're cut into packets, and drops them whole. Then tc qdisc replace puts netem on eth zero as its root qdisc, the one that all of its outgoing packets go through. Replace swaps out whatever is there already, so running the script again with other numbers changes them, and zero and zero give the server its network back. Then the delay, the loss, and limit, a hundred thousand packets. And the comment over it says that by default, netem drops anything past a thousand waiting packets, and a busy server goes past that. Last, tc qdisc show prints what's set now, so that we can see it took. In compose dot yaml, a netem service is built from that stage, and network mode, service colon server, puts it in the server's network namespace. It sees the server's eth zero as its own, so what tc changes there is the server's network. Cap add gives it NET ADMIN, and the server keeps running without it, so a bug in the server can't touch the network. And profiles puts the service in a profile called netem, like the load profile, so it stays out of a plain docker compose up. Instead, it runs when we ask, with docker compose run netem and the numbers. Then the benchmark, loss dot s h, next to load dot s h in bench, and built like it. It starts a swarm of bots, makes the network worse in steps, and asks Prometheus at each step how old the bots' worlds are. Bots is the first argument, two hundred by default, which gives the server some work while the machine running them all stays well within its limits, so what we measure is the network. Delay is fifty milliseconds, hold is sixty seconds, how long each step lasts, and the ready timeout is two minutes. Query asks Prometheus for a single number, like load dot s h's, and jq picks it out of the answer. This time, curl's dash dash fail and jq's dash e make it fail when Prometheus answers with an error, or has no number yet. Wait for query keeps asking, two seconds apart, until there is one, as a new swarm's metrics take a scrape or two to show up. Past the ready timeout, it says it timed out waiting, and returns an error, so that a broken stack doesn't leave the benchmark hanging. Otherwise, it prints the number it got, for the caller to keep. Stop removes the containers with our label, like before, and it also runs netem with no delay and no loss, so that the server gets its network back. Trap runs stop when the script exits, so a Control C halfway through doesn't leave the server on a bad network. Then we build the bots and netem, like in load dot s h, so that the run starts them from this checkout. Then the swarm starts with docker compose run, in the background, and dash dash use aliases gives the container the service's name, bots, on Compose's network. Docker compose run leaves that out by default, and without it, Prometheus's DNS lookup wouldn't find these bots. Our label is there too, for stop, and the bots get the login server, the game server, how many bots, and ten seconds to arrive. Window is the hold less ten, so we measure the last fifty seconds of each step, once things have settled after the change. Age is the sum of the bucket rates over that window, by l e, like the tick's in load dot s h. Before the first step, we wait for the bots' metrics, until Prometheus can give us a p50 of the age. So we start measuring once the bots are in and being scraped, instead of sleeping for a time we'd have to guess. Then the loop, over the losses: none, one, two, five and ten percent. Docker compose run netem changes the network, with the delay and this step's loss, and we sleep for the hold. The p50 and the p99 of the age come from wait for query and histogram quantile, times a thousand for milliseconds. Result is a JSON object that jq builds, like before, with the loss, the p50 and the p99 passed in with dash dash argjson. The object gets three fields, the loss in percent and the two ages in milliseconds, and map values rounds each of them to one decimal place. The next jq prints a line we can read to standard error, so that we can follow along in the terminal. And echo sends the object to standard output, which goes to a file. After the loop, jq dash s collects the objects into an array. Last, CI builds the netem image too, next to the others, so that a change that breaks it fails the build.

20:23 Let's run it, starting with docker compose up, which builds and starts the server, Prometheus and Grafana, with the new dashboard. The first build compiles everything, so that part is sped up. Then loss dot s h, with two hundred bots, into tcp dot json. Each step takes a minute, so this is sped up too. With no loss, the p50 is seventy-nine milliseconds, and the p99 a hundred and ten. At one percent, the p99 is a hundred and seventy-three already, and at five percent, nine hundred and sixty. At ten percent, the median world the bots see is five point zero seconds old.

21:00 Here's tcp dot json, without the ten percent step, which would squash the rest of the chart. With no loss, a world is up to a tick old, fifty milliseconds, plus the fifty it took to arrive. So the p99 sits just above the dashed line, at a hundred. As packets get lost, the p50 barely moves up to two percent, and at five, it's one hundred, as most of the time, no world is waiting. The p99 is what a player notices, as a stutter, and it climbs. The right side shows why it climbs, starting when a packet is lost. The client's TCP sees later packets arrive without it, and its acknowledgements tell the server, or the server's own timer runs out, at two hundred milliseconds or more on Linux. The server sends it again, which takes another trip, and all that time, the worlds behind it wait. TCP also takes a loss as a sign that the network is overloaded, and sends less for a while. For a download, backing off keeps the network usable for everyone, while for us, it means each world arrives later still. At ten percent, it can't send our worlds as fast as the server makes them, two hundred clients times twenty worlds a second. So they queue up in the server, and the bots see the world seconds late, and later and later. So far, the numbers match what we said in episode three: for a game like ours, TCP's occasional hiccup is fine on a good network. Next, we're going to see whether UDP does better on a bad one.

UDP

22:29 Here's how the world is going to get to a client over UDP. The client logs in, and says hello to the game server over TCP, with its token, as before. And that connection stays open, as the server keeps a player in the world for as long as it's connected, and it carries the input. Then the client sends its token again, in a UDP datagram, to the server's port, seven seven seven seven, which for UDP is a port of its own. Your router shares a single public address between all your devices, which is called NAT. And it lets a datagram in when it comes from somewhere one of your devices has already sent one to, and drops the rest. So the client has to speak first, and the server answers to the address the datagram came from, which the router passes on. The token tells the server which player it is, as the address alone could be anyone's. From then on, the server sends that player's worlds over UDP. Until the datagram arrives, the worlds keep coming over TCP, and if a firewall blocks UDP altogether, the player stays on TCP. As the hello can get lost too, the client sends it four times a second, until worlds arrive over UDP. And the input stays on TCP, as a client sends it when a key changes, so it's a few small messages, and each of them has to arrive. If a key release went missing, your player would keep walking.

23:50 The whole world in a single datagram is too large for one packet, as with two hundred players, it's about five kilobytes. Most networks carry packets of up to fifteen hundred bytes, which is called the MTU, and Docker's networks do too. IP sends a larger datagram anyway, by splitting it into fragments, here four, and the other end puts them back together. However, it hands the datagram over once all of its fragments are in. So losing any one of them loses the whole world, and at five percent loss, that's close to one world in five. Some routers and firewalls drop fragments altogether, too, so a player behind one of them would get no world at all. So, like most games, we split the world in our own code, into parts of up to twelve hundred bytes. Each part is a World of its own, with some of the players, and it fits in a packet with room to spare for tunnels and VPNs. A lost packet then costs us the players in that part, for one tick, until the next one arrives.

24:49 We start with the protocol again, where in nanommo dot proto, World gets two more fields. Part, field four, says which part of the world this is, counting from zero. And parts, field five, says how many parts the world was split into, so the other end knows when it has them all. Over TCP, the world comes whole, and both fields stay at zero, which protobuf doesn't send, so a world over TCP costs no more bytes than before. The splitting and the putting back together go into the protocol library, as the server and the client both need them, in a new header, parts dot h. First cstddef, for size t, and cstdint, for the ids and the time. Then three containers from the standard library, starting with map, which keeps values sorted by a key, as we're going to keep the players in one, by id. Set keeps values sorted, with no duplicates, for the parts and the players we've seen, and vector is for the list of parts. Then the generated nanommo dot p b dot h, for World and Player. K max datagram size is a constexpr of twelve hundred bytes, which split uses as its default. Its comment says that most networks carry fifteen hundred bytes a packet, less what tunnels and VPNs take, so IP doesn't have to split a datagram this size. The IP and UDP headers take twenty-eight of those bytes, and a VPN often sixty to eighty more. And QUIC, the protocol under HTTP/3, counts on twelve hundred getting through too. Then split, which the server calls each tick, takes a world and a maximum size, twelve hundred unless we say otherwise. It returns the world as a vector of smaller Worlds, each of which fits in a datagram. Then a class, Parts, for the other end, which puts the world back together. Add takes a part as it arrives, and world returns the world as far as we know it. The private part has what that takes: you, which player is ours, and sent, the tick we're on. Then the players, in a map by id, so that each part can find and replace the players it carries. Then two sets: arrived, the parts of this tick we've had, and seen, the players that were in them. They're private, so that add is the way in, and keeps them in step with each other. In parts dot c c, we include its own header first, then algorithm, for max, and vector, as split returns one. In an unnamed namespace goes k room, thirty-two bytes, as split fills each part up to the maximum, and each datagram also carries a few other fields. The comment over it lists them: room for the fields besides the players, sent, part, parts, and the client's own you. Sent is a large number and you can take five bytes, so thirty-two covers them all, with some to spare. Now split, which starts with a vector of one empty World, and a size of zero, for the bytes in the current part so far. We fill the parts by bytes, not by a number of players, as a longer name makes a larger player, so a fixed count could still go over. So for each player, bytes is what the player is going to take in the message. Byte size long is protobuf's count of the player's own bytes, and we add three to it. The three are a byte for the field's number, and up to two for its length, as a player in a list goes in with both. If the current part has players already, and this one would take it past the maximum, with the room counted, emplace back starts a new part, and the size goes back to zero. Checking for players first means that each part gets at least one player, even one that's too large on its own. Then we copy the player into the last part, and add its bytes to the size. Once all the players are in, each part gets the world's sent, so all the parts of a tick carry its time, and its index and the count. Now add, for the receiving end, where a part whose sent is older than ours is from a tick we've moved past. UDP can deliver datagrams out of order, so an old one can turn up after a newer one, and we drop it. A newer sent starts a new tick, so we keep its time, and clear both sets, arrived and seen. Then we keep you, note that this part has arrived, and copy each of its players into the map, by id. Each player's id goes into seen as well, so we know who was in this tick. Then, if as many parts have arrived as the tick has, we know who's in the world this tick. Max makes a whole world from TCP count as one part, as its parts field is zero. Erase if, from C++20, removes each player that the lambda returns true for, and the lambda captures this, to look in seen. It returns true for the players we haven't seen in any part this tick, as those have left. If a part doesn't arrive, the tick doesn't complete, and its players stay where the last tick put them. Last, world builds a World from you, sent and the players in the map. The map keeps the players sorted by id, so they come out in a steady order, whichever parts they arrived in. Then the tests, in parts test dot c c, which include parts dot h, cstdint, string, and GoogleTest. In an unnamed namespace, a helper, make world, takes how many players, the sent, and a shift, which is zero unless we give one. It sets the world's sent, and then a loop adds the players, numbered from one, with the loop's counter as each one's id. Each player also gets a name, bot and its number, and stands at x equal to its id plus the shift, which lets a test tell one tick from the next. The yaw and the motion are set too, so that each player takes as many bytes as one from the server. The first test, splits a world into datagrams, pins that each part fits in a packet. It splits two hundred players, with a sent from around now, in microseconds, which is a large number, and asserts that there's more than one part. Then a World with a you of four billion, an id that takes as many bytes as a you can. For each part, the part's bytes and that you together have to fit in twelve hundred bytes, which is what k room is there for. Each part has its index, the count and the world's sent, and the players in all the parts add up to two hundred. The second test, leaves the players of a lost part where they were, pins the behaviour we built Parts for. It puts a whole tick of two hundred players together, then splits the next tick, with a sent of two, and each player shifted by a half. The loop adds its parts from the second one on, so its first part is lost on the way, and the result still has all two hundred players. Player one, in the lost part, is still at x one, from the tick before, while player two hundred has moved on to two hundred and a half. And the third test, drops players who left and parts that arrive late, covers the two cases where add leaves something out. Assembled gets a world of three players, then a newer one of two, so a player has left, and then the older one arrives, too late. That leaves two players, as the late world is ignored, and the player who left stays gone. Last, the protocol's CMakeLists adds parts dot c c to the library, which the server and the client link already, so they get split and Parts with no other change. The test executable gets parts test, next to the protocol's other tests. So these few lines decide what a lost datagram costs us. Nobody waits for a missing part, and the cost is that a player who left can stay on screen until a tick arrives whole, usually a tick or two later.

32:16 Once it's built, we run the protocol's tests, and all ten pass, with the three new ones for parts among them.

32:24 Now the server, which has to take the hellos over UDP and send the parts, and first, it needs a UDP socket, open on our port. So in listen dot h, next to listen from episode three, we include asio's ip slash udp, with UDP's socket and its endpoints. Then bind takes a UDP socket and a port, and returns an Abseil status, like listen does. And its comment says that it opens the socket on the port, on all addresses, for datagrams. In listen dot c c, bind takes listen's steps for UDP, of which there are fewer, with ip udp from our header. The endpoint is our port, on any of the machine's IPv4 addresses, and the error code works like in listen. Open opens the socket, and if that worked, bind ties it to the port, with no listen step, as UDP has no connections to accept. If either step failed, we return an Unavailable error, with the port and asio's message, like listen does, and otherwise, the status is OK. In server dot h, we include array, for a buffer of a fixed size, and ip udp, for the socket. There are two new private functions: receive waits for datagrams from the clients, and send to sends a part of the world to a client's address. Send to takes the part as a shared pointer, so that all the clients share its bytes, like the TCP sessions share the players. And it takes own, the client's own you, by value. Then the members, where udp underscore is the UDP socket, and datagram is a buffer of a kilobyte for an incoming one. A kilobyte is enough, as all a client sends us over UDP is a hello with a token. Sender is an endpoint, an address and a port, for where the last datagram came from. Lower down, udp tokens maps the token each player joined with to its id, and udp clients maps a player's id to the address its worlds go to. In server dot c c, we include array and vector, then asio's buffer and socket base, for the socket's options, and parts dot h, for split. The constructor makes the UDP socket on the I O context, next to the acceptor, and leaves it closed, as opening it can fail, and a constructor can't hand back an error. Once listen has the acceptor, bind opens the UDP socket on that port, and if that fails, start returns its status. A TCP port and a UDP port are separate things, so both can be seven seven seven seven, and a player has a single port number to deal with. Then an error code called ignored, like the one for Nagle's no-delay in episode three, for an option that can fail without harm. Its comment explains that one socket sends all the datagrams, so it gets more room for those on their way out than the two hundred kilobytes Linux gives by default. That room is the socket's send buffer, and when it's full, a send has to wait, so the worlds would pile up in the server. With netem, each datagram also sits in the server's network stack for fifty milliseconds, and it counts against that room until it leaves. So set option asks for a send buffer of four megabytes. Then start starts receive, next to accept, so that the server waits for datagrams from the moment it's up. In handle hello, once a player's in, udp tokens keeps the token it joined with, so that a datagram with that token can find the player's id later. In leave, erase if takes the player's token out, and the player's address goes from udp clients, so the maps don't hold on to players who've gone. And if that player comes back from somewhere else, its worlds don't go to the old address. In receive, async receive from waits for a datagram, into our buffer, and fills in sender with where it came from. When one arrives, we try to parse it as a Hello, and if that works, we look its token up in udp tokens. If the token belongs to a player, that player's worlds go to the sender's address from now on. And that's the address the client's router showed us, so our replies get back through its NAT. We ignore a datagram that isn't a hello, or has a token we don't know, and receive calls receive again, to wait for the next datagram. Now broadcast, where the players are serialised for TCP like before, and then comes a vector for the parts. If any client is on UDP, split cuts the message into parts, and each part is serialised once a tick, into a shared pointer, like the players. And with no one on UDP, we skip that work. Then for each session, own is the client's you, as before, and if the player has an address in udp clients, we go over the parts. The comment over that loop reads: a datagram arrives whole or not at all, so it needs no size in front. Over TCP, a stream has no edges between messages, so each one needed its size in front. For each part, the bytes sent go to the metrics, and send to sends the part and own to the address. Otherwise, the session sends the whole world over TCP, as before. Last, send to, where own goes into a shared pointer too, called tail, as the send finishes after this function has returned. Buffers is an array of two buffers, the part and the tail. And async send to gathers them into a single datagram, like the session's write over TCP, so the part's bytes aren't copied for each client. The handler is empty, but it captures part and tail, and as its comment says, the handler keeps the bytes alive until the datagram has gone. If a datagram can't go, it's lost, like any datagram can be, and the next tick sends a newer world anyway.

38:01 Then the bots, which have to ask for their worlds over UDP too, and measure them. In bot dot h, we include vector, and asio's ip udp, for the UDP socket. The constructor takes udp, a bool, which makes the bot ask for its worlds over UDP. Newest changes, as over UDP, the world comes in parts, and each part can be lost on its own. So newest is a vector now, with the latest sent for each part. Over TCP, the world comes whole, as a single part, so the vector has one entry. Newest returns a const reference, so the swarm can read it without a copy. Then the new private functions: hello over UDP asks for the worlds, and receive over UDP waits for them. And then stop, as a bot now has a second socket and a second timer to stop. The members get a UDP socket, next to the TCP one, and a hello timer, for sending the hello again. Udp keeps the constructor's choice, and token keeps the token, for the hellos. A datagram buffer of two kilobytes fits a part with room to spare, newest becomes the vector, and udp worlds turns true once worlds come that way. In bot dot c c, we include chrono, for the hello's timer, and asio's error, for the error that a cancelled wait comes back with. The constructor takes udp, and sets up the UDP socket and the hello timer on the I O context, next to the others. Once the bot is connected over TCP, and it's on UDP, we look up the TCP socket's remote endpoint, its far end. The lookup fills in the error code we already have, as the TCP side could have closed in between. We reached the server on seven seven seven seven, and it takes UDP there too. So if that worked, the UDP socket connects to that address and port. Connecting a UDP socket doesn't send anything, and instead sets where our datagrams go. It also has the socket take datagrams from there and nowhere else. If that works too, we keep the token, say hello over UDP, and start receiving, and otherwise, the bot carries on with its worlds over TCP. In read, the two places that gave up on the connection call stop now, as stopping has more to it. Hello over UDP returns if the bot has disconnected, or if worlds come over UDP already, as then the server has our address. Otherwise, it sends a Hello with the token, in a datagram. The server may not have our token yet, as our TCP hello might still be on its way, or the datagram may get lost. Send takes an error code that we ignore, as a hello that fails is followed by another one. The hello timer expires after two hundred and fifty milliseconds, and when the wait is over without an error, we call hello over UDP again, until udp worlds is true. Receive over UDP waits for a datagram on the UDP socket, and returns if the bot has stopped, or the wait was cancelled, which asio calls operation aborted. Otherwise, a datagram means the worlds are coming over UDP, so udp worlds turns true, and the bytes go to receive, like a message from TCP. Then it calls receive over UDP again, to wait for the next datagram, like the server does. Now receive, which both transports feed, returns if the bytes don't parse, and otherwise keeps the latest sent for each part. Parts is at least one, as a world from TCP has none set, and newest takes that size. If the part's number fits, its place keeps the larger of what it had and this sent, and we check the number first, as it comes off the network. Stop is new too: it sets connected to false, cancels both timers, and closes the UDP socket, which cancels a receive that's waiting. Close takes an error code that we ignore, as the socket may not be open. In the bot's main dot c c, the swarm's constructor takes udp, after the number of its first bot, and before the metrics. The swarm keeps it, and passes it to each bot it makes. Look changes, as newest is a list now, and it skips a bot that isn't connected, with continue. Then for each part's sent in the bot's newest, if there's been one, it observes how old it is. So with five parts, a bot gives five ages a frame, each for a fifth of the players, like a screen where some players are a tick behind and the others aren't. In main, transport is the sixth argument, tcp by default for now, and it labels the metrics, so the dashboard can tell the transports apart. The line we print at the start says which one the worlds come over, and the swarm is on UDP when the transport is udp. In compose dot yaml, the server publishes seven seven seven seven over UDP too. A published port is TCP unless it says slash udp, so without this line, a hello from outside Docker wouldn't reach the server. In the Dockerfile, the server's image exposes it as well, and so does the profiling image. And loss dot s h takes the transport as its first argument, tcp by default, as it's the one we're going to change, and the bots move to the second. The swarm gets it after the ramp and the first bot's number, which stays one.

The benchmark again

43:11 Now we run the benchmark again, this time with the world over UDP. The stack is built from the new code, and loss dot s h runs with udp, into udp dot json, sped up again. With no loss, the p50 is eighty-one milliseconds and the p99 a hundred and ten, close to TCP. At one percent, the p99 is a hundred and eighteen, at five, a hundred and forty-seven, and at ten percent, a hundred and sixty.

The numbers

43:38 Here are the two side by side, again without ten percent. With no loss, they're level: a hundred and ten milliseconds at the p99 over TCP, and a hundred and ten over UDP, so on a good network, UDP doesn't make the world any fresher. Once packets get lost, they part ways, and the gap grows with the loss. At one percent, TCP's p99 is a hundred and seventy-three, and UDP's a hundred and eighteen. At five, it's nine hundred and sixty against a hundred and forty-seven, so UDP's p99 is fifteen percent of TCP's. A lost part costs its players a tick, fifty milliseconds, and no world waits for it. On the right, the p50 barely moves for either until TCP falls behind, so the difference is all in the tail. And at ten percent, which the charts leave out, TCP's worlds were five point zero seconds old at the p50, while UDP's p99 was a hundred and sixty milliseconds.

The decision

44:33 So the numbers decide it, and the worlds go over UDP, by default, in the client and in the bots. TCP stays as the fallback, for a network that won't let UDP through. The input stays on TCP, and so do logins, and anything later on that has to arrive and arrive in order, such as chat or a trade. That's how a lot of games split it, picking the transport by what a message needs, not one for the whole game. Input could go over UDP too, which games do by sending the last few inputs in each datagram, so that a lost one is covered by the next. Resending inputs that way is more machinery than we need right now, and we'd measure it first.

45:14 Now the client, where the parts get put back together into a world, and drawn. In connection dot h, we include asio's ip udp, for the UDP socket, and steady timer, for the hello's timer, and parts dot h, for putting the parts back together. Create takes udp, a bool, which makes the connection ask for the worlds over UDP. The private constructor takes the token now, as the hellos over UDP are going to need it after start has returned, and with a single argument, it's explicit. Start takes udp too, and the private functions get hello over UDP and receive over UDP, like the bot's. And receive, which takes a World from either transport, so that TCP and UDP end up in one place. The members get a UDP socket, a hello timer and the token, like the bot, and a datagram buffer of two kilobytes. Then a Parts, which puts the world together, and udp worlds. In connection dot c c, we include chrono, for the timer. Create takes udp, makes the connection with the token, and passes udp on to start, and if start fails, create returns its status. The constructor sets up the UDP socket and the hello timer on the I O context, next to the TCP socket, and keeps the token. Start resolves the host, opens the TCP connection and says hello, as before, and after that hello, if it's on UDP, it does what the bot does. It looks up the TCP socket's remote endpoint, the server's address and port, and connects the UDP socket to it. If either step failed, start returns an Unavailable error, which the login screen shows, like a failed TCP connect, and otherwise, it says hello over UDP and starts receiving. Read changes a little, as it used to parse each message from TCP directly into world underscore. Now it parses into a World of its own, and hands that to receive. Hello over UDP works like the bot's, and it stops once worlds arrive over UDP. Until then, it sends the token in a datagram, four times a second. When the connection closes, the client throws the whole Connection away, timers and all, so it doesn't need the bot's connected check. Receive over UDP waits for a datagram, and if it parses as a World, it's a part. So udp worlds turns true, which stops the hellos, and the part goes to receive. Then it waits for the next one, and as the client runs these handlers with poll, once a frame, the datagrams are handled in the frame loop, like the reads from TCP. And receive adds the World to parts, and keeps the world Parts puts together, for the renderer. A whole world from TCP goes through Parts too, as a single part, so the rest of the client doesn't depend on which transport the world came over. In the client's main dot c c, enter takes udp, and passes it to Connection's create. In main, transport is the second argument, after the host, and it's udp unless we say tcp, which is how we fall back by hand, or compare the two, like we're about to. When the login screen hands us a request, enter is on UDP when the transport is udp. And the bots default to UDP too, so a swarm plays the way players do now, while loss dot s h still says which one it wants.

48:29 Let's see it with two clients, side by side, on a network that loses packets. Here, both clients, the servers and thirty bots run on one machine, with netem on its loopback interface. And netem adds twenty-five milliseconds each way, and drops five percent of the packets in both directions. The left client asks for TCP, with tcp after the host, and the right one takes the default, UDP. On the left, now and then, the bots stop and then catch up in a jump, as the worlds wait behind a lost packet. On the right, they keep walking, as a lost part costs its players a tick, and the client's smoothing covers most of that. Both players walk up the board, and as the input goes over TCP on both sides, a lost input packet can hold up either of them for a moment.

Docs

49:17 Last, the README, which I'm pasting in, where the client's part says it gets the world over UDP, and that tcp after the host asks for TCP. The bots get their new argument, the transport, and serve their world age on port nine four six five. A new section on packet loss has the commands: the netem service with a delay and a loss, netem zero zero to put the network back, and loss dot s h for each transport. And it notes that the bots and the server have to share a clock to measure the age, which they do on one machine.

Wrap-up

49:49 So, today our bots started measuring what a player sees: how old the world on their screen is, about sixty times a second. netem made the server's network worse, with a delay and a share of the packets lost, and the offloads off, so that it loses them one by one. Over TCP, the tail of the world's age grew with each step of loss, as worlds waited behind lost packets, and at ten percent, TCP couldn't keep up at all. Then the world went over UDP, after a hello with the token, in parts that each fit in a packet. On a good network, the two are level, while with ten percent lost, UDP's p99 was about a hundred and sixty milliseconds. So the worlds go over UDP by default now, with TCP to fall back on, and what has to arrive stays on TCP.

Playing it

51:27 Let me know in the comments how a game you play or work on splits its traffic between TCP and UDP, and what you'd like to see next. And if you want to follow along as the game grows, subscribe to the channel. I hope you find this useful, and thanks for watching.