Deploy a C++ game server to a VPS with Docker

Intro

0:00 Hello and welcome back to episode nineteen, where our game goes onto a server on the internet, so that you can connect to it from anywhere. Let's begin with a quick look at where we're going to be by the end of the episode.

0:12 Our client connects to a server we rent, by its address on the internet, about ninety milliseconds away from my machine, there and back. The login server, the game server, MySQL and Redis all run on it, in Docker, and the internet reaches the game's ports and the website, and none of the rest of our stack.

0:32 On the website of the series you can find more information about the project. The code for the project is open source on GitHub, tagged at each step of each episode so that you can follow along.

Last time

0:44 Now, before we get to today's work, let's quickly go over what we did last time. Flyway started keeping our tables in versions, as SQL files in the db folder, and it records which ones each database has run. Docker Compose runs it before the servers, the saver, the website and the tests, and they don't start if it fails. When we put the health column into version one, Flyway refused, as that file had run already. So the column went into version two, and the bots kept their accounts and their progress through the change. The tests run through the migrations too, on a fresh database, on each push. So far, all of it has run on my machine, and today it goes onto a server of its own.

Today

1:26 Today, before we deploy, we're going to look at how our server copes with a crowd, as that tells us what to expect from a smaller machine. Then we rent a VPS for our game, from DreamHost. We decide which parts of our stack the internet gets to reach, in a compose file for the server. A script sends a commit over SSH, and starts it there, with certificates that name the server's address. And last, we connect to it from my machine, and put it under load too. All right, let's get started with a crowd on my machine.

Under load

1:59 Here, a thousand bots play on my machine, on the server from the end of last episode. The other players move in steps, and slower than they should, and so does mine, as the server can't keep up its twenty ticks a second. Each tick moves everyone a tick's distance, so with fewer ticks a second, the whole world runs slower than real time.

2:24 Let's measure it, with the stack from the end of last episode. Load dot s h, our benchmark from episode eight, adds two hundred and fifty bots a step, for six steps. Each step takes close to a minute, so this is sped up. The server keeps its twenty ticks a second up to seven hundred and fifty players, and with a thousand, it manages fifteen. With twelve hundred and fifty players, it's down to ten point five, and with fifteen hundred, six point six.

2:53 On the chart, the server keeps up to seven hundred and fifty players, and somewhere between that and a thousand, it falls behind. Each tick sends each player the whole world, so the work grows with the square of the players, as we saw in episode four. A VPS like the one we're going to rent has two virtual cores, each of them slower than one of my machine's, so we expect it to fall behind with fewer players. We're going to fix that in a future episode, and for now, let's see where a VPS stands.

A server of our own

3:21 There are a lot of places to run a game server, and platforms that run containers for us are the first that come to mind. They're made for web services, which answer a request and are done, and a lot of them don't take UDP, or connections that stay open for hours. Kubernetes runs containers across a cluster of machines, and that's a lot to look after for one server. A VPS is a virtual machine of our own, in someone's data centre, with root access, and Docker goes on it like on any Linux machine. It takes our UDP and our long connections, and it's a flat price a month, so it's where a lot of small online games start.

4:01 I rented mine from DreamHost, and there's an affiliate link in the description, so if you rent one through it, it supports the series. The plan is their Stack 4, with two virtual CPUs, four gigabytes of memory and seventy-five gigabytes of disk, by the month. For its configuration, I picked Docker with Portainer, so Docker and Compose come installed. Portainer is a web interface for Docker, which we're not going to use, as our deployment is going to be a script that we can read and run again. And for access, it takes an SSH key instead of a password, so the server lets in whoever holds the private key, and there's no password to guess.

4:40 SSH runs a command on the server, as the debian user, with my key. The server has two cores and nearly four gigabytes of memory, and Docker and Compose are there already. Its firewall, U F W, lets in SSH, with limit, which refuses an address that opens six connections within thirty seconds. It also lets in the web's ports, for Caddy, a web server that DreamHost's setup puts in front of Portainer, and its rules let in nothing else. And a ping comes back in about ninety milliseconds, which is the way to its data centre and back.

What the internet reaches

5:12 Players need two ports: the login server's, seventy-seven seventy-eight, and the game server's, seventy-seven seventy-seven, over TCP and UDP. MySQL, Redis and the game server's metrics are for our own services, so they stay on Compose's network, where our containers reach them by name. We can't count on U F W to keep them off the internet. Docker writes firewall rules of its own for the ports it publishes, and a published port goes past U F W. So a port that shouldn't be online mustn't be published at all. Redis, as we run it, takes commands from anyone who connects, without a password, and there are bots that scan the internet for open Redis ports. Prometheus and Grafana listen on the server's localhost, and we reach them through SSH, which can forward a port on my machine to one on the server. And the website goes online too, with its rankings, on port eight thousand and eighty like on my machine, as DreamHost's Caddy has the web's usual ports.

6:10 The server's changes go into a file of their own, compose dot prod dot yaml. Compose merges it over compose dot yaml when we give it both, like the profiling file from episode eight. Under MySQL, the password comes from a variable, NANOMMO MySQL password, instead of the one in compose dot yaml. The colon and the question mark make Compose stop with that message when the variable isn't set, so it can't start with an empty password. Ports set to reset, with an empty list, drop the ports compose dot yaml published, as reset is a YAML tag that tells Compose to throw away what the file before it set. And restart unless stopped starts MySQL again if it crashes, or when the server reboots, until we stop it. Flyway logs in with that password too, and Redis loses its port as well, and restarts like MySQL. The login server gets the password through the setting our servers have read since episode six, and it restarts like the others. The game server gets the password and the restart policy too, and its ports use override, which replaces the list from compose dot yaml with this one. The new list has seventy-seven seventy-seven, over TCP and UDP, and leaves out the metrics port, which Prometheus reaches over Compose's network anyway. The saver gets the password and a restart policy too. The website's key comes from a variable too, as Laravel encrypts its cookies with it, and the key in compose dot yaml is public, in our repo on GitHub. The website gets MySQL's password like the others, and it keeps its port, eight thousand and eighty, from compose dot yaml. And Prometheus and Grafana publish their ports on the server's localhost, so nothing outside the server reaches them. The password and the key are going to go into a file called dot env, next to the compose files, which Compose reads variables from, and git ignores it. The Docker builds leave it out too, along with the server's compose file and the deploy script, so changing either doesn't rebuild our C++.

A deploy script

8:12 To deploy, a script sends the server a commit of our repo, as files, so the server needs no git, and no access to our repo. The certificates go with it, from a folder on my machine. The authority's key stays here, as the README has said since episode fourteen, and the server gets its own certificate and key, and the authority's certificate, for bots that run there. The first deploy makes dot env on the server, with a random password for MySQL and a random key for the website, which stay there. Then Compose builds the images on the server, next to the release that runs, and our game's images come from the test stage, so a build or a test that fails leaves that release alone. Once they're built, the new release starts, and the release before stays next to it, so going back is a move and a rebuild. With several servers, CI would build the images once, and push them to a registry for each server to pull. With one server, building where it runs keeps it to SSH and Docker, though each deploy builds our C++ from the source again, on the two cores the game runs on.

9:17 Deploy dot s h is a shell script, so it starts with the shebang for a plain POSIX shell, like our certificates script. Set dash e u, like in our other scripts, stops it at the first command that fails, or at a variable that hasn't been set. Then, over it, a comment says what the script does. It ships a commit of this repo to a server over SSH, with certificates from a folder our certificates script made, and it starts the game there, with compose dot prod dot yaml on top of compose dot yaml. It takes the host, such as debian at the server's address, then the folder with the certificates, and the commit, which is HEAD unless we name one, such as a tag. Git archive writes the commit's files as a tar stream, and a commit has none of the files git ignores, such as our builds and our certificates. So add file adds the three that the server needs, the authority's certificate, and the servers' certificate and key, under the prefix tls. The second prefix, with nothing after it, puts the commit's own files at the top level of the archive. The pipe hands the stream to SSH, and the rest of the script runs on the server, in that SSH session. The comment over it says why it's a single connection: so that a deploy right after other SSH commands isn't refused, as U F W's limit refuses a sixth connection within thirty seconds. On the server, with set dash e u again, tar unpacks the stream into a fresh folder, nanommo dot next. If there's a release already, we copy its dot env into the new one. Otherwise, it's the first deploy, and printf writes dot env, a line for each of its arguments. The compose file variable names both of our compose files, so each docker compose command on the server reads them. The password is sixteen random bytes from openssl, as hex, and the website's key is thirty-two random bytes, in base sixty-four, with base sixty-four and a colon in front, which is how Laravel takes a key. The parentheses run all that in a subshell, with a umask of zero seven seven, so that nobody but our user can read the password and the key. Then compose builds the images, in nanommo dot next, while the release that runs keeps running. Over it, we write what a failure does: if a build or a test fails, the script stops there, and the release that's running stays as it is. Once the build is done, the release before the current one goes, and the current one becomes nanommo dot previous, and the new release takes the folder's name. Docker compose up starts it, from the images we've just built. Dash dash wait waits until each service is healthy, so that the script fails if one isn't, and force recreate starts each container again, even when its image hasn't changed. The reason goes above it: a container left running would keep the folder it started from, which is nanommo dot previous now. Docker gives a container the folders it mounts when it starts, so a container that kept running after the move would still read the old release's tls and db folders.

The first deploy

12:19 First, our certificates script makes the certificates in a folder of their own, vps, in the tls folder, with the server's address on them. Then deploy dot s h runs, with the host and that folder. The server builds all of it from source, protobuf and all, on its two cores, and runs our tests, before anything starts. That took about fifteen minutes, so this is sped up a lot. Then docker compose ps lists our services on the server, up, with the game's two ports and the website's published on all its addresses, and Prometheus and Grafana on its localhost. MySQL's and Redis's ports, and the game server's metrics port, have no address in front of them, as nothing publishes them, so nothing outside Compose's network reaches them.

A certificate that fails

13:04 Let's connect, with a bot from my machine, which logs in like the client does, trusting our authority's certificate from the vps folder. It can't log in, as the login server's certificate failed to verify. Open S S L x five oh nine prints the certificate's names, and our address is on it, but as a DNS name. A client checks a host that's a name against the certificate's DNS names, and an address against its IP names. Our script puts each name after the folder on as DNS, as it was written with a domain in mind, and our server has an address for now.

13:40 So, let's go back to our certificates script, to the loop over the names after the folder. A case statement now looks at each name before it goes on, and above the case, we write that an address goes on as an IP, which clients check apart from domain names. A name with a colon in it is an IPv6 address, which goes on as an IP. A name with a character that isn't a digit or a dot is a domain, and goes on as DNS, as before. Otherwise, it's an IPv4 address, and it goes on the certificate as an IP too. A game server often has an address and no domain, so the script takes either now.

14:17 The script leaves certificates that are there already alone, so we delete the folder, and make them again, with a new authority. This time, the address is on the certificate as an IP address. Then we deploy again, and the server builds again, as the certificates script is part of the source our images build from, so this is sped up too. And three bots from my machine log in, and join the game on the VPS, over TLS, with their worlds over UDP.

Playing on the VPS

14:48 Now let's start the client, with our authority's certificate, and the server's address. I make an account, alice, on the VPS's login server, and I'm in, on the island. When I press a key, my input goes to the server, and my character moves once the world comes back. The delay is a round trip of about ninety milliseconds, plus the wait for the next tick. Together, that's about two ticks, and our client's smoothing from episode seven hides the steps between them. Games that need a quicker response move your character as soon as you press, and correct it when the server's position differs, which we touched on in episode three.

15:27 The website runs on the VPS as well, on port eight thousand and eighty, and its rankings list the bots playing there. My account from the client is there too, as the website and the login server share the accounts table, in MySQL on the VPS.

Load on the VPS

15:44 Let's put the VPS under load, with the benchmark, run on the server through SSH. Load dot s h needs jq on the server, which I installed with apt, as the README says. The bots run on the server too, so they take some of its two cores, which real players wouldn't, and that makes this a little pessimistic. The results come back through SSH, into vps dot json on my machine, and each step takes close to a minute, so this is sped up. The server keeps its twenty ticks a second with three hundred players, and with four hundred, it manages nineteen point five. With five hundred, it's down to twelve point six, and with six hundred, nine point two.

16:25 On the left, my machine, which keeps up to seven hundred and fifty players, and on the right, the VPS, which keeps up to about four hundred, and falls behind past that. As we expected, the VPS falls behind sooner, since our game server runs its world on a single thread, and the speed of a core matters more to it than how many cores there are.

16:45 And this is what a player sees from my machine when the VPS is past its limit. Everyone moves in steps, a little slower than they should, and my own character answers late, as each key goes to the VPS and back before it moves. For now, our VPS holds about four hundred players before that happens.

Docs

17:06 Last, the README gets a section on deployment, which says what compose dot prod dot yaml changes, how to make the certificates and deploy, and how deploy dot s h builds a release before it swaps it in. It says how to reach Grafana through SSH, how to go back to the release before, and that the benchmark needs jq on the server. And the section on TLS now shows an address among the names a certificate can have.

Wrap-up

17:32 So, today we saw our server fall behind on my machine past seven hundred and fifty players, and rented a VPS from DreamHost, with two cores and Docker. A compose file for the server publishes the game's ports and the website's, and none of the rest of our stack, as Docker publishes past the firewall, and MySQL's password stays in dot env, on the server. Deploy dot s h sends a commit and the certificates over SSH, builds it next to the release that runs, keeps the release before, and starts each container from the new one. Our certificate named the server's address as a domain, so it failed to verify, and now an address goes on as an IP. We connected from my machine, about ninety milliseconds away, and the VPS keeps up with about four hundred players, and falls behind past that.

Playing it

19:13 Let me know in the comments what you'd like to see next. And if you want to follow along with the series, subscribe to the channel. I hope you find this useful, and thanks for watching.