Distributing your game content via Torrenting

In my previous article, we covered the processes to distribute your game via CDNs and the costs involved with that process. Thankfully quite a few people read the article, and a number of them pointed out to me that torrenting is also a valid, and cost-effective way to distribute large game content to users too.

Which was a great point. There’s some upper limit that you’ll eventually run into when distributing your data from CDNs, namely the cost involved with distribution of content. As we showed previously, this is a pretty low-cost operation overall, but if you’ve got a massively successful product, then this can create un-need costs that eat into your overhead.

Torrenting is a process that creates a Peer-To-Peer network that allows your users to transfer your data files between each other, rather than solely relying on your CDN for the data transfer. This can be a great benefit to your system, as it reduces distribution costs on your side, as well as gives your users an available option if your CDN goes down: they can still get the files from each other.
Torrenting isn’t new, and large companies, like Blizzard and Microsoft have been using it over the past few years for the very same reasons I mentioned.

disclaimer : Torrenting for piracy usage is not advocated by myself or any organizations I’m associated with. It’s an interesting technology to distribute files, and that’s where this article starts.

How torrenting works

• A user who wants to upload a file first creates a small torrent descriptor file that they distribute to other users by conventional means.
• They then make the file itself available through a BitTorrent node acting as a seed.
• Those with the torrent descriptor file can give it to their own BitTorrent nodes which, acting as peers or leechers, download it by connecting to the seed and/or other peers.
• The file being distributed is divided into segments called pieces. As each peer receives a new piece of the file it becomes a source (of that piece) for other peers, relieving the original seed from having to send that piece to every computer or user wishing a copy.
• Pieces are typically downloaded non-sequentially and are rearranged into the correct order by the BitTorrent Client, which monitors which pieces it needs, and which pieces it has and can upload to other peers.
• Pieces are of the same size throughout a single download (for example a 10 MB file may be transmitted as ten 1 MB Pieces or as forty 256 KB Pieces). Due to the nature of this approach, the download of any file can be halted at any time and be resumed at a later date, without the loss of previously downloaded information,
• When a peer completely downloads a file, it becomes an additional seed.

Utilizing torrenting for your game

Creating a private torrent network

I’m going to go out on a limb, and assume that you don’t want your game data files lounging around all over the internet; and as such, you’ll most likely want to create a private torrent system to use. I highly suggest not rolling your own code here:

Setting up a seeding server

Rather than reinventing the wheel, you can take advantage of existing work for torrenting. For example, using uTorrent will allow you to run a torrent seeder on a linux/windows box easily, which brings the question of where to host the box?

Simple web-hosting won’t suffice to create a torrent seeder, you’ll actually need a compute layer, meaning that you’ll need some machine sitting out in the intertubes. Jeff Atwood has a great article on comparative costs between EC2 Compute units ($17k/yr) vs building his own high-end server for $2,750. in which he argues:

Of course, that figure doesn't include the cost in time to build and rack the server, the cost of colocating the server, and the ongoing cost of managing and maintaining the server. But I humbly submit that the one-time cost of paying for three of these servers, plus the cost of colocation, plus a bunch of extra money on top to cover provisioning and maintenance and support, will still be significantly less than $17,000 for a single year of EC2 web application hosting.

Of which, you have a separate question: How much performance do I really need? Jeff is certainly targeting web applications which need to execute high performance computations, where a simple torrent seeder will only need a minor fraction of that performance, since the lion’s share of the file transfer process is between peers.

This limited-performance need may allow you to branch out to other types of server technologies, for example torrent-server is a torrent seeder that runs on Google App Engine and allows you to scale production, reduce costs, and avoid having to maintain a large server infrastructure. Which, as a small developer could be a great way to get started.

Adding a torrent system to your game client

As we discussed in the article about patching, you’ll most likely have some separate system for bringing new build data off the web, and caching it locally, which is a great place to put tons of heavy torrenting / transfer code that you won’t want munging up your game code. Which should be an easy process, a quick search yields hundreds of client libraries for doing torrenting, most of which are open source, and exist in whatever language you happen to be targeting. Once integrated, a simple modification to your file-transfer logic should allow your client to wait for file completion, regardless if it’s coming from a torrent network, or a CDN.

One down side is that to use this type of setup, you need the user to agree to allow their bandwidth to be used for the greater good. Otherwise you can run into some fussy legal problems; which I am not allowed to comment about, because I am not legal council.

When to torrent, when to CDN?

It’s worth noting that torrenting may provide a worse experience for distributing content to your users. Especially if the number of seeds are low, or if connections between peers is less than ideal. There’s certainly a break-even point, in which it makes more sense to utilize one over the other, or even find ways to use both where ideal.

For example, the farmville folks have two seperate servers; They found if the number of daily users is below 5 million, it was cheaper to route those games through their own servers, than using a scalable CDN and the costs involved with it.

By all means the best solution here is a combination of CDN and Torrenting. For instance, using the CDN to help seed new patches and content updates, while letting torrenting do the lions share. Or even if there’s  a low-peer network for torrenting, it’s better to swap over to the CDN structure.

And if rolling-your-own isn’t your thing, I highly recommend Highwinds’ Game Delivery Network, which provides all of the items in this article in a nice, awesome package. In general, for distributing assets, the Torrent-CDN approach is a great middle ground, as it allows you the flexibility and cost structures that are used by lots of powerhouse game developers.

~Main

You can find Colt McAnlis here: