The Gathering is coming closer (12 days left!), and like last year, I'm planning on publishing a few thoughts on how it all is from the inside.
I think it's appropriate to start out with the same kind of disclaimer I made last year: Even though I'm the chief of Tech:Server (like last year), this is in absolutely no way any kind of official information from TG; you will find that on www.gathering.org only. If you try to take anything I've said here to the info-desk, they will look at you and say “Huh”.
Now, with the disclaimers out of the way... :-)
As with almost any computer party, there's bound to be a lot of people who want to play games on the LAN. However, you don't really want 4752 computers on the same broadcast segment; they're segmented into groups of 24 and 24 (giving a /26 to each table switch, essentially), with layer 3 switching between them.
This means that if you enter your favourite game, open up "LAN games" and search for local games, you most likely won't find anything. (Well, you'll find the people on the same switch as you, but that's about 0.5% of the hall, which is pretty boring.) The only real alternative is to use the Internet mode in your game, which might be very useful or it might not be useful at all, depending on the game in question. It's always been done that way, but can't it be done any better?
In retrospect, the solution is incredibly simple, and it's sort of odd why we didn't think of it earlier. (Actually it's based on what another computer party did last year, but the implementation is quite different in many aspects, and there's no code sharing at all.) Let me first explain how an average game does this searching:
- The client sends a broadcast UDP packet to 255.255.255.255 on a given port. (Some games use more than one port.)
- The server responds by unicast to the client.
- The client and server talk unicast from there.
So, the only problem is that the very first packet won't reach the server. The solution is simple:
- Use the ip-helper functionality of the core switches, which takes UDP broadcasts on a given port (you can select the ports yourself; by default it does DHCP and a few others, so you can have one DHCP server instead of 200 of them), turns them into unicast packets and send them to one or more designated servers.
- On the server receiving the unicasts, listen for UDP packets. Whenever something that looks like a game packet arrives (filtered by port and size, mostly, but we can add stuff like data matching and rate limiting as well), forward it (faking the source address to the same of the original packet) to all the other table switches using directed broadcast (ie. send to 10.0.0.255, 10.0.1.255, 10.0.2.255, etc. -- except that we have official IPs and /26 networks). At the last hop, the switch will change the address to 255.255.255.255 and flood it to all the clients.
The code to do the last step is a Perl script of under 200 lines. (Net::RawIP is ideal for what we wanted; in C I'd probably have to construct the UDP packet by hand before sending it out on the raw socket, which is always a pain.)
Of course, the network becomes a bit more chatty from all of this, but nothing we can't handle -- it'll be fun to see how well it works out in practice, though. We've already tested a few games in the lab, and it works amazingly well for such a simple hack. (On a sidenote, getting the average gamer to understand how to use Ethereal so we can get packet samples to add to our ACLs is surprisingly hard. That might sort of explain why we're among the first actually doing this.)
It should be noted that Cisco also has a solution called multicast helper. It does about the same, only with multicast instead of unicast, and no Perl scripts involved (you just do broadcast → multicast on the originating switch, and multicast → broadcast on all the other ones), but it seems to be a little less flexible when it comes to filtering and logging, and one site warned about high CPU usage. In any case, it doesn't really change how the system works, so I guess I'm happy either way.