GameNode

[bzar]

Back story


While coding wars I've stumbled upon many problems with creating rich multiplayer web-based games. Some of them exist because the communication supported by browsers is not really game-friendly. The HTTP protocol is inherently stateless, so some overhead is required to keep sessions using cookies and whatnot. Each communication requires a new connection, raising lag. Servers cannot push data to clients, because the system was designed for request-based applications, like web pages mostly are.


The ones more experienced with web programming are now thinking: "you can get around all that using long polling/comet/whatever!". Yes, but most web servers are not really up to the task of serving a huge amount of long-polling clients. Most assume only a handful of concurrent connections with short lifespans. They are not suitable for long-lived connections. In addition, long polling techniques have their own set of disadvantages, such as missing data when reconnecting and http overhead and difficulties in communication between clients.


Now I imagine some of you think "Well, I can use a publish-subscribe messaging server such as Orbited for the data pushing!", but those bring a whole bunch of other problems, like separate logins for the web content and game data server. This is how we currently do things in Wars, and I got to thinking that there must be a nicer way.


Enter WebSockets. A nice, clean socket connection to a backend server. Although it's currently having some security issues, it is in my opinion the future of any two-way communication in web applications. This got me thinking: what if ALL dynamic data for an application was available only through a websocket server, and all html/css/js was served as static files. This means no server-side generation of HTML, only static files and a websocket to fill the page with data. A nice clean separation between the view and the model.


This model also solves the login issue, because one only needs to log into the web socket interface. The static pages are only templates, so getting them won't gain you anything. After tinkering with this idea for a while I noticed that if the login happens through a websocket, I of course cannot switch pages without losing my session. First I thought about creating a small view framework to keep the user actually on the same page and only dynamically modify the DOM to show different views. This idea was short-lived, because I wanted direct support for nice URL based navigation and back button and so on. So back to the drawing board. Next I thought about the current common implementation using cookies to send a session ID with each request. I didn't want cookies to have anything to do with my framework, but that reminded me of an interesting HTML5 feature: WebStorage or more accurately the SessionStorage part of it. This allowed me to store the session ID to the browser after successful login, and use it to restore my session after loading each page.


Now I had something I could use.


What have I done?


It's still a bit raw, but it already works. GameNode is a web application framework based on HTML5, websockets, node.js and node-websocket-server that provides a nice base for making both the server and client part of a web game (or application). The framework supports two main ways of communication: RMI (Remote Method Invocation) and messaging.


The RMI works by creating a "skeleton" object on the server, which is instantiated for each client. The client can then call the skeleton object's methods from the browser using an automatically generated stub object. The return values are automatically conveyed to a designated callback function. This is a nice way of handling stuff usually done with AJAX calls, or sending player actions (set game piece to these oordinates etc).


For a more speedy and two-way communication, we have messages. Both the server and client can implement message handlers, which process incoming messages. This is best for spontaneous client updates (an arrived chat message, locations of other players) and fast update messages for the server (location of player etc).


These things are in my experience best shown with examples. I have a few basic examples in the version control, which you can check out. The "test" example shows a very simple RMI/messaging test application, "chat" a basic chat server with dynamically creatable chat rooms (or "channels") and "auth" is a work-in-progress example of using the framework for login and sessions.


If you want to try any of the examples, you need node.js and a websocket-enabled browser such as chromium (or firefox 4 beta with the secret websocket switch set). Start the server (eg. "node testServer.js), and open the client html with your browser. Some examples require your browser to also support console logging, so be sure you have that (try typing "console.log" to your browser's javascript console).


If anyone's interested, give me a shout!


PS: I'm inclined to rewrite Wars using this thing when websockets get more common, so I'm writing this framework with mostly games in mind, although I'm sure it's good for so much more.


EDIT: typo

 

[edwin eefting]

Hi,


You might want to take a look at http://open.syn3.nl/syn3/trac/default/wiki/projects/synapse, especially at the Internet-paper-project.


I did implement a full framework with a build in event-webserver that does use generic polling.


The advantage being that it also works in restricted environments and through proxys.


I also implemented message-queueing, so that if you send too many messages/second for a browser-client, the messages will get queued and send in one big message body.


Let me know if you are interested so we could work together.


Edwin

 

[bzar]

Hi! Thanks for replying, it was getting lonely in here . I took a cursory glance at your framework before work, and will take a better look later today when I get back home.


Your solution looks like it can handle quite a load, have you tried client spamming it (something like 1000 simultaneous clients)? I'd be interested in the results. Do you have a hello world example for the simplest possible use case? My framework is concentrated on rapid development and minimum boilerplate code, so a hello world would provide a good comparison in this respect. How are you handling messages sent to a client who's currently reconnecting after a timeout? You also mention threaded event handling, so I'm guessing you have a polling server that delegates clients to threads. Does this cause any difficulties regarding concurrency and shared data? Node.js uses a purely event-based concurrency model, which has the advantages of smaller client footprint and zero concurrency issues.


As for proxy traversal, websockets do that as well, as they're basically long-poll connections that notify the server and proxies of the fact.


I'm intrigued by your framework, though it does seem to have a bit different philosophy than mine.

 

[milkshake]

Im interested in how many hours its taken you so far to get to where your at on this project?

 

[bzar]

^ me? Let's see... maybe... 15-25? I started last saturday IIRC. It's still a very early version.

 

[milkshake]

^ me? Let's see... maybe... 15-25? I started last saturday IIRC. It's still a very early version.

wow I must be the worlds slowest when it comes to coding then lol


how many lines of code have you wrote in that time?

 

[bzar]

Edwin: The internet paper thing is very nice, reminds me of a similar app I did with python, django and websockets. I'm not hosting it anywhere since it was just a test, but it worked nicely . That quick scribble wouldn't hold too many users though, because the django websocket connector is accessed through apache .


I checked out some of the example applications in your source tree. Synapse looks like a nice piece of code, and not too difficult to get into. It also gave me an idea about doing a reverse RMI for my project (server has a stub for each client). There was a bit more verbosity in the server/client interface definition part (event types) than with gamenode, no doubt due to lack of introspection in C++. I like to keep the number of macros and manual framework calls to a bare minimum, which is why almost everything is done with callbacks and introspection in gamenode. In general, if I was starting a more performance heavy web application that required two-way messaging, I'd probably go with synapse. It would take a bit longer to develop the app, but the end result would in some cases be faster (I say most cases, because the V8 engine running node.js is in many cases fast enough for all practical purposes, so the difference would be small). However, for smaller performance requirement and a bit faster, iterative development I'd go with GameNode.


I'll try to get a couple of example games (one synchronous and one asynchronous) out, which will probably communicate my point better


Your thoughts?

 

[bzar]

wow I must be the worlds slowest when it comes to coding then lol


how many lines of code have you wrote in that time?

Well, I've rewritten the code a few times while refactoring (iterative development, you know ), but the combined current line count in the *.js files is around 600. There's a bit more in the example htmls, but it's not really much. Most stuff is done in a very generic fashion, which keeps the line count down.

 

[milkshake]

Well, I've rewritten the code a few times while refactoring (iterative development, you know ), but the combined current line count in the *.js files is around 600. There's a bit more in the example htmls, but it's not really much. Most stuff is done in a very generic fashion, which keeps the line count down.

ah right, cos im working on an app-repo for pandora in php and mysql (havent even started thinking about the html, javascript and css).


Because of the complexaty of what im attempting (plus its the first time I have attempted anything like this), its taking me many hours, many re-writes (neaten and simplyfy as much as I can).


I have been writing functions testing them then converting them to methods then adding them to class's once I know they work(I know it will mean less code later but it seems to be taking forever).


When I say commplexaty thats because my repo will check each file to make sure its a valid pnd file with PXML and icon appended ot the pnd binary and also that the PXML conforms to specs. We then check to make sure there are no duplicates etc before allowing any db entry.


In the end I'm trying to make it as simple as possible for the end user to upload an app without needing to fill in any details (my repo will pull needed title, diescription etc from the PXML) although they can edit them if they wish.


then it will provide a full list of apps via JSON which will mean a native client can read and process whats in my repo.


feels like ive spend 70+ hours so far and wrote over 1000 lines of code easy, and as i have mentioned I havent even started on the html for the front of the site.

 

[bzar]

Well, compared to that I've had the advantage of needing to only be compatible with my own code. You're writing data scrapers, which always take their own space (except for trivial ones, like my webcomic feed generator). Keep it up! You only get faster by training

 

[bzar]

Oh, and BTW, I implemented a first version of the reverse RMI today. Took about 35 minutes .

 

[chosen]

This sounds excellent, great work! I may like to use it for a project, although that won't be for a while yet.


Will you be releasing this when you are happy with it? If so, under which license, LGPL maybe?

 

[bzar]

This sounds excellent, great work! I may like to use it for a project, although that won't be for a while yet.


Will you be releasing this when you are happy with it? If so, under which license, LGPL maybe?

Thanks . Probably BSD or something of the kind. I tend to release games and such as GPL and libraries with BSD or similar.

 

[bzar]

I've added a couple of quick simple examples on how to write asynchronous (tictactoe) and synchronous (flyer) games using gamenode. I'll probably still work on these and nothing in the framework is nailed to the floor, so this is just a sample.


The TicTacToe example shows authentication, sessions (I have an idea how to make these even simpler), publish/subscribe, game lobby, and two-way RMI. It's basically a multiplayer tictactoe (who would've guessed!) server with several game rooms. Users later can log in using the same username and continue playing their games. There's no persistence, so games disappear when the server shuts down. I'm planning on a mongodb example to show how to persist stuff.


The Flyer example is a simple start for a multiplayer asteroids- or wings-type game. I'll probably add some content to this, as right now it's quite basic. Connect with several clients and fly around using arrow keys. If you're the only one on the server you won't seem to move as you have no point of reference . A nice thing I noticed here is that the server actually used less CPU with a few clients than the console outputting debug messages

 

[bzar]

Added a tiny web server that can be used for development and examples. The server can serve specific files from root directory and entire directories by mapping them to subdirectories of root. I wouldn't recommend it for any real file serving, since it doesn't really support much anything (setting content length or MIME type etc), but it's enough to serve files for the examples or light development. Also it integrates nicely to the rest of the framework.

 

[bzar]

Added AdvancedFlyer example. I decided to leave the Flyer example as it currently is, because if I made it any more complicated the point as an example of a simple synchronous multiplayer game would suffer. I still wanted to make a better example with a shared data model and different framerates for client and server, frame prediction and so on. The current implementation runs smoothly with client(s) running at 60fps and server running at 15fps, and uses very little cpu. I added a framerate select box and a FPS counter for testing.


Among others, I'm targeting pandora with this framework. I think at least simple multiplayer browser games (simple canvas based or just images and DOM) are a nice fit, so I'd like to get some test results. Currently the advanced flyer demo uses around 9KiB/s for a single client (with around 5KiB/s/client after that), so I don't think that's an issue, even though I could make it smaller. As this is an example, I'm aiming for clear rather than efficient code. What I would like to know is framerates on a pandora as indicated by the client with all three of the settings: 60fps, 30fps and 20fps and general choppiness/smoothiness. GameNode requires WebSockets, so chromium is preferred (ff4b has websockets, but they are disabled by default for security reasons).


Instructions:


Setting up a server on a desktop:

1. Get the latest version of node.js
   
2. Checkout GameNode from SVN (svn co http://bzar.iki.fi/svn/random/javascript/gamenode)
   
3. Move to examples/advancedFlyer directory with a terminal
   
4. Run server with node server.js
   

Running client on a pandora

1. Browse to http://<your desktop ip here>:8080
   

Disclaimer: GameNode is still under heavy development so I wouldn't recommend using it yet except for tinkering and quick prototypes (example application submissions will be greeted with joy). For now I might change anything at any time and no backward compatibility between svn commits is guaranteed.

 

[MarkoeZ]

Hey B-Zar, how have you been?


i should check up on Wars, been a while since i visited the forums frequently.


Just see your concept for the first time now, and it sounds very interesting. No direct plans for network gaming at this moment, but if Panjoust goes online, i'll look for you ; )


Cheers!


MarkoeZ

 

[bzar]

Good, good. Thanks for asking . Wars is still in development and active gameplay testing by me and a few friends. I'm really considering a rewrite with this gamenode I'm working on, as it's shaping up so nicely.


I'll be glad to be of assistance with Panjoust network, at least as a consult, though I'm really not very experienced with online multiplayer. The AdvancedFlyer demo is kinda my first attempt at implementing some of the techniques I know to achieve a smooth gameplay experience, like shared data model and asynchronous updating. I've been searching the net for good multiplayer implementation tutorials, but haven't found much anything. I'm sure there are some standard techniques that I'm not aware of that could make my multiplayer a whole lot better. If anyone has any tips or good tutorials (advanced, not the basics) they'd like to share, I'd be thankful

 

[edwin eefting]

Hi! Thanks for replying, it was getting lonely in here . I took a cursory glance at your framework before work, and will take a better look later today when I get back home.


Your solution looks like it can handle quite a load, have you tried client spamming it (something like 1000 simultaneous clients)? I'd be interested in the results. Do you have a hello world example for the simplest possible use case? My framework is concentrated on rapid development and minimum boilerplate code, so a hello world would provide a good comparison in this respect. How are you handling messages sent to a client who's currently reconnecting after a timeout? You also mention threaded event handling, so I'm guessing you have a polling server that delegates clients to threads. Does this cause any difficulties regarding concurrency and shared data? Node.js uses a purely event-based concurrency model, which has the advantages of smaller client footprint and zero concurrency issues.


As for proxy traversal, websockets do that as well, as they're basically long-poll connections that notify the server and proxies of the fact.


I'm intrigued by your framework, though it does seem to have a bit different philosophy than mine.

1. I didnt try spamming it yet, but i do have regression tests that can be modify to do this. I didn't yet optimize some routines, because so far the performance was more then sufficient. (there are some places that can be optimized, but i will probably use a profiler to do this correctly)


2. i dont have a minimum example yet, but you can look at the test-module. It has testcode to talk to other various modules.


3. real timeouts normally do not occur since the webserver is written to allow unlimited connection times (no timeouts). If the client or proxy is using http 1.0, then very new poll can however be a new connection which is no problem. the synapse http server recognizes each individual synapse javascript instance, by ways of an authentication code in a special header. so you can have an administrator session in one browser window, and a guest session in another without any conflicts.


4. the server is multithreaded. when you write a server module in C++ you can choose wether you want a session to be single threaded or perhaps the whole module. offcourse if you enable multithreading, the programmer is responsible for locking and concurricy issues. (i use boost locking which is the best for mutexing). The server has advanced scheduling that creates/destroys threads on the fly. (no need to worry about it when you create a module)


the javascript client is single threaded, so no concurrency issues.


note that the framework is a generic framework which is not specialized to one specific purpose like web programming. however there are offcourse modules that do this, like the http module.


edwin

 

[bzar]

Oh, hey Edwin. I was going to PM you after I didn't hear from you in a while.


I don't doubt the performance, I'm just interested in the readings. Mainly average memory consumption overhead per client (threading tends to increase this) and lag (this should be no problem, but try serving a file to a lot of clients and see if the average wait time increases). I'm sure it will pass with flying colors, but I just like numbers . Of course I'm not demanding you to do this or anything, only do this if you find it interesting yourself


Yeah, as you probably read from my latter post, I got the hang of how synapse is structured and I can pretty much imagine what a hello world would look like. It's got a bit more boilerplate as I suspected, partly because it's more generic, partly because C++. Don't take this the wrong way, it looks very lean for what it does from an application developer's point of view. I'm comparing oranges to apples here


Proxy timeouts is what I actually meant, because obviously you would make you web server non-timeouting . Having several sessions in one browser is a nice touch, but does it work with opening new tabs (say by copypasting a url) or does each tab require a separate login?