Simon Tatham's Portable Puzzle Collection

[ledow]

I was wondering if someone would be kind enough to have a look at the possibility of a GP2X port of "Simon Tatham's Portable Puzzle Collection". It's a dozen of so mini puzzle games which are designed to be portable (as in between architectures) and has full source available - plus a hell of a programmer behind it, the same man who wrote PuTTY for Windows.

I know that they are based on GTK but I also read through and they are designed so that any type of frontend (e.g. SDL, framebuffer, Windows GDI or DirectX etc.) can be created quite simply by filling out some basic functions for shape drawing etc. Also they have been ported to the Palm series of handhelds (again, with the full C source code available) so they are obviously not dependent on Windows-isms or even Linux-isms.

The website is http://www.chiark.greenend.org.uk/~sgtatham/puzzles/ and there is a developer documentation, full SVN source access and also (from the sounds of it) a very helpful original author who is very familiar with Linux. I can only hope that all this means that these puzzles can be easily ported!

Most of them are small-screen compatible already (because of the Palm ports) and the only "controls" are a cursor and up to three buttons (they are designed for mice, but the Palm port uses a touchscreen/stylus or a set of "cursor keys" on the Palm).

Anyone up for a bit of source code hacking to get them working nicely on the GP2X? I imagine they'd be fantastic on the F200 with it's touchscreen, if my Palm experience of them is anything to go by. And you'd only have to "make" one frontend (i.e. an SDL interface) for them all to work on the system (including any future puzzles that the author adds).

What do people think?

 

[senquack]

What do people think?

I love how clean and unified the whole look is.. Very very intriguing.

edit I just looked at the source, and it's like the guy is begging people to port it. The HACKING file is a complete guide to how the program works.

 

[xcen]

This looks pretty cool, good suggestion!

 

[Parkydr]

Some stuff was done the last time we discussed it.

I can compile GTK stuff now but a lot of the puzzles are small but still not the right size for the GP2X.

 

[onumad]

Some stuff was done the last time we discussed it.

I can compile GTK stuff now but a lot of the puzzles are small but still not the right size for the GP2X.


Really nice games... Go on with this port!

 

[ledow]

I've had a quick look. Having played the Palm versions, I can state that a 320 wide screen, especially if it's scaled from a "virtual" 640 screen is good enough for quite a lot of the games - everything relies on primitive drawing (circles, lines, rectangles etc.) so scaling works well and things like the map-colouring, net etc. are only in need of a very small screen. Plus the interface is simple "movable cursor + up to three buttons" for each game.

Delving actually into the source (damn, it's a long time since I did any C), it actually looks very do-able. I think the first step is an SDL-frontend, which shouldn't be much trouble. I've tried but my programming brains burned out long ago - all I want in a language now is quick results from when I start coding, not worrying about pointers and structures.

As a proof-of-concept, I have managed to get some very primitive, ugly, dirty elements of an SDL conversion going - it's NOT a port, it's not even a game at the moment, it's some simple drawing elements rewritten to use SDL to show the games and it works. Basically "frontends" in the sense that the author means them are nothing but a single file which defines functions to draw circles, squares etc. and send keypresses/menu events to the backend, which is platform-independent. Using SDL_gfx, such functions are easily created and are usually nothing more than wrapper functions with some error-checking.

I started from the GTK port and started developing the SDL functions in parallel inside the same file - so you run the games and it draws both a GTK window and an SDL window and makes and debugging quite easy... you can literally compare the GTK and SDL code side-by-side and the GTK and SDL results side-by-side. And when you're done, you can strip out the GTK statements and just have an SDL.c file which not only makes it compatible with the GP2X but as an extra added bonus doubles as another Linux/Windows frontend!

However, my code is terminally crap. It doesn't do 25% of what it should, it crashes (because I'm aware it's perfectly crap and probably does horrible, nasty things to my computer each time I run it), it's horribly-written, it has no user-interface - SDL provides simple mouse, joystick and keyboard input functions and they can map rather nicely to control the games. There are no menus, but you can easily have each executable pick up settings from command-line params only and use a "menu" executable to send out specific options plus map two keys - save and quit. The frontend is REALLY nicely seperated from a coding point of view and you don't have to deal with any specifics of the individual games - you just put in some graphics primitives, provide some sort of way for the user to issue commands (again, dependent on however you want to do it - full menus, command-line params, particular keys being pressed etc.) to start an individual game, restart them, generate a new one etc. (all the code for that is platform-independent, all you have to do is code how the user can initiate it).

But if my addled brain can get that far, I'm sure any halfway decent C programmer can do this casually in a matter of weeks of "when I have five minutes". I'm a computer science graduate (many years ago) and for a while used to program business software for a living but my mind can't work the way it used to any more.

Please, someone who can code in C without having to hit themselves on the head to remember how it all works, who has a proper development environment and still has a brain that's not been corrupted by a myriad other languages over the years, could you have a look at porting this? At my pace, it'll take me until next year to get one game "fully" working. And then the code would be as ugly as sin.

 

[senquack]

But if my addled brain can get that far, I'm sure any halfway decent C programmer can do this casually in a matter of weeks of "when I have five minutes". I'm a computer science graduate (many years ago) and for a while used to program business software for a living but my mind can't work the way it used to any more.

Nah, you just need some confidence, coding a UI is really just tedious drudgery, all the hard work of writing the game has been done for you. I'd say that if you want to make a polished SDL port of this game it'd take me 2 months casual development, .5-1 month if I was locked in a room and forced to do it full-time. The fella that wrote it made EXCELLENT documentation.. sometimes you just have to print out 200 pages of source code and thumb through it for days on end to figure it all out.

I think ParkyDR's working on the port, but he's probably using X11/GTK.

 

[Parkydr]

I've only got as far as compiling them as is for GTK/X11, just as proving my Debian ARM QEMU setup. They run, but most are too big for the screen.

I don't know when I'll do anything with them, I'm working on other things at the moment.

 

[sparr]

just want to point out that these are available as debian arm packages. if pandora runs a deb-based distro, this whole package will be mere clicks away

 

[ledow]

Yeah, I'm not too happy about an X- or GTK- based port. Unnecessary gumph. SDL is coming along very nicely, although I take no responsibility for finishing this off or, for that matter, if it kills anyone's GP2X. And I intend to post the code somewhere once it's standalone-playable on a GP2X without having a development environment on standby, no matter how ugly the code and the game look. If I don't manage to get any further, I'll post the code here anyway as a starting point for other people. This really isn't my thing any more and I can't stand programming in things I don't know any more.

From my experience now playing with the code, you might have performance problems using GTK, too. Generating puzzles is obviously CPU-intensive for the larger ones (I might even "overclock" slightly for that part of the game) but that's all non-interactive at that point anyway - but even the video hardware on my laptop struggles with some of the GTK cruft. The programs, especially Net, do a lot of poly-line-circle drawing and update all the time, especially when "rotating" - I've tried to keep updates to a minimum but there's only so far you can go and make things like the box-rotation in Net still work nicely.

Having said that, I have a display on SDL (software-only, no hardware acceleration because I haven't yet tried it with the GP2X hardware SDL ports yet) at the moment. It took me about three-four hours, most of which was re-learning C and reading up on SDL, SDL_gfx and the actual program mentioned. I've got display of virtually all elements (drag-dropped items, game backgrounds, game elements, hiding and unhiding of the circles in blackbox, rotation of the cubes in Net, which displays perfect by the way) in all games, because of the lovely seperation of code. I have some "off-by-one-pixel" problems but they don't affect gameplay at all - it's things like the "grid" boxes in Net line up perfectly but they are one-pixel to the right of where they should be so they look "closer" to their contents (the blue boxes are actually poly's) than they should. I also have colour problems (as in everything HAS a colour, but it's not necessarily the correct one) but that's because I'm an idiot and converted 16-bit integers to 8-bit incorrectly - it didn't occur to me until I was on the way to work this morning what I'd done.

So... I have playable-quality graphical display using SDL and SDL_gfx which should port quite nicely to hardware-acceleration-SDL on the GP2X. It works on ALL games, from start to finish - the most intensive part is the "congratulations" flash in some games, e.g. Net - they run slower than anything else! I don't have to worry about sound, cos there isn't any. I do have to implement a user-interface (in the most basic meaning of the word) but knowing me that's going to be "A starts a random game, B restarts, Start quits, possibly L and R to change size of the puzzle. And most importantly, and the only "alpha-release" blocker, I need cursor control - I'm faking that at the moment but looking at the SDL library, that shouldn't be hard at all. Basically, all that's required is a cursor position and a "what button was pressed where" function. I've also got to implement a "clipping" function but, again, it has a perfect SDL equivalent and it isn't used that much in the games - Net doesn't use it at all, neither does Map, or Blackbox from what I can tell.

It also looks like with the hardware-accelerated SDL it will be possible to zoom-in, out, pan etc. around the puzzles with hardware assistance. I don't think I'll play with that, though. For the moment, I'll get standard size puzzles working and let someone else take it and run if they want - it looks incredibly simple but my experience of C is that as soon as you start messing with pointers into large blocks of memory, things start crashing.

I've ignored all the "menu" rubbish that's in the GTK port - I'm not sitting and programming an SDL in-game menu from scratch. You'll just have a button assigned to each function. You probably won't be able to do things like, say, play game 1175784 that you played on the PC - all the functionality is there but I can't be bothered to create the input facilities to let you do that. Each executable is self-contained and you just choose what one to play from the GP2x menu. They all compile perfectly and run within the above constraints. I've not mastered Makefiles yet so I compile and link by hand, the way I used to program. If someone could help with that, it'd be a plus. It looks like it uses a Perl script to make a Makefile. I need to be able to compile sdl.c as a frontend and link it with -lSDL -lSDL_gfx but switching from C to Perl to Bash is a bit too much for my poor brain. At the moment, I've just used gtk.c (handy when programming to have the GTK code as reference, even if there is twice as much GTK code as the SDL equivalent) and I use a bash script to compile (although it throws up loads of gcc errors, it all works fine once it's linked).

Anyway, progress is good. Mostly because of the excellent quality of code. I'm going to have to email Simon Tatham and thank him. Don't expect a polished game AT ALL and don't expect a playable game anytime soon - I get frustrated when things don't compile and so far everything's compiled properly all the way through. If that stops, it's much harder for me to find the gumption to keep going.

 

[Parkydr]

The GTK libraries make the executable bigger, but the performance is fine, X is pretty fast (fast enough for 2XRally and certainly fast enough for puzzle programs) and Net works fine.

As I said before, I only compiled it to test out the building GTK applications, I'm not going to do anything more with it if you're doing an SDL port.

 

[ledow]

The GTK libraries make the executable bigger, but the performance is fine, X is pretty fast (fast enough for 2XRally and certainly fast enough for puzzle programs) and Net works fine.

As I said before, I only compiled it to test out the building GTK applications, I'm not going to do anything more with it if you're doing an SDL port.

Thanks for trying, Parkydr, really. I wasn't going to get into an SDL port but so far it's been a lot easier than I ever thought it would be, or I wouldn't have made the request in the first place - I've done a lot of programming but don't have the heart to pore over thousands of lines looking for a stray asterisk any more. Executable sizes *are* quite small when using SDL, but obviously I'm programming an entire interface from scratch rather than just recompiling for an existing GTK library, which does tons of other stuff, so you expect that.

I see me getting something "playable-enough" and then stopping. I'll post the code when I have something useful to the casual player, obviously, and contribute it back to Simon Tatham if he wants it (he probably won't, we have entirely different coding styles - I just get stuff working, he actually plans ahead for other programmers to use his code!) but after that I won't be doing much more. It's only because it's been so easy-going at the moment that I'm sticking with it - any other piece of software and I'd have got stuck a long time ago.

Even silly things - the backend demands circle, poly, rect and line drawing primitives, plus an update function. For SDL, excluding variable-type conversion, two of those are just literally a single line of code. For filled circles you have to "draw" a filled circle and then "draw" an outline, rather than just having one function for the both, so that function is just two lines of code and some variable fiddling. Poly's are easily done with some jiggery-pokery - converting from an array in the format X,Y,X,Y,X,Y to an array of X and a seperate array of Y before passing to the function. And updates are literally just SDL_Flip(screen).

I'm being overzealous on the memory-locking at the moment because SDL plays about if you don't do it properly and the only real problem that wasn't me being utterly stupid (drawing to SDL screens before they were initialised because I wasn't familiar with the code-paths, converting 6-byte RGB colours to 3-byte the wrong way and messing up the colours, etc.) was solved by copy/pasting example code off the SDL wiki.

In 320x240, it's not actually as bad as you might think. I think scaling will really help though and the accelerated SDL libs seem to offer some lovely functions to do that.

I might have to delve deeper into a couple of areas - is gettimeofday() a good enough random number seed on the clock-less GP2X? Is puzzle-generation going to be too slow for the larger puzzles (I can kill a 600MHz laptop for minutes by asking for a large Bridges game, for instance, but Net seems to go as large as you like without hardly pausing)? Can we strip out the "solver" functions without affecting gameplay? Do ALL of the games play/display as they should all the way through (I only play a handful of them very thoroughly and can't solve some of the puzzles at all - my brain just can't understand them)? If hardware acceleration isn't enough to keep them playable when everything is running, is there a faster alternative to SDL_gfx? Would alpha-blending (built in to SDL_gfx) help with how the games operate? Is drag-and-drop (as Map uses) going to be problematic with SDL/the GP2X's joystick? Is it easy to use external USB devices like mice (although I would never make a GP2X game that DEMANDED them - that's the point of the GP2X, it's portable)?

Anyway. Progress is way ahead of what I ever expected - don't expect a perfect port, though, because it ain't gonna happen. It might be that by the time I've finished, it's easier to just recompile the GTK version!

Oh, screenshot forthcoming, just to compete with that one a few posts above! Don't expect it to look pretty.

C - the programming language created by people that love computers and hate human beings.

 

[ledow]

Okay, I have made ABSOLUTELY NO PROGRESS on this since my last post but I have a screenie at least - that's not because it's hard, or impossible, I'm just very lazy and stopped programming (my wife announcing she was pregnant didn't help!). This also isn't the latest "version" in the screenshot, because I'm tinkering whenever I have five minutes.

What you see in the screenshot is the original GTK version on the left each time and on the right the SDL version (i.e. what the GP2X displays when you link against the GP2X SDL libraries). You'll notice that it's a bit buggy (stray lines, circles, bulges, etc) - most of the artifacts are "off-by-one errors", where I draw a pixel too much or try to draw a line too many. Once I find out where they are, they are quite easy to fix. The circles are a "drag artifact" where the window doesn't properly remove the dragged mouse cursors but "smears" it... that shouldn't be too hard to fix.

But it runs, it's quite fast, it's a damn sight smaller than a full X+GTK installation and it works on a GP2X if you have the right SDL libraries. And obviously, the screenshot is done within X for my convenience but what you see in the right-hand windows is just pure SDL... no X dependencies.

You'll also notice that they are missing fonts - because I have to "replace" the font functions with calls to SDL_Font and I hadn't yet read up on the interface. That's no biggie, either.

Anyway... it proves it's possible. If **I've** done it then it proves it might be extremely easy. At the moment I'm working on joystick use, font drawing and those damn off-by-ones. Once they are done, we are at the "testing" stage and it's just a matter of slapping an interface around it.

 

[ledow]

:lol:

Since my last post I got fonts working in all games. I used SDL_ttf and DejaVu fonts, and they are just as good as the ones that GTK tends to pick up on a basic system... colour - took four lines of code to seperate out the RGB's and recombine them in SDL_Color format. Font loading, rendering etc. - about four lines each. Alignment - a little more tricky to do properly but a little bodge works in 90% of the games perfectly (basically, most of them choose to centre the font). So now all elements of the games are visible, in the correct colours, in a position where they don't affect gameplay (i.e. it's simple to see what's going on, but there are minor imperfections in layout)

In the same amount of time I also threw in some joystick-code to "warp" the mouse cursors... hoping that will work nicely on the actual GP2X. Will test that in my next batch of GP2X-trials of this code.

If only programming were always this easy. It's amazing how quickly good code can be manipulated.

Ran into a few issues where I got my "&" mixed up with my "*" and my "." with my "->" (doesn't sound much but BOY does it make a lot of difference - one works, the other will write random data to your RAM, which is why I don't like C...) but it still going nicely.

 

[onumad]

Please continue with this conversion! .

 

[ledow]

I am! But although progress is blindingly fast when I'm actually working on it, I don't get much time to work on it. I still can't get 100% accurate display (although it's more than good enough to be playable in all games and if you hadn't played the games before, you wouldn't even notice that it was wrong). I spent a lot of time implementing a function that none of the games actually used, and it turned out that the problem I was trying to fix was completely unrelated. But the off-by-ones in the display code are going to go on a backburner because I've spent too much time trying to fix them and got nowhere... I really need a good graphical step-by-step debugger but I haven't got one, nor the time to set one up and fix non-issues like the off-by-one drawing.

Currently, I'm a little stuck because the SDL Timer routines (which are required for most games) are crashing the program because of some of the new code doing SDL stuff inside the Timer callback routine - it doesn't like it and the SDL libraries say not to do it anyway. I can't see a way to eliminate that at the moment, and that's the main bits like controlling the cursor etc. I may have to look into spawning seperate threads and do some cheating (i.e. a lot of primitive variable locking etc. in order to ensure that SDL routines are not actually called from within the SDL callback, but instead the callback sets a flag which makes the SDL routines get executed straight afterwards). That means messing a lot with the main loop and timings so it's likely to cause me all sorts of problems.

So basically, once the timer problems are solved and the joystick can still move the cursor (they are a bit mutually-exclusive at the moment because of the timer callback crashes), then it'll be playable and it'll probably be the end of my contribution.

The code is possibly the ugliest I've ever seen in my life, and this is coming from someone who's always coded the ugly way first.

 

[ledow]

Just a quick update.

I have replaced quite a lot more code to make it more SDL / GP2X friendly. This has been a step forward in terms of bug-fixing. I have already fixed hidden bugs where I had omitted what I thought were unnecessary functions, but the off-by-ones are still proving elusive so I'm not devoting time to them.

I have control on the joystick/buttons, while playing, but at the moment the joystick "emulates" the keyboard which is a bit more inconvenient. It's not hard to change it to mouse control *now* but it's just more work. Before I couldn't get anything but automated control for the SDL version (i.e. I used to tell it to "click" at x,y at time t and programmed a list to test the SDL interface because it couldn't interactively control itself - that's now changed). I'm still having problems with the timing routines and I have to find a way around them before anything is playable to anyone other than myself.

I'm still chasing down a couple of bugs - the timer routines are still giving me problems but I have a possible solution now that I've replaced a lot more code. There is something strange happening on one particular screen update type but I've bodged a fix for that (basically, I "press" a key twice to counter-act it's effect and the updates that should have happened take place, but what a bodge that is!). And, obviously, I have to get mouse code working properly but now that I have proper SDL event control working on my terms, that shouldn't be too hard (I estimate 10-20 lines of code, and it should also fix the drag/drop problem in "map"). I've also found a bug that seems to be related to drawing a rectangle over the top of a polygon, which causes the vertical lines in "map". I think the only way to fix that is to map what it does pixel-by-pixel on some paper, so that can wait.

Before, I was basically trying to shortcut and reuse as much GTK-style code from the existing GTK interfaces as I could and that was giving me lots of problems. It worked but it led to no end of problems once you got some way down the road. At first, every function was dual GTK/SDL depending on compiler options and the SDL component basically fudged all the GTK-style code and data that it was given and plugged it into SDL functions. That's not slowly disappearing for SDL-only code/data and it's making stuff easier.

This picture doesn't look much different from the last but notice the fonts. A lot of what's different in this version is in the control interface - before I had get the joystick & buttons to pretend to be a mouse, now they are seen as keypresses by the programs. That helps a bit and meant I could get some interactive control over the programs. Also, in case I haven't mentioned it, the programs are just as stable as the desktop variants. The only crashes I've ever got are from my own stupidity with pointers.

In this screenshot, the windows with "X" in the top-left corner of the window are the SDL versions.

 

[ledow]

About an hour after posting, I got the mouse controls working perfectly. Now the joystick buttons (that's how the GP2X sees it's buttons/D-pad/thumbstick) can control just about anything. I have primtive mappings to allow you to move the cursor with the D-pad/thumbstick and left/middle/right-click with other buttons. I have one button for quit, one for new game etc. That also means that, theoretically, there's no problem with using the games on the F200, and a lot of them are perfect for touchscreens as the Palm-port of the games has shown me before.

I'm having graphical problems dragging at the moment but that's because something along the line isn't doing what it says it should - hopefully some simple breakpoints will narrow down exactly what line is doing it but it seems to be because the SDL clip/blit functions aren't a perfect match for the GTK clip/blit functions that they've replaced. Things work but look ugly in drag-drop games.

I was using "Net" are my first testbed (keyboard or mouse control, timed animation, simple graphics/poly's, no dragging but three different types of clicks) and that's working fine now, just timer probs (common to all games at the moment and provide smooth animation among other things) and off-by-ones (common to all games, provides slightly skewed positioning as can be seen in the above screenshots). When I exhausted "Net", I moved onto "Bridges" (fonts, dragging) and that seems okay because the dragging is done differently. Now I've exhausted both "Net" and "Bridges", I've moved to "Map" as my next testbed (more poly's, more colours, fonts, more advanced drag-drop) and also "Untangle" - the drag-dropping is the only "new" problem with those.

If I can solve the timer problems (or bodge them), then Net and Bridges can be tested and released. If I can solve drag-drop drawing, then "Map" and "Untangle" can follow. The other games I'm not familiar enough with to play through all the way yet but I've tested them all and a lot of them "just work". By the time that Map and Untangle are finished and working, I'll have 99% of the "SDK" behind the games working properly anyway, so all other games (including any new ones built on that framework) should just follow suit - and experiments show this to be true - "BlackBox" is virtually perfect too, despite never having any code really tested on it previously.

 

[ledow]

I'm still having graphical problems dragging (which ruins "Map") but I have now got every operation moved safely over to SDL-only-functions and working, and in such a way that the SDL timers no longer exclude certain control mechanisms (because now I have full control over the event queue, which I didn't before).

Joystick control - Check (pretends to be mouse, drags properly, buttons assigned for left, middle, right-click - I have to pick a set of button mappings)
Mouse control - Check (so hopefully it'll work on SDL for F200's that see the touchscreen as a mouse)
SDL-only functions for coloured rect, circle, poly and line - Check (but with off-by-ones and trouble dragging)
SDL-only timer functions - Check (and seem to be the most reliable part of the whole program now, proven by smooth animation in all games)
SDL-only functions for coloured fonts - Check (although I've cheated and just used one font, which doesn't affect anything because games only use one font at the moment but it's not "in keeping" with the way the framework was designed).
Screen updates that no longer require any trickery - Check
Start, stop, restart - Check (implemented as seperate joystick buttons - no functionality to play particular games from random seeds or save games because I'm just not interested enough in that)

Unfortunately, the next step is one of the biggest - clean up, remove debugging code (lots of it) and lots of testing to make sure it doesn't blow up people's GP2X's (I'm not that happy at the prospect that I could have missed a pointer somewhere and having to test it on mine, so I wouldn't wish it on a stranger just yet either).

It also happens to double-up as a primitive SDL port for Windows. I don't think that in it's current state it would ever be accepted into the main program as a port, mainly because of the lack of a GUI and certain assumptions and shortcuts, but it's certainly a good starting point for anyone who wanted to do that.

 

[Kramy]

Neato. Looks like you're making progress.