GP2X Gp2x Executable Format

[Inopia]

As for Squidge's idea, let me propose something: why not keep the format simple as possible, without icon data etc, and just put the exe along with any metadata inside a ZIP file?

- Any metadata a launcher might need can be placed inside, or omitted at will.
- Extensible, you can always add more stuff like screenshots as launcher technology progresses
- Compression, your executables are automatically compressed to save space and upload time
- You can put more than one executable inside. This is _very_ useful for instance if you are releasing a program with different optimizations (overclocked/non-overclocked) or if we have another BLU problem in the future.
- Because the entire executable has to be loaded into memory anyway, unzipping it doesn't really slow things down. In fact, it might even decrease launch times because the bottleneck is probably loading from NAND/SD.
- Universal, because everybody knows how to use a zip/unzip tool it's easy to just open up the file, add, remove or change some data.

As for the 940 support, I'm probably more for leaving out the 940 support in the executable. Although I can undertand rob's point, and I think it's important we make it easy for programmers to use the second core, I propose a library of functions that make it easy to upload and run code on the copro from the 920.

 

[Robster]

Inopia: well, it's an interesting idea. One thing I'm not sure of: when you open a zip file, how much of it actually gets loaded before any particular thing can be extracted?

As an example, with information in the header, you can load a small portion of the file to start with (in a loader, say) to get all the information you need for loading the rest of the file. And if your program is a loader, you never actually have to load the whole file.

Now, just to take this to an extreme, imagine a zip file along your lines, with MAME inside and a bunch of icons etc. The executable file for MAME could easily be 5MB, zipped down to (say) 2MB. Does zlib have to load in the entire 2MB zip file to be able to extract a file called 32x32.ico? If so, that could make loaders quite slow

Having the executable uncompressed does allow the possibility of mr_spiv doing something extra clever along the lines of b2fxec

Also, just because I'm interested, I can't imagine "a library of functions that make it easy to upload and run code on the copro from the 920". How is the 940's code to be generated? As a completely separate project, and stored as a file on SD? Can someone please give some concrete examples of how this will work - as in, linker scripts, makefiles, library functions etc.

I imagine that in 99% of cases there will be one image for the 920 and one for the 940 - for example, Doom2x may have the game and rendering code on the 920 and the sound engine on the 940. These won't change at all during runtime. I still think the simplest way to support this scenario is to have the 920 and 940 images both in the one executable file.

 

[Inopia]

The 940 and the 920 cores are completely similar when it comes to instruction set, so anything that runs on one, runs on the other. So you can just compile everything and simply point the 940 to any code you want to execute. I don't see why that shouldn't work? Maybe I'm missing something important though (probably)

As for the zip file, I should check that first. I know a rar file can be opened even if it's only partially downloaded But rar is a closed format. If zip needs to unpack everything that might pose a problem, but I doubt it since java uses the same technique (.jar == .zip) without problems. Very good point though

 

[Inopia]

From wikipedia:

ZIP is a fairly simple archive format that compresses every file separately. Compressing files separately allows for individual files to be retrieved without reading through other data; in theory, it may allow better compression by using different algorithms for different files. However a caveat to this is that archives containing a large number of small files end up significantly larger than if they were compressed as a single file (the classic example on Unix-like systems is the common tar.gz archive which consists of a TAR archive compressed using gzip).

 

[Squidge]

From a launcher point of view, we could always add the icon data first, so it's relatively easy to pull out of the archive. Or we could go even simpler, and just have our own simple archived format:

int num_of_files_in_archive;
int filehdr_start[num_of_files_in_archive];

struct filehdr
{
unsigned char filename_len; // to make it easier to read just the filename (no need to guess and hunt for null terminators)
unsigned char filename[filename_len];
unsigned int filesize; // size of file, MSB is set if compressed
unsigned char data[filesize];
}

files could be compressed seperately using the standard, portable and easy to use gzip library.

icon data would always be the first file in the archive, therefore no searching is required.

Although this is not as flexible as zip, there's no need to change the archive after it's built, and no need for date stamps, etc.