Pandora pandora repo/website/native app-manager development


Testcase:
Code: 
[urjaman@phenom ~]$ modprobe squashfs
FATAL: Error inserting squashfs (/lib/modules/2.6.35-ARCH/kernel/fs/squashfs/squashfs.ko): Operation not permitted
[urjaman@phenom ~]$ ls testdir
file_a  file_b  file_c
[urjaman@phenom ~]$ mksquashfs testdir test.sqfs
Parallel mksquashfs: Using 4 processors
Creating 4.0 filesystem on test.sqfs, block size 131072.

Exportable Squashfs 4.0 filesystem, data block size 131072
	compressed data, compressed metadata, compressed fragments
	duplicates are removed
Filesystem size 0.23 Kbytes (0.00 Mbytes)
	82.17% of uncompressed filesystem size (0.28 Kbytes)
Inode table size 55 bytes (0.05 Kbytes)
	42.31% of uncompressed inode table size (130 bytes)
Directory table size 39 bytes (0.04 Kbytes)
	69.64% of uncompressed directory table size (56 bytes)
Number of duplicate files found 3
Number of inodes 4
Number of files 3
Number of fragments 0
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 2
Number of uids 1
	urjaman (501)
Number of gids 1
	users (100)
[urjaman@phenom ~]$ unsquashfs -l test.sqfs
Parallel unsquashfs: Using 4 processors
3 inodes (0 blocks) to write

squashfs-root
squashfs-root/file_a
squashfs-root/file_b
squashfs-root/file_c
[urjaman@phenom ~]$ lsmod | grep squashfs
[urjaman@phenom ~]$ cat /proc/filesystems 
nodev	sysfs
nodev	rootfs
nodev	bdev
nodev	proc
nodev	cgroup
nodev	cpuset
nodev	tmpfs
nodev	devtmpfs
nodev	binfmt_misc
nodev	debugfs
nodev	securityfs
nodev	sockfs
nodev	pipefs
nodev	anon_inodefs
nodev	devpts
nodev	ramfs
nodev	mqueue
	ext3
nodev	usbfs
	ext2
nodev	fuse
	fuseblk
nodev	fusectl
EDIT2: Notes on above.
1. I dont have the rights to directly load the squashfs module. This would also propably be the case with your php/webserver privileges.
2. I can still easily use mksquashfs to create a test sqfs file.
3. I can also list the contents of that file easily with unsquashfs.
4. The kernel has not autoloaded the squashfs module when I did these tests.

EDIT: Additionally:
Code: 
[urjaman@phenom ~]$ pacman -Ql squashfs-tools
squashfs-tools /sbin/
squashfs-tools /sbin/mksquashfs
squashfs-tools /sbin/unsquashfs
[urjaman@phenom ~]$ pacman -Qo /lib/modules/2.6.35-ARCH/kernel/fs/squashfs/squashfs.ko 
/lib/modules/2.6.35-ARCH/kernel/fs/squashfs/squashfs.ko is owned by kernel26 2.6.35.4-1

squashfs-tools contains mksquashfs and unsquashfs. In my case the squashfs kernel module is owned by the kernel package. It could also be in a squashfs-modules package (or something like that) depending on the distribution.
 
lol nice but what does all this mean lol would it be easier to write a bash script and have
Code: 
exec('bash.sh',$output);
echo $output
run a bash script then return output? I belive exec allows to call bash files problem is id need to write one lol havent got a clue.
 
update:

this is the repoly I had from my isp about squashfs, we got it installed ect but...

I've run the command manually and it looks like the squashfs tools don't support the version being used by your archive:

Major/Minor mismatch, filesystem on xournal.pnd is (4:0) I only support Squashfs 3.0 filesystems! Later releases will support older Squashfs filesystems

You'll need to consider alternate access methods for that file, I'm afraid.

Kind Regards,

Support
Click to expand...

I wonder why they cant install a later release?
 
milkshake said:
I wonder why they cant install a later release?
Click to expand...
At a guess, they're using a distro which has ceased active updates, like Ubuntu Hardy or something. That's probably fine, if it ain't broke don't fix it sort of thing, but it does keep you from getting the latest goodies until they update.
 
Last edited by a moderator:
This is getting ridiculous. Here's how to do it, kids:

Code: 
16:02:11-torpor@w1xer.at:~/Pandora/hak/

$ dd ibs=1 bs=1 skip=`isosize ~/Desktop/PandOricutron_108.pnd` if=~/Desktop/PandOricutron_108.pnd status=noxfer 2>/dev/null | sed -ne '/<\?xml/,/<\/PXML>/p' 

<?xml version="1.0" encoding="UTF-8"?>
<PXML xmlns="http://openpandora.org/namespaces/PXML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="PXML_schema.xsd">
<application id="PandOricutron" appdata="PandOricutron">
<title lang="en_US">Oricutron on your Pandora -- port by torpor</title>
<title lang="de">Oricutron aufs Pandora -- als torpor geport'ed ..</title>
<description lang="en_US">This is the Oricutron Oric-1/Atmos 8-bit emulator ported and packaged for Pandora, and including some nice new-school games from champions of the homebrew, Oric/Atmos scene .. ported by torpor for Oric and Pandora lovers everywhere!</description>
<version major="1" minor="0" release="0" build="8"/>
<exec command="oricutron_PandoraOS" arguments="--fullscreen"/>
<author name="Peter Gordon" website="http://code.google.com/p/oriculator/" />
<icon src="PandOricutron.png"/>
<categories>
<category name="Emulators"/>
</categories>
</application>
</PXML>

EDIT: Redirect stderr to /dev/null so that output is not corrupted by dd ..
 
torpor said:
skip=`isosize ~/Desktop/PandOricutron_108.pnd`
Click to expand...

I'm guessing this wont work with squashfs images.
 
Last edited by a moderator:
Brilliant! Good ol' sed.
It surely will work with squashfs. Once the filesystem is made, the PXML is tacked onto the end in plain text, that's why Jeff and I were trying to suggest reading backwards from the end. This works as well, though it may take longer for particularly large files.

edit: oh, I see now. Nevermind, I misunderstood what isosize did. I figured it would return 0 for non-ISO files, but it doesn't, returns whatever arbitrary value happens to be given at that location in the file.
So you need to correct it slightly to have a "file blah.pnd | grep ISO" on there. Or just remove the skip completely and let it take its time.
 
Proper .PND's are supposed to have .png as well, this strips it, and will only work on "properly-formed" .pxml's, which I have to admit to having made here in this case alas ..

EDIT: ooops .. yes, I guess non-ISO .PND's exist, I was under the impression we're "supposed to use .iso" or something, but I guess not. But of course, the 'isosize' can be grok'ed elsewhere, for other things, in the pipeline ..
 
If all PNDs were ISO, we wouldn't need squashfs tools or fancy data stripping, we could just extract what was needed from the ISO with standard tools.
The PND spec also says tools should attempt to be filesystem independent, because while ISO and squashfs are the current standard, it may grow to allow other formats.
 
So I guess this means I'm stuck as they will not allow me to use the mount command.
anyone else got any ideas because Im stuck and I do want stick to php for the site.

also as well as getting the pxml out of the pnd I also want to get the icon and the screenshot image (if it exists).

7z I have access to I belive but some .iso's are squashfs created, is there no way we can we can put in and official standard to use iso when creating pnd files, then we could just go through the existing pnd files and make sure they are all iso pnd files?

lol am I going mad here or what? lol
 
milkshake said:
So I guess this means I'm stuck as they will not allow me to use the mount command.
Click to expand...
Not at all. Just use Torpor's method without the skip option. There is no way you don't have access to dd and sed. It takes longer for particularly large files without being able to skip, but shouldn't be too bad
Code: 
$pxml = system("dd ibs=1 bs=1 if=file.pnd status=noxfer 2>/dev/null | sed -ne '/<PXML/,/<\/PXML>/p' "
Tweaked the sed line a bit. Not all PXML files are guaranteed to xml version line, and even if they do, you don't need it.
It takes my PC about a minute to scan a 35MB file and strip out the pxml data. Do that in a separate processing thread or something, if you can.

PokeParadox said:
ISO as standard for PND? Why no sir I don't like it.
squashfs is nice because it greatly reduces file size of many PND files and it is basically standard for anything I release...
Click to expand...
Not to mention longer file name support, longer directories, case sensitive filenames if a program requires it, file level security... anything else I'm forgetting?
My Java PND is impossible in an ISO, not just because of how big it was, but also because of filename and directory length limitations.
 
Last edited by a moderator:
Yikes, filesystems-within-a-file-extension .. are you really, really, really sure you don't just want to disguise tarballs in that .PND? Please sir, do not throw me into that briar patch!

;)
 
WizardStan said:
filename and directory length limitations.
Click to expand...
I'm not advocating ISO, but see http://en.wikipedia.org/wiki/RockRidge (or simply mkisofs -R).
Personally, I use squashfs for everything, makes for really compact pnds.
 
Last edited by a moderator:
ok Ive written this small script that command doesnt seem to pull all the xml data unless there is an issue with the xournal xml file see code.

Code: 
//search for .pnd filetype in current dir
$filetype = "pnd";
$dir = opendir('.');
while (false !== ($file = readdir($dir))) {
        $pnd = pathinfo($file);
        //if filetype is pnd then create variable name of pnd archive
        if($pnd['extension']==$filetype){
                $pndfile = $file;
        }
    }
closedir($dir);
echo "the current .pnd file in test dir is $pndfile <br/>The contents of the xml array are as follows:<br/>";

//run Torpor's method and pull the xml details
exec("dd ibs=1 bs=1 if=".$pndfile." status=noxfer 2>/dev/null | sed -ne '/<PXML/,/<\/PXML>/p' ",$output, $return);
print_r($output);

check output here
 
Here is some PHP I knocked up and tested on my web server (note I've not really written PHP before):

Code: 
<html>
 <head>
  <title>PHP Test</title>
 </head>
 <body>

 <?php 

$filename = "/path/to/pandorapanic.pnd";

$readBufferSize = 2048;
$fileSize = filesize( $filename );
$read = 0;

$handle = fopen($filename, "rb");

$begin = -1;
$end = -1;

while ( $read < $fileSize )
{
    $toRead = $readBufferSize;
    $pos = $fileSize - $read - $readBufferSize;
    if ( $pos < 0 )
    {
        $toRead += $pos;
        $pos = 0;
    }
    fseek( $handle, $pos );
    $data = fread( $handle, $toRead );
    $read += $toRead;
    if ( $end == -1 )
    {
        for ( $i = 0; $i < $toRead - 6; $i += 1 )
        {
            if (   $data[ $i ] == '<' &&
                   $data[ $i + 1 ] == '/' &&
                 ( $data[ $i + 2 ] == 'P' || $data[ $i + 2 ] == 'p' ) &&
                 ( $data[ $i + 3 ] == 'X' || $data[ $i + 3 ] == 'x' ) &&
                 ( $data[ $i + 4 ] == 'M' || $data[ $i + 4 ] == 'm' ) &&
                 ( $data[ $i + 5 ] == 'L' || $data[ $i + 5 ] == 'l' ) &&
                   $data[ $i + 6 ] == '>' )
            {
                $end = $pos + $i + 7;
                break;
            }
        }
    }
    for ( $i = 0; $i < $toRead - 4; $i += 1 )
    { 
        if (   $data[ $i ] == '<' &&
             ( $data[ $i + 1 ] == 'P' || $data[ $i + 1 ] == 'p' ) &&
             ( $data[ $i + 2 ] == 'X' || $data[ $i + 2 ] == 'x' ) &&
             ( $data[ $i + 3 ] == 'M' || $data[ $i + 3 ] == 'm' ) &&
             ( $data[ $i + 4 ] == 'L' || $data[ $i + 4 ] == 'l' ) )
        {
            $begin = $pos + $i;
            break;
        }
    }
    if ( $begin != -1 && $end != -1 )
    {
        fseek( $handle, $begin );
        $data = fread( $handle, $end - $begin );
        echo "<p>" . htmlspecialchars( $data ) . "</p>";
        break;
    }
}

if ( $begin == -1 || $end == -1 )
{
    echo '<p>Not found</p>';
}

?> 
 </body>
</html>

The output I get is:

Code: 
<PXML xmlns="http://openpandora.org/namespaces/PXML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="PXML_schema.xsd"> <application id="ppanic101"> <title lang="en_US">PandoraPanic!</title> <exec command="PandoraPanic_R"/> <icon src="./images/ppicon.png"/> <description lang="en_US">PandoraPanic! A frantic, frenetic and occasionally mind bending minigame collection created for Pandora by the GP32X community.</description> <description lang="nl_NL"> PandoraPanic! Een panische minigame collectie voor de pandora door leden van het GP32X Forum</description> <previewpics> <pic src="./preview/PanicPreview1.png"/> <pic src="./preview/PanicPreview2.png"/> <pic src="./preview/PanicPreview3.png"/> <pic src="./preview/PanicPreview4.png"/> <pic src="./preview/PanicPreview5.png"/> <pic src="./preview/PanicPreview6.png"/> </previewpics> <info name="PandoraPanic! Manual" type="txt/html" src="index.html"/> <author name="Minigames: the GP32X Community, Framework: PokeParadox and Pirate Games" website="http://projectinfinity.org.uk/doku.php?id=homebrew:games:pandorapanic"/><!--Optional email and website, name required--> <version major="1" minor="0" release="1" build="3"/><!--This programs version--> <categories> <category name="Game"><!--category like "Games", "Graphics", "Internet" etc--> <subcategory name="ArcadeGame"/><!--subcategory, like "Board Games", "Strategy", "First Person Shooters"--> </category> </categories> </application> </PXML>

Steve
 
I still want to be able to pull the icon file out and the screenshot image if it exists which will be much more difficult than the pxml.

I wonder why that terminal command doesn't work then wizardstan?
 
Well if the icon is just appended after the PXML, then there should be the PNG magic data after '0x89 0x50 0x4e 0x47', so you can just read that in (I've checked pandorapanic.pnd and there is PNG data after the PXML). I don't know how screenshots work, are they put at a specific point in the file?

Steve
 
so i can use image magic to pull the icon if the icon is appended after the xml file?

ok i have amended the test script to check for multiple pnd files just for testing purposes of course lol.

Code: 
//search for .pnd filetype in current dir
$filetype = "pnd";
$dir = opendir('.');
while (false !== ($file = readdir($dir))) {
        $pnd = pathinfo($file);
        //if filetype is pnd then create variable name of pnd archive
        if($pnd['extension']==$filetype){
                $pndfile[] = $file;
        }
}
closedir($dir);

$pndnum = sizeof($pndfile);
echo "there are $pndnum .pnd files in the current directory<br/><br/>";

//for each pnd found return xml
foreach($pndfile as $val){
        echo "$val<br/>The contents of the xml array are as follows:<br/>";

        //run Torpor's method and pull the xml details
        exec("dd ibs=1 bs=1 if=".$val." status=noxfer 2>/dev/null | sed -ne '/<PXML/,/<\/PXML>/p' ",$output, $return);
        print_r($output);
        echo "<br/><br/>";
        unset($output);
}

output
 
You must log in or register to reply here.

Similar threads

D
Pandora Making a pnd, PXML, and General PND problems
Replies
16
Views
3K
mcobit
mcobit
milkshake
Pxml Specification
Replies
8
Views
3K
milkshake
milkshake
milkshake
PXML specification
2 3
Replies
44
Views
8K
elvy
E
porg
Special attribute "Native Pandora App" for the repo or PND-XML, so that native apps get the exposure
2
Replies
23
Views
3K
porg
porg
ivanovic
PXML.xml Spec Changes - Versions
2
Replies
26
Views
5K
ivanovic
ivanovic
Back
Top