Release PND Manager

[Speaker Ender]

All,


I've created a simple GUI Front end that allows you to extract a PND file on Windows and then recreate the PND file. The tool uses 7-zip and mksquashfs.exe in the background to open/create the PND files (no need to mess with the command line). If you don't have 7-zip installed it prompts you to install it, and if you don't have mksquashfs.exe downloaded it can attempt to download it for you.


Just run setup to install, it will then create a program group "speakerender" in your start menu.


Install is attached


EDIT: 6/15/2011 @ 11:56am


Latest version: v1.0.1.5 PNDManager-V1_0_1_5.zip

Spoiler

Change Log


v1.0.1.5


Fixed bug on detection of CHMOD in default cygwin path (c:\cygwin\bin) and recommended path c:\mksquashfs


v1.0.1.4


Includes Self Extracting copy of mksquashfs/cygwin1.dll/chmod.exe/cygintl-8.dll/cygiconv-2.dll/cyggcc_s-1.dll


Added step to chmod Files being added to squashfs to 755


Using Cygwin style paths for mksquashfs/chmod to allow proper function(cygdrive/<path>, provided by foxblock)

I've tried it on a couple of PNDs and the PNDs it creates appear valid (but are smaller in size),


Please give it a try and let me know what you think. Thanks

 

[freamon]

OK - gave it a try with get-iplayer.


The good news is that it extracted the contents perfectly well. The bad news is that when I copied the pnd it created to my Pandora, it wouldn't run. The error message in /tmp/pndrun_getiplayer was:

error determining fs, output was data

I tried in both XFCE and MiniMenu (which also failed on showing the preview). I'm guessing that the fact the new pnd was 68 kb smaller than the old one isn't a good thing. Maybe have a look at pnd_make.sh and see if it adds anything you're missing?


EDIT: just realised you might not have a Pandora: pnd_make.sh can be found at


http://git.openpandora.org/cgi-bin/gitweb.cgi?p=pandora-libraries.git;a=tree;f=testdata/scripts;hb=HEAD


*---*


Some other thoughts:


1) It didn't seem to try to download mksquashfs.exe, and when I downloaded it (from ftp.slax.org), I originally didn't download cygwin1.dll. The app reports the missing dll, but then still says it successfully created the pnd (which it didn't, of course).


2) Maybe when you get it working, the app can read the PXML file to find where the icon is, so the Optional Icon path gets filled in automatically?

 

[milkshake]

one thing to remember, it is not enough to simply package pnd with squashfs you must also append the PXML and icon to the end of the generated PND if you want it to show up in the menu or desktop.

 

[freamon]

one thing to remember, it is not enough to simply package pnd with squashfs you must also append the PXML and icon to the end of the generated PND if you want it to show up in the menu or desktop.

I'd wondered that too, but I checked the file it produced in a hex editor and I could see all the PXML stuff, followed by 'PNG' in the file header of the icon file.


Speaking of headers though, the original starts with 'hsqs' (for zlib compression), and the new one starts with 'shsq' (for lzma compression), so that explains why the new one is smaller. I'm guessing that the PND system in the Pandora relies on PNDs being compressed with zlib?

 

[Speaker Ender]

Thanks for the feedback and trying the pnds it makes.

one thing to remember, it is not enough to simply package pnd with squashfs you must also append the PXML and icon to the end of the generated PND if you want it to show up in the menu or desktop.

It actually does that, currently it runs the mksquashfs.exe creates "newFiles.sfs"


Then runs (if no png icon file):


copy /B newFile.sfs+PXML.xml newFile.pnd


del newFile.sfs


if icon png provided:


copy /B newFile.sfs+PXML.xml newFile.step1


copy /B newFile.step1+iconFile.png newFile.pnd


del newFile.sfs


del newFile.step1


I found if I don't specify its a binary copy the new file results in a couple kb that isn't usable at all.

I'd wondered that too, but I checked the file it produced in a hex editor and I could see all the PXML stuff, followed by 'PNG' in the file header of the icon file.
Speaking of headers though, the original starts with 'hsqs' (for zlib compression), and the new one starts with 'shsq' (for lzma compression), so that explains why the new one is smaller. I'm guessing that the PND system in the Pandora relies on PNDs being compressed with zlib?

Hmm, I'll see what I can do with that, mksquashfs does have a switch for nolzma.

Some other thoughts:
1) It didn't seem to try to download mksquashfs.exe, and when I downloaded it (from ftp.slax.org), I originally didn't download cygwin1.dll. The app reports the missing dll, but then still says it successfully created the pnd (which it didn't, of course).


2) Maybe when you get it working, the app can read the PXML file to find where the icon is, so the Optional Icon path gets filled in automatically?

1) I'll have to check, did you have mksquashfs.exe already in a folder "c:\mkSquashfs"? Currently it checks intially for mksquashfs.exe in that folder and cygwin1.dll in that folder or in the system32 folder. I could restrict it to just the same folder with mksquashfs.


2) I'll give that a try.

 

[milkshake]

It actually does that, currently it runs the mksquashfs.exe creates "newFiles.sfs"


Then runs (if no png icon file):


copy /B newFile.sfs+PXML.xml newFile.pnd


del newFile.sfs


if icon png provided:


copy /B newFile.sfs+PXML.xml newFile.step1


copy /B newFile.step1+iconFile.png newFile.pnd


del newFile.sfs


del newFile.step1

woah, that seems like overkill try this.


when making the squashfs file you can specify the name of the archive it produces so you dont need to rename it afterwards:

mksquashfs ./pnd_folder name_of_pnd.pnd

this way it already has the right extention.



also with the dos binary copy command you dont need to copy to new binary and delete the old one you can just append to existing binary like so.



assuming you followed the above steps:

copy /B name_of_pnd.pnd+PXML.xml

copy /B name_of_pnd.pnd+iconFile.png

and thats it, both files appended without creating additional files along the way then deleting at the end.


3 commands in total, no redundancy.


works with windows 7 havent tried preview windows versions but im assuming it will work the same.

 

[freamon]

Some other thoughts:
1) It didn't seem to try to download mksquashfs.exe, and when I downloaded it (from ftp.slax.org), I originally didn't download cygwin1.dll. The app reports the missing dll, but then still says it successfully created the pnd (which it didn't, of course).

1) I'll have to check, did you have mksquashfs.exe already in a folder "c:\mkSquashfs"? Currently it checks intially for mksquashfs.exe in that folder and cygwin1.dll in that folder or in the system32 folder. I could restrict it to just the same folder with mksquashfs.

Oops - I should clarify: I didn't have mksquashfs.exe anywhere on my system, when I run setup.exe it recognised this and opened a dialogue box saying 'specify program location?' but I was expecting a dialogue box saying it would attempt to download the files.


I've just re-run it and figured it out: if you press 'No' to the 'specify program location?' dialogue *then* it asks about attempting to download the files.


I found this a bit counter-intuitive to be honest, maybe it would be easier on my brain if setup.exe popped up a single dialogue box with 2 buttons on it: one saying 'specify program location' and the other saying 'attempt download from ftp.slax etc'. I realise this is still a beta though, and all the stuff you end up doing for the hard-of-thinking comes later.

 

[Speaker Ender]

Updated version, now compresses with hsqs (uses the -nolzma option of mksquashfs). The file size now closer resembles the original PNDs I've tried. If someone could test the PNDs it now makes.

Oops - I should clarify: I didn't have mksquashfs.exe anywhere on my system, when I run setup.exe it recognised this and opened a dialogue box saying 'specify program location?' but I was expecting a dialogue box saying it would attempt to download the files.


I've just re-run it and figured it out: if you press 'No' to the 'specify program location?' dialogue *then* it asks about attempting to download the files.


I found this a bit counter-intuitive to be honest, maybe it would be easier on my brain if setup.exe popped up a single dialogue box with 2 buttons on it: one saying 'specify program location' and the other saying 'attempt download from ftp.slax etc'. I realise this is still a beta though, and all the stuff you end up doing for the hard-of-thinking comes later.

I've fixed the dialogue to be clearer for the mksquashfs. It provides a two radio button option, 1 to look on the local device for the files, and the other to attempt the ftp download.

assuming you followed the above steps:


copy /B name_of_pnd.pnd+PXML.xml


copy /B name_of_pnd.pnd+iconFile.png


and thats it, both files appended without creating additional files along the way then deleting at the end.


3 commands in total, no redundancy.


works with windows 7 havent tried preview windows versions but im assuming it will work the same.

Thanks, I'm not used to merging files via copy, I've updated the code so no need to create the extra files or delete them. It works the same on at least Windows 7 and XP.

 

[freamon]

Finally got around to testing your new version. Original pnd: 529kb, new pnd: 528kb.


Unfortunately, the pnd it creates is not recognised as such when put on a Pandora. It doesn't appear in the menu, and when you try to manually mount it, it complains about 'wrong fs type'. I had a quick look at the new file in a hex editor and there's no 'PXML' or 'PNG' in there, so I'd guess it's not appending them or not appending them properly.

 

[Speaker Ender]

Finally got around to testing your new version. Original pnd: 529kb, new pnd: 528kb.


Unfortunately, the pnd it creates is not recognised as such when put on a Pandora. It doesn't appear in the menu, and when you try to manually mount it, it complains about 'wrong fs type'. I had a quick look at the new file in a hex editor and there's no 'PXML' or 'PNG' in there, so I'd guess it's not appending them or not appending them properly.

Okay, it looks like cutting out creating the temp files (and not specifying an output file) results in the xml and png file not being appended.


This:

c:\mksquashfs\mksquashfs.exe "C:\temp\pnd\test 1\gpfce-gp2x" "C:\temp\pnd\test 1\gpfce-gp2x-1.pnd" - nolzma

copy /B "c:\temp\pnd\test 1\gpfce-gp2x-1.pnd"+"c:\temp\pnd\test 1\gpfce-gp2x\PXML.xml"

copy /B "c:\temp\pnd\test 1\gpfce-gp2x-1.pnd"+"c:\temp\pnd\test 1\gpfce-gp2x\gpfce.png"

Original: gpfce-gp2x.pnd is 3,250KB

New File size: gpfce-gp2x-1.pnd is 3,348KB





But run like this:

c:\mksquashfs\mksquashfs.exe "C:\temp\pnd\test 1\gpfce-gp2x" "C:\temp\pnd\test 1\gpfce-gp2x-2.pnd" - nolzma

copy /B "c:\temp\pnd\test 1\gpfce-gp2x-2.pnd"+"c:\temp\pnd\test 1\gpfce-gp2x\PXML.xml" "c:\temp\pnd\test 1\gpfce-gp2x-2.pnd" 

copy /B "c:\temp\pnd\test 1\gpfce-gp2x-2.pnd"+"c:\temp\pnd\test 1\gpfce-gp2x\gpfce.png" "c:\temp\pnd\test 1\gpfce-gp2x-2.pnd"

Original: gpfce-gp2x.pnd is 3,250KB


New File Size: 3,250KB


So I've updated again. Assuming this works, next I'll work on processing the xml to try to find the default icon.


Thanks again for giving this a try.

 

[freamon]

Well the PXML and PNG info is back, but it still won't run. Output from /tmp/pndrun_getiplayer.out

Filetype is squashfs

Mounting PND (mount -t squashfs)

mount: wrong fs type, bad option, bad superblock on /dev/loop3,

missing codepage or helper program, or other error

Dunno what to suggest - sorry.

 

[Speaker Ender]

Well the PXML and PNG info is back, but it still won't run. Output from /tmp/pndrun_getiplayer.out

Filetype is squashfs

Mounting PND (mount -t squashfs)

mount: wrong fs type, bad option, bad superblock on /dev/loop3,

missing codepage or helper program, or other error

Dunno what to suggest - sorry.

If you manually run mksquashfs, and the copy command on getiplayer? Does it result in the same problem?

mksquashfs c:\temp\getiplayer c:\temp\getiplayer-1.pnd -nolzma

copy /B c:\temp\getiplayer-1.pnd+c:\temp\getiplayer\PXML.xml c:\temp\getiplayer-1.pnd

copy /B c:\temp\getiplayer-1.pnd+c:\temp\getiplayer\icon\get-iplayer.png c:\temp\getiplayer-1.pnd

Looking at the hex view of the new PNDs I'm outputting (and from manually running the above), I've got a padding of whitespace between the squashfs and the xml file. The is probably the problem, but not sure of the cause.


EDIT: So actually the white space is padded to the intial squashfs created. I'll see if there is another switch I need to apply


EDIT2: so need to add -nopad to mksquashfs.exe to avoid that white space padding.

 

[Speaker Ender]

EDIT: So actually the white space is padded to the intial squashfs created. I'll see if there is another switch I need to apply


EDIT2: so need to add -nopad to mksquashfs.exe to avoid that white space padding.

Okay, new version, creation now parses the PXML.xml file to find the icon file, and mksquash.exe now uses -nopad (this removes the whitespace gape in the PND between the squashfs, xml, png).

 

[freamon]

Hmmm. So I could have saved you some time if I'd read dmesg when the mount failed, but it only occurred to me when I saw mksquashfs running from the command line. The version from slax is old now, and produces files with a 3.1 filesystem. This isn't supported on the Pandora, which only supports version 4.0 (this also seems to be the case with a lot of modern Linux distros).


So I downloaded a new version of mksquashfs from here. It doesn't work with your GUI because -nolzma is no longer required, so I used it from the command line. First I tried

mksquashfs c:\temp\getiplayer c:\temp\getiplayer-1.pnd -nopad -no-recovery

as this is how pnd_make.sh invokes it. However, when I tried it out, the pnd mounted successfully but ran into a 'permission denied' when trying to change to the directory it had just mounted. I noticed that the Windows version creates a file with a uid of 400 and a gid of 401, whereas if I created a pnd on my Pandora it would have a uid of 1000 and gid of 1001, so I tried to replicate that behaviour with

mksquashfs c:\temp\getiplayer c:\temp\getiplayer-1.pnd -nopad -no-recovery -force-uid 1000 -force-gid 1001

and YAY!!!! That worked.

 

[Speaker Ender]

Hmmm. So I could have saved you some time if I'd read dmesg when the mount failed, but it only occurred to me when I saw mksquashfs running from the command line. The version from slax is old now, and produces files with a 3.1 filesystem. This isn't supported on the Pandora, which only supports version 4.0 (this also seems to be the case with a lot of modern Linux distros).


So I downloaded a new version of mksquashfs from here. It doesn't work with your GUI because -nolzma is no longer required, so I used it from the command line. First I tried

mksquashfs c:\temp\getiplayer c:\temp\getiplayer-1.pnd -nopad -no-recovery

as this is how pnd_make.sh invokes it. However, when I tried it out, the pnd mounted successfully but ran into a 'permission denied' when trying to change to the directory it had just mounted. I noticed that the Windows version creates a file with a uid of 400 and a gid of 401, whereas if I created a pnd on my Pandora it would have a uid of 1000 and gid of 1001, so I tried to replicate that behaviour with

mksquashfs c:\temp\getiplayer c:\temp\getiplayer-1.pnd -nopad -no-recovery -force-uid 1000 -force-gid 1001

and YAY!!!! That worked.

Okay, I'll see what I can do with this new version, and best way to get it onto a device. Someone should update the wiki to reference a different download of mksquashfs rather than slax.

 

[Speaker Ender]

Okay, I'll see what I can do with this new version, and best way to get it onto a device. Someone should update the wiki to reference a different download of mksquashfs rather than slax.

New version: v1.0.1.2


Included Self Extracting copy of mksquashfs/cygwin1.dll (Version 4.0 of mksquashfs.exe)


Removed -nolzma switch (not required with version 4.0 of mksquashfs)


Added -no-recovery -force-uid 1000 -force-gid 1001 switches to mksquashfs.exe

 

[Nintendo]

 

[llcameron]

I'll still setting permission denied messages when it tries to run the startup script.

 

[foxblock]

The problem most likely is because Windows does not know most of the UNIX file attributes and uses a different system.


Actually NTFS and newer version of Windows are compatible in some way, but accessing and modifying that information is from code is pretty complicated, see here: http://msdn.microsoft.com/en-us/library/aa379283%28v=vs.85%29.aspx


If you open the PND file created with the tool in 7-zip you will see that only root has rwx access to the files, but not the user (and it will be run on a user-level on the Pandora).


The only way I have found around this in PNDTools is calling chmod 755 (from the cygwin package) on the files, which will set the access rights accordingly.

 

[Speaker Ender]

The problem most likely is because Windows does not know most of the UNIX file attributes and uses a different system.


Actually NTFS and newer version of Windows are compatible in some way, but accessing and modifying that information is from code is pretty complicated, see here: http://msdn.microsoft.com/en-us/library/aa379283%28v=vs.85%29.aspx


If you open the PND file created with the tool in 7-zip you will see that only root has rwx access to the files, but not the user (and it will be run on a user-level on the Pandora).


The only way I have found around this in PNDTools is calling chmod 755 (from the cygwin package) on the files, which will set the access rights accordingly.

Are you talking about running chmod via cygwin on all the extracted files prior to opening? It would be nice if calcs could be used to set all these permissions.


I'm wondering why it worked for freamon. I was under the impression when forcing the uid/gid to 1000/1001 force it for the first user created on the pandora.


Also, it looks like others have had similar problems with mksquashfs, I found one user that compiled it and forced the mode to 755 (but I can't figure out how looking at their source), and the binary is for an older version than 4.0