Release PNDTools

[foxblock]

So some people suggested it should be easier to browse and create PNDs as there currently is no way to properly create them on Windows (without fighting with cmd.exe) and you can only use 7-zip to view, but not edit, them (though there is nothing wrong with).

Because of that I present
PNDTools (in the lack of a better name)
which is a simple and easy-to-understand GUI for opening, browsing, editing and creating PND files (as well as PXML files in the future).

What does it do?

• It can open a PND and display its contents in a tree structure
• You can then extract data from the PND or add new stuff
• It extracts the PXML and icon data
• You can create a new PND from scratch or modify an existing one and save the result again as a PND
• It has options
• GUI for viewing and editing the PXML file
• Partial PXML validation
• PXML Creation wizard

What does it currently not do? (aka things on the todo-list)

• Full PXML file validation check

What will it (probably) never do?

• Have a browsing interface as good as 7-zip's and the same functionality

This is a beta release, so a lot of bugs are to be expected and the program has a few known "quirks":

• Currently it calls chmod 755 on all the PND data to ensure all necessary files are given the "executable" flag, this works, but obviously is not the "nice" way to do it
• You will need to place it somewhere it has direct write access or execute the Program as administrator
• It needs half of the cygwin dlls and the cygwin chmod because Windows does not know the "executable" file-flag (well it does, but modifying is very complicated and calling chmod was easier to do for now)
• Only for Windows atm!
  
• Drag&Drop is always recursive

Download it here (Windows only) (mirror)
To "install" just extract the contents of the archive anywhere (the program will have write access, see above).
Current version: 0.7.3 built 13.03.2015
Source: Public GIT --- ZIP (master branch, automatically generated by github)

License: GPLv3
Beta: Version with xz compression (updated squashtools)

Changelog:

Spoiler

Version 0.7.3 - built 13.03.2015
* Fixed some bugs in the PXML created by the PXMLCreator
* Fixed errors when loading GIF images
* Changed PXMLCreator to follow the PXML specification more closely in terms of optional tags (gives warnings instead of errors)
* Changed PXMLCreator behaviour (does not close automatically after creating the PXML)
* Added icon tag to default PXML in the advanced editor (and mentioning that it's optional)
* Improved "Open containing folder" context menu action
* Improved placeholder graphics in PXMLCreator
* Added some licenses

Version 0.7.1 - built 03.06.2014
* Fixed file dialogues saving files without extension
* Fixed stuff being dropped on files being added to the "root" of the PND and not causing file conflict dialogues
* Added a few popular licenses to the PXMLCreator
* Minor improvements and fixes

Version 0.7.0 - built 23.04.2013
* Added 7zip tool to load PND using the ISO filesystem
* Added code to extract non-PNG icons
* Updated VirtualTreeview component to version 5.1.2

Version 0.6.2 - built 10.04.2013
* Added dialog for file conflicts in the PND tree
* Improved UI behaviour in terms of error display and logging
* Fixed several UI bugs
* Updated PXML schema and documentation

Version 0.5.0 - built 26.01.2013
* Added PXMLCreator - a simplified GUI for PXML creation
* Improved help and error text
* Fixed .bat files not selectable in options menu
* Fixed error in PXML schema file

Version 0.3.8 - built 22.01.2012
* PXMLtools branch
* Added PXML creator (basic graphical, node-based XML editor with built-in help and partial validation)
* Added functionality to automatically open files passed by command-line or drag&drop (meaning you can now set PNDTools as default program to open PND files)
* Added batch files for squashtools and chmod and functionality to use those instead of direct exe calls (see included readme.txt for more info)
* Added Lazarus compatibility fixes
* Fixed several bugs, which have accumulated over the time (like big icons not being attached correctly)
* Fixed exe-calls failing on Windows XP (and lower?)

Version 0.2.0 - built 05.06.2011
* Added "smart" file adding, detecting PND, PXML and icon files and asking for an appropriate action
* Added options menu, options include
* Toggle "smart" adding
* Toggle file specific icons
* Toggle binary file size units
* Paths to the tools
* Tool parameters
* Fixed buttons disappearing on Vista and Win7 when ALT-key is pressed
* Fixed tab order

Version 0.1.3 - built 01.06.2011
* Fixed icon being appended before PXML

Version 0.1.2 - built 30.05.2011
* Fixed path bug (PNDs could not be loaded or saved from a path other than one next to the program)

Version 0.1.1 - built 29.05.2011
* Fixed error messing up the extracted PXML file

Media:
Main window:

PXML editor:

Easy PXML Creator:


How it works:
When creating a PND no actual data is handled until you press the "Create PND" button, after which it will copy all data to a folder called "temp" and then call chmod and mksquashfs on that.
When opening a PND it will extract (unsquashfs) the contents to a folder called "temp2" and adds that folder to the tree-view, also it extracts the PXML and icon to a folder called "meta".
If you want to access the data contained in the PND you can simply copy it from those folders.
The folders are located next to the program exe and will be wiped before each respective operation.

Please try it, but be careful when doing so - I am not to be held responsible when data is lost or damaged, make back-ups.

Any bug report, feature request, layout request, general comment or anything else is greatly appreciated
Also the program currently uses the default Delphi-icon, which looks pretty shitty, a proper replacement icon would be great

foxblock out

 

[Tam Toucan]

Just gave this a quick go. Regarding copying the files to the same directory, it's not just a directory seperator problem is it? The GUI shows '/' for the command filename, but if I got to the terminal and run the same unsquashfs command as the GUI, but using '\' then it works. Just a thought.


Other than that looks good. Nice one!

 

[foxblock]

Just gave this a quick go. Regarding copying the files to the same directory, it's not just a directory seperator problem is it? The GUI shows '/' for the command filename, but if I got to the terminal and run the same unsquashfs command as the GUI, but using '\' then it works. Just a thought.


Other than that looks good. Nice one!

When using backslashes '\' in the parameters (filenames, pathes) passed to mksquashfs or unsquashfs it reports a warning regarding the use of DOS paths instead of POSIX paths, which is why I convert them to forward slashes and also try to pass the path as a relative one.


I actually just changed that today, before I just passed the DOS paths and it also worked, but a lot of time the squashFS tools would simply not do anything, which did not happen after the change so I thought it was related (it might be a combination of ShellExecute and the called program, though)


Not completely sure about that, though, but now you know my reasoning


Also thanks


Edit: also please send me the lines in the log about "Calling program" and the actual path to the files if you encounter an error. It might help to find and fix the error (or work around it).

 

[Tam Toucan]

Calling program: D:\Tools\PNDTools\tools\unsquashfs.exe -f -d "temp2" "../../Tam/Down/cdevtools.pnd"

Adding files to tree, this might take a while...

No files have been added, this most likely is due to an error while extracting the PND

The command terminal only flashes up for a second so I can't see what it reports. I do get a temp2 and a meta directory created in PNDTools, but that's all.



If I cd to PNDTools and run the above I get

Could not open ../../Tam/Down/cdevtools.pnd, because No such file or directory

but using '\' works, that's why I wondered about the slashes.

 

[foxblock]

Calling program: D:\Tools\PNDTools\tools\unsquashfs.exe -f -d "temp2" "../../Tam/Down/cdevtools.pnd"

Adding files to tree, this might take a while...

No files have been added, this most likely is due to an error while extracting the PND

The command terminal only flashes up for a second so I can't see what it reports. I do get a temp2 and a meta directory created in PNDTools, but that's all.



If I cd to PNDTools and run the above I get

Could not open ../../Tam/Down/cdevtools.pnd, because No such file or directory

but using '\' works, that's why I wondered about the slashes.

Yep, that is to be expected as the Windows cmd.exe works with DOS paths while squashfs works with POSIX paths (one of the pitfalls to get this to work on Windows).


Also I don't know of a way to keep the terminal open, any help on that is appreciated.


Last but not least, the source-code can be found here: https://github.com/foxblock/PNDTools (packed ZIP)


I never used GIT before and it was a pain to get to work on Windows and I am pretty sure I am doing stuff wrong as I am used to centralized code control like SVN, so please point any error I make out.


Feel free to branch or submit patches, but please notify me before you start to make any major changes so we don't work on the same thing.

 

[PokeParadox]

So far so good! Looking forward to the day when you have integrated PXML editor.


Also rather than making all files 755, you could have a separate button that says "Add Binary" and only set to exec when that button is used... just a suggestion.

 

[foxblock]

So far so good! Looking forward to the day when you have integrated PXML editor.


Also rather than making all files 755, you could have a separate button that says "Add Binary" and only set to exec when that button is used... just a suggestion.

Yep, I could do that, but all folders need 755, too and some PNDs use scripts and a binary - also this would not work as nice with the drag&drop.


But I could let the user "mark" files as binary (so he has to mark the scripts and binaries) and set the rest to 644 or set all files to 755 if none was marked.


This however would have to be done manually for every loaded PND, too as there is no way to automatically detect these attributes as they get lost on unpacking.


For the moment I think nothing is wrong with all files having 755, in fact if you look at some of the PNDs you will see that they even have 777 for all files (UAE4ALL and RGB for example).


But I will add that to the todo-list, thanks for the suggestion.


EDIT: Okay just fixed a pretty stupid bug in the PXML extraction code, fixed version has been uploaded (Version 0.1.1), Source will be updated later.

 

[EvilDragon]

Also I don't know of a way to keep the terminal open, any help on that is appreciated.

On Windows? Well, instead of running the command directly, you could use a batch script that runs the command and a "pause" after that.

 

[Linopoly]

Also I don't know of a way to keep the terminal open, any help on that is appreciated.

You can use the /k flag to cmd.exe to get it to run a command and then stay open. You might have to play with quotes and backslashes to escape the string you're giving it, but I know it's possible to do whatever you want via that route, I was using it to run scheduled tasks and leave them open.

 

[thatgui]

As often these days - it seems that I'm doing something wrong. I tried to open three different pnds but all I get is this:

Welcome to the PNDTools


To get started just drop some files in the top-most area or load an existing PND


Deleting old temporary files


Looking for PXML data...


PXML data found, writing to file C:\Users\___\Desktop\PNDTools\PNDTools\meta\PXML.xml


Icon data found, writing to file C:\Users\___\Desktop\PNDTools\PNDTools\meta\icon.png


PND metadata successfully extracted


Extracting PND... this might take a while


Calling program: C:\Users\___\Desktop\PNDTools\PNDTools\tools\unsquashfs.exe -f -d "temp2" "../../lighttpd.pnd"


Adding files to tree, this might take a while...


No files have been added, this most likely is due to an error while extracting the PND


Deleting old temporary files


Looking for PXML data...


PXML data found, writing to file C:\Users\___\Desktop\PNDTools\PNDTools\meta\PXML.xml


Icon data found, writing to file C:\Users\___\Desktop\PNDTools\PNDTools\meta\icon.png


PND metadata successfully extracted


Extracting PND... this might take a while


Calling program: C:\Users\___\Desktop\PNDTools\PNDTools\tools\unsquashfs.exe -f -d "temp2" "../../linapple_1.1b-v1.3.5.0.pnd"


Adding files to tree, this might take a while...


No files have been added, this most likely is due to an error while extracting the PND


Deleting old temporary files


Looking for PXML data...


PXML data found, writing to file C:\Users\___\Desktop\PNDTools\PNDTools\meta\PXML.xml


Icon data found, writing to file C:\Users\___\Desktop\PNDTools\PNDTools\meta\icon.png


PND metadata successfully extracted


Extracting PND... this might take a while


Calling program: C:\Users\___\Desktop\PNDTools\PNDTools\tools\unsquashfs.exe -f -d "temp2" "../../scummvm-1.2.1-1.pnd"


Adding files to tree, this might take a while...


No files have been added, this most likely is due to an error while extracting the PND

After the "Select File" Dialog closes I always have to acknowledge the execution for unsquasfs - maybe I'm not fast enough doing it?


Remark: the login name in paths is replaced by three '_'

 

[foxblock]

On Windows? Well, instead of running the command directly, you could use a batch script that runs the command and a "pause" after that.

Good idea, I certainly could do that.

You can use the /k flag to cmd.exe to get it to run a command and then stay open. You might have to play with quotes and backslashes to escape the string you're giving it, but I know it's possible to do whatever you want via that route, I was using it to run scheduled tasks and leave them open.

I will look into that, I don't run cmd.exe directly, but run squashfs.exe which is a command line too, maybe it simply works when I call it on that, too.

As often these days - it seems that I'm doing something wrong. I tried to open three different pnds but all I get is this:

Well as I wrote in my initial posts, squashFS seems to have problems with certain relative paths, try copying the PND next to the program (or vice versa) or into a sub-folder of the program directly (except for temp or temp2, which will get deleted).


Sorry about that, I am looking into fixing this problem, but for now it seems like I have to pass a relative path as the absolute path sometimes screws up, but the relative one does so, too apparently.


foxblock out

 

[foxblock]

I think I fixed the mentioned path bug, please download the new version and try it (0.1.2) - load a PND from anywhere


Basically I just try to convert the paths to those Cygwin recommends when called with DOS paths and it seems to work (obviously, maybe).


PS: Oh and the "/k" flag sadly only work when calling the command line directly, now I either can do that or work with batch scripts, dunno what is better.

 

[EvilDragon]

PS: Oh and the "/k" flag sadly only work when calling the command line directly, now I either can do that or work with batch scripts, dunno what is better.

How are you calling squashfs.exe?


AFAIR, you can run executable through cmd.exe as well (cmd.exe <executable name>)


Then it should also listen to parameters.

 

[foxblock]

PS: Oh and the "/k" flag sadly only work when calling the command line directly, now I either can do that or work with batch scripts, dunno what is better.

How are you calling squashfs.exe?


AFAIR, you can run executable through cmd.exe as well (cmd.exe <executable name>)


Then it should also listen to parameters.

I am calling unsquashfs.exe directly and yes I know I can run it through cmd.exe, that is why I was wondering whether that or batch files are the way to go

 

[milkshake]

pandora version coming at all?

 

[Granitehead]

pandora version coming at all?

And Linux version?


Don't exactly need it, but would be nice.

It does not work with PNDs using the ISO filesystem (and I don't see a good reason for why it needs to do so as SquashFS is the superior FS imo)

I agree that SquashFS is better than ISO, but maybe you should allow to at least /load/ Iso though.

 

[foxblock]

pandora version coming at all?

Not likely, Dunny mentioned it is a pain to get Lazerus working on ARM, so I don't know if it is worth the effort (you always have a Win/Lin PC around and don't necessarily need to pack everything on the Pandora - topor might disagree ).

pandora version coming at all?

And Linux version?


Don't exactly need it, but would be nice.

It does not work with PNDs using the ISO filesystem (and I don't see a good reason for why it needs to do so as SquashFS is the superior FS imo)

I agree that SquashFS is better than ISO, but maybe you should allow to at least /load/ Iso though.

Linux version is much more likely, I am going to work on that, but want to finish proper PXML support first.


Parts of the code are heavily using the Windows API and would need to be rewritten, but I have put all those parts in separate units, so that process should be fairly straight-forward.


Any help of this is greatly appreciated and I invite anyone to give it a try, the code is public (see first post for the links) and if you manage to get it working on Linux, give me a shout and I will pull your changes.


Though that reminds me I have to include a proper licensing scheme, probably MPL or GPL.


ISO loading is on my todo-list, too.


foxblock out

 

[Granitehead]

Parts of the code are heavily using the Windows API and would need to be rewritten, but I have put all those parts in separate units, so that process should be fairly straight-forward.


Any help of this is greatly appreciated and I invite anyone to give it a try, the code is public (see first post for the links) and if you manage to get it working on Linux, give me a shout and I will pull your changes.

Is this Pascal? I believe to remember it had those strange comments in {}. ^^

 

[foxblock]

Is this Pascal? I believe to remember it had those strange comments in {}. ^^

Yes and the comments are not the strangest things about this language

 

[thatgui]

Works like charm know


Thanks a lot, intresting to see how different pnds from different authors a structured. Good to know for my project