python 2.6 Folder on SD almost empty?

[Maschinenherz]

Hmmm... could you check on the python 2.6 folder in your pandy? Perhaps i have a problem because I startet with the Os on SD that had 2.7. My thought is, other people with no problem startet with an OS prior to 1.62 and the upgrade didnt remove the folder. I alsways had a fresh image, and on nand is 1.55.


I dont understand the pnd system/linux file system enough as I am a beginner on linux and have no experience programming, but it seems like a rather strange situation to me

 

[levi]

Okay, I've had a poke at the code now.  I've not been able to reproduce the bug on my laptop running python 2.7, but I've traced through the stack trace in the actual code at least:


File "/mnt/utmp/keepnote/keepnote/extensions/new_file/__init__.py", line 168, in save_default_example_files
shutil.copy(os.path.join(base, fn), os.path.join(data_dir, fn))
File "/usr/lib/python2.7/shutil.py", line 118, in copy
copymode(src, dst)
File "/usr/lib/python2.7/shutil.py", line 91, in copymode
os.chmod(dst, mode)
OSError: [Errno 1] Operation not permitted: '/mnt/utmp/keepnote/keepnote/extensions_data/new_file/plain_text.txt'


shutil.copy takes src and dst, and passes them to copymode as is, so the dst in the os.chmod command is the same as the second argument to shutil.copy in new_file.save_default_example_files.

On my system however, the dst evaluates to ~/.config/keepnote/extensions_data/new_file/plain_text.txt, which is writeable and under my ownership, so the os.chmod doesn't fail as it appears to do in your stacktrace.

I've not yet been able to dig through how data_dir gets set in save_default_example_files fully.  If you've an environment variable 'XDG_CONFIG_HOME' set it seems to use that instead of the user home, appended with '.config/keepnote', but that doesn't match what it's set to according to your stacktrace, and nothing in the PND sets that variable according to a quick grep of its contents.

So, I've not completely got my head around what's happening on your system, so I've added a few debug statements to give me more of an idea what it's doing.   Unzip this into your pandora/appdata/keepnote folder, and run my hacked keepnote again, capturing the pndrun.out logs afterwards.  If I've set the paths up properly it'll replace one of the source files and add a little debugging to the log which should be enlightening.

debug.zip

 

[Maschinenherz]

Ok. I will try it tonight after work. Your help is really awesome.

 

[Maschinenherz]

=======================================================================================
PND             : /media/64GB/pandora/menu/KeepNote.pnd
PND_FSTYPE      : Squashfs
APPDATADIR      : /media/64GB/pandora/appdata/keepnote
APPDD_FSTYPE    : vfat
PND_CPUSPEED    : <unset>
EXENAME         : keepnote
ARGUMENTS       : <unset>
=======================================================================================
[ START ]--- Mount the PND ----------
Mounting : mount -t squashfs -o ro "/dev/loop1" "/mnt/utmp/keepnote"
Mounting the Union FS : mount -t aufs -o exec,noplink,dirs="/media/64GB/pandora/appdata/keepnote=rw+nolwh":"/mnt/pnd/keepnote=rr" none "/mnt/utmp/keepnote"
[sUCCESS]--- Mount the PND ----------
[ START ]--- Starting the application ( keepnote  ) ----------

==============================================
KeepNote 0.7.5: Fri May  8 23:27:09 2015
enabling extension 'backup_tar'
enabling extension 'new_file'

Traceback (most recent call last):
  File "/mnt/utmp/keepnote/keepnote/__init__.py", line 1184, in init_extensions
    enabled = ext.enable(True)
  File "/mnt/utmp/keepnote/keepnote/extension.py", line 234, in enable
    self.enabled.notify(enable)
  File "/mnt/utmp/keepnote/keepnote/listening.py", line 60, in notify
    listener(*args, **kargs)
  File "/mnt/utmp/keepnote/keepnote/extensions/new_file/__init__.py", line 85, in on_enabled
    self.load_config()
  File "/mnt/utmp/keepnote/keepnote/extensions/new_file/__init__.py", line 103, in load_config
    self.save_default_example_files()
  File "/mnt/utmp/keepnote/keepnote/extensions/new_file/__init__.py", line 168, in save_default_example_files
    shutil.copy(os.path.join(base, fn), os.path.join(data_dir, fn))
  File "/usr/lib/python2.7/shutil.py", line 118, in copy
    copymode(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 91, in copymode
    os.chmod(dst, mode)
OSError: [Errno 1] Operation not permitted: '/mnt/utmp/keepnote/keepnote/extensions_data/new_file/plain_text.txt'
enabling extension 'editor_insert_date'
enabling extension 'command_basics'
enabling extension 'export_html'
enabling extension 'python_prompt'
enabling extension 'keepnote'
/usr/pandora/scripts/pnd_run.sh: line 522:  2682 Segmentation fault      "./$EXENAME" $ARGUMENTS "$@"
[ FAILED]--- Starting the application ( keepnote  ) ----------
[ START ]--- Restoring the frame buffer status ----------
[sUCCESS]--- Restoring the frame buffer status ----------
[ START ]--- uMount the PND ----------
[ START ]--- Waiting the Union to be available ----------
[sUCCESS]--- Waiting the Union to be available ----------
auplinklink.c:223: AUFS_CTL_PLINK_MAINT: Inappropriate ioctl for device
rmdir: failed to remove `/mnt/utmp/keepnote': Device or resource busy
[ START ]--- Waiting the PND mount dir to be free ----------
[sUCCESS]--- Waiting the PND mount dir to be free ----------
cleanup done
[sUCCESS]--- uMount the PND ----------
=======================================================================================
Return code is : 3
 

 

[Maschinenherz]

so. I extracted the file. and startet your hacked PND via script. that is the actual pnd.out

 

[levi]

Did you extract the file, or the whole path structure in that zip?  It should have produced a 'keepnote' folder inside appdata/keepnote containing the other folders and file to make it work.

Either way, it doesn't seem to have worked.  If you can confirm you see the keepnote folder and not an __init__.py file inside pandora/appdata/keepnote, I've made the zip wrongly (which wouldn't surprise me, as I've not done many hacks like that before).

 

[Maschinenherz]

sorry. my bad. I was one folder to far -.-

new out follows

 

[Maschinenherz]

the file path is: appdata/keepnote/keepnote/extensions/new_file/_init_.py

there are also .pyo and pyc

the new .out:

=======================================================================================
PND             : /media/64GB/pandora/menu/KeepNote.pnd
PND_FSTYPE      : Squashfs
APPDATADIR      : /media/64GB/pandora/appdata/keepnote
APPDD_FSTYPE    : vfat
PND_CPUSPEED    : <unset>
EXENAME         : keepnote
ARGUMENTS       : <unset>
=======================================================================================
[ START ]--- Mount the PND ----------
Mounting : mount -t squashfs -o ro "/dev/loop1" "/mnt/utmp/keepnote"
Mounting the Union FS : mount -t aufs -o exec,noplink,dirs="/media/64GB/pandora/appdata/keepnote=rw+nolwh":"/mnt/pnd/keepnote=rr" none "/mnt/utmp/keepnote"
[sUCCESS]--- Mount the PND ----------
[ START ]--- Starting the application ( keepnote  ) ----------

==============================================
KeepNote 0.7.5: Sat May  9 00:14:57 2015
enabling extension 'backup_tar'
enabling extension 'new_file'

Traceback (most recent call last):
  File "/mnt/utmp/keepnote/keepnote/__init__.py", line 1184, in init_extensions
    enabled = ext.enable(True)
  File "/mnt/utmp/keepnote/keepnote/extension.py", line 234, in enable
    self.enabled.notify(enable)
  File "/mnt/utmp/keepnote/keepnote/listening.py", line 60, in notify
    listener(*args, **kargs)
  File "/mnt/utmp/keepnote/keepnote/extensions/new_file/__init__.py", line 85, in on_enabled
    self.load_config()
  File "/mnt/utmp/keepnote/keepnote/extensions/new_file/__init__.py", line 104, in load_config
    self.save_default_example_files()
  File "/mnt/utmp/keepnote/keepnote/extensions/new_file/__init__.py", line 170, in save_default_example_files
    shutil.copy(os.path.join(base, fn), os.path.join(data_dir, fn))
  File "/usr/lib/python2.7/shutil.py", line 118, in copy
    copymode(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 91, in copymode
    os.chmod(dst, mode)
OSError: [Errno 1] Operation not permitted: '/mnt/utmp/keepnote/keepnote/extensions_data/new_file/plain_text.txt'
enabling extension 'editor_insert_date'
enabling extension 'command_basics'
enabling extension 'export_html'
enabling extension 'python_prompt'
enabling extension 'keepnote'
/usr/pandora/scripts/pnd_run.sh: line 522:  3580 Segmentation fault      "./$EXENAME" $ARGUMENTS "$@"
[ FAILED]--- Starting the application ( keepnote  ) ----------
[ START ]--- Restoring the frame buffer status ----------
[sUCCESS]--- Restoring the frame buffer status ----------
[ START ]--- uMount the PND ----------
[ START ]--- Waiting the Union to be available ----------
[sUCCESS]--- Waiting the Union to be available ----------
auplinklink.c:223: AUFS_CTL_PLINK_MAINT: Inappropriate ioctl for device
rmdir: failed to remove `/mnt/utmp/keepnote': Device or resource busy
[ START ]--- Waiting the PND mount dir to be free ----------
[sUCCESS]--- Waiting the PND mount dir to be free ----------
cleanup done
[sUCCESS]--- uMount the PND ----------
=======================================================================================
Return code is : 3
 

 

[Maschinenherz]

Errm... it looks exactly the same... but I am sure to have started your hacked pnd because the original is in another castle... and your file is in the proper location... Oo

 

[levi]

Hmm, that path looks right, but as you say there's nothing happening.

Instead of this palaver, I've built another hacked pnd.   Hopefully that'll produce something worth looking at

It's probably worth clearing out your appdata/keepnote folder entirely at this point.

keepnote_hack2.pnd

 

[Maschinenherz]

and here it is. it seems like every creation of a file through the pnd is forbidden. even the error log Oo

=======================================================================================
PND             : /media/64GB/temp/temp/keepnote_hack2.pnd
PND_FSTYPE      : Squashfs
APPDATADIR      : /media/64GB/pandora/appdata/keepnote
APPDD_FSTYPE    : vfat
PND_CPUSPEED    : <unset>
EXENAME         : keepnote
ARGUMENTS       : <unset>
=======================================================================================
[ START ]--- Mount the PND ----------
Mounting : mount -t squashfs -o ro "/dev/loop1" "/mnt/utmp/keepnote"
Mounting the Union FS : mount -t aufs -o exec,noplink,dirs="/media/64GB/pandora/appdata/keepnote=rw+nolwh":"/mnt/pnd/keepnote=rr" none "/mnt/utmp/keepnote"
[sUCCESS]--- Mount the PND ----------
[ START ]--- Starting the application ( keepnote  ) ----------
Traceback (most recent call last):
  File "./keepnote", line 428, in <module>
    main(sys.argv)
  File "./keepnote", line 397, in main
    start_error_log(options.show_errors)
  File "./keepnote", line 153, in start_error_log
    keepnote.init_error_log()
  File "/mnt/utmp/keepnote/keepnote/__init__.py", line 448, in init_error_log
    open(error_log, "a").close()
IOError: [Errno 13] Permission denied: u'/mnt/utmp/keepnote/keepnote/error-log.txt'
[sUCCESS]--- Starting the application ( keepnote  ) ----------
[ START ]--- Restoring the frame buffer status ----------
[sUCCESS]--- Restoring the frame buffer status ----------
[ START ]--- uMount the PND ----------
[ START ]--- Waiting the Union to be available ----------
[sUCCESS]--- Waiting the Union to be available ----------
auplinklink.c:223: AUFS_CTL_PLINK_MAINT: Inappropriate ioctl for device
rmdir: failed to remove `/mnt/utmp/keepnote': Device or resource busy
[ START ]--- Waiting the PND mount dir to be free ----------
[sUCCESS]--- Waiting the PND mount dir to be free ----------
cleanup done
[sUCCESS]--- uMount the PND ----------
=======================================================================================
Return code is : 7
 

 

[levi]

Hmm yes.  What are the permissions on your appdata folder and the keepnote folder inside it?  Is it on an ext-formatted partition which can have detailed permission settings, or a FAT/NTFS partition that just has a read-only flag?  Either way, check the appropriate permissions.

If you identify a problem there, try the original (non-hack) keepnote pnd again - that could have been the problem all along!

 

[Maschinenherz]

its a fat 32, and the permissions of the folder are as follows:

owner:root

group:root

everything is on read and write

the folder was empty after the hack 2 pnd, and just one folder and 4 .pyc files after the original.

when i start the old OS from nand, then the folder is completely populated....

 

[Maschinenherz]

just for the lolz.... I started the old OS from nand.... i started the original keepnote... this is the pnd.out from the RUNNING keepnote.... notice something?

=======================================================================================
PND             : /media/64GB/pandora/apps/KeepNote.pnd
PND_FSTYPE      : Squashfs
APPDATADIR      : /media/64GB/pandora/appdata/keepnote
APPDD_FSTYPE    : vfat
PND_CPUSPEED    : <unset>
EXENAME         : keepnote
ARGUMENTS       : <unset>
=======================================================================================
[ START ]--- Mount the PND ----------
Mounting : mount -t squashfs -o ro "/dev/loop0" "/mnt/utmp/keepnote"
Mounting the Union FS : mount -t aufs -o exec,noplink,dirs="/media/64GB/pandora/appdata/keepnote=rw+nolwh":"/mnt/pnd/keepnote=rr" none "/mnt/utmp/keepnote"
[sUCCESS]--- Mount the PND ----------
[ START ]--- Starting the application ( keepnote  ) ----------

==============================================
KeepNote 0.7.5: Fri May  8 23:19:55 2015
enabling extension 'backup_tar'
enabling extension 'new_file'

Traceback (most recent call last):
  File "/mnt/utmp/keepnote/keepnote/__init__.py", line 1184, in init_extensions
    enabled = ext.enable(True)
  File "/mnt/utmp/keepnote/keepnote/extension.py", line 234, in enable
    self.enabled.notify(enable)
  File "/mnt/utmp/keepnote/keepnote/listening.py", line 60, in notify
    listener(*args, **kargs)
  File "/mnt/utmp/keepnote/keepnote/extensions/new_file/__init__.py", line 85, in on_enabled
    self.load_config()
  File "/mnt/utmp/keepnote/keepnote/extensions/new_file/__init__.py", line 103, in load_config
    self.save_default_example_files()
  File "/mnt/utmp/keepnote/keepnote/extensions/new_file/__init__.py", line 168, in save_default_example_files
    shutil.copy(os.path.join(base, fn), os.path.join(data_dir, fn))
  File "/usr/lib/python2.6/shutil.py", line 89, in copy
    copymode(src, dst)
  File "/usr/lib/python2.6/shutil.py", line 66, in copymode
    os.chmod(dst, mode)
OSError: [Errno 1] Operation not permitted: '/mnt/utmp/keepnote/keepnote/extensions_data/new_file/plain_text.txt'
enabling extension 'editor_insert_date'
enabling extension 'command_basics'
enabling extension 'export_html'
enabling extension 'python_prompt'
enabling extension 'keepnote'
 

 

[levi]

You mean that when you run it with the old OS it lanches correctly, and fills the appdata folder with files?  That's the expected behaviour, so it's good it still does that I guess.

Now that it's generated the error log file, perhaps it will get to the point where I've inserted the logging messages if you run my hack2 version from the latest OS.  That probably won't tell us much if there are basic problems creating files, but it's the only thing to do that immediately comes to mind.

If it doesn't produce the DEBUG: messages, it might be worth checking that it hasn't created a config.xml file in appdata/keepnote/keepnote/extensions_data/new_file/.  The code checks for that file before even trying to generate the files it was complaining about before (where my debug statements are located), so by deleting that file you can make it regenerate them.

The permissions on my appdata folders are the same by the way; root:root, read-write for all.  It doesn't look like a straightforward permissions problem then, but at the same time it's looking less like a straight python bug.  I don't really know what it looks like at the moment.

If anyone else running the latest OS fancies testing out my hacked pnds (either version really) and reporting whether it launches or not, it would help rule out a number of theories floating around.  My Pandora's out of action currently, so I can't test this in situ unfortunately.

Edit: So that error about plain_text.txt is a red herring after all?  Still doesn't explain the error.log error, and it doesn't explain the segfault that is actually killing the process.

 

[Maschinenherz]

Not a red herring, but a moby dick,it seems to me.

I try to summarize.

OS: 1.55 NAND, Keepnote on fat32 SD in right Slot: running, error in pnd.out without meaning

OS: 1.62 SD in Left Slot, Keepnote on fat 32 SD in right Slot, not running, importerror libpython 2.6

OS: 1.71 SD left Slot, Keepnote hacked version 1 and 2 get segfault.

OS: 1.71 SD left Slot, Keepnote original, python 2.7, importerror libpython2.6.

OS: 1.71 SD left Slot, Keepnote original, python 2.6 copied from NAND to SD, importerror libpython2.6.

OS: 1.55 NAND, Keepnote hacked version 2 on fat32 SD in right Slot: NOT running

first: jepp, i managed to copy the 2.6 to the new OS. (lets try /media/ROOT/ instead of /media/root, should we? Stupid thing I am -.-)

second: OS 1.71:

fresh deleted appdata/keepnote is empty after starting hackedpnd 2

fresh deleted appdata/ keepnote has one program specific folder and  4 python files after start of original PND

OS 1.55: appdata/keepnote is fully populated

3: differences I found:

as the hacked pnd wouldnt show up in the menu, i made a script by copiing the command from the desktop file and changed the path, because "run pnd application" in menu didnt open either. script was in home folder.

I did the same on NAND, and the script did nothing. LED was not even blinking.

on NAND i just have the "filesystem", on SD ROOT and BOOT

I have no effing idea.... what the hell is the difference between NAND and SD other than the OS version?

 

[levi]

There shouldn't really be any difference between an the same OS on an SD card and on NAND.  The only thing I can think of is that it's not guaranteed than an SD card is mounted when booting from NAND, but as soon as you browse to it, it should mount it.

Note PNDs aren't designed to be run from directories other than those under /pandora.  They're also not really designed to live on the same partition as your OS, but the OS does include a /pandora directory, and I've never tested what happens if you populate it.

I think if you put it in /pandora/menu my hacked versions ought to turn up, but I've not been able to test that.  If you don't want to do that you can run the runpnd script (or something called something like that) which ships with every pandora OS since the year dot.  I suspect that's the important line in the desktop files too, but I've never actually looked in those to my best recollection.

I'm not surprised my hack doesn't work on 1.55 - that _ctypes.so file I eliminated was clearly doing something on the python2.6 shipped in that OS, but has been less helpful since python got updated to 2.7 in later OSes.  I suppose it might have been fixing this segfault we've been beset by since running without that on py 2.7, but without having source for it it doesn't really help much.  I know it works on Python 2.7 on x86 without hacking _ctypes.so, but perhaps there's a bug in the ARM builds.

There's still the mystery of why it failed to write the error-log.txt when you'd cleared out appdata/keepnote.  I note looking at that stack trace, it's the only file path which seems to be in unicode, but it's a 7-bit clean string so that really shouldn't cause any problems.

If nobody else can try the hacked keepnote pnd on 1.71 and see if they also get the segfault, I'll have to assume that ctypes hack is necessary, but have no means of reproducing it that I've noticed.  You mentioned other python pnds that fail to work on python2.7 - it may be worth looking at those and seeing if any of those can be fixed.

 

[Maschinenherz]

Nope. Wasnt in the menu the official way. Therefore the copy and paste script.


some of the seem to have been corrupted at download, because i used the website and a shitty connection. Reinstalling worked.


I have to somehow get to know wich pnds also run off python to test. the mist not working pnds i never looked at the pnd.out and removed them. With the gourmet recipe pnd i am not sure. But maybe there is the problem on the fat partition. But the maintainer mentioned that problem as fixed.


Its a long road to a solution. Also i have to try to use my brain more often. Sometimes i need captain obvious to help me think, as seen with my copy problems -.-

 

[Maschinenherz]

Ok. I had much work the last weeks. I havent found any app that I know is running on python and wont work. I think, that I have to stay with removing the sd card and booting from nand. I hope, that it wont be too bad for my sd slot -.-

 

[FBnil]

Question: As someone else pointed out a page ago... can you get a new SD card, create empty pandora directories, and only copy some .pnd's and not anything residual (so an empty ./pandora/appdata/) and if that works, slowly start copying back things...