I Finally Finished My Building Calibre

[chuckr]

Finally, I have a Calibre (a recent one, ver 0.7.20), and it seems to work fine and dandy. I'd kinda like to give it away, but I am facing some problems with this, and I'm inviting comment.

1st, if a recent Calibre is already done, then stop reading right now, ok?
2nd, Calibre is dependent on a hugely big list of other pieces of software, so distributing something like an opkg for it would be a pretty sick joke. The largest I did was a recent qt4, and probably the second was python itself (when I began the porting, Pandora's python was busted ... I reported it several times, but when nobody responded, I just built my own version). AND I really do mean a huge list, even the dependencies of the dependencies is a big list.
3rd, I built everything I put together to go into /usr/local. I saw some complaints averring that getting /usr/local to work would be scarily hard, but that's uninformed garbage, it's pretty simple, and doesn't interfere with anything you installed with opkg (into /usr). Can't make an opkg of my stuff because I was never successful in getting opkg to install into /usr/local. The documentation says it does, but I sure couldn't get it to work that way.
So, the only way I can see to give calibre away is to be willing to copy my whole work flash to any flash which gets mailed to me. My 32G working flash (obviously, this ain't never going to install into onboard flash, fellas) has nearly 7G on it (which includes gcc). The only thing you'd need to do is boot it up. My flash is 32G big, but I can't see any reason it wouldn't install into, say, an 8G flash. So if you want this, email me at chuckr@telenix.org and I'll give you a mailing address. I'm also going to ask that you include a suitable, self addressed and stamped mailer so I can return it to you. I will just grin if you send me a bad flash. I can't get you a mailer, I'm disabled, and have trouble getting to the USPO.

Oh, I just realized something ... Calibre is a *mainly* python app, so if you still have the pandora-standard, busted version of python installed, it's probably not going to work for you. You can tell by starting up python and entering "import platform", and if that fails, you have the busted python on your machine. What's busted is mostly that python-code lib in /usr/lib/python2.6, it's missing a bunch of stuff. Who knows, maybe they finally fixed it? Anyhow, if you destroy your python, then the version I include will do you just fine.

I tried to post this from my Pandora, but although I can log into gp32x from there, and it says "successful", it never seems to actually work. Lucky I have other machines.

Nice thing about my copying the flash is, you get that really long list of required dependencies with no extra work, and it's not going to bother your previously installed stuff, as long as you aren't already using /usr/local (then it probably still will work fine, but I make no promises on what I haven't tested). I can say that this recent version of Calibre has a really slick GUI, and autoconverts a nice long list of book formats (nearly everything that's not a secretly kept format, but there ARE a lot of those ... go read the calibre site, if it worries you).

If you have better notions of how I oughta handle this, I'm happy to listen, but I'm dusabled, so if you ask for something that will take a lot more work, it's gonna be at my work rate, which is pretty slow. Luckily, I really enjoy porting (for the fantastic Pandora gaming platform, I nearly never run games).

 

[hdonk]

The way I did things for the firefox pnd:
1. On a bootable SD card, build firefox and get it installed and running in a separate directory (/pnd_dev/firefox in my case)
2. Booting from NAND, which I keep as a base install, with just the hotfixes on it, copy across the /pnd_dev/firefox to somewhere
3. Find any absolute softlinks (find . -type l) & fix
4. Try running firefox and see what libraries are missing. Copy those missing libraries & their softlinks in to the lib directory for firefox
5. Build pnd of this working directory!
6. ...
7. Profit

EDIT: This was for the first pass - after that I knew which libs need copying.

 

[chuckr]

The way I did things for the firefox pnd:
1. On a bootable SD card, build firefox and get it installed and running in a separate directory (/pnd_dev/firefox in my case)
2. Booting from NAND, which I keep as a base install, with just the hotfixes on it, copy across the /pnd_dev/firefox to somewhere
3. Find any absolute softlinks (find . -type l) & fix
4. Try running firefox and see what libraries are missing. Copy those missing libraries & their softlinks in to the lib directory for firefox
5. Build pnd of this working directory!
6. ...
7. Profit

EDIT: This was for the first pass - after that I knew which libs need copying.

Well, I have two problems you didn't have ... 1, Calibre has an enormous list of dependencies, huge things like qt4 and python, and you need a full X11 development environment (all the libs and all the include files). If I were to even try to build a separate package, it wouldn't work for anyone. Maybe, using a complete environment like either FreeBSD's ports or Gentoo's portage, where you build from binaries and have complete control, then (even though it's be really hard) I *could* do that, but surely not with just the opkg things. Number two is that I've built everything to run underneath /usr/local, and (at least for me) opkg firmly refuses to install to anywhere excepting under /usr directly. That means I can't even install to test here, because I WON"T pollute my original installation dirs that Pandora populates.

I really do think that the only good way to distribute this thing is as a 6.8G tarball (that'd be before I compressed it, you'd save some in bzip compression).. If anyone wants to host such a thing, I'd be happy to package it up and put it into place. But it'd be a tarball, not a opkg. Actually, be much less likely to interfere with folks preinstalled work.

Oh, I forgot one point: I needed to add swap here to get it to work, it's pretty hefty. I have a 32G flash, so I made partition mmcblk0p2 be a 1g swap partition. Loading new books is a bit slow, but reading itself is fast enough. Very nice GUI.

PS. Your last point is profit. Not sure how to read that, but I am NOT selling this, I'm giving it away (you worry about the licenses of all the stuff I built all on your own). I'm no lawyer, and don't care to become one.

 

[hdonk]

The way I did things for the firefox pnd:
1. On a bootable SD card, build firefox and get it installed and running in a separate directory (/pnd_dev/firefox in my case)
2. Booting from NAND, which I keep as a base install, with just the hotfixes on it, copy across the /pnd_dev/firefox to somewhere
3. Find any absolute softlinks (find . -type l) & fix
4. Try running firefox and see what libraries are missing. Copy those missing libraries & their softlinks in to the lib directory for firefox
5. Build pnd of this working directory!
6. ...
7. Profit

EDIT: This was for the first pass - after that I knew which libs need copying.

Well, I have two problems you didn't have ... 1, Calibre has an enormous list of dependencies, huge things like qt4 and python, and you need a full X11 development environment (all the libs and all the include files). If I were to even try to build a separate package, it wouldn't work for anyone. Maybe, using a complete environment like either FreeBSD's ports or Gentoo's portage, where you build from binaries and have complete control, then (even though it's be really hard) I *could* do that, but surely not with just the opkg things. Number two is that I've built everything to run underneath /usr/local, and (at least for me) opkg firmly refuses to install to anywhere excepting under /usr directly. That means I can't even install to test here, because I WON"T pollute my original installation dirs that Pandora populates.

I really do think that the only good way to distribute this thing is as a 6.8G tarball (that'd be before I compressed it, you'd save some in bzip compression).. If anyone wants to host such a thing, I'd be happy to package it up and put it into place. But it'd be a tarball, not a opkg. Actually, be much less likely to interfere with folks preinstalled work.

Oh, I forgot one point: I needed to add swap here to get it to work, it's pretty hefty. I have a 32G flash, so I made partition mmcblk0p2 be a 1g swap partition. Loading new books is a bit slow, but reading itself is fast enough. Very nice GUI.

PS. Your last point is profit. Not sure how to read that, but I am NOT selling this, I'm giving it away (you worry about the licenses of all the stuff I built all on your own). I'm no lawyer, and don't care to become one.

Points 6 & 7 are a looooong running internet meme I'm afraid B)
As for the other bits, it's a slow job, but it's usually possible to override all the default settings in most apps. Use LD_LIBRARY_PATH to change where it looks for libraries. Find the configuration files, or lines of code, that tells it where it should look for info. Change those. I'm not saying it's easy, and from what you're saying Calibre is full of lots of dependencies, but that's what's needed to convert it in to a self contained pnd. If you don't do it, I'm sure someone else will get around to it eventually. I would but my next project is my last project - more work on Cube.

 

[Chaser]

Despite where your project ends up, i just wanted to say thanks for putting this together. I was unsure what Calibre was but have had a look at the windows version and it would be a mighty fine app if it hits the Pandora. Cheers for all the effort involved so far!

 

[chuckr]

Despite where your project ends up, i just wanted to say thanks for putting this together. I was unsure what Calibre was but have had a look at the windows version and it would be a mighty fine app if it hits the Pandora. Cheers for all the effort involved so far!

Thanks. I just tried updating it to the latest version (from 0.7.20 to 0.7.28) and it went very smoothly. Note please that this ISN'T the version that's been available (and broken, from what I can see) as an opkg. Maybe that package wasn't so heavily salted with dependencies, though. As far as that comment about changes required to run a /usr/local, I know how to do it seeing as I HAVE done it, and believe me, it's a lot simpler than you think it is. Only two changes needed in your environment: fix your startup bash script to have the PATH know about your new bin directories, and in /etc/ld.so.conf, list all directories that you want to have ldconfig run on, so your shared libs get noitced. When you build things, IF THEY WILL LIVE in the new /usr/local, then pass a --prefix=/usr/local to the configuration script. That's it, that's all. Nothing else, and no little config scripts for you to have to hunt down and fix. Anyone tells me that they are going to try it, I'll even post both the parts of my ~/.bashrc and the /etc/ld.so.conf that I use (actually, I decided to use a separate dir for the qt4, and also I built my own X11 stuff, because I didn't want the X11 include stuff pulluting my /usr). The idea is for me to have a REALLY EASY time identifying things which are part of the Pandora distribution.

About Calibre, you might know that some things work slowly, like starting up Calibre takes a few minutes, and adding a long list of books can take 30 minutes to do. Reading goes as fast as either the arrow keys on the left side of the pageup/pagedn keys on the right side can be pressed. That's the part that would really bother me, if it went too slowly, but that part's quick enough. Also, if you run this, note that I use a 1G added swap, and not doing that is a bad idea. I think 1G is overkill, but since my flash is 32G, I have space to waste. Lastly, forget any idea you might have had about getting away without booting from flash, it won't even come close to working, my present environment (with my books) is well under 8G. I know a 16G flash will work, but I don't know if a 8G will squeak through or not.

If anyone can arrange me server space, I'll compress it and send it for you, wherever you say. Be kind to the Calibre people, they obviously put a lot of work into this, and it shows.

 

[chuckr]

The way I did things for the firefox pnd:
1. On a bootable SD card, build firefox and get it installed and running in a separate directory (/pnd_dev/firefox in my case)
2. Booting from NAND, which I keep as a base install, with just the hotfixes on it, copy across the /pnd_dev/firefox to somewhere
3. Find any absolute softlinks (find . -type l) & fix
4. Try running firefox and see what libraries are missing. Copy those missing libraries & their softlinks in to the lib directory for firefox
5. Build pnd of this working directory!
6. ...
7. Profit

EDIT: This was for the first pass - after that I knew which libs need copying.

Well, I have two problems you didn't have ... 1, Calibre has an enormous list of dependencies, huge things like qt4 and python, and you need a full X11 development environment (all the libs and all the include files). If I were to even try to build a separate package, it wouldn't work for anyone. Maybe, using a complete environment like either FreeBSD's ports or Gentoo's portage, where you build from binaries and have complete control, then (even though it's be really hard) I *could* do that, but surely not with just the opkg things. Number two is that I've built everything to run underneath /usr/local, and (at least for me) opkg firmly refuses to install to anywhere excepting under /usr directly. That means I can't even install to test here, because I WON"T pollute my original installation dirs that Pandora populates.

I really do think that the only good way to distribute this thing is as a 6.8G tarball (that'd be before I compressed it, you'd save some in bzip compression).. If anyone wants to host such a thing, I'd be happy to package it up and put it into place. But it'd be a tarball, not a opkg. Actually, be much less likely to interfere with folks preinstalled work.

Oh, I forgot one point: I needed to add swap here to get it to work, it's pretty hefty. I have a 32G flash, so I made partition mmcblk0p2 be a 1g swap partition. Loading new books is a bit slow, but reading itself is fast enough. Very nice GUI.

PS. Your last point is profit. Not sure how to read that, but I am NOT selling this, I'm giving it away (you worry about the licenses of all the stuff I built all on your own). I'm no lawyer, and don't care to become one.

Points 6 & 7 are a looooong running internet meme I'm afraid B)
As for the other bits, it's a slow job, but it's usually possible to override all the default settings in most apps. Use LD_LIBRARY_PATH to change where it looks for libraries. Find the configuration files, or lines of code, that tells it where it should look for info. Change those. I'm not saying it's easy, and from what you're saying Calibre is full of lots of dependencies, but that's what's needed to convert it in to a self contained pnd. If you don't do it, I'm sure someone else will get around to it eventually. I would but my next project is my last project - more work on Cube.

I think you missed my point ... I wasn't asking how to do it vis-a-vis Linux, that part I have working totally fine and stable. It's always more simple that most people think it is. The part I regret not getting to work is getting opkg to operate. Opkg's docs say it WLL allow you to set a DESTDIR, but in all of my experimentation, opkg stubbornly refuses to use /usr/local. That's a problem of a sort, yes, but to be really honest, I dislike opkg as a application installer, even if DESTDIR could be made to work. It's not only broken with regards to installation directories, but it also has no support for building from source. My personal favorites are either Gentoo's portage, or FreeBSD's ports. I'm pretty sure I could (with a bit of work, I admit, but I like that kind of work) make FreeBSD ports work with our Linjux environment, and doing it with Gentoo's portage ought to be fairly easy also (actually, easier). BUT doing such a thing requires somebody on the core OP team agree with me, and until or unless that happens, I'll just not make packages of my new Calibre. Man, it's a huge matrix of various & sundry interdependencies.

 

[Chaser]

Just wanted to apologise for not getting back to your detailed reply Chuckr.

Sadly as a linux n00b, your work is like most stuff around here, way above my head but I read and put into practice that which I can and I've had a rich Pandora experience thus far as a result and learnt enough to know the Pandora is a dream to use and linux is nowt to be afraid of for general use of this machine.

Calibre would be a great app in a simply installable form, either PND or a few steps required and it sounds like you've got there but the OP team need to do something that helps it work in your favour. I hope you get there, as I will support the calibre team/op team and yourself for sure, as no doubt countless people to whom this app would be an incentive for, in future will.