Can somebody (I) host a repo mirror?


milkshake said:
That's a cap on pnd manager I believe not the repo bzar could answer that.
Click to expand...
There's a cap resulting from CPU limiting I set for the downloads. It's a tradeoff between download speed and UI performance.


The responsible line is here. Making that sleep smaller raises the download speed while degrading UI experience. On desktop you can put that sleep to 1ms or smaller and have full DL speeds and full UI speed, but on pandora I had to make a compromise there :(
 
Wouldn't that work better if downloads were sequential (not parallel) ?
 
Neelix said:
@:  You could try changing ISP...    I don't have any such issues most of the time.  (and my traceroute doesn't go through singtel)
Click to expand...
That's the stupidest thing I've ever heard... "Screw you, I got mine". try changing ISP because it's that simple.. I might as well move country too huh?

If you're not technically able to help build support for the global members of our community - Please stay out of this. there are way too many selfish nerds with no empathy in this thread already. OOOOH It works for me so that means everything is fine I SAID GOOD DAY SIR...  ;)
 
Last edited by a moderator:
Thanks to you guys for compressing the masterlist for a start.

I don't really see a problem with maintaining a replicated mirror because the repo is actually pretty small and there are only a few changes every day.
We don't need to over-engineer anything but there are easier ways to deal with this.

The idea of setting up a bunch of anycast repos is overkill when the ability to change repos in a tool would do fine.
 
Last edited by a moderator:
Theres several goals..

i) do we want full repo clones; if the main repo goes down (meteorite!), then do you want to just change to another repo website and good to go (sans backups and commercial pnds)?

ii) do we want just a file backup, and then the main repo could if it wishes send out a link to the alternate repos url (ie: for faster download, no change to normal website performance of course)

iii) do we just want backups; ie: a rsync/wget of the repo each night, so that the files are somewhere else; list woudl be a long directory listing only, so it works for people in a pinch and trivial to set up

.. do you want PNDManager and so forth to be able to query the backup repos, and talk to them as if they were the real deal?

.. or are the mirrors just for faster download, or file availability in case Germany is swallowed by Cthulhu?

Need to define the purpose otherwise the conversation will fly all over the place perhaps..

jeff
 
Could someone tweak PNDmanager so that the package listing and downloading went like this:

The config file contains a list of known servers in order of user preference (probably repo.openpandora.org first)

On startup, download a package list from all known servers.

Use this download to evaluate the connection to each server (which is fastest?)

Look up the package to download to see which servers have a copy

(depending on user preference either:

--- download from server furthest up list

--- download from server with fastest connection

(obviously do not choose a sever which does not send a package list, because it is either overloaded or broken)

That allows for:

  1. mirrors for speed
  2. mirrors in case ED's server explodes
  3. devs hosting their own content
 
Last edited by a moderator:
FWIW it's possible for servers to be configured to offer gzipped versions of any files you ask for, and it's up to your http client to negotiate to get the gzipped files, which are dynamically zipped up on the server and unzipped as part of the http client. Not sure, but I think it should be possible to offer bzip2ed files the same way. If the server were configured to do that (and your web client is intelligent enough) you shouldn't see much difference between the bzipped 2 master list and just pulling down the full master list.
 
ParadisoShlee said:
Neelix said:
@:  You could try changing ISP...    I don't have any such issues most of the time.  (and my traceroute doesn't go through singtel)
Click to expand...
That's the stupidest thing I've ever heard... "Screw you, I got mine". try changing ISP because it's that simple.. I might as well move country too huh?
Click to expand...
That's not what I said,  nor what I meant.    You were directing ED to take things up with his ISP when the reality is that the issue is closer to your ISP's end of the connection.  You could take things up with your ISP,  but frankly, what speaks loudest to ISPs in this country these days is churn. 

- Neelix
 
Squashfs is already compressed. Compressing on the fly for transfer will accomplish nothing but CPU waste, unfortunately.
 
WizardStan: I think Levi was talking about the masterlist, not the PNDs.

Binky: it's probably better (and easier to implement) to just use a sensible default repo server based on your continent, which can be overridden manually if needed.

skeezix: I think your third option would be good. Basically a read-only copy of the entire thing, excluding the commercial PNDs and maybe the archived old versions. To leave comments, rate stuff, upload stuff, etc. you still have to go to the main server, but to just download some PND or its meta-data (screenshots etc), you can use the mirror.
 
milkshake said:
As far as im aware PNDManager supports the use of multiple repo's already assuming they follow the REPO json spec
Click to expand...
The package management backend library of the pndmanagement Panorama plugin PNDManager uses (called libpndman) has that capability. It's not implemented in pndmanagement-plugin nor PNDManager because at the time there were no other repos nor any in sight.

Binky: That would require changes in all libpndman, qtpndman (Qt wrapper for libpndman), pndmanagement-plugin and PNDManager. Anyone is welcome to do that of course, but it's quite a bit more work than it sounds ;)
 
Last edited by a moderator:
Neelix said:
ParadisoShlee said:
Neelix said:
@:  You could try changing ISP...    I don't have any such issues most of the time.  (and my traceroute doesn't go through singtel)
Click to expand...
That's the stupidest thing I've ever heard... "Screw you, I got mine". try changing ISP because it's that simple.. I might as well move country too huh?
Click to expand...
That's not what I said,  nor what I meant.    You were directing ED to take things up with his ISP when the reality is that the issue is closer to your ISP's end of the connection.  You could take things up with your ISP,  but frankly, what speaks loudest to ISPs in this country these days is churn. 

- Neelix
Click to expand...
The_Internet_Is_A_Series_Of_Tubes.jpg


My day job is a sysmin at the datacentre of the most awesome amount of bandwidth (it's suck there too) and I'm very much aware that the internet is a series of tubes.. we need more dump trucks.

I think my original intention was pretty much in the topic.. Can we please move towards multiple repos because it's the right thing to do.
 
Last edited by a moderator:
I have a Server with gigabit up/down(yes, real gigabit Internet) oft you want to set up a mirror PM me;)
 
Last edited by a moderator:
You must log in or register to reply here.

Similar threads

skeezix
Minimenu Tab Plugin Spec .. Discuss Before I Code :)
Replies
9
Views
4K
skeezix
skeezix
Askarus
Maintained list of all remaining issues. Call to help.
3 4 5
Replies
96
Views
12K
pyrat
P
Back
Top