Repository

[Cassiel]

Guys, I think there is a problem with the repository.


When downloading some packages (I can give you a fairly chunky list of examples if needed) via different means, you can end up with different sized/different CRC32'd packages. This seems to specificity affect Firefox, though could potentially be other browsers/clients (have not extensively tested).


Using Atari800 as an example:


936743bb from http://www.scrameta.net/a800.html with FireFox


936743bb from http://www.scrameta.net/a800.html with downThemAll! (download manager) or Internet Explorer


944b245d from openpandora.org with FireFox


936743bb from openpandora.org with downThemAll! (download manager) or Internet Explorer


This would appear tp be a recent issue however, since I downloaded a bunch of packages (with Firefox) 2 weeks ago but now if I redownload (with Firefox) I get 'different' packages, both size wise (a few bytes) and obviously checksum wise.


Both the 'old' CRC packages and the 'new' CRC packages both validate perfectly (as in decompressed/tested using 7Zip).


This issue was brought to my attention by a third party, so it's not something specific to me (though I am experiencing exactly the same incorrect behaviour).


Has anything been changed recently with the repo? Potentially I guess it could also be a recent change with Firefox that's not playing nice with the repository anymore, but I don't recall upgrading in the recent past.....

 

[ibisum]

I've also noticed this problem with the repo recently - the downloads sometimes stop at odd places. I've been using mostly PNDManager to keep my stuff in sync with the repo, but also use firefox with the repo directly, and it 'appears' that downloads just pause or quit, sometimes.

 

[Linux-SWAT]

It's not the first time someone reported strange things with downloads.

 

[Mr Rob]

I've noticed it, too, though I've always assumed it was due to large files (wesnoth, for example).


It would be nice to have MD5 hashes available in the repo for each package. Maybe not under the main interface, but under a "more information" button or whatnot.

 

[milkshake]

sounds strange main be this only happens when the server is underload? I.E. When eds resource hungry back up scripts are running?

 

[Pickle]

I mentioned behavior like this to ED about the SZ beta 3 download, it would reconnect many times and I had to start it up a couple times. wget can start from the last byte received so eventually i received it. I suspect the same thing is happening with the repo.

 

[EvilDragon]

Hmm... this is very hard to figure out, since it never happened to me.


I can download from home, work, etc., all downloads are intact, so it's hard to find the issue :/


Someone got any idea?


It doesn't really make sense...

 

[mindlord]

I'd like to add that this behavior does not seem to be isolated to just the repo. Downloading Sebt's toolchain took several attempts to actually get from dozens of different computers and browsers.

 

[EvilDragon]

Well, they're all on my server, so it has to do something with that... I just don't know what could cause this :/

 

[gruso]

Yeah, I get this occasionally. I just wait until later and it generally comes good.

 

[Linux-SWAT]

The first thing that comes to me is a bad ram.


Mixed memory ?

 

[EvilDragon]

Doubt it's bad RAM, as this would lead to a VERY crashy server...

 

[Linux-SWAT]

I've seen cases when strange things happened after a certain amount of ram was used, and when swapping the dimm, it was impossible to boot.

 

[EvilDragon]

I've seen cases when strange things happened after a certain amount of ram was used, and when swapping the dimm, it was impossible to boot.

Well... we got 24GB RAM in a server with a loooong uptime, and it's really used a lot


Every Linux server I had so far never had an uptime longer than 2 days maximum when the RAM was bad (unless you never did anything on that server).

 

[Prometheus]

Just a thought: I wonder if this phenomenon coincides with the times when the forums (both here and at GP32X) become unresponsive and keep throwing server configuration errors?

 

[Linux-SWAT]

Was just a first idea. Anyway, keep in mind that if your ram is mixed, it increases this possibility.


I try to remember all the times someone reported a server error, but there's nothing clear. Removing (or adding) the www worked once.


Check ifconfig if collisions or errors are reported ?

 

[EvilDragon]

It's a professional 19" rack server, no mixed RAM


ifconfig has 11TiB TX without a single collision, so that's fine.


It shouldn't be caused by the backups it does, but you never know... I'll disable them and will just do simply rsync updates.

 

[sebt3]

It shouldn't be caused by the backups it does, but you never know... I'll disable them and will just do simply rsync updates.

Well bitbake isnt especially cool on CPU and memory... (having 6+ g++ instances doing large IO / CPU computation is kind of demanding)


I for one would move the bitbake building environnement to a non-webost server and spread the backup across the clock

 

[DaMummy]

you people went 17 posts deep without realizing the real cause.....aTc

 

[Cassiel]

Dodgy RAM.... WTF?


Don't want to sound like an arsehole, but I'd recommend going back to my original (detailed) post.


Looks like it's a specific (and repeatable!) issue depending on what client/browser being used. Looks like a Firefox specific issue, which points to a configuration issue. Also, this is a RECENT issue that didn't exist before (I'm talking no more than two weeks maybe).


Downloads (especially of large files) crapping out before successfully completing is a COMPLETELY separate issue, and one that's unfortunately always existed, for whatever reason.