Can't download "Unknown Horizons" via pnd Manager (general filesize problem ?)

[Asmo]

I just tried Widelands again, with the latest version of qupzilla, saving to a very fast USB stick for good measure.

The download ended with the saved pnd only being 103.6mb (instead of 170+), no error messages or anything.

 

[thatgui]

Downloaded with Qupzilla on the Pandora, worked fine as well.

Same issue with the repo though. How does PNDManager download?

I still think the time it takes to download a file is essential => download rate. I could download the widelands pnd without a problem with Firefox on my Pandora, but I can't download it with PNDManager. According to PNDManager I got about 70 kB/s download speed, Firefox tells me it is downloading with about 650 kB/s.That is why I asked for a tool that lets me control the download rate (in another thread), I was hoping to reproduce PNDManagers speed on a normal PC

 

[_jr_]

Ok, one last test run with rate limited wget on a small pnd:


$ wget --server-response --ignore-length --content-disposition --limit-rate=10k http://repo.openpandora.org/includes/download.php?app=scummvm.djwillis.0001
--2013-06-24 22:43:36-- http://repo.openpandora.org/includes/download.php?app=scummvm.djwillis.0001
Resolving repo.openpandora.org... 5.9.50.149
Connecting to repo.openpandora.org|5.9.50.149|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Date: Mon, 24 Jun 2013 20:45:22 GMT
Server: Apache
X-Powered-By: PHP/5.4.4-14+deb7u2
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Accept-Ranges: bytes
Content-Description: File Transfer;
Content-Disposition: attachment; filename="scummvm-op.pnd";
Content-Transfer-Encoding: binary;
Cache-Control: must-revalidate, post-check=0, pre-check=0
Set-Cookie: PHPSESSID=maae3q8jsg0a1t72hpcjk3chn2; path=/
Content-Range: bytes 0-10254803/10254804
Content-Length: 10254804
Connection: close
Content-Type: application/x-pnd;
Length: 10254804 (9.8M) [application/x-pnd]
--2013-06-24 22:43:37-- http://repo.openpandora.org/includes/download.php?app=scummvm.djwillis.0001
Connecting to repo.openpandora.org|5.9.50.149|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Date: Mon, 24 Jun 2013 20:45:23 GMT
Server: Apache
X-Powered-By: PHP/5.4.4-14+deb7u2
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Accept-Ranges: bytes
Content-Description: File Transfer;
Content-Disposition: attachment; filename="scummvm-op.pnd";
Content-Transfer-Encoding: binary;
Cache-Control: must-revalidate, post-check=0, pre-check=0
Content-Range: bytes 0-10254803/10254804
Content-Length: 10254804
Connection: close
Content-Type: application/x-pnd;
Length: 10254804 (9.8M) [application/x-pnd]
Saving to: `scummvm-op.pnd'

46% [========================================> ] 4,727,354 10.0K/s in 7m 42s

2013-06-24 22:51:19 (10.0 KB/s) - Connection closed at byte 4727354. Retrying.

--2013-06-24 22:51:20-- (try: 2) http://repo.openpandora.org/includes/download.php?app=scummvm.djwillis.0001
Connecting to repo.openpandora.org|5.9.50.149|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 206 Partial Content
Date: Mon, 24 Jun 2013 20:53:05 GMT
Server: Apache
X-Powered-By: PHP/5.4.4-14+deb7u2
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Accept-Ranges: bytes
Content-Description: File Transfer;
Content-Disposition: attachment; filename="scummvm-op.pnd";
Content-Transfer-Encoding: binary;
Cache-Control: must-revalidate, post-check=0, pre-check=0
Content-Range: bytes 4727354-10254803/10254804
Content-Length: 5527450
Connection: close
Content-Type: application/x-pnd;
Length: 10254804 (9.8M), 5527450 (5.3M) remaining [application/x-pnd]
Saving to: `scummvm-op.pnd'

94% [+++++++++++++++++++++++++++++++++++++++++===========================================> ] 9,667,726 10.0K/s in 8m 2s

2013-06-24 22:59:23 (10.0 KB/s) - Connection closed at byte 9667726. Retrying.

--2013-06-24 22:59:25-- (try: 3) http://repo.openpandora.org/includes/download.php?app=scummvm.djwillis.0001
Connecting to repo.openpandora.org|5.9.50.149|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 206 Partial Content
Date: Mon, 24 Jun 2013 21:01:10 GMT
Server: Apache
X-Powered-By: PHP/5.4.4-14+deb7u2
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Accept-Ranges: bytes
Content-Description: File Transfer;
Content-Disposition: attachment; filename="scummvm-op.pnd";
Content-Transfer-Encoding: binary;
Cache-Control: must-revalidate, post-check=0, pre-check=0
Content-Range: bytes 9667726-10254803/10254804
Content-Length: 587078
Connection: close
Content-Type: application/x-pnd;
Length: 10254804 (9.8M), 587078 (573K) remaining [application/x-pnd]
Saving to: `scummvm-op.pnd'

100%[+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=====>] 10,254,804 10.1K/s in 57s

2013-06-24 23:00:22 (10.0 KB/s) - `scummvm-op.pnd' saved [10254804/10254804]

The first error message (like the one quoted above) ends exactly at offset 4,727,353. A second message (identical except for a different status code: 206 partial content) ending at offset 9,667,725. That means that the error appears at the end of each downloaded chunk, sending the error message instead of actual content. I don't know enough HTTP to determine whether this appending of a clear text server error mesage to actual preceding content is allowed by the spec, bt it confuses the hell out of at least wget and PNDManager.

It appears that the current firefox pnd does somehow detect the beginning of the error message, since it is not saved to the partial pnd. This might be more correct, but doesn't help, since it does not retry after these errors and therefore doesn't download the complete file (and does not indicate to the user that there was an error).


But this doesn't help much with finding the culprit, since the error clearly happens on the server.


Searching the error messages shos that repo.openpandora.org is not the only one suffering from this phenomenon. It might be related to broken .htaccess files, but I'm not sure these are relevant here (and I already mentioned that I don't know anything about Apache/PHP


I hope this help to reproduce and fix the problem.