Input/output error when writing a big file onto USB external hard drive

[Kainekokun]

Hello,


my apologies is this has been addressed somewhere before and I could not find that.


So here's the problem: I've got an external drive (an SSD to be precise), formatted as FAT32 (I know, I know, but it's the only *easy* way to get it to work on all the systems I need it for). When I try to store a file of about 25 MB size from the Pandora (one of the SD cards) to the drive in the USB port, I get an "input/output error" and the writing fails, resulting in just an empty (zero bytes) file. I have tried the same operation on my Mac, transferring the exact same size onto the same external drive, and it worked fine. Then I tried the reverse, copying the file from the external drive onto one of the SD cards of the Pandora, and that worked fine, too. Smaller files are no problem, but once you hit a certain size (about 16MB), the input/output error crops up. This happens reliably, i.e. every time I try to write out to the USB drive.


I then tried re-formatting the SSD to NTFS, but I got the exact same problem then, only it also started getting weird, i.e. if my working directory was on the SSD and I attempted to copy, after the input/output error, ls would return an empty directory, and I'd have to switch directory to be able to see the contents of the SSD again.


I've gone and tried with a USB thumb drive (also FAT), and lo and behold, this time it worked! The file copied over. So I went and examined the output of the `usb-devices' utility, and the biggest differences I can see (other than the usual vendor and serial number strings, etc.) are as follows.


Thumb drive:


...


C: #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=100mA


...


E: Ad=02(out) Atr=02(Bulk) MxPS= 512*1 Ivl=31875us


SSD:


...


C: #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=2mA


...


E: Ad=02(out) Atr=02(Bulk) MxPS= 512*1 Ivl=0ms


All the other numbers were the same.


I can't find enough information about the meaning of this output, so I am posting here in the hopes that someone with more know-how could help. Could it be that the SSD's controller runs out of power when trying to write a lot of data at once? Does the pandora not provide the same amount of power over USB as a regular computer?


If anyone has any ideas on how to investigate further and/or solve this, I would really appreciate!


I am using a 512MB pandora (brand new, got it last week), and I am running the vanilla image it came with, all I changed other than the usual configuring of the system, was following the user manual's instructions (http://pandorawiki.o...from_an_SD_card) on how to copy the system to an SD card and run it from there instead of running it off the NAND (I went for the option where I copied the contents of the NAND onto the SD card). I also installed some apps, including the Community Codec Pack.

 

[Kainekokun]

UPDATE:


Since the USB enclosure for the SSD came with this feature, I thought I'd give it a try. The enclosure comes with an additional USB cable, which can be used to add more power input in case the attached SATA device needs it. I attached this cable to my Mac, so the SSD/USB controller were fed power from both the Pandora and the Mac, and this time the file transfer succeeded! So it seems definitely to be a power issue, but the big question now is: is there another solution? The whole idea was to use the SSD over USB with the Pandora, and I was under the impression that the Pandora should be able to provide just as much power over the USB 2.0 port as a normal laptop or PC, but in this particular case, it does not seem to be the case? Is there any setting that can be switched, to ensure that the Pandora provides enough power over the USB port? Just to re-iterate, when I connect the SSD to a USB port on my Mac, the file transfer works fine *without* needing the additional power input.


As always, any help and info will be much appreciated.

 

[patnik]

Perhaps it would help to get a faster,more direct response if you could post the name/model number of said SSD drive so that someone with similar experience could give better help.


You may have hit the nail on the head with the suspicion that the SSD`s hardware controller power requirement may be the culprit. But again, as of this post we don`t know the model # of your portable SSD drive so we cannot say for certain. It might be best to wait for another member reading this topic to offer some help or solution to your issue.


My question to you in the meantime:


1) You just received your Pandora recently. Exactly what firmware # is installed in it as of present?


2) Have you considered upgrading the firmware to the latest Super Zaxxon Final ( if it is not presently installed on your Pandora)?


3) Have you tried another USB cable to see if that is the reason? (for troubleshooting)


4) When attempting Pandora to SSD via USB, is USB-HOST Mode selected to "ON" when you attempt the transfer? Have you selected USB-HOST Mode "ON" as default in your Pandora settings?


5) What CPU speed is your Pandora set to when you try transferring/connecting to your SSD?


Just a few questions that popped in my head. Hopefully someone with a previous experience can shed some light/offer a solution. Or one of the knowledgeable gurus can guide us better.


...Patnik.

 

[Kainekokun]

Hi Patnik,


thank you very much for your reply.

Perhaps it would help to get a faster,more direct response if you could post the name/model number of said SSD drive so that someone with similar experience could give better help.

The SSD drive is a Samsung 830, the USB enclosure is a BIPRA BIP0901, the manual names the chipset as "JM20329", the usb-devices output identifies it as a "USB to ATA/ATAPI bridge" and names the manufacturer as "JMicron".

You may have hit the nail on the head with the suspicion that the SSD`s hardware controller power requirement may be the culprit. But again, as of this post we don`t know the model # of your portable SSD drive so we cannot say for certain. It might be best to wait for another member reading this topic to offer some help or solution to your issue.

I actually did another test this morning using an older external HDD, a Western Digital "MyPassport" 250GB HDD. The output of usb-devices was identical to the output I get for the SSD, but this time the file transfer worked just fine! This confuses me, because I would have thought that, if anything, the HDD would pull more power than an SSD, no? Does anybody know of a tool I could use to measure the amount of power being pulled over the Pandora's USB 2.0 port?

My question to you in the meantime:


1) You just received your Pandora recently. Exactly what firmware # is installed in it as of present?

Release 1.5 - Super Zaxxon

2) Have you considered upgrading the firmware to the latest Super Zaxxon Final ( if it is not presently installed on your Pandora)?

For as far as I can tell, 1.5 is the latest, right?

3) Have you tried another USB cable to see if that is the reason? (for troubleshooting)

Interesting proposition. Since the file transfer worked fine on my Mac, I would not expect the cable to be the issue, since it seems to work there, but I went and tried with two other cables I have lying around that fid the plug on the enclosure and which I know to work fine: both of them yielded the same "input/output error", so I am confident in excluding the cable as the issue.

4) When attempting Pandora to SSD via USB, is USB-HOST Mode selected to "ON" when you attempt the transfer? Have you selected USB-HOST Mode "ON" as default in your Pandora settings?

I did not consciously set that, but I went and double-checked in the startup settings, and I can confirm that USB-HOST Mode is NOT "ON" by default. I even went and re-booted my Pandora, just to make extra-sure, but still no joy.

5) What CPU speed is your Pandora set to when you try transferring/connecting to your SSD?

800 MHz, which, if I understand it correctly, should be the "factory" setting for the 512MB model, right? For the sake of completeness, when setting up my Pandora, I chose the profile "800 MHz, OPP5".

Just a few questions that popped in my head. Hopefully someone with a previous experience can shed some light/offer a solution. Or one of the knowledgeable gurus can guide us better.


...Patnik.

Thanks again for thinking along!

 

[kuru]

Standard setting should be 600MHz/OPP3, OPP5 is max overvolting as I understand it.

 

[Kainekokun]

Standard setting should be 600MHz/OPP3, OPP5 is max overvolting as I understand it.

The UI says that this profile is meant only for the "first wave" Pandora's ... or did I misunderstand that?

 

[patnik]

I just tested my Pandora with my 2TB external 3.5" SATA "USB Enclosured (Sabrient brand)" NTFS formatted backup drive as I was curious. Sure enough, when USB host Mode ON is selected, the drive + all files are fully accessible to my Pandora.When USB Host Mode is OFF, the Pandora doesn`t detect it at all. Then I tried with a generic USB SD reader "thumbdrive" and same results. Done at 352MHz, 600MHz and 800MHz speed options.


USB HOST Mode was greatly improved to my understanding by the latest ( or just prior to the latest ) Super Zaxxon firmware with the 3.2 kernel ( which one needs if one is to use Notaz`s Android Gingerbread application on their Pandora) and older kernels might produce some iffy experiences for a few Pandora units when choosing Host Mode ON. One of the knowledgeable techies (Notaz, WizardStan, Mcobit, Android, M.Weston himself, etc) could correct me as I am not 100% certain. In any event, since you said you were still using the vanilla firmware a la SD card install, could you tell us at your boot screen, what kernel is identified? Is it Kernel 3.2, or 2.6.x or something else? I currently have the prior to latest Super Zaxxon Final1.5, 3.2 Kernel flashed to NAND.


Also, in checking back the Wiki USB compatibility list, it seems the MyPassport models have good compatibility for the Pandora as well as a few other platter drives; little to no feedback on SSD compatibility, apart from your present topic at hand. So it seems that your particular enclosure, this BIPRA brand, requires a secondary power source,like a powered hub for example,so that the Pandora can then communicate and transfer files properly with little to no errors. Any chance you can get your hands on another brand of SSD-compatible USB enclosure by any chance? Like maybe a friend who can allow you to test theirs on your Pandora?


Anyone reading this topic, have you had success with SSD + enclosure, particularly 2.5" sized units in question. Please give feedback, recommendations and solutions for us here.


...Patnik.

 

[Kainekokun]

Thanks again for your reply, your input is very much appreciated.

I just tested my Pandora with my 2TB external 3.5" SATA "USB Enclosured (Sabrient brand)" NTFS formatted backup drive as I was curious. Sure enough, when USB host Mode ON is selected, the drive + all files are fully accessible to my Pandora.When USB Host Mode is OFF, the Pandora doesn`t detect it at all. Then I tried with a generic USB SD reader "thumbdrive" and same results. Done at 352MHz, 600MHz and 800MHz speed options.

I might have gotten the whole idea of the USB host mode wrong ... are you suggesting I should switch it ON? I thought it's only used when you want to connect your Pandora to another PC and have them access the Pandora like a USB device? Did I get that completely wrong?

USB HOST Mode was greatly improved to my understanding by the latest ( or just prior to the latest ) Super Zaxxon firmware with the 3.2 kernel ( which one needs if one is to use Notaz`s Android Gingerbread application on their Pandora) and older kernels might produce some iffy experiences for a few Pandora units when choosing Host Mode ON. One of the knowledgeable techies (Notaz, WizardStan, Mcobit, Android, M.Weston himself, etc) could correct me as I am not 100% certain. In any event, since you said you were still using the vanilla firmware a la SD card install, could you tell us at your boot screen, what kernel is identified? Is it Kernel 3.2, or 2.6.x or something else? I currently have the prior to latest Super Zaxxon Final1.5, 3.2 Kernel flashed to NAND.

The boot screen says: Kernel 2.6.27, Release 1.5 - Super Zaxxon. I am surprised you speak of a 3.2 kernel ... I was not aware there were images available with that kernel version. Naively, I thought my Pandora would be sporting the newest firmware, seeing as it's brand new. Would you recommend flashing with such an image? And if so, would that be the latest firmware update available on the main page?

Also, in checking back the Wiki USB compatibility list, it seems the MyPassport models have good compatibility for the Pandora as well as a few other platter drives; little to no feedback on SSD compatibility, apart from your present topic at hand. So it seems that your particular enclosure, this BIPRA brand, requires a secondary power source,like a powered hub for example,so that the Pandora can then communicate and transfer files properly with little to no errors. Any chance you can get your hands on another brand of SSD-compatible USB enclosure by any chance? Like maybe a friend who can allow you to test theirs on your Pandora?

I saw the MyPassport entry on the compatibility list as well. And indeed, I am starting to think it has to do with the enclosure. I've seen that most USB enclosures come with the option to plug them into two USB ports at the same time: one for data and power and the second for additional power, if the first connection does not provide enough to run the enclosed drive. The particular thing about the BIPRA model I have, is that the second connection plugs into a special charger inlet on the PCB, while many other enclosures use a special Y-split USB cable that connects both connections into the same mini-B (data + power) plug on the PCB. This might very well be part of the problem. It could be that, if the host system does not provide enough power over the mini-B connector, the BIPRA tries to pull the additional over the other inlet, while an enclosure with a Y-cable (and this is, again, pure guessing, as I have too little experience with USB hardware design) would still try to pull that over the same mini-B connector. I'll see if I can get my hands on a different enclosure, to see if that changes anything.


I've also done some further reading, and I found that the data one gets back from the usb-devices command is -- if I understand it correctly -- the configuration as advertised by the connected devices. If that is so, then it would appear that the BIPRA is advertising it's power needs wrong? I have checked what my Mac has to say about the BIPRA when connected, and interestingly, while it agreed with the Pandora about the 2mA power requirement, it also mentioned another number, namely the "available power", at 500mA ... which leads me to guess that (for whatever reason) the Mac believes the device says it only needs 2mA, but allows it to take up to 500mA, so when the device pulls more, it does not result in errors -- is anybody reading this who could confirm this to be possible?


I've also been looking into possibilities to "correct" the power requirements in the configuration of a USB device ... ideally there'd be a bit of code I could put together to just override whatever the connected device is telling the system. I found some pointers in an issue some people were having while trying to charge their Blackberry device over USB on a linux machine. If I get anywhere following that lead, I'll report it here, but I'm afraid the solution in that case was pushing the device to change "mode" and thus advertise different needs to the USB controller. I'm not even sure one *can* tell the system to offer a different power output to a connected USB device in software. Again, if anyone with more knowledge could shed some light on this, I'd be very grateful.


Thanks again to everybody for reading and helping along!

 

[C0smikD3bris]

SSD manufacturers periodically release firmware upgrades for their drives as memory controller errors etc come to light out in the wild.


I know a couple of my SSD's (OCZ Vertex 2's) have had a few, have you checked the manufacturers website to see if there is any listed for your particular drive?


If it says backup your data before flashing the new firmware then i HIGHLY recommend doing so.

 

[Kainekokun]

Alright, I got my hands on a different USB enclosure, popped in the SSD and hooked it up to the Pandora. The very first test was very promising: I transferred a test file of ~25MB size, and it went across with no error messages, plus diff confirmed that the file transferred successfully. All happy with this I went home to test some more, and surprise-surprise: it didn't work anymore. With this new enclosure, I sometimes do and sometimes don't get the input/output error on the console, but the file never transfers successfully. The next best thing I got was a 14MB file on the SSD, which obviously was not the whole original. It seems a bit more hit and miss than with the previous enclosure, which failed every time, but it's still no use.


I have noticed something else, though, and I can confirm that this goes for both enclosures: when the file transfer fails, the USB drive gets "lost" for a moment, as if it was briefly disconnected and then re-connected. So, if I list /media right after a failed copy attempt, the device is gone (it is also gone from the desktop of the Pandora). But after a few seconds it comes back, presumably because the USB system re-detects that there is something attached.


I've checked the output of dmesg, and it confirms that the USB drive is being re-detected and re-mounted. But I get no messages about the device being disconnected, or any power issues.


I've read around on the network and I am getting the impression that when the data transfer goes heavy, the drive requests more power and for some reason the USB system refuses to deliver it and disconnects the device. From what I've seen around the net, the "default" power offered by the USB port is 100mA, but if the device requests more, the device is switched to another "category" and is offered 500mA. Also, some systems seem to protect themselves from devices that request too much power, and disconnect a device (or refuse to connect it up in the first place) if it requests more power than is available.


The issue seems then to be, that all these enclosures advertise only one USB configuration, in which the "max power" requirement is set to 2mA. This is the lowest possible non-zero value: the field is stored as a single byte and represents the power requirements in 2mA units, so value 0 equals 0mA, 1 equals 2mA and it goes all the way up to 255, which equals 510mA, although the USB specification states that the maximum allowed is 500mA. In fact, I've found forums where people had problems with devices that were requesting 510mA, and that used to work fine, until a kernel update changed the system's behaviour to refuse such requests. They mentioned a way to override this refusal and force the USB bus to connect to the device anyway, and it worked fine.


The above fix does not work in my case, because the Pandora happily connects to the device, but seems to refuse it the full 500mA power it requires, I suspect due to the fact that the configuration the device broadcasts states it only requires 2mA.


Is there anyone reading this who could tell me, whether (and how) it would be possible to override this configuration, and basically tell the Pandora's USB bus to provide the full 500mA to the connected device, despite what the device itself is saying?


I have tested this SSD with the newer enclosure on both my Mac as well as my linux workstation at work, and both had absolutely no problem in mounting and transferring the test file to and from the device. This leads me to assume that there should be a way to get the Pandora to play along as well.


@C0smikD3bris -- Good idea! I've checked the page of Samsung, but there does not seem to be a new firmware available for my SSD.

 

[Kainekokun]

Another update: while I had the new enclosure hooked up to my linux workstation for some testing, I saw that the /sys/bus/usb filesystem showed a folder called "power" for every connected device. Inside I found some promising files like "level", so I set out to see, if I could find that on the Pandora as well. Alas, the Pandora does not seem to have these folders/files in it's /sys/bus/usb tree.


Remembering how patnik mentioned kernel 3.2, I did some more digging around, and discovered that there is indeed a 3.2 kernel on the image I have transferred from the NAND to the SD card. So I changed my autoboot.txt to point to the 3.2 kernel, and booted my Pandora with that kernel. I know other people have issues with that kernel, at least that's what I understood from reading around the forums about this, but in my case the result was terrible, as I simply have NO USB HARDWARE when booting the 3.2 kernel. 'usb-devices' outputs nothing, /sys/bus/usb is empty and 'lsusb -v -t' crashes with a segfault!


Another thing that cropped up last night, though, is that my Pandora seems to sporadically boot from the NAND instead of the SD card, despite the autoboot.txt file being in place. Is there a magic key combination I might have been inadvertently pressing during system startup that could cause this? Or should I start to wonder, whether there's some malfunction in my Pandora?

 

[Kainekokun]

I've done some more testing by now. Having an older Western Digital MyPassport (250GB Model) and a less old 500GB model, I decided to try with those as well. The 250GB model did not work at all, the drive kept re-starting (which I could tell from the speeding up of the drive's spin), and didn't even show up on the output of "usb-devices". So I went and took it apart (which turned out much more painful than anticipated), but managed to extract the HDD and replace it with the 128GB SSD. When used in this configuration, this setup behaved just like all the precedent enclosures I tried with the SSD: the drive mounts, but big file transfers fail, and the output of "usb-devices" claims the device only requires 2mA. Just for completeness' sake, I even tried the 250GB HDD in the newer of the enclosures I had tried so far, but I got the exact same behaviour as in the WD "enclosure". So even this was no joy.


So I turned to the 500GB WD. This one was formatted in HFS, so when I connected it, it showed up on "usb-devices", and claimed it needed 500mA (joy!), but obviously it did not mount. At first, not wanting to re-format this drive, I decided to perform the same operation as on the 250GB. While this model turned out much easier to dismantle, the innards bore a very disappointing surprise: the SATA-to-USB bridge was built into the HDD! There was no PCB to disconnect and re-connect to the SSD!


To get to the bottom of this, I made a backup and re-formatted it as FAT32. Back on the Pandora, it mounted correctly ... and worked! I've copied quite some data over from a second SD card onto the 500GB HDD and it worked just fine (after every copy my script used the md5sum to verify the copy was successful).


The major difference appears to be, that all the combinations which failed use a SATA-to-USB bridge/adaptor, while the only working device (the 500GB HDD) does not. Coincidentally, all the adaptors "lie" about their maximum power intake being 2mA, while the HDD with built-in adaptor properly advertises a maximum power intake of 500mA.


I am almost certain by now that this "false advertisement" is the source of the problem, but I have no clue how to solve this. There ought to be a way to tell the Pandora to ignore the advertised power requirements. The fact that the 500GB HDD works just fine proves to me, that the Pandora is perfectly capable of providing enough power, so it must be possible to make it do so for a device connected through a SATA-to-USB adaptor as well!


Is there anybody out there who could help with this, or any address/forum/something where I could ask this question?

 

[notaz]

I remember MWeston also tested several drives a while back, and they all violated USB spec by drawing way over 500mA, some peaking over 1A, tripping overcurrent protection. You can write a small script to 'cat /sys/class/power_supply/bq27500-0/current_now' in a loop, this file reports battery draw current. Subtract the value you get on pandora alone from the value with a drive connected and you get USB current draw.


You should also try 3.2 kernel more, the USB driver is not loaded there by default because it breaks powersaving of the whole chip. If you flash the latest firmware properly, you'll get UI options for enabling/disabling USB host that do the right thing, else you can "modprobe ehci_hcd".


In general I think it's best to use such drives with external power sources..

 

[Kainekokun]

Thank you so much for your very useful reply notaz!

I remember MWeston also tested several drives a while back, and they all violated USB spec by drawing way over 500mA, some peaking over 1A, tripping overcurrent protection. You can write a small script to 'cat /sys/class/power_supply/bq27500-0/current_now' in a loop, this file reports battery draw current. Subtract the value you get on pandora alone from the value with a drive connected and you get USB current draw.

I've done some preliminary testing, I plan to do some more thorough testing and get back to you with some graphs, but so far, I can see that at it's peak, the SSD definitely pulls less than the HDD, including when it's writing a lot of data, just before it "bails out" and gets disconnected. The disconnect seems to happen when it reaches a value of about -897000, with a "baseline" without anything attached on the USB of about -350000. BTW what unit are the values in /sys/class/power_supply/bq27500-0/current_now? For comparison the HDD peaked at -1127000 while still functioning perfectly fine.

You should also try 3.2 kernel more, the USB driver is not loaded there by default because it breaks powersaving of the whole chip. If you flash the latest firmware properly, you'll get UI options for enabling/disabling USB host that do the right thing, else you can "modprobe ehci_hcd".

Aha! That explains the whole enabling/disabling USB host a bit better ... on the 2.6 kernel that UI entry seems to do nothing, so I ignored it further, but on the 3.2 kernel indeed after enabling USB host I get access to the devices I plug in.


Again, I've only done some preliminary testing, and only just plugged in the SSD. Unfortunately, even with this kernel, a big write operation causes the drive to get disconnected, but interestingly, the values I got out of /sys/class/power_supply/bq27500-0/current_now were lower, the "baseline" without anything attached to USB (but USB host being ON) was around -210000, and the SSD got disconnected when it reached about 791000, which would seem to point at a consistent power draw from the SSD between the two measurements.

In general I think it's best to use such drives with external power sources..

That would get rid of all my problems, but it kind of defeats the purpose of a device like the Pandora that can actually power these drives on its own. Besides, these preliminary results seem to show that the Pandora should be more than capable of providing the power the SSD needs.


DISCLAIMER: I limited the sampling to one sample a second, to limit data sizes, but this is just some preliminary testing. When I get around to it, I plan to do some more precise testing, trying to pinpoint at what moment the SSD gets disconnected on the measurement graphs, as well as sampling more than once per second.

 

[notaz]

The units are in uA (microamps), negative for current flow out of the battery and positive for flow into it, so for example in your second measurement, 791000-210000 would be 581mA taken by the drive. This is measured by battery monitoring chip, voltage is something like 3.9V at that point (there is a voltage_now file for that), so on USB end the current is probably a bit lower since it's supplying 5V there (Ohm's law). However this doesn't mean the drive doesn't go way over 500mA, there could be sudden spike during write operation that trips overcurrent protection. Battery monitoring chip only updates it's reading once per second, so it might not even notice that spike..


If it's really a current spike I don't think anything can be done by software as that overcurrent protector is not software controlled. If you feel comfortable with modding and soldering perhaps attaching large capacitor on the drive's or cable's USB power/ground pins would help to reduce that current spike, but don't quote it on me, I don't have much experience on these things.

 

[mcobit]

The spike would maybe even occur earlier when loading the capacitor in the first place


Edit: even if this would work, you would still really stretch the usbspec a bit. I think theres a good reason why they have those safety mechanisms.