Support Portal

How can we help you today?

USB drive usage errors

J

Jonathan Klay

started a topic
9 months ago

On the TS-4200 (TS-8160-4200), booting to busybox, getting fs errors writing data to USB drives. I would like to know what types and sizes should work.

I have tried a couple boards and a variety of USB drives, and formatted them (FAT/vfat) with both Linux and the Verbatim app. I have not seen any errors with 4, 8 and 32 GB USB 2 devices. With 64 GB and 256 GB USB 3 flash drives, the device would mount and be usable for anywhere from a few minutes to a couple days. Oddly, one of these was used last summer and ran for a couple months, and we thought there were no issues, so now I have a real time crunch. We expect to collect about 100GB of data.

I ordered some 128 GB USB 2 devices, thinking it was incompatibility with USB 3, but after some initial test that looked good, it stopped writing over the weekend with dmesg errors such as

[314622.810000] FAT: Directory bread(block 60028) failed

[314622.810000] sd 0:0:0:0: rejecting I/O to offline device

(I couldn't get the first messages anymore). The drive seemed to be mounted still, but in reality the /dev/sda device no longer existed.

Kris Bahnsen

said
8 months ago

Jonathan,

Honestly, based on your description I would look at the flash technology itself. NAND flash is rapidly getting more dense. Some device are even using Tri-level cells, which means each flash cell is storing 3 bits of data. This newer technology requires intelligent flash translation layer, more processing power in the controller, and more sensitive sense electronics. Even as MLC NAND technology gets smaller and more dense, the same amount of caveats will apply to the actual hardware.

Consumer components like USB flash memory devices are pretty notorious for putting out the latest and greatest with little concern of longevity and long term support. Many of the lower cost and larger devices are not rated for fully writing the whole disk over and over.

For consumer devices, we strongly recommend SanDisk brands, bought from a reputable distributor/retailer. Above that, I would recommend a spinning magnetic disk, or even a high-end SSD with a USB converter for the TS-4200. In addition, I would recommend against using FAT32 for the device. While it can support a large volume size, it does not have any kind of journaling features. I would recommend using ext3 if possible.

>[314622.810000] FAT: Directory bread(block 60028) failed

>[314622.810000] sd 0:0:0:0: rejecting I/O to offline device

>(I couldn't get the first messages anymore). The drive seemed to be mounted still, but in reality the /dev/sda device no longer existed.

This error means that a key part of the FAT32 metadata was invalid. Followed by a message indicating that the kernel is taking the device offline due to the error. An fsck may fix the problem, it may have just been an intermittent problem.

>Earlier errors with USB 3 drives have included

>[ 6.810000] sd 0:0:0:0: [sda] Attached SCSI removable disk

>[ 17.300000] vfat: Unknown symbol fat_dir_empty (err 0)

>[ 17.300000] vfat: Unknown symbol fat_time_unix2fat (err 0)

I'm unsure of why you are seeing these errors. It may be related to the fact that it is a USB3 device. They indicate that some parts of the kernel modules for vfat were not found. It may also be that all of the necessary modules were not properly loaded. If you boot up to Debian before inserting the USB device do you still see the same errors?

J

Jonathan Klay

said
8 months ago

Thank you for the quick reply today. I've been testing with Kingston and SanDisk. The 128GB USB2 are SanDisk. I do not see any issues in dmesg when I boot on into debian. If I copied a lot of data to the USB 3 drives in debian, I would still get errors and the device would be taken offline.

In use, the system will be out in the field logging data continuously for several months, so no user interaction will be available.

I'm trying the Seagate Diskwizard to format with MBR rather than GUID PT. I will also look into a SSD with adapter. I'll even consider ext3, though the engineers won't like the idea.

Well, as I sit here, the test just had issues copying files to the 128GB drive with MBR FAT32.

Kris Bahnsen

said
8 months ago

Jonathan,

The MBR vs GUID wont make a difference once the partition is mounted.

I am still apt to blame the flash technology itself, however I would not be surprised if there are other factors related directly to the TS-4200 causing issues. Newer USB drives likely consume more power, if the TS-4200/baseboard are unable to provide that much power the USB devices may brown out. This may not be enough to disconnect them, but could be enough to cause invalid data to be written. Additionally the USB controller in the CPU could just have incompatibilities with the USB devices.

After thinking about this more, in addition to recommending an SSD with a USB adapter, I would also recommend that it be externally powered. If you still have issues with this, let us know and we can offer some further suggestions. However, due to the fact that this only seems to appear with larger/newer media, there may be very little we can do to resolve this. If that is a hard requirement of your application, I would recommend looking at our current product lineup and using a platform that has a newer CPU; newer USB peripheral implementations may be able to better handle the requirements of newer USB mass storage drives.

For reference, this is something we often fight on our older products. A couple of times now, the SD cards and USB mass storage devices we ship with our kits have gone EOL. Because of this we've had to spend a great amount of time to qualify new parts just to ensure there are no strange incompatibilities. Though we've never had to deal with such large USB mass storage devices.

J

Jonathan Klay

said
8 months ago

Thanks for the additional info. I tested booting straight into debian with linuxrc-sdroot, then inserting the freshly formatted media, mounting, and copying some files but after a few minutes of file writes, it was once again offline with "FAT: Directory bread(block 59962) failed" errors.

I'm going to order a SSD, and research what connectors/adapters are available. The MicroSD slot is limited to 32GB, isn't it?

K

Kris Bahnsen

said
8 months ago

Jonathan,

The TS-4200 only has support for SDSC and SDHC devices. SDHC specification goes up to 32 GB; however there are some SDXC devices that will start at 32 GB as well. So long as the SD card you are using is SDHC (or SDSC), it will work.

J

Jonathan Klay

said
8 months ago

Just for info, while waiting for shipping of a SSD I tried formatting ext3 and using it in case this is all related to vfat. This was much worse. I mounted it and that looked OK, made a directory, looked OK, made a second dir and it disconnected - I tried this twice, other time made 3 directories.

ext2fs_mkdir: Attempt to read block from filesystem resulted in short read while creating root dir

root@ts4200:~# mount -t ext3 /dev/sdb1 /mnt/cf_usb/

mount: /dev/sdb1 is not a valid block device

K

Kris Bahnsen

said
7 months ago

Jonathan,

Just wanted to follow up on this to see if you've had any success with an SSD or other external drive options.

Let me know if you have any questions or further comments.

J

Jonathan Klay

said
7 months ago

There are a few varieties of SSD now, I avoided the newer types and tested a 120GB Crucial SSD. Had trouble finding a quality USB-to-SATA adapter. The IDE/SATA units have external power but are poor quality, 1 was using a lot of power and only lasted a day, another worked fine but used more power than we were happy with. Plain cheap USB-to-SATA on USB power actually ended up working and used only slightly more power than a USB flash drive. We saw no filesystem or USB errors with this SSD while continuously writing for several days..