Saturday, January 03, 2009

Safely remove an USB hard drive in Linux

I bought a new USB hard drive, Western Digital My Book® Essential Edition™ 1TB, and use it with Linux. Here I'm discussing about how to safely remove (then disconnect) it from a Linux system.

The problem: according to its user manual, you should disconnect the drive safely when it is not active, but in a Linux system how to do this is not very straightforward. You may notice that after you unmount it (whether through command-line or a desktop environment), the drive is still spinning and it's LED on. If you read it's user manual carefully you'll find that the manufacturer never said it's safe to disconnect it from your system in such condition.

The quick solution to this problem:

send SYNC, then STOP command to the device, this can be done easily in Linux by unbinding the device

suspend the USB port by echoing a "suspend" to the "/sys/bus/usb/devices/$DEVICE/power/level", where $DEVICE corresponding to the device of your USB device.

To put it simple, I wrote a script to do this for you. Download and try it. For developers, you can get it from the github repo.

From the hardware aspect, the drive should only be removed when the following requirements are met:

there's no pending I/O request and software cache flushed

it's hardware cache flushed

it's driver spins down (means "not spinning")

(optional) the USB port is put into "suspend" mode

The user's manual read (page 6, section "Turning Off/Disconnecting the Device"): "(for Windows systems): Right-click the Safely Remove Hardware icon in your system tray and select Safely Remove Hardware. You may hear the drive power down before the Power LED turns off. The drive is now shut down properly, and you may disconnect the drive safely. (for Macintosh systems) Drag the My Book icon to the Trash icon for proper dismount. You may hear the drive power down as the Power LED flashes. When the Power LED is steady, you may press the Power button once or disconnect the drive’s power cord to turn off the drive safely." (there's a mistake, this device doesn't have a Power button) If you read carefully you can tell that Windows and Macintosh don't do the same things when told to remove an USB device. As the bold text as shown, in a Windows system it will be turned off while in a Macintosh system you hear the drive power down but the Power LED would finally become steady rather than off. Therefore we know that Windows does all 4 steps and Macintosh do the first 3 steps only when told to remove a USB device. (BTW, it's funny that you have to use your ear before disconnecting a device.)

(This is only the case for Windows XP. For Windows Vista seems only the first 3 steps are done thus the drive remained active after you removing it from the system.)

Some may argue that they always disconnect their drives directly after unmounting (step 1 only) and never experience any damage, but as an operating system developer and a serious user, I always follow the manufacturer's manual or specification whenever it make sense and possible because I know electronic devices are fragile, rigid and obtuse if you don't use them as the way they are designed. For a well-designed and robust piece of hardware, theoretically you can disconnect the device safely when the first two steps are done. The firmware on the device would notice the USB cable was unplugged and shut down the disk drive properly. If not, the drive would be stopped sharply as if you pulled it's power when it's running and this damages the hardware gradually. As an outsider of the manufacturer, we hardly know whether it's well-designed or not so the only safe principle is to follow it's manual, means we should not disconnect the device until at least all three steps are done.

At the Linux kernel part. To finish all the 4 steps, you have to unmount the device, unbind the device from the driver, then tell the USB core driver to put that device into suspend mode, as shown in the solution above. And you have to be running a kernel with CONFIG_USB_SUSPEND enabled.

CONFIG_USB_SUSPEND is not enabled by default in a vanilla kernel. I'm pushing it in this discussion.

Properly removing a device from a running Linux should be done by either the HAL or desktop manager. If you can, please push the developers you know working on any DM that missing this feature to implement this function properly.

To crumja:You are right, command "sdparm --command-stop /dev/sdx" stops the drive, as listed as the 3rd step in the blog. But that alone is not enough before unconnecting the hard drive. The USB port should be put into "suspend" state.

According to Alan Stern, the best approach is "to send a SYNCHRONIZE CACHE command followed by START-STOP (if the device supports it), and then to disable or suspend the port. In Linux, those two commands will be sent automatically if you unbind the device from usb-storage. The suspend has to be done manually unless you have set up a udev rule (or something equivalent) to enable autosuspend for the device."

I've had this same problem for sometime, I'm glad that I found your fix. One thing that I'm seeing is that after my drive is suspended, the power lights go out but when I shutdown my system with the usb drive still connected, the lights come back on. Do you have any ideas on how to correct this?

Let me tell you my findings with the script, too: after running it, it takes about 5-10 seconds before the light goes off and starts flashing in a 5 seconds intervall. That's it. To stop it completely I have to remove the power.

BTW: Yan Li, I wrote you a mail with a small change in newer versions of udev. I have no more udevinfo. Instead I have to use "udevadm info".

My usb drive is the WD MyBook 500g model. On my machine I run Window XP and openSUSE 11.1. I do not see this problem with Windows. On windows the drive will power down completely (lights go off) when my machine is off. Your change does work, it spins down and suspends and the lights go out, until I power the machine down. Then they come back on. The only other way that I know how to correct it is to make a Kernal change, but the last two times I tried that on openSUSE didn't go well. Grub got messed up.

I am having a little trouble with your script and consider myself pretty new to linux. I saved your script as wd_shutdown.When I run the script root# ./wd_shutdown /dev/sda or sda1 it can not find my wd 1TB my book.as root ./wd_shutdown /dev/sda"./wd_shutdown: line 78: udevinfo: command not foundcannot find it's parent USB device,perhaps it's not an USB device?"

Hi Yan Li,First of all, thank you for your script!Just a remark... The routine to check if the device is mounted is not working here on my Ubuntu Gutsy. I was able to fix it by removing the extra space found in the grep expression ("^$DEV_NAME " changed to "^$DEV_NAME").Also your script exits with a "Permission denied" error when it tries to suspend the USB port. I have figured out such error occurs because the "level" file was removed after unbinding the device. Is "unbind" really necessary? Why it can't be executed after suspending the USB port?Thank you so much!BR,Marcos.

I tested my WD 1TB on a ThinkPad T61 laptop and hadn't found the problem you've described. The HD sat silently when the laptop was shutdown. So I can't help you in this case until I found a machine and can reproduce this problem.

You said you knew a way to correct this problem by changing the kernel, could you please share more on this? Thanks!

Many thanks for the script. I have been struggling to get spindown working on my Freecom Hard Drive 500Mb (internally its a SAMSUNG HD501LJ). Presently I am on Ubuntu Hardy Heron HP laptop, using on-board USB ports to the external drive.

And I had tried everything to get spindown to work - sg_start, scsi_stop, spindown, sdparm, hdparm, scsi-spin and eject. None of them actually got the device to spin down as it does if you remove the USB cable... and yet your script does exactly that. Thanks!

One interesting thing on my platform - the "unbind" part of your script *removes* the level file (and a few others from the power folder), thus tricking the last part of your script into thinking that it is not supported. However if I plug in the drive, umount any automatically mounts, I can do the suspend thing manually without problems.

Is it possible then that the unbind part of the process is unnecessary?

Thanks again, anyway - I will next be trying this on my NSLU2 ARM device :-)

i'm extremely pleased to have found someone who seems to have a solution to my problem. i cannot get ubuntu to effectively remove my wd passport before i unplug. as you said: the led is on and the drive is spinning. i've actually had my data corrupted, and i think improper removal is the problem. while i'm glad to have found your page, i am new to linux, and have no idea how to use your solution. if you could offer me some help, a step by step, on how to execute this, i'd really appreciate it, as i'm not sure how to use your script. thanks in advance.

To Jon:Thanks for your advice. As to my testing, both unbinding and suspending are necessary for fully bringing a device into proper mode for removing. Unbinding along won't tell the device to go into suspend mode, and for my WD Book 1T this won't shut the LED. And you shouldn't suspend a device without unbinding it since the cache may not be flushed correctly.

As to your case, if the power/level file is missing after unbinding, then I guess it's a bug of your kernel. Unbinding a device from a driver won't remove that device. If you could provide more information of your kernel version, distribution, etc., we might be able to dig into that issue.

Thank you so much for the post and program. I am so impressed with the last couple years improvements in the desktop experience and I hope to see it continue.

I think the suspend before 'safely remove' is an important element of this. I am much more confident my usb disk is ready to be unplugged when the power light has turned off or gone dim as it does when I 'safely remove' in MS Windows.

I hope to see a standard mature version on every distro like the eject command.

To Marcos:> Just a remark... The routine to check if the device is mounted is> not working here on my Ubuntu Gutsy. I was able to fix it by> removing the extra space found in the grep expression ("^$DEV_NAME "> changed to "^$DEV_NAME").

This is fixed in the script. Please update. Thanks for pointing this out.

> Also your script exits with a "Permission denied" error when it> tries to suspend the USB port. I have figured out such error occurs> because the "level" file was removed after unbinding the device. Is> "unbind" really necessary? Why it can't be executed after suspending> the USB port?

Jon reported a similar issue. Please refer to my reply to Jon. Basically I think it might be a bug in your running kernel. I'm using latest stable kernel release from upstream (2.6.29.1 now) and I haven't seen such issue.

To Rakesh Radhakrishnan:> In my hardy system, support is there for USB> suspend. /sys/bus/usb/devices/7-3/power/level is present also. But> since after unbinding, this "level" file will be deleted.

Other people have met similar issues. But since I haven't observed this issue in latest stable kernel release (till 2.6.29.1) nor 2.6.26 Debian Lenny kernel, it might be a bug of the kernel used by Hardy.

Unfortunately I have no Hardy system for testing now (all people around me are running Intrepid or Jaunty). Could you please try a latest upstream kernel by using KernelMainlineBuilds? And if the new kernel is OK, filing a bug against Hardy's kernel?

> Is there any problem executing unbind after suspend. It is working> fine. But in my opinion, it does not make sense to unbinding driver> after suspending the device. What do you think?

You have to unbind the device before putting it into suspend mode since unbinding flush the cache. Directly putting a device into suspend mode without flushing cache may lead to data corruption.

Does anyone know how I could use udev to start the script? The problem is that a "ACTION==remove" in a udev rule is only triggered when the device is removed (i.e. disconnected) from the computer, and that would be too late for the script to do it's work.

To Chris: Since the operating system must be told that you wish to remove an USB device, this event must be initiated from a GUI tool. Though I'm not an GNOME nor KDE expert so I'm not sure what is the best way to implement this.

Very nice and more or less what I was looking for. Having an encrypted filesystem on my USB disk makes me want to be sure that I have safely powered it down before disconnecting it.

However, I encountered a problem in that the script switches off more than it should. In particular, I had the disk hanging off a USB hub (a Logitech G11 keyboard -- using an external power source for the disk is mandatory in that case btw). Running the script then powered-down the keyboard as well as the mouse and the disk connected to it. Not good.

Similarly, when targeting a disk connected to the main chassis, running the script powered off the USB connector for good; there seemed to be no way to get it working again except rebooting.

I think the script should look for a USB device "deeper down" in the USB device tree. I have modified it somewhat, making it call a Perl program that parses "udevadm" output such that the parent device of the device with DRIVER=="usb-storage" and SUSBYS=="usb" is targeted. This is "deeper down" than the device targeted by the original script. Which seems to work.

Additional minor changes are: added a "verbose" flag, moved the test for a zero $DEVICE upwards, and fixed the list of options in getopts.

The modified script can be obtained from my employer's site:

http://public.m-plify.net/suspend_disk/suspend-usb-device.sh

The Perl script from:

http://public.m-plify.net/suspend_disk/usb_device_filter.pl

One may have to correct the FILTER variable in the shell script to make it point to the perl script.

Hello, I've tried to make your scrip work on my WD Passport 2.5" 250gb, but it seems not to work... After launching the script, the drive's light started to flash really quick, like it does when it's reading or writing files! It was also spinning with its usual vibration!Is there something I can do to make it spin down the drive? Thanks a lot!

To pollanza: that's weird, I have never seen that. Could you please try using another USB port? And could you please run that script with "bash -x ./suspend-usb-device" (this just turns on debugging output) and tell me what is displayed when your drive's light is flashing abnormally?

I've modified your script with a suggestion from a friend of eeepc.it forum and now it works! Now it spins down the plate of the drive!!Here there is the modification:# send SCSI sync command, some devices don't support this so we just# ignore errors#sdparm --command=sync "$DEV_NAME" >/dev/null || truehdparm -f "$DEV_NAME" >/dev/null || true# send SCSI stop command#sdparm --command=stop "$DEV_NAME" >/dev/nullhdparm -Y "$DEV_NAME" >/dev/null

To d.tonhofer@m-plify.com: I've carefully studied your script (usb_device_filter.pl). It was very well-written. Thank you. I'm thinking that we can use a simpler way to get the correct parent of the usb-storage device. In a devpath, it's always the one two slashes before "host*", eg. for a devpath like:

I've tested this way and it worked well with my USB hubs. By this way the modification to current script is trivial and no need to use another Perl script, which would bring in more dependencies. However I'm not sure whether it works in all situations so it would be kind of you to test the latest script as you can find in the git repo.

suspend-usb-device script updated: per David's advice this script now works better with USB hubs, only the designated device will be shutdown instead of the whole USB hub. Also includes Christian Schmitt's patch for supporting Firewire devices (need more testing). Please write to me or leave comments if you saw any bugs. Thank you.

When after some time I connect the drive again and want to detach it for a second time, then the script does what it should (sync, stop) but the drive continues running. Removing the wires gives that ugly forced HDD shutdown noise.

There's a weak discussion on LP https://bugs.launchpad.net/ubuntu/+bug/117713 with no working solution

Hi, I've got quite a problem. Ant I'm quite new to linux. Your script works fine, at least for a second. It does succesfuly power off my hard drive, then immediately it powers on, but it doesnt mount. /dev/sdb is not created. It seems system doesn't seem to be knowing that something is plugged at all..

On "Fedora 11" (Fujitsu-Siemens laptop), things go wrong. The same USB disk is not powered off reliably (sometimes it is powered off and sometimes it isn't). It also sometimes goes into a state where it doesn't become visible to the system after disconnect/reconnect. This may just be a problem with the laptop hardware (which is cheap and nasty), or maybe there is something wrong in the OS. I am building another F11 system and may be able to test on that one soon.

To Evaldas: hum... people are reporting similar issues and those problems seem to be machine-specific. I haven't met similar issues here so I can't debug. David is trying and hope he could find the root cause for this.

Being a Gateway Laptop Hard Drive repair I have never encountered with such a system where I had to remove an USB Hard drive in Linux. You have explained it well and certainly it will not only help me but it will be useful to million people out there.

With the new kernel 2.6.33 echo "suspend" wont work any more. According to kernel power management documentation, suspend is only for kernels 2.6.32 and lower and it was replaced by "auto". Setting to auto the device will autosuspend after 2 secs. This can be also be changed by setting autosuspend file value to 0

Many thanks for the script Yan Li and to everyone else who has contributed but maybe some USB HDDs just don't behave!

I tried with a Hitachi SimpleDrive mini, model HTS545050B9A300, using the latest version of the script from the git (enhancement request: a version number in the script) and with a kernel with CONFIG_USB_SUSPEND enabled.

No error messages but the HDD went right on spinning. I also tried quiescing it with sdparm first. Here's the last attempt:

David's idea to get the parent by DRIVER=="usb-storage" and SUBSYS=="usb" is not "a simpler way" but it is more correct one. If you do not check DRIVER or/and SUBSYS you will do some bad things with SATA devices. What is /dev/sdd? Is it a USB-device or a hot-plugged SATA-device? I think you would better filter devices by asking udevadm.

It may be prudent to explicitly set the $PATH in the script. On Slackware 13.0 32-bit running udevd 141, when a bash script is run from udev rules the PATH has bash' default, "/usr/gnu/bin:/usr/local/bin:/bin:/usr/bin:.", presumably because udevd does not provide a PATH environment variable. A good choice might be "/usr/sbin:/sbin:/usr/bin:/bin".

Hi, great script! I discovered it after looking into sdparm myself. I found that --command=stop didn't spin down my USB disk.

When I run your script I get ./suspend-usb-device: line 180: echo: write error: Invalid argument

This comes from the last command: echo 'suspend' > "$POWER_LEVEL_FILE". If I find the level file and try to echo to it myself (as root) I get the same error. I have CONFIG_USB_SUSPEND built into my kernel.

The disk is an Hitachi HTS545016B9A300 in an Astone USB/eSATA external case (I'm using the USB port since I don't have eSATA on my laptop). The case is listed as:152d:2339 JMicron Technology Corp. / JMicron USA Technology Corp.

Nice to see that somebody is concerned with this issue. I've been using Linux during 6 years and I've got 3 HD (1 external and 2 internal) broken. They've always shut down while still spinning, making an annoying noise.

The script doesn't work with an iomega USB-2.5"HD. However, after (only!) unmounting the disk, the following seems to spin it down (only LED light is kept):$ sudo hdparm -Y /dev/sdc1/dev/sdc1: issuing sleep command HDIO_DRIVE_CMD(sleep) failed: Invalid exchangewhere /dev/sdc1 is the USB-HD device.

Is it safe? If note, How should I adapt your script to support my disk too, so the other/missing steps are done?or, maybe, is there any cleaner and automatic solution (probably with udev rules)?

I have tried to adapt your script to my iomega USB-HD. But something really weird is happening: the hdparm -Y instruction is very slow when it is run from the script. I don't know why.

Thus, I've been playing with the following (you already know the full commands):- sdparm-sync- sdparm-stop- hdparm-y- unbind

From the command line, the following seems to work:1. sdparm-sync2. sdparm-stop3. hdparm-y (spindown)4. sdparm-stop again to power down (switch led off)5. unbindIf step 2 is not done, it doesn't work.If step 5 is not done, after a few (~10) seconds the disc is resumed (spinning again).

Unfortunately, if I adapt the script to do the 5 steps above, it doesn't work: hdparm-y takes so much time and the disc is resumed. Any idea, why the script behaves differently?

@Enrique: an obviously difference between running commands from command-prompt and within a script is the pause between commands. Perhaps you can try to add a few "sleep 2" between the commands in the script. Also I'd suggest remove "2. sdparm-stop" and try again.

There's still one little problem: the led doesn't switch off. I'm not sure why, because sometimes it does. Anyway, being on or off, later when the device is unbound, the led get switched on again.Not a big problem, I think? But, the usb link is not in suspend state then, isn't it?

I'll like to make this automatic when I press unmount or remove-safely. Do you know if this can be done with udev rules? (I've been looking here: http://reactivated.net/writing_udev_rules.html#external-run)

Incidentally, is it possible to reverse the process, i.e. power the USB drive back on and have it reconnect? The reason is that I want to use USB drives for backup, and don't want them running when not doing backup (ideally I'd have one drive per day used in rotation, the way we used to do with tapes), and want them to be powered on by software. At the moment I do this with a timer switch, kill the power to the drive for 15 minutes once a day and let your script power it off after backup, but that's rather inelegant.

This is brilliant, thanks for taking the time to write this post. I've struggled with this problem for quite a while until coming across this post. It's quite easy to follow your steps and I haven't come across any problems, thanks again.

I found your explanations useful, but the script is not working as expected for me (or is it my drive ?).

I'm using a 2.6.36 kernel and, as BlackFateGR said, I had an error with this line (at the end) :echo 'suspend' > "$POWER_LEVEL_FILE"I replaced the 'suspend' by 'auto' (as suggested) and the error message disappeared...

@kr it's hard to say. Perhaps you can try the driver on an officially supported OS (hint: Windows) and see whether it keeps spinning after being removed from the OS. If it spins similarly then I speculate it should be safe to be removed then.

Actually after GNOME has built-in "Safe remove" function this script is no longer useful anymore. Moreover, I heard it doesn't work with latest kernels due to changes in the USB power management knob.

Maybe you should report the Ubuntu/GNOME bug to them (that sounds serious to me). Personally I'm using various versions of Debian on many machines and I haven't observed such an issue so maybe it's Ubuntu-specific. Actually it's because I feel the GNOME is doing well enough on this little task so I don't have to maintain my ad hoc script any more.