SIRIS, ALTO, and NAS: Error: Backup failed due to I/O error from the protected system during the backup process

Issue

When you try to start a backup for a system protected by the ShadowSnap agent, you receive the error message, "Backup failed due to I/O error from the protected system during the backup process."

In legacy versions of the Datto IRIS environment, this error message was "STC -31 on Final."

Environment

ShadowSnap Agent

Cause

Causes of this error include:

Networking / Samba issues

Server tuning variables that require adjustment

Bad install of the agent

Protected volume is out of space

System writer is not present on the protected machine

VSS subsystem does not work

Hardware problems on the system

Bad sectors on the protected disk

Disk errors and warnings in the event log

ATAPI errors and warnings in the event logs

The AppAssure application can cause this error. It is incompatible with the Datto backup solution. When uninstalling this program, Datto suggests following AppAssure's process(external link) to remove the AVolFlt family driver properly. This driver has been found to remain on a protected system even after removing AppAssure (also known as Rapid Recovery or Replay).

3. Perform a chkdsk /r on the production machine for all volumes with the filesystem unmounted. Do not attempt this in read-only mode. If Windows has not identified corruption on the disk, a chkdsk with Windows mounted will not either. Running with the system unmounted ensures direct access to drive I/O and will provide a more accurate indication of the disk state. Even though errors might not appear in the event log, this only means the file system has not reported the corruption yet.

4. If these steps do not solve the problem, and disk errors can be ruled out, refer to the Further Troubleshooting section below.

Check System Resources

1. Check the protected volume to make sure there is enough space to perform a backup. The production machine requires a minimum of 20% free space per protected volume and 1 GB of RAM. If resources fall below these thresholds, backups will fail. Reducing system overhead during backups can also help prevent this issue.

Volume Use

Status

Comments

0 – 70%

Good

Use is good.

71 – 80%

Warning

Use is outside optimal limits. Monitor the volume growth and look at cleaning up unnecessary files and moving non-essential data to other storage.

2. If the IP address of the protected machine has changed, update it from the Datto appliance GUI by following the steps described in this article. If the Datto appliance needs to follow special routing to reach the protected machine, contact Datto technical support to assist with the changes.

Further Troubleshooting

1. Ensure that a stable route from the Datto device to the ShadowSnap agent exists, and can communicate over TCP port 25566.

2. Check for any stuck file restores or virtual machines that do not show up in the web interface. You can destroy these as long as they are not in use.

6. Make sure that ports 25566 and 139 are open on the local Windows Firewall. Port 25566 will be open if the raw_agent_svc is running as part of the ShadowSnap agent. If another program uses any of these ports, the agent may malfunction.

7. Make sure that anti-virus exceptions exist for the StorageCraft ShadowSnap application

9. Check the Windows Event Logs for the error. If it does have an error, verify and disable possible conflicting services, then uninstall and manually reinstall ShadowSnap and ShadowProtect.

10. Check network connectivity between the protected system and the Datto device. Rule out any IP address conflicts with the protected machine or the Datto device. Replace the network cable for the protected server, and try a different switch port.

11. Run the ShadowProtect Diagnostic (external link) runsupportmode.cmd tool provided by StorageCraft and consider the recommendations it makes for server tuning.