mkretzer wrote:No. In his mail there was not one mentioning of something about the 4 k cluster size... That is also a reason i am kind of caucious about this recommendation.

We more or less found a way to circumvent the filesystem from being stuck as shown in the screenshot below.

Edit: Yes it looks like I just drew a white box, it's actually white-space where the windows kernel forgets the disk is ReFS.

We notice that the backup repo's (now newly formatted to 64KB cluster size) still cause the filesystem to be unresponsive, so we tried throttling the repositories to a lower throughput. Surprisingly, this significantly improved our performance. Since the volume doesn't become unresponsive, Veeam can now backup consistently without being interrupted by the unresponsiveness of the volume.

The throughput is still significantly slower than they would have been on NTFS (we are going to log a case for this as well), but at least, it's stable.

We suspect that the lower block size on the previous formatted volume, resulted the volume to get stuck even faster (in fact, 16x faster). We are backupping from All flash storage arrays, so our bottleneck almost always is our destination storage target.

We are currently monitoring the incoming IO's and as soon as it reaches its limit and causes storage latency on the backup target, the filesystem becomes unresponsive. So throttling temporarily circumvents this issue. This, however, isn't a permanent solution since, even with storage latency, the filesystem should keep on working. A 20-30 ms hiccup on the storage lun causes a 20 second unresponsiveness of the ReFS volume, which in turn brings Veeam to a halt...

I have 2 repo's 64KB cluster size and was testing some backup jobs and some backup copy jobs. Everything was acting normal until I did 2 backup copy jobs at the same time (to the same repo) Then I saw drops (veeam job timeline) in both the jobs at the same time. In the windows resource monitor I saw at disk level alot of writes but no file (and the memory consumption went straight up) - stopping one of the jobs was a solution for the other job to proceed as normal.

@rendest No we did not try to throttle as our main problem was that with 4 K and without the (bad) patch the whole system crashes. But you might be right as the unresponsiveness happened as soon as there is some kind of load.

Do you already have KB3216755 installed? I have the feeling this update makes the volume much more stable under load - but crashes Veeam services after 12 hours or so...

Good to hear, so now it's up to Veeam by patching whatever Microsoft broke. Since you mentioned it causes Veeam to crash, and our setups are identical, we are waiting for feedback from Veeam before attempting to install the patch.

mkretzer wrote:But can you throttle a fast-clone? In our system the fast-clone still lead to high load and the problem described here

As I understood, fastclone are just commands to move block pointers, so that shouldn't be that intensive.

But there are other maintenance tasks, which do not follow the throttling (for example cleanup/rollback tasks after a failed backup). Those were quite intensive & took our repository hostage overnight.