We are suffering from a script slowness due to too much data coming out of the Get-VBRBackupSession command. We have a big Veeam B&R with a LOT of jobs and we need to slim down the processing of the following script below, or accelerate it's processing somehow. The usual suspects have been accounted for (Veeam B&R server has a lot of available resources, All-Flash storage, SQL Standard instead of express etc).

Currently, this script takes over a minute to execute, which is unacceptable for our scenario:

The problem is Get-VBRBackupSession alone is probably taking almost all of this time, it's a slow command because it always returns every single session and is non-optimal on the backend. Unfortunately, it looks like it might be even worse in 9.5, at least based on my lab results. Are you on 9.5 already?

I'm trying to think of another way to get at the data you're looking for, but it's not coming to me. Any change you are trying to call Get-VBRBackupSession mulitple times, for example once per job, and perhaps you could move it outside of that code. I know you're not doing that in the code you posted, but that code appears to be running for a specific job, so I was just hoping that it was perhaps just a subset of a bigger loop. That's the only thing I can think of off the top of my head.

tsightler wrote:The problem is Get-VBRBackupSession alone is probably taking almost all of this time, it's a slow command because it always returns every single session and is non-optimal on the backend. Unfortunately, it looks like it might be even worse in 9.5, at least based on my lab results. Are you on 9.5 already?

I'm trying to think of another way to get at the data you're looking for, but it's not coming to me. Any change you are trying to call Get-VBRBackupSession mulitple times, for example once per job, and perhaps you could move it outside of that code. I know you're not doing that in the code you posted, but that code appears to be running for a specific job, so I was just hoping that it was perhaps just a subset of a bigger loop. That's the only thing I can think of off the top of my head.

Opening a case wouldn't be a bad idea.

Thanks for the input Tom, case opened #01999855.

PS: Nice meeting you at VeeamPAC, very informative and great sessions.

Did some digging into this and there's definitely something going on in the code underneath this cmdlet that is non-ideal. I've passed my findings on to PM/R&D and we'll see what they have to say soon enough. I've also let the support person working on this case know that this issue is not unique to your environment.

In the interim, I wanted to share the following workaround which I hacked together based on what I found. It's ugly, as it's calling directly into the backend .NET code, but it's doing things in a little different way from the cmdlet and this method returns only the sessions for the job very quickly (in my lab the differences is ~150 seconds vs ~5 seconds) and allows them to be filtered down to the most recent 25 very quickly before entering the inner loop. Unfortunately this call returns a slightly different object type than the Get-VBRTaskSession needs, so I have to make a call to another .NET function to return the correct object. It's something about this function that is actually the core problem with Get-VBRBackupSessions, in my lab calls to this function take ~25ms each time, and Get-VBRBackupSessions calls this function for every single session in the database while, with my hack, we only have to call it 25 times for the specific sessions we need.

This is just a hackish workaround and I'll be trying to get a fix out of dev, but it made the script run 20x faster in my lab, so I thought you might want to give it a spin and let me know how it works for you and if there's any bugs that I didn't catch.

Another question, we have a similar delay behavior with Find-VBRHvEntity cmdlet, which takes now a serious amount of time when executed. Could it be a similar/related issue or it's in another ballpark? I think it's a separate matter since this cmdlet is looking into the backup infrastructure instead of Session in the DB.

Another question, we have a similar delay behavior with Find-VBRHvEntity cmdlet, which takes now a serious amount of time when executed. Could it be a similar/related issue or it's in another ballpark?

Nope. The problem related to performance drop of Get-VBRBackupSession cmdlet has been caused by additional piece of code added behind the curtains to the underlying procedure. The said piece does not seem to be present in Find-VBRHvEntity. Thanks.