Tag Archive SSIS

I had a situation recently that an instance (not one I managed I hasten to add!) had started to grow in storage, to the point that it was at nearly 200% capacity. I was asked to look at it. Turned out that Plugin Tracing had been switched on, and a plugin was throwing errors. Unfortunately there were now so many records in the Log file that it wouldn’t open (Usual generic unhelpful SQL Error).

Taking a quick look at organisation insights confirmed that this was consuming nearly 60gb of storage, and a quick record count (Using Record Counter in XrmToolBox) indication there was approaching 10,000,000 records.

Normally I would use the standard Bulk Delete option, however for a couple of reasons I didn’t think this was a viable option:

It would take too long

It kept falling over after 1000 deletes (probably due to the number of records, but I didn’t get to the bottom of it)

In the end I decided to go back to trusty old SSIS, and (again) the lovely KingswaySoft solution for Dynamics 365.

So I don’t want to get in to the whole GDPR discussion in this article, but with GDPR in mind, I think it’s important, and potentially reduces risk, to anonymise data in a non production environment.

Again, there are plenty of articles on whether it should be a process of anonymising or pseudonymisation, but for the purpose of this article, I am going to go through the steps of using SSIS for the purpose of Anonymising Dynamics 365 Data.