The team at Microsoft Labs did not let this go unnoticed and created an updated version of the Attachment Management solution that not only addresses the Portal issue but also added a couple of other features to the solution.

Essentially, Microsoft sent me the following note just days after my original blog post:

Setting Up the Solution

For the rest of this post, I will assume that this might be your first time setting up the Attachment Management solution so I will start from the beginning. If you have already set up the solution, you will need to first update it and then go here to see what you need to do to make it Portal friendly.

The first thing you will need to do is set up an Azure Storage account in Azure. As I had mentioned in the past, Azure provides a lot of extras and features to your Dynamics 365 system so it should almost be default that you deploy Azure with Dynamics 365.

In Azure, highlight Storage Accounts and then choose “+Add”. Give the storage account a name and provide relevant details for the type of implementation. I would strongly suggest enabling secure transfer and try to ensure that your resource group location is in the same data center as your Dynamics 365 instance.

Setting Up Azure Storage Account

You will need to configure 2 containers, one for notes and for email attachments. Once the storage account is created, click on “+Container” to add a new container.

Add Container

Give the container a name that reflects what it stores. Also set the access to Private.

Create email attachment container

Create a container for notes as well.

Create notes container

You will also need to generate a Shared Access Signature so Dynamics 365 will know where to store and retrieve the files. In the Storage Account blade, click on “Shared Access Signature”

Shared Access Signature

With the Shared Access Signature, you can specify the time range for access to the attachments. Make sure that the start date is a few days in the past and your end date is far enough in the future. If you set it in the next few months or years, maybe setting a reminder in your Outlook will be a good idea so your users won’t start getting errors.

Ensure date range fits and also ensure HTTPS protocols.

Once the SAS token is created, keep the window open or copy the SAS as you will need it in the next few steps. In the event you do close the window, you will need to generate a new SAS token.

The SAS token

Installing on Dynamics 365

Navigate to your Dynamics 365 system to install the Attachment Manager solution. Navigate to Settings and then go to the Dynamics Marketplace. You will need system customer rights to install the solution.

Navigate to Dynamics Marketplace

Locate the “Attachment Management” solution.

Attachment Management

The solution will need your account information in order to install the solution.

Providing access to your Dynamics 365

You will see the information regarding the solution and you will need to designate which organization you want to install the solution to.

Designate which organization to install to

Once you have agreed, the solution will begin to be installed. You can monitor the installation in the Dynamics 365 Administration Center in Office 365.

Attachment Solution installing

After 5 to 10 minutes, the solution should be installed. In Dynamics 365 navigate to Azure Attachment Storage. If you don’t see the area, log out and log back in to refresh the sitemap.

Azure Attachment Storage

Click on Azure Blob Storage Settings to enter in the Azure settings. Key in the storage account name, the SAS token and the names of the notes and email attachment containers. Click Confirm to save the information.

Azure Attachment Storage Settings

Be sure to update the security roles so end users will be able to attach files without getting an error message:

Update the Security Roles

The Portal Fix

The original version (1.1) that was all you needed to do. The updated version (1.5) there is now an extra, important configuration step. Click on Notes Attachment Entities.

Note Attachment Entities

Choose the entities for which you want note attachments to move to Azure storage. If you plan on using the Dynamics 365 Portal, then UNCHECK THE adx_webfile ENTITY! Miss this step and your Portal could turn into a real mess!

This was due to the fact that the Dynamics 365 Portal uses file attachments to store images and CSS files. If these move to Azure, the portal cannot read the attachment and will not apply the files to the Portal.

If there are entities for which you would rather not move into Azure storage, uncheck those as well.

Choose the entities for which to move attachments.

Now when you attach a file to a Dynamics 365 record (that is set to have attachments move to Azure) you should see in the Azure Storage Account the attachments appearing.

Attachment stored in cheap Azure Storage

Existing Attachments

The previous version would only work on a “go forward” basis meaning that if you have attachments already in Dynamics 365 these would not be moved to Azure. Navigate the to Azure Attachment Area and choose “Reports and Admin”.

Reports and Admin

You now will be able to see how many attachments have moved over and you also can select and move existing attachments (assuming the entities are selected) to Azure Blob storage. This is a great feature for existing Dynamics 365 installations that may have a lot of existing attachments.

Mass move Attachments to Blob

At this point I haven’t tested a large amount of records, so I would be hesitant to run this against a 200GB database.

Storing attachments in Azure is one way to keep your Dynamics 365 database to a reasonable size. With the latest improvements and fixes, the Attachment Management and the Dynamics 365 Portal can co-exist.

The solution also comes with web resources to allow users to drag and drop attachments and an interface to download multiple attachments. I will review these in an upcoming blog post.

Thanks to Suresh Guduru and the Microsoft Labs team for the quick response and release of this latest version of the Attachment Manager.

Nick Doelman is a Microsoft Business Solutions MVP and will be participating in the CRMUG/D365UG Summit in Nashville from October 9th-13th and will be presenting at Extreme365 in Long Beach, California November 6th-9th. If you are attending either of these events, please come say hi!

14 thoughts on “Dynamics 365 Attachment Storage Revisited”

We are also trying to integrate CRM with azure for document storage. While evaluating different add-ons and custom implementations, i read in almost every post that attachment management add-on is not reliable and there are people who reported issues after deploying the solution into their production environment (though i dont know exactly when it happened).
My question in short is can i reliably deploy this add-on solution into my production?

Hi Yash, I got it working, but the caveats apply that it has limited support (but any ticket via AppSource I put in got a fast response). You need to do your own homework and evaluate risks based on your situation. If mission critical, then implementing a commercial solution with SLA driven support would likely be a much better option. There are also recent postings about using Flow and SharePoint to save attachments which also might be a good alternative. I have used this solution for smaller projects where if there is an issue, the overall business is not affected (might be a nuisance to find a particular document). I have also used something else (MSCRMAddOns, or North52) where it was critical that this always worked and if not could be resolved via professional support. Hope this helps.

Thanks a lot Nick for immediately responding to my query. Your answer helps me a lot to evaluate different approaches. I have one last question if you dont mine answering.

We have only production instance of CRM and we dont have dev or stage CRM. From organization insights, i found attachment table to occupy 20 GB, annotation entity occupy 13 GB and there were more than 5000+ email and notes attachments records.
1. After i select “Report and Admin” and click move to blob, i heard a plugin will run and move all my attachments to azure. My only worry is since plugin threshold time is 2 minutes, will the plugin ever timeout and throw error while moving attachments of large size (more than 30 GB) on a single go (I know u mentioned it would be unrealistic to use for 200 GB but will it work for 30-40 GB attachment data)? Or my understanding of the architecture behind plugin is incorrect? My one & only concern is will it move existing 40 GB attachment data to azure successfully without throwing any error?

I used a logic app to move the attachments in smaller chunks over time. You can do this by changing the $top parameter to only grab so many attachments at a time, and schedule the logic app to run every few hours, etc.

Now with the large attachments, I would be concerned that these would still time-out. Another option would be to potentially look at something like Kingswaysoft to move your attachments.

Thanks you Nick for your quick response. As you brought up SSIS kingswaysoft connector, i got one more question (hopefully this is last question and i dont want to take any of your valuable time).

Using SSIS i can very easily download all the email & notes attachments to a local SQL DB or excel and upload the attachments back to azure. But how do i establish the connection between uploaded attachments in azure to respective email records so that whenever we open a past email record (i guess we have an URL in the email and if we click it will redirect us to azure to download the attachment) i must be able to directly browse to the respective attachment in azure.

Finally, if i upload all the email attachments to azure and somehow able to uniquely identify and establish connection from attachments to respective emails, my problem is solved (I guess this is what the attachment manager is doing to establish that link between email record and azure attachments)

Sorry for the delay on this, you will need to add a shortcut/file as a note attachment that is hyperlink to the document stored in Azure. if you take a look at the logic app code from here https://github.com/gsureshqa/LogicApp I believe it will give you an idea how to do it.

However which testing this logic app, it only works fine if i set the $top (number of attachments) to less than 200. If i set to any value more than that, the package would obviously fail for me. How to resolve this issue since we have 50000 activitymineattachment records? Its insane to run the logic app code 250 times.

Also nick, using SSIS kingswaysoft i retrieved all email attachments to local disk and uploaded to separate azure container. But i am stuck on how to reestablish the connection to existing notes records. I went through the logic app code, but i am not a matured coder to understand it. Can you please make me understand what is going behind scenes of updateactivitymineattachment in foreach loop? So that i can replicate the same function using SSIS package that i built.

This only works for the emails and note attachments, but id does come with a migration tool (so you can move stuff out of CRM db and back. For 100’s of gigabytes, this will of course take some time, best to use an azure VM in the same datacenter for this).

Just to verify (because the documentation of the solution itself is not clear): You can set the access level of the storage to private, such that the documents are not publicly available without the SAS token og direct access through the Azure Portal?

Search

Power Platform Stuff

Welcome to my personal blog! My name is Nick Doelman. I am a Microsoft Business Applications MVP and I have been working with the Power Platform and Dynamics 365 since version 1.0.

I am currently a Power Platform and Dynamics 365 Freelance consultant, trainer, blogger and speaker.

This blog is mostly about the Power Platform and Dynamics 365 (formally known as CRM). I also post a lot about Power Apps Portals. Views expressed here are mine, and are not that of my current clients or Microsoft or the MVP program.

Any information or techniques described here are done at your own risk, please keep out of reach of children and pets.

Support and Help

If you have a question on any of these posts, please leave a comment. I will try my best to respond or try to point you in the right direction, but it may at times take a few days. You could also submit a support ticket with Microsoft and put in a link to my blog to help explain your issue to them (they sometimes point people to my blog posts, so its all fair game.) Thank you!