Category: Azure

I received a question today via Twitter, and although I know the information needed to answer it is available online, I don’t believe there’s a single concise answer anywhere[1]. This is the question, along with a brief elaboration following my initial response:

Here’s the short answer: When you use an organizational ADLSg2 account to store dataflow data, your Azure subscription will be billed for any storage and egress based on however Azure billing works[2].

Here’s the longer answer:

Power BI dataflows data counts against the same limits as Power BI datasets. Each Pro license grants 10 GB of storage, and a Premium capacity node includes 100 TB of storage.

Integrating Power BI dataflows with ADLSg2 is notlimited to Power BI Premium.

When you’re using Power BI dataflows in their default configuration, dataflow data is stored to this Power BI storage, and counts against the appropriate quota.

When dataflow data is saved to Power BI storage, it can only be accessed by Power BI – no other services or applications can read the data.

When you configure your dataflows to use an organizational ADLSg2 account, the dataflow data is saved to the Azure resource you specify, and not to the Power BI storage, so it doesn’t count against the Pro or Premium storage quota. This is particularly significant when you’re not using Power BI Premium, as ADLSg2 storage will scale to support any scenario, and not be limited by the 10 GB Pro storage limit.

When dataflow data is saved to ADLSg2, the CDM folders can be accessed by any authorized client via Azure APIs, and by Power BI as dataflow entities. This is particularly valuable for enabling collaboration between analysts and other Power BI users, and data scientists and other data professionals using Azure tools.

Hopefully this will help clear things up. If you have any questions, please let me know!

[1] Please note that I didn’t actually go looking to make sure, because I was feeling lazy and needed an excuse to blog about something vaguely technical.

[2] I add that final qualifier because I am not an authority on Azure or Power BI billing, or on licensing of any sort. For any specific information on licensing or billing, please look elsewhere for expert advice, because you won’t find it here.

In the last few weeks I’ve seen a spike in questions related to the integration of Power BI dataflows and Azure Data Lake Storage Gen2. Here’s a quick “status check” on the current state of this feature to get the answers out for as many people as possible.

Power BI dataflows are generally available (GA) for capabilities that use the built-in Power BI-managed storage.

Power BI dataflows integration with Azure is currently in preview – this includes the “Bring your own storage account” capabilities where you can configure Power BI to use your ADLSg2 storage account for dataflows storage, instead of using the built-in Power BI-managed storage.

Only a single ADLSg2 storage account can be configured for a Power BI tenant.

The storage account, once configured, cannot be changed.

The setup process to connect Power BI with ADLSg2 is somewhat lengthy and step-intensive.

To grant users other than the owner of the dataflow access to the dataflow in Power BI, you must grant them permissions to access the workspace in Power BI and grant them access to the CDM folder in ADLSg2.

These limitations will be addressed when this capability hits GA, but you should definitely be aware of them in the meantime. (You may also want to take a look at this MBAS session for an overview of the roadmap for the rest of this calendar year.)

I’ve seen customers take different approaches:

Some customers delay their integration of Power BI and ADLSg2, and are waiting for these limitations to be removed before they move forward.

Some customers adopt within the constraints of the preview, and choose a workload or set of workloads where the current limitations are acceptable.

Some customers set up a demo tenant of Power BI and use it to test and validate the ADLSg2 integration while deciding on option 1 or option 2.

I hope this helps. If you or your customers have any questions on this topic that aren’t answered here, please let me know!

[1] And they’re all documented. Nothing in this blog post is new, but hopefully it will help to have this summary online and sharable.

Last week Microsoft held its annual Microsoft Business Applications Summit (MBAS) event in Atlanta. This two-day technical conference covers the whole Business Applications platform – including Dynamics, PowerApps, and Flow – and not just Power BI, but there was a ton of great Power BI content to be had. Now that the event is over, the session recordings and resources are available to everyone.

This session is probably my favorite dataflows session from any conference. This is a deep dive into the dataflows architecture, including the brand-new-in-preview compute engine for performance and scale.

Common Data Model sessions

As you know, Power BI dataflows build on CDM and CDM folders. As you probably know, CDM isn’t just about Power BI – it’s a major area of investment across Azure data services as well. The session lineup at MBAS reflected this importance with three dedicated CDM sessions.

This session covers how CDM and CDM folders are used in Power BI and Azure data services. If you’ve been following dataflows and CDM closely over the past six months much of this session might be review, but it’s an excellent “deep overview” nonetheless.

This session is probably the single best resource on CDM available today. The presenters are the key technical team behind CDM, and goes into details and concepts that aren’t available in any other presentation I’ve found. I’ve been following CDM pretty closely for the past year or more, and I learned a lot from this session. You probably will too.

[1] I have a list of a dozen or more sessions that I want to watch, and only a few of them are dataflows-centric. If you look through the catalog you’ll likely find some unexpected gems.

[2] If this is all you need to know, why do we have these other two sessions?

[3] Including Jeff Bernhardt, the architect behind CDM. Jeff doesn’t have the rock star reputation he deserves, but he’s been instrumental in the design and implementation of many of the products and services on which I’ve built my career. Any time Jeff is talking, I make a point to listen closely.

Last week I delivered two online sessions on the topic of integrating Power BI dataflows with an organizational Azure Data Lake Storage Gen2 storage account. I’ve blogged on this topic before (link | link | link | link) but sometimes a presentation and demo is worth a thousand words.

On April 30th I delivered a “Power BI dataflows 201 – beyond the basics” session for the PASS Business Intelligence Virtual Chapter. The session recording is online here, and you can download the slides here.

On May 4th I delivered a “Integrating Power BI and Azure Data Lake with dataflows and CDM Folders” session for the SQL Saturday community event in Stockholm, Sweden. I was originally planning to deliver the Stockholm session in person, but due to circumstances beyond my control[1] I ended up presenting remotely, which meant that I could more easily record the session. The session recording is online here, and you can download the slides here.

Each of these sessions covers much of the same material. The Stockholm presentation got off to a bit rocky start[2] but it gets smoother after the first few minutes.

Please feel free to use these slides for your own presentations if they’re useful. And please let me know if you have any questions!

[1] I forgot to book flights. Seriously, I thought I had booked flights in February when I committed to speaking, and by the time I realized that I had not booked them, they were way out of my budget. This was not my finest hour.

[2] The presentation was scheduled to start at 6:00 AM, so I got up at 4:00 and came into the office to review and prepare. Instead I spent the 90 minutes before the session start time fighting with PC issues and got everything working less than a minute before 6:00. I can’t remember ever coming in quite this hot…

After having a prolific first few months with this blog, the year-end holidays disrupted my routine, and I’ve been struggling to get everything balanced again. 2019 has been great so far, but its also been stupid crazy busy, and while blogging has been on my backlog, it just hasn’t made the bar for implementation.

Until now, I guess…

Last week I was in Darmstadt, Germany for the SQL Server Konferenz 2019 event, where I presented on Power BI dataflows. My session was well-attended and well-received[1] but I realized that I’d never actually told anyone about it. Now it’s time to correct this oversight for some upcoming events. These are the public events where I’ll be speaking over the next few months:

Pro tip: If you’re attending Intelligent Cloud, be sure to attend Wolfgang Strasser‘s “Let your data flow” session earlier in the day. This session will provide a great introduction to Power BI dataflows and will provide the prerequisite knowledge necessary to get the most out of my session.

Gabi and Oliver presented last month at the PASS Summit conference in Seattle on this very topic. Their session “Azure Analysis Services or Power BI – Which service fits you best?” looked at the history of the two services, their current capabilities and strengths, and future capabilities that Microsoft has announced in its business intelligence release notes. They even included a decision flowchart!

If you weren’t able to attend their session in November, I have good news and I have bad news and I have more good news.

If you’re in a position where you need to select a BI platform, I highly recommend checking out these resources, as they includes both valuable information, and a framework for using that information in different common scenarios.

And if after reading these posts and this slide deck you still have unanswered questions, please seek professional help. Specifically, please find a Microsoft Partner who specializes in business intelligence, or a similar expert consultant who can help evaluate your specific needs and how the different available technical options can be applied to address them.

[1] The conference had many awesome sessions, so this should not be the only one you’re interested in.