During the writing of The Definitive Guide to DAX I wanted a simple way to analyze the content and distribution of data compressed in the VertiPaq engine, used by Analysis Services Tabular, Power Pivot and Power BI models. I always relied on BISM Memory Report (thanks Kasper!), but when you focus on a single database there are a number of details available in other data management views (DMVs) other than the one used by BISM Memory Report.

I created VertiPaq Analyzer, which is a Power Pivot data model that collects data by these other DMVs and shows them in pivot tables that provide you information about compression, size of data and related structures (such as relationships and hierarchies), and column selectivity (very important to understand how to optimize DAX queries.

You can download the workbook here, and read the article that describes all the metrics used.

After months of public preview, Microsoft today is releasing the Power BI service to general availability. The preview was really a beta that evolved in these months, and I personally liked the approach that MS developers have now. I did not read the “it’s by design” answer to the many comments and suggestions provided in support forums and community areas, I have seen a continuous commitment to help users and partners in creating analytical solutions.

You can read the official announcements and many details in the Power BI blog. In this blog post, I want to focus on Power BI Desktop (formerly known as Power BI Designer) and then make a few considerations about the entire platform. Chris Webb wrote a good comment about the role of this tool in the Microsoft BI stack. I would like to add my personal point of view about that.

Power BI Desktop is a tool that you can download for free. It includes the features of Power Query, Power Pivot, and Power View, so you can create an analytical solution that works locally without any licensing cost. Someone read this as a strange move from a company like Microsoft, but things are changing and going to the cloud is not a question of “if” but of “when”. Thus, you start using Power BI Desktop today for a local model that works only on a single desktop. Whenever you will be ready to access cloud features (including the ability to use Power BI mobile apps to navigate your data on your mobile device), you will upload it to Power BI service. Which is still for free for personal use with 1GB of data uploaded to the cloud. You start to pay only to use the Power BI Pro features, which include collaboration (sharing with other people), on-premises gateways, hourly scheduled refresh, and larger data capacity (10GB/user). The cost is 9.99$ user/month, which is a competitive price from my point of view. More details in the pricing page of Power BI site.

However, the fact that Power BI Desktop is completely free is not what is more interesting to me. What is important is that I have seen what Microsoft did in the last 9 months, and if they will continue to keep the pace of new release and improvements, this tool will be something you will be ready to pay even on a desktop, because you will depend on it. The fact that it is free is just a welcome additional benefit. Let me list a few of the important facts I see here, not all of them are so highlighted in official announcements.

New in-memory engine: Power BI Desktop runs a local instance of the Analysis Services engine, which is the same engine that also runs Power Pivot for Excel (however, the engine here had 3-4 years of evolution since the version that runs current versions of Excel and Analysis Services). An important difference is that Power Pivot is an add-in that runs in-process, and its release is managed by the Office team. Consequence: slow update cycle, priority to compatibility with older versions, more complex tests. Since Power BI Desktop has an independent release cycle and runs locally in the user context, you will be able to update it more frequently. The engine we have now is the same that runs on Excel 2016, but in reality it already exposes features that are not part of Excel 2016, such as bidirectional cross-filter for relationships in the data model. The new engine has many performance improvements, but what I like more is the ability to get improvements with monthly updates, instead of waiting for service packs of a bigger beast like Office.

New DAX: all the DAX code you wrote is good and works, but there are new DAX functions and the syntax for DAX variables which are very important to simplify DAX code and improve performance.

New graphical engine: if you used Power View in the past, or you used Power BI Designer during the preview, be prepared. Power BI Desktop has a completely new graphical engine, based on D3.js, which is amazingly fast. All the graphical components created by Microsoft are also open-source (available on GitHub), and you can extend Power BI data visualizations by extending these classes. There is food for system integrator and ISVs here, but first of all the results in terms of user experience are very good. Maybe you will find some bug and some feature to improve, but the roadmap is clear and is very good. Power BI Desktop is not only a product, it is also a platform, it can become an entire ecosystem (adoption of Power BI service and Power BI Desktop will be important for that)

Complete design experience: if you used the Power BI Designer preview, forget the limitations you had. The Power BI Desktop released now has the ability to create DAX measures, calculated columns (seeing the preview of the result of your formula in a table-view similar to Power Pivot), relationships (you have the diagram view). You can edit longer DAX expressions in a decent window with a decent editor (only one formula at a time by now, I hope they will implement a sort of complete-script editor in the future).

Ready for hybrid solutions: if you have your data on-premises in Analysis Services Tabular, you can connect Power BI directly to the service, from both Power BI Desktop (which will be only a set of reports in that scenario, without a copy of the data in the PBIX file) and Power BI on the web (so the cloud service only renders the report, but data is not persisted on cloud servers and you can log and audit all the incoming queries to your on-premises server). As soon as the same feature will be available for Analysis Services Multidimensional (Microsoft already announced it, even if we still do not have a release date), the number of semantically-rich data models ready to be used in Power BI will grow exponentially in one day. Be ready for that.

Direct publishing on Power BI: you can publish your data model and reports from Power BI Desktop to the Power BI Service by clicking on a button in the Power BI Desktop user interface. The alternative is to upload the PBIX file to the server, not a big deal, but this integration will just make it easier and faster.

Integration with other vendors: a big news is that you can publish your report also on other servers. The Power BI Desktop has a Publish button that open a list of options (see Chris Webb's post here). Power BI service is the first, and the second one will be Pyramid Server. Pyramid Analytics is the first vendor that implements the ability to publish a Power BI Desktop file to their server, which runs on-premises. This integration is not available yet and it will be released in the next few months (read more details about announcement from Microsoft and Pyramid Analytics). My understanding is that, once available, it will be possible to publish reports you design with Power BI Desktop without any access to any cloud service. I am really curious to see all the details, because you can imagine how many questions I have.

Integration with other services: the number of connectors (especially for other cloud services) available increased at a very fast pace, I have seen a new connector every week in the last months, and there are more to come. Probably Power BI is already the easiest way to get data from other cloud services and publish dashboards. One example over all is Google Analytics: Power BI is so quick and simple to get the most common used information that I use it on regular basis now, and I access to Google Analytics only when I need particular details.

As you see, this is a very technical and feature-oriented point of view. If you read between the lines, what is more important to me is that this is the foundation of a new ecosystem for business analytics. Involving partner in extending connectors, publishing, and visualizations is a key to achieve that goal. This was not here in previous versions of Power BI strongly integrated with Office 365. The connection with Office 365 is still here (you have more features available when you use One Drive for Business for navigating in Excel data models), but is not a precondition to start using the service.

This is the real challenge. Creating the reference ecosystem for Business Analytics. If you look at the market from this point of view, I would say that nobody has a clear leadership today. There are strong players in certain sectors, but nobody controls an ecosystem here. The next 12-18 months will say if the bet is right.

Microsoft is going to release the new Power BI service on Friday, July 24th. The number of new features is huge, but remember that this is just the beginning of a new wave of continuous updates, similarly to what we have seen for Power Query in the last months, just at a larger scale.

I will cover in this blog some more details about the impact of Power BI on many different point of views. In the meantime, I collected a few useful links to get some more anticipations of what is coming:

The title of this blog post is dedicated to Power BI Desktop, because I think we are going to see a first version of a complete environment to design data models, etl, and visualizations, completely detached from Excel. It will be also free, but I am more interested to the features that will be available. Having a distribution unrelated to Office will help those environments where Office update requires years...

Highlight unnatural hierarchies (read here why this is so important for performance)

A more complete description of the new features is available in the Darren's post.

I think that this release is a revolution for Power Pivot users. Until now, you had to use Analysis Services to restore a Power Pivot model and then run your query using DAX Studio to analyze performance. Now you don't need anything else other than Excel. This is amazing.

If you are wondering about how to use this feature, simply follow these steps:

The table name (Sales in this case) should correspond to a table of your model, use the table where you defined the measure originally. Now your definition of Sales Amount overrides the one of the data model in this query and you can easily change the following DAX code of the measure definition and test the entire query again (maybe clearing the cache before) until you obtain a better version. Then, simply copy the code & past it into your Power Pivot model, replacing the previous definition of the same measure.

You will see that this is way more productive than changing the code in Power Pivot and refreshing the pivot table every time!

The 2015 is a special year for Italy, because the country hosts Expo 2015, which is the current Universal Exposition. For this reason, the Italian PASS chapter promotes a special edition of SQL Saturday, a free training event for SQL Server professionals. The SQL Saturday #454 in Turin on October 10, 2015 has the following characteristics:

More than 20 sessions, on SQL Server, Business Intelligence and Azure Data Platform.

All the sessions will be in English language.

The venue is in the center of Turin, close to the train station:

You can be at the expo in 40 minute

You can travel to Milan in less than 1 hour

Turin is usually less expensive than Milan and you might stop for at least one night, dedicating the Sunday after SQL Saturday to visiting the Expo or Milan.

We want to provide the best experience to the attendees, and we also want to help those of you traveling with family and/or friends that might not interested to technical content. For this reason, we are planning a web page containing information for side and/or alternative activity during the SQL Saturday. You will get more information about that starting in July.

However, we first need a good estimation of the number of attendees, in order to correctly size the venue and to verify the interest in side activities, so we will module the time to allocate in such a section of the upcoming web site. These operations have to be completed months ahead of the event.

For this reason, we ask you to fill the survey at http://www.sqlsatexpo.com/, providing us important information about your intention of visiting Expo 2015 and about the number of people who will travel with you.

If you are a speaker, please submit your sessions, considering that the agenda will prioritize three topics: SQL Server 2016, Power BI, and Azure Data Platform.

In the last months I assisted many companies implementing solutions based on Analysis Services Tabular. There is not so much difference between the versions 2012 and 2014, because SQL Server 2014 didn’t introduce new features to the BI services. Thus, my considerations are valid for both.

One issue observed in different cases was a general performance degradation after a few days of work. Restarting the msmdsrv.exe service was enough to restore normal performance. The problem might affect both query and process operations. Microsoft released a hotfix (KB2976861) that mitigates the problem for slowness of full process, but it is not something that completely solve the problem.

The real reason of the issue is the fragmentation of the memory heap. Analysis Services can use its own heap algorithm, or the standard Windows one. It seems that the workload generated by Tabular creating objects of a dynamic size is an issue for the Windows Low-Fragmentation Heap, which is the default setting in Analysis Services (because of a better scalability).

In the Heap Memory Settings for Analysis Services Tabular 2012 / 2014 article on SQLBI you can find a complete description of the settings to control heap memory used by Analysis Services. If default values produces the symptoms described above, then consider changing them with the suggestions included in the article.

The last update of Power BI Designer allows you to create measures (not calculated columns yet). Download the new version of Power BI Designer and you will see the New Measure button. The editor is much better than anything you have seen in Excel 2010/2013, but it can be improved (larger real estate is the first request).

The real important fact is another. You have a new version of DAX in your hands. It is not just because you have a some new functions or because the engine is faster (way faster). No, the big change (which is not a breaking change, but just a new feature) are "variables". I'm not sure this is the right name, but it is the intuitive name you give to a feature where you use the keyword VAR before specifying an identifier. What are we talking about? Look at this example:

You can assign an expression to an identifier within a larger DAX expression. The evaluation context is the one where you write the definition. You can avoid repeating the same expression multiple times within the same measure, and you can simplify the writing of code avoiding too many nested evaluations and avoid using EARLIER in most of the cases. For example, consider this expression

In the last few weeks I and Teo Lachev took the ownership of DAX Editor code. It is an add-in for Visual Studio that allows you to edit DAX measures in a text editor instead of using the Measure grid. It also provides other features (such as editing measures of an online Tabular database and performing a few queries). The user interface is not so user-friendly, so I suggest you to carefully read the documentation in order to understand how you can edit DAX measures in your own Visual Studio project.

Support for MDX Script extension (useful for setting default member and custom color and format for MDX clients)

Several bug fixes

We also re-published the DAX Editor in Visual Studio Extension Gallery. If you had a previous version, please uninstall it and then install the new one. We had to change the internal VSIX ID so the upgrade to this new version doesn’t work automatically. However, once you install this new build, upgrade should be easier in future builds.

In the last months, I’ve been trying to suggest a direction for Power BI, but I start to realize that in these days Microsoft is really prioritizing features based on customer feedback. This can make things harder for a new idea to be prioritized, because nobody ask for something completely new. A few days ago, Jamie Thompson asked support for PBIX files saved on OneDrive. I think it’s a good idea, Excel is already supported there, and you can add your vote here: http://support.powerbi.com/forums/265200-power-bi/suggestions/7259274-view-pbix-files-on-onedrive-onedrive-4-business

But I want to ask community support for requesting a feature in Power BI Designer that would help thousands of ISV and millions of users. So, Microsoft, please:

You can click on the link and vote it. If you are in a hurry, just to these 2 clicks and receive a big thank you! If you have more time, Please, let me elaborate it – I will include the description I published on UserVoice adding a few comments.

Power BI Designer saves a local PBIX file, which can be a file to export data and data model – in other words, it’s a format containing a complete semantic model. All the applications that today export data in several formats (CSV, Excel, XML), might provide a richer semantic model exporting a PBIX file.

Many ISV/SI that have OLTP and other applications that stores data in some database, usually struggle to offer a compelling BI story to their customers. The smaller they are, the more they feel this pressure because probably the effort they can put in their custom software is minimal.

Today these ISV/SI integrate their solution with external vendor technologies (QlikView is a common choice here). However, the cost of such a solution for the end user is not always appealing, and for this reason the MS partner ecosystem always look for components (charts and pivot tables) to integrate in their solutions.

Providing them an easy and inexpensive way to produce PBIX files “ready to use” straight from their product/solution would provide several benefits:

Customers would have something ready to be uploaded to Power BI service

ISV/SI would be able to provide a BI solution integrated with MS ecosystem

ISV/SI can implement solutions like “send a PBIX file via mail every week to all the agents including only the data of their prospects/customers”

Today they already do that using the .CUB format, which can be consumed by both Excel and custom applications

Microsoft would increase the number of Power BI users very quickly - Small ISV/SI would be able to implement such integration very fast

What I propose to do is, in descending order of importance:

Support Power BI Designer as a local engine with an API that can be used by anyone and officially support local connections by other programs (starting from Excel)

The API should provide the ability to create a data model and to populate it with data by just using API, without any manual interaction

Providing the ability to connect from other clients (today it is possible but not officially supported) would increase the adoption.

Document and “open” the PBIX file, so that it can be generated by anyone

I think that this is easy for the data model, but not for the data.

But without the data, this model would be not so useful, requiring a manual refresh to be populated.

Move Power BI Designer to open source

Not really a priority in my opinion, but if the first two wouldn’t be possible, this one could be ok

If you think this is a good idea, here is the point. The top ideas are SQL Server on-prem and SQL Server Analysis Services cubes. Very important features. They have more than 1,000 votes. We need to go there to receive attention from Microsoft. Thus, your vote is not enough. Please, forward the message, convince other people to vote, add your comments, talk very loud.

Microsoft is prioritizing cloud services, but getting quick and large adoption for small databases of thousands of applications, each one with hundreds of users, means generating a huge volume of data models ready to use. Yes, we need DAX in Power BI Designer in order to make it useful, but we know it is already in the roadmap. Microsoft released yesterday support for Google Analytics, and by now it works only in Power BI Designer. Working in a desktop app has its own advantages. I never had a so powerful tool to navigate in Google Analytics data. I’m waiting for DAX. But being able to generate a PBIX file from within an application would be a great leverage to Power BI adoption also for people who are not ready to query data or create a new data model, but that want to analyze their data.

And, of course, if you think it’s a bad idea… comments are open, I’d like to hear other point of views.

After the initial release of DAX Studio 2.0, in December 2014, I started working on a few features to analyze performance of DAX queries. Darren Gosbell implemented many other features, and Daniele Perilli helped us with the graphics of the UI. We certainly can do more, but I think that the current feature set of the new release is good enough to use DAX Studio only during your daily job with DAX, without opening SQL Server Management Studio (SSMS) and SQL Profiler anymore.

By using DEFINE MEASURE at the beginning of the query, you can define new DAX measures and see the effects in the same MDX query

Most important, you can see the profiler events for executing DAX measures in a MDX query, which is very important for performance tuning

Connect to Power BI Designer

Just open Power BI Designer and then open DAX Studio

You can choose to connect to Power BI Designer

At the moment, we don’t support multiple instances of Power BI Designer and if you close Power BI Designer, you lose the connection without any warning.

There are also many other bug fixes and small improvements. In this release, we still didn’t solved an issue we have in getting trace events when connected to Power Pivot, but debugging work nicely when using Power BI Designer. Yes, you read it correctly!

In the last weeks of beta testing, I never used SQL Server Management Studio anymore to debug DAX measures and queries. In particular, I no longer open the SQL Profiler, unless I want to capture a query generated by other programs (this is a feature we might integrate in DAX Studio in the future, sniffing all profiler events and debug queries run by other processes).

Remember: DAX Studio is free and open source. If you want to contribute, post and vote bugs and feature requests on CodePlex. If you are also a .NET Developer, join us writing other features. There have been more than 2500 downloads in 3 months since 2.0 release. It’s a good number, but I expect to reach such a number much sooner with this release. Spread the word!

One important thing to note is that also the mobile app for iPad and iPhone are also available worldwide. Jus go in the Power BI Download page and select the app you want. For a PC, I strongly suggest to download the Power BI Designer. You will find another good reason to do that later this week on this blog... but no spoilers by now!

I published another article in the “From SQL to DAX” series I launched many months ago on www.sqlbi.com. This time, the goal is to analyze DAX alternatives for IN and EXISTS operators that you have in SQL. The name of the article is From SQL to DAX: IN and EXISTS.

I hope that a syntax corresponding to the SQL operator IN will be available in future versions of DAX, it would simplify the syntax to write for long list of values to include in a selection.

You are probably used to think to FIRSTNONBLANK and LASTNONBLANK as DAX functions related to time-intelligence calculations (usually for semi-additive measures). However, these functions have a syntax and a behavior that can be useful in other scenarios.

For example: you might be aware that MIN and MAX cannot be used with text columns. However, you can use FIRSTNONBLANK and LASTNONBLANK instead of MIN and MAX, respectively. In fact, this is the corresponding syntax:

In the last two years, I and Alberto Ferrari delivered many courses about Power Pivot, SSAS Tabular, and DAX. We had DAX content in many of the courses, and this created some overlapping and the lack of a clear guidance for studying DAX, regardless of the tool you were using.

With the upcoming release of Power BI, the number of tools where you can use DAX to write formulas is going to increase. We reviewed the structure of the courses, moving DAX in a separate course (Mastering DAX – 3 days), removing most of the DAX content from the SSAS Tabular course (now is 2 days) and creating a new course about DAX optimization (Optimizing DAX – 2 days).

Previously, we had an Advanced DAX course that included both DAX (for more complex topics) and optimization. After two years of delivery, we have seen an increased demand for a more complete DAX course (not just an advanced one), and we accumulated a lot of knowledge and content about DAX optimization. A single course of 3 days mixing all together was becoming too dense, so we extended the DAX course and moved the optimization part in a dedicated course. In the meantime, we also adapted the SSAS Tabular Workshop content, reducing its length to 2 days, because all the DAX content has been moved to the Mastering DAX workshop, and there is more time for Analysis Services specific issues, like processing, memory, partitioning, security, and so on.

This is the new learning path starting from scratch:

SSAS Tabular Workshop – covers the Tabular modeling in Microsoft SQL Server Analysis Services 2012/2014 and includes exercises to make practice with DAX and Tabular modeling.

Length: 2 days

Hands-on-labs: yes

Target: BI Architects and BI Developers that already have an experience in building Business Intelligence solutions.

Prerequisites: knowledge of SQL language, no previous experience in Analysis Services is required.

Prerequisites: experience in data modeling with Power Pivot, or Power BI, or Analysis Services. A basic knowledge of DAX syntax is suggested even if not strictly required. Attending the SSAS Tabular Workshop or Power Pivot Workshop satisfy the prerequisites.

Target: BI Developers, Advanced Power BI and Excel Users that want to optimize measures and queries written in DAX.

Prerequisites: good experience in writing DAX formulas and queries. Attending the Mastering DAX Workshop should satisfy the prerequisites. We will provide a test to check prerequisites.

We planned many courses in Europe from April to July, and when possible we scheduled 2 courses in the same week (SSAS Tabular + Mastering DAX, or Mastering DAX + Optimizing DAX), allowing a full immersion for the entire week. We are also planning more online and recorded courses later this year (and other public classroom next Fall). At the moment, the SSAS Tabular Workshop online will still include the DAX content and will be modified when we will provide a Mastering DAX course online.

Here is a list of planned courses divided by country. We’ll try to participate to user group meetings when we are visiting other countries.

As usual, if you attend one of these conferences, don’t be shy, stop us and say hello!

There is a lot of buzz going on about Power BI – we are waiting the consolidation of certain features before starting to write about it. However, thanks to Project Botticelli, you can already get a very good training with a 10% discount using the coupon SQLBI2015 to access the entire library, including the Power BI Video Course and the DAX Video Course available there.