Friday, August 03, 2007

Apparently, Mike's recent vacation recharged his batteries because he's kept the promise he made in his June 22, 2007 blog.Suspend() post. He's moved on from his previous exhaustive coverage of LINQ to SQL to the more challenging task of analyzing the Entity Framework (EF) and LINQ to Entities.

LINQ to SQL is very much a consumer of ADO.NET in that it fits into the existing ADO.NET picture by sitting on top of the SQL provider (SqlClient).

The Entity Framework is a different beast in that it plugs in new bits to ADO.NET. It adds a new provider (the Entity Provider) altogether and a new language that this provider understands (Entity SQL). So, you can expect to see EntityCommand, EntityDataReader, EntityAdapter and so on.

The unusual thing about this new provider is that it's a layered provider in that its job is to sit above existing providers.

Regardless of the correctness of the [diagram], we've got a query captured as an expression tree from a LINQ perspective and then that needs to be turned into what I'd call a conceptual command tree which is a transformation from the LINQ query to an Entity Provider query dealing with the conceptual and then there's a transformation from that to a logical command tree which is then handed to the "real" provider which generates the right kind of SQL for that thing.

Based on my understanding of database-specific EntityClient providers, Mike's narrative sounds right to me, although I believe that Microsoft calls the conceptual command tree the Canonical Query Tree (CQT). There's more about QCTs and their role in query processing in Murali's August 21, 2006 Queries in ADO.NET vNext post.

An Earlier Attempt to Diagram Data Flow in the Entity Data Model

For reasons unknown, Microsoft has never published an official, detailed data-flow diagram for the EF and Entity Data Model (EDM). The Microsoft Patent Application for the Entity Framework and EDM includes several simplified diagrams (with extreme overuse of the term "rich") but nothing one could use to better understand how EF, EDM and LINQ to Entities interact.

I made a stab at diagramming the EF and EDM for my "Objectify Data With ADO.NET vNext" article in the October 2006 issue of Visual Studio Magazine. The article was based on the ADO.NET vNext August 2006 CTP, which predated the ADO.NET team's replacement of DataProviders with Entity Providers use of the term EntityClient.

Courtesy of 1105 Media, Inc.

Changing the System.Data.SqlClient Data Provider to an EntityClient Provider for SQL Server 200x legend and changing the T-SQL to eSQL legend on the arrow between the Map Provider Layer and the Data Provider box might bring this diagram into closer alignment with today's EF and EDM implementation. Comments, suggestions or both from Danny Simmons, Zlatko Michailov, or both would be appreciated.

What Entity Framework Pilgrims Need from Microsoft

Prior to Tech*Ed 2007, I made a list of what I believed was needed from the ADO.NET Team for the delayed and orphaned Entity Framework to gain any traction at all with developers of data-intensive .NET projects. I published the list in a May 29, 2007 Defining the Direction of LINQ to Entities/EDM post to which Microsoft's Mike Pizzo responded with a detailed comment.

So here's another request: Mike Taulty and I currently must guess at our data flow diagrams. It's more than about time for Microsoft to publish a detailed, annotated EF/EDM/LINQ to Entities data flow diagram with the proper names of the components. Diagrams that illustrate how EF integrates with Projects Jasper and Astoria (and why) would be useful, too.

Note: The diagrams in John Papa's ADO.NET Entity Framework Overview in MSDN Magazine's July 2007 "Data Points" column don't provide sufficient detail. However, his column does provide a summary of the Orcas Beta 1 version's features.

So far, I'd give the ADO.NET team a "C" or lower grade for responding to my requests. The Entity Framework June 2007 CTP was useful, despite its inability to run on Orcas editions other than Web Developer Express. (See my Entity Framework June 2007 CTP Visual Basic Samples post for example eSQL queries with results in DataGrid controls.)

Since the Entity Framework is now shipping separately from Orcas, the Orcas beta 2 does not include any Entity Framework bits, and unfortunately the existing CTP will also not work with beta 2. We will, however, have a beta version of the entity framework sometime in August that will work with beta 2 (and have some additional goodies <grin>).

However, it's August 3 and there's nothing official so far in the ADO.NET Team Blog about the forthcoming EF CTP or Beta, and not even a hint from Danny whether this drop will finally include a preview of the long-awaited EDM Designer.

Update 8/3/2007 1630 PDT: Mike Pizzo responds in a comment to this post.

As promised, we will have Beta releases of the Entity Framework that correspond to the Beta 2 and RTM releases of Orcas. A Beta 2 release of the Entity Framework that works with the recent release of Orcas Beta 2 will be available later this month, along with a list of the new features available in that release.

2
comments:

As usual, your summary of the Entity Framework (despite our apparent best attempts at under-documenting the architecture) are unusually accurate.

One small change in your diagram; while the application can issue eSQL queries at either the Object Services or Mapping Provider (now renamed “EntityClient”) layers, the communication between layers is always done through CQTs. So Object Services generates a CQT from either the LINQ expression or the eSQL query and passes that CQT to the EntityClient. The EntityClient expands the CQT using the mapping views and passes the expanded CQT (not T-SQL) to the store data provider (i.e., SqlClient). As Mike correctly describes in his post, only the provider knows how to generate provider-specific syntax from the canonical CQT representation.

This architecture is designed to be symmetric such that, in the future, for default mappings, the EntityClient could be removed and the ObjectServices layer could talk directly to the underlying provider.

Also, note that we didn't rename "Data Providers" to "Entity Providers", the only rename was to change the "Mapping Provider" to "EntityClient". Other providers (i.e., "SqlClient Data Provider") remain unchanged.

Mike's summary is scarily accurate, though that doesn't excuse us from posting something. I'm working on such a posting now...

The dual Web role application has been running in Microsoft's South Central US (San Antonio) data center since September 2009. I believe it is the oldest continuously running Windows Azure application.

About Me

I'm a Windows Azure Insider, a retired Windows Azure MVP, the principal developer for OakLeaf Systems and the author of 30+ books on Microsoft software. The books have more than 1.25 million English copies in print and have been translated into 20+ languages.

Full disclosure: I make part of my livelihood by writing about Microsoft products in books and for magazines. I regularly receive free evaluation software from Microsoft and press credentials for Microsoft Tech•Ed and PDC. I'm also a member of the Microsoft Partner Network.