ADO.NET Data Services: In parallel with the ASP.NET Extensions release we will also be releasing the ADO.NET Entity Framework. This provides a new modeling framework that enables developers to define a conceptual model of a database schema that closely aligns to a real world view of the information. We will also be shipping a new set of data services (codename "Astoria") that make it easy to expose REST based API endpoints from within your ASP.NET applications.

Note: Joe Wilcox says "'Astoria' will be included in next week's ASP.NET 3.5 Extensions preview release. Astoria is Microsoft's incubation project for exposing data as a service" in his Microsoft Updates Web Developer Roadmap post of November 29, 2007.

As "Something to watch," Joe adds:

Microsoft's data model emphasis, which is more than just about rich Internet applications delivery. The emphasis directionally reveals something about Microsoft's forthcoming Web services platform. Data-driven may not be a new concept, but Microsoft is positioning for a leap frog over competitors. That's topic for another post. :)

Other Elements of the ASP.NET .NET 3.5 Extensions Preview

Scott's post also states that the extensions release will include:

ASP.NET MVC: This model view controller (MVC) framework for ASP.NET provides a structured model that enables a clear separation of concerns within web applications, and makes it easier to unit test your code and support a TDD workflow. It also helps provide more control over the URLs you publish in your applications, and more control over the HTML that is emitted from them. You can learn more about it from Part 1 of my ASP.NET MVC Tutorial series. I'm hoping to find time this weekend to write and post Part 2 of the series. [Read more about the MVC Framework here.]

ASP.NET AJAX Improvements: New ASP.NET AJAX features in the ASP.NET 3.5 Extensions release will include better browser history support (back/forward button integration, and server-side history management support via a new <asp:history> server control), improved AJAX content linking support with permalinks, and additional JavaScript library improvements.

ASP.NET Dynamic Data Support: The ASP.NET 3.5 Extensions release will deliver new features that enable faster creation of data driven web sites. It provides a rich scaffolding framework, and enables rapid data driven site development using both ASP.NET WebForms and ASP.NET MVC.

ASP.NET Silverlight Support: With the ASP.NET 3.5 Extensions release we'll deliver support for easily integrating Silverlight within your ASP.NET applications. Included will be new controls that make it easy to integrate Silverlight video/media and interactive content within your sites.

Update 11/30/2007: Rick Strahl's What's Ailing ASP.NET Web Forms essay provides the view of the current state of ASP.NET and a look at the future of ASP.NET MVC. It's written by an independent Web developer with many years of ASP and ASP.NET experience.

Rob's working at Microsoft (but living on Kaua'i) and still maintaining his Weke Road personal blog. My post's "SubSonic's Continuing Open-Source Status" section described Rob's and Microsoft's commitment to continuing development of SubSonic as an open-source project.

We’re shelving the query tool update that we’ve been planning. In fact we’re shelving any further feature change/innovation in favor of moving forward with .NET 3.5.

There are soooo many language enhancements to .NET 3.5 that we have a chance to refactor and trim our feature set really nicely. For instance, our entire Sugar library can be redone as extension methods - which is what they should be.

In addition, as many know, we have a lot of love coming for the new MVC framework.

It doesn’t make sense to rev two branched versions of SubSonic that run on a different platform, so the next major version will be a version rev to 3.0, and will work on .NET 3.5 only.

This doesn’t mean we won’t support/fix/tweak/patch 2.0 - we’ll always try to make it better. But in terms of features, it’s pretty much set.

With regard to incorporating LINQ in the query tool and meshing with the MVC Framework, the "New Features for [SubSonic] 3.0" topic says:

Eric and I have been talking a lot about the query tool and how to make it work with LINQ. I think we have a nice scheme planned, so expect to see a new query tool as we’ve discussed in the past.

We’re going to throw all kinds of love at the new MVC toolset. All Kinds. It’ll be magic. And we’ll cook up some nice reference apps for you to play with. More on that later (when I figure out just what features I’ll be working on in the MVC toolset.

Judging from this post, if Dynamic Data Support is based on SubSonic and the preview arrives next week, it would probably be an unmodified or slightly modified version of SubSonic 2.0. When the bits arrive next week, I'll update this and the 11/11/2007 post.

Silverlight 1.1 Becomes Silverlight 2.0

According to Scott, the next public Silverlight preview will include major upgrades to the WPF UI framework, add a "rich set of controls," including a DataGrid, provide REST, POX, RSS, and WS-* (plus, presumably ATOM Pub), and include LINQ to XML support. The number and significance of new features warrants a full version upgrade, so Silverlight 1.1 will become Silverlight 2.0.

Microsoft plans a Silverlight 2.0 beta version for 2008Q1 with a GoLive license. VS 2008 Express Edition users will be glad to hear that the Silverlight toolset for VS 2008 will work with the free editions. (VS 2008 Professional Edition is required for the just-released Silverlight 1.1 tools.)

JD ran into a serious performance problem in his Photo FeedsFacebook app, which uses LINQ to SQL, that was causing his page to load slowly. It turned out that three page loads caused 1,176,879 calls to SqlVisitor.Visit and pinned the CPU. Compiling the query decreased load time by a factor of five and the app now "barely registers on the CPU when the page loads."

Rico says compiled queries are thread-safe, which is reassuring.

Daniel Moth Demonstratrates and Explains the Parallel Exensions

Microsoft UK's Daniel Moth posted the following short video clips and introductions to Channel9 on November 29, 2007:

In this 22:27 tour of the Parallel Extensions sample code, you will learn where to focus your energy when you explore the samples. Watch this video in order to see how Parallel Extensions to the .NET Framework fit in the large picture and what goal it aims to solve.

Achieving "Declarative Data Parallelism" with the Parallel Extensions is achieved through Parallel LINQ (PLINQ). This 18:50 video explains how to parallelise your LINQ queries, how it works under the covers and how to configure it for advanced scenarios

The .NET incarnation of the solution to "imperative data parallelism" lives in the static Parallel class of the Parallel Extensions assembly. If that sentence confuses you a bit, watch the 20:15 video for the full explanation.

Declarative data parallelism in the form of a data parallel implementation of LINQ-to-Objects. This allows you to run LINQ queries on multiple processors.

PLINQ samples include the ray-tracing project that I described in the "Luke Hoban Gets the Most Complex LINQ Query Award (and PLINQ Compatibility)" topic of my LINQ and Entity Framework Posts for 10/2/2007+ post, which also describes the Parallel FX program and task parallel library. However, there's no sign of the project described in the "Joe Cheng Solves the Wide Finder Problem with LINQ and PLINQ" topic of my LINQ and Entity Framework Posts for 10/5/2007+ post.

So far, I haven't been able to find MSDN's "Concurrent Programming Developer Center" that Soma mentions in his post.

ADO.NET Data Services: In parallel with the ASP.NET Extensions release we will also be releasing the ADO.NET Entity Framework. This provides a new modeling framework that enables developers to define a conceptual model of a database schema that closely aligns to a real world view of the information. We will also be shipping a new set of data services (codename "Astoria") that make it easy to expose REST based API endpoints from within your ASP.NET applications.

Diego Vega: Another New Entity Framework PM and Stealth Blogger

The Entity Framework (EF) team is staffing up program management in advance of the Beta 3 drop. The latest addition is Diego Vega whose been on the job but under the radar since mid-October. Diego's program area is LINQ to Entities.

Diego's latest post, Entity SQL Non-Quirkiness of November 27, 2007, describes the unwanted (but not unexpected) result of omitting the VALUE modifier in an aggregate (MAX()) subquery of an Entity SQL SELECT statement.

His first EF-related post, Choosing an Entity Framework API of November 7, 2007 contrasts the Entity Services (relational-to-entity mapper) and Object Services (object-to-entity mapper) layers, both of which accept Entity SQL Queries. Only the ObjectServices layer and its ObjectQuery type support Entity SQL and LINQ to Entities queries.

Like the other recent PM hire, Alex James (see the "Alex James Joins the ADO.NET Team as Program Manager for Metadata" topic of Link and Entity Framework Posts for 11/19/2007+), the ADO.NET team didn't announce that Diego had signed on and was blogging about EF. Thanks to Julie Lerman (again) for the heads up.

The next public preview of Silverlight will include a ton of new runtime features, as well as a significantly enhanced VS 2008 tooling support. I'll be blogging more details about this shortly.

Hopefully, the promised LINQ to XML addition will be in the next public preview.

Added: 11/27/2007

Scott Wisniewski Generates a US Map from TIGER Files, LINQ to XML, and WPF

In what's probably the most complex jobs of data binding attempted so far with LINQ to XML and WPF, the VB Team's Scott Wisniewski describes how to draw a US map in his Map LINQ: Create Dynamic Maps with Visual Basic 9.0 and WPF article for MSDN Magazine's December 2007 issue. The map includes color-coded population data for all counties in each conterminous US state. Moving the mouse over a state emphasizes its outline.

The following screen capture has been widened to more closely represent the aspect ratio of a conventional US map. Click the image to display a full-size capture.

The sample code (30 MB, expands to about 150 MB) is available for download here. After you unzip the files to their designated folders, move the PolygonData and PopulationData folders from the ...\WPFApplication1 folder to the ...\WPFApplication1\bin folder and set WPFApplication 1 as the Startup project.

The XML files (133 MB) take a while to load and process with LINQ to XML queries, so be patient after you press F5.

To mark his renewed presence on the conference circuit, Pat has posted backgrounders, abstracts, and links to slides for his two technical sessions and the closing general session at Tech*Ed 2007 Developers Barcelona:

The series is definitely worth a read, especially if you've never ridden a cruise ship.

Bill McCarthy Uncovers XAttribute Namespace Issue in LINQ to XML

Bill warns in his Don't "p" on your XAttributes post of November 26, 2007 not to assign pn (where n is an integer) as a namespace prefixe when programmatically adding attributes in LINQ to XML. You're likely to incur an exception caused by using the same namespace prefix twice.

Ken Tucker Creates an Excel Spreadsheet with the Open XML SDK and LINQ to XML

Wednesday, November 21, 2007

Windows Communication Foundation (WCF) defaults to the DataContractSerializer (DCS) for serializing services' object graph payloads to well-formed XML Infosets. DCS is capable of serializing 1:many associations of an object graph, such as Customers -> Orders -> OrderDetails, into a hierarchical XML Infoset, which has a structure that any WS-* SOAP client should be able to deserialize correctly.

An alternative to DCS is the NetDataContractSerializer (NDCS), which can serialize graphs with 1:many and many:1 (bidirectional) associations by preserving association references in ID/IDREF pairs of Id and Ref values. NDCS delivers graph fidelity at the expense of interoperability because it requires clients to support CLR types. A factor in lack of interoperability with other vendors' Web service toolkits is the complexity of schemas genrated by NDCS and Microsoft's patent that appears to cover NDCS's method of serializing/deserializing objects.

The online help topics for DCS and NDCS use trivial classes with two or three members; no help topics that I've seen demonstrate serializing generic lists. To verify that DCS and NDCS behave as expected with object graphs of moderate to high complexity generated from relational databases, I used my LINQ In-Memory Object Generation (LIMOG) utility. LIMOG autogenerates C# and VB classes designed for serialization with DCS and NDCS. LIMOG also creates object initializers to generate customized List<T> or List(Of T) instances for serialization and deserialization. The three default data sources for classes are Northwind, AdventureWorks, and AdventureWorksLT (Lite) sample databases. Click here to download the AdventureWorks (OLTP)'s entity-relationship diagram in HTML or Visio format; click here to open an HTML entity-relationship diagram of AdventureWorksLT.

Note: The original impetus for creating the LIMOG app was to generate mock objects that could substitute for entities generated by LINQ to SQL and the Entity Framework and speed unit testing. Using mock objects also frees the unit test from infrastructure dependency, which conforms to the separations of concerns principal of software design. Serializing files generated by LIMOG can provide provide payload size and complexity estimates for WS-* Web services.

Here's a screen capture of the LIMOG utility after generating the VB code for the Order class and a List(Of Order) instance with 100 members (click for full-size image):

Serializing Partial Object Graphs with DCS

DCS uses opt-in serialization; DCS serializes classes decorated with a [DataContract] attribute and properties with a [DataMember] attribute only. These two attributes have properties that let you customize the resulting document, such as specifying object and member names and member sequences. (DCS sequences properties alphabetically by default.)

By default, DCS doesn't preserve references by default so it creates duplicates of objects that normally would share a common pointer to the same object in memory (i.e., ReferenceEquals = True). This means that the default DCS implementation can't handle cyclic references (cycles) created by bidirectional 1:many and many:1 relationships. I raised this issue in my LINQ to SQL and Entity Framework XML Serialization Issues with WCF - Part 1 post of October 9, 2007.

DCS offers a PreserveObjectReferences property to solve the preceding problem. However, there are no commonly observed standards for serializing and deserializing the ID/IDREF attributes in XML Infosets required to handle bidirectional references, so Microsoft's XML team purposely made it difficult to set DCS's PreserveObjectReferences property to true.

LINQ to SQL handles this problem by not applying the <DataMember()> attribute to many:1 properties, such as Order.Customer, Order.Employee, Order.Shipper, Order_Detail.Order or Order_Detail.Product. Thus the serializer doesn't follow the paths necessary to include the Products, Categories, Suppliers, Shippers, and Employees classes in the resulting document, which has this structure:

Click here to open a 39.1 kB text file with autogenerated VB code for a set of Northwind classes suitable for serialization with DCS. (The class code is autogenerated from database metadata by a custom VB project, not by LINQ to SQL.) Notice that the [DataMember] attribute is missing from the last one to three members of several classes.

Added 12/13/2007: The C# version of the DCS-formatted class is here and NCDS-formatted class is here.

Click here to open the 125-KB, 5,012-line XML file generated from 100 Order and related Customer and Order_Detail objects by a test harness that executes the preceding functions.

Added 12/13/2007: The C# version of the three functions that generate the three-tier object graph are here and the VB version is here.

Click here to open its 69-line, easily read XSD schema generated by Visual Studio 2008's schema inference engine. (Schema files are renamed with an xml extension so they open in IE.)

EF Beta 2 doesn't serialize many:1 or 1:many associations. It's up to the programmer to write code to download EntityCollections<T>, which correspond to LINQ to SQL's EntitySets. If the client application depends upon many:1 associations, for example to replace surrogate (usually numeric) foreign key values with readable names, it must request the missing EntityReference. EF hides foreign key values whether or not they are surrogate keys, which makes reconstituting many:1 associations unnecessarily difficult.

In either case, you must regenerate the associations with code. The entity classes contain a single read-only property, MemberName As EntityCollection(Of EntityType) for 1:many associations. Many:1 associations generate a read-write MemberName As EntityType property and a MemberName As EntityReference(Of EntityType).

All three properties are decorated with <XmlIgnoreAttribute(), SoapIgnoreAttribute()> attributes and lack <DataContract()> or <DataMember()> attributes, so the WCF doesn't searialize them. Therefore, SOAP clients don't receive the information required for a SOAP request to obtain the entities.

LINQ to Objects simplifies writing the code for client-side reconstructions of associations. However, the code is very "chatty" and thus isn't well suited to service-oriented architecture.

Serializing Complete Object Graphs with NDCS

XML Infosets are well-suited to representing acyclic tree data structures, such as the preceding Customer -> Order -> Order_Detail example, but have serious problems with representing edge-labeled directed graphs that result from the cyclic references (cycles) introduced by adding many:1 relationships. Cyclic references lead to endless loops and an XML document of infinite size. NDCS handles serialization of graphs with cycles by assigning an integer ID value (Id attribute) to each unique element when it's first encountered and then substituting a REF[erence] (Ref attribute) as a pointer to the element when it's encountered again.

Code to serialize the CustomerList graph with NDCS is similar to that for DCS except that DCS specifies the generic class name in the constructor while NDCS's XML document contains the CLR type declaration (System.Collections.Generic.List`1[[NwindObjects.MainForm+Customer, NwindObjects, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]). Here's the NDCS serialization code:

It's easier to view the serialized XML Infoset than to describe its structure, so click here to open the 410-kB, 15,299-line XML file with all 2,626 references preserved. The autogenerated 39.5 kB text file for classes designed for serialization with NDCS is here.

The primary and imported XML schemas inferred for the XML document are available here (203 KB, 7,136 lines) and here (600 bytes). Whitespace added by VS 2008 contributes 90% of the primary schema's original 2 MB file size as inferred; whitespace has been removed from the linked version.

Note: This is a very large schema for a SOAP WSDL file and it uses the xs:import feature that some Web service clients don't support in WSDL files.

Using NDCS for WCS Serialization

Unfortunately, it's not so simple to substitute NDCS for WCF's default DCS on either the service or client component. The Indigo team deliberately obfuscated the substitution process by requiring additional user-written code to enter the serialization and deserialization pipelines, remove the DataContractSerializerOperationBehavior objects from and create and add NetDataContractSerializerOperationBehavior objects to each Operation (see Configuring and Extending the Runtime With Behaviors and WCF's NetDataContractSerializer by Aaron Skonnard.) Autogenerated service proxies also have problems with the service's CLR type declaration for generics (shown above).

Microsoft's party line appears to be: If we can't serialize LINQ to SQL or EF objects in an interoperable manner we'll enforce unidirectional serialization by not serializing many:1 associations (LINQ to SQL) or not serializing any associations (EF). This policy is likely to result in EF sharing LINQ to SQL's failure to provide an "out-of-the-box multi-tier story."

Note: NCDS appears to serialize NHibernate's Bag objects, which provide custom collection types that implement IList<T>. Tim Scott and Greg Banister's Remoting Using WCF and NHibernate post of May 9, 2007 describes a unit test that serializes an NHibernate.Collection.Bag MemoryStream and deserializes it back to the same type. However, this test, which is the in-memory analog of the preceding code snippets, doesn't duplicate WCF serialization. There is nothing in the post that suggests that Bags contain cyclic references. (See Frans Bouma's comment of May 28, 2007 to this post.)

Note: The preceding section was added on November 26, 2007

Probable Patent Coverage of NDCS Serialization

The process by which the NDCS converts a cyclic edge-labeled directed graph to an acyclic edge-labeled directed tree appears to be covered by US Patent 20040239674 "Modeling graphs as XML information sets and describing graphs with XML schema" published on December 2, 2004 and assigned to Microsoft Corp. The inventors are luminaries of Microsoft's early XML Web services efforts: Tim Ewald, Don Box, Keith Ballinger, Martin Gudgin and Stefan Pharies. Here's the abstract of the patent:

Systems and methods for modeling graphs as XML information sets and describing them with XML schema. An edge labeled directed graph is converted to an edge labeled directed tree representing some of the edges directly and some of the edges indirectly. A graph is completely traversed such that all nodes are visited and all edges are traversed. Nodes are included by value initially and then by reference. A schema is provided that describes the structure of an XML tree tha[t] contains graph data.

The patent provides a detailed description of the serialization process with a simple XML Infoset example.

The 3-Tier Model is just a pattern. And, like all patterns, It is meant to serve our needs, not the other way around.

He then goes on to examine the issue of the purity of separation of the data and logic layers:

Contractual purity (there is a set of well-known interfaces between the tiers)

Binary purity (the types are completely separate in each tier, *and* there is a contract between each)

and explains the advantage of abandoning binary purity in favor of contractual purity. He concludes:

The encapsulation of the data context is the creation of the logical layer.

His follow-on Encapsulation and LINQ to SQL post of November 13, 2007 provides a concrete implementation of encapsulating an IQueryable with a TableView class and an abstract class for wrapping a DataContext with a DataContextWrapper class. The article concludes with a sample implementation of DataContextWrapper.

Added: 11/25/2007

Bill McCarthy Excises LINQ to XML's Duplicate Namespace Declarations

Bill's Cleaning up your XML literal namespaces post of November 24, 2007 includes the code for a short utility program that removes redundant xmlns declarations from elements other than the root when you use the Element1.Add(Element2) method.

Added: 11/24/2007

CoDe Magazine Focuses on Data Access with LINQ and the Entity Framewok

Adam Sobieski, a Phoster Incorporated developer, currently is completing development of System.Collection and System.Collection.Generic implementations for MPI.NET, which is a high-performance message-passing interface for .NET. According to his Update: LINQ to MPI.NET post of November 24, 2007, Adam plans to write a LINQ implementation for the collections.

Here's the University of Indiana's description of MPI.NET:

MPI.NET is a high-performance, easy-to-use implementation of the Message Passing Interface (MPI) for Microsoft's .NET environment. MPI is the de facto standard for writing parallel programs running on a distributed memory system, such as a compute cluster, and is widely implemented. Most MPI implementations provide support for writing MPI programs in C, C++, and Fortran. MPI.NET provides support for all of the .NET languages (especially C#), and includes significant extensions (such as automatic serialization of objects) that make it far easier to build parallel programs that run on clusters.

Adam's interest in LINQ to MIT.NET stems from his work on distributed AI applications using C# 3.0 and .NET 3.5. According to Adam's RETE, CEP, ESP post of the same day:

There are two types of machine reasoning popular today: CEP/ESP and RETE. There’s also combinations (Bruno Berstel, Alexandre Alves). I’m thinking P2P distributed applications with RETE/ESP using LINQ programming practices will allow for rapid-prototyping of some sophisticated AI applications. I’m presently brainstorming some developer libraries in this area (Semantic Grid, Knowledge Grid).

It's not yet clear to me how LINQ to MPI.NET would fit in with the ParallelFX program and PLINQ. PLINQ is directed to multi-core processors. My PLINQ Gathers Momentum post of December 8, 2006 (last updated 7/30/2007) includes many links to early PLINQ articles.

LINQ to LLBLGen Pro is coming along nicely using the LLBLGen Pro API to emulate LINQ to SQL but with the API's database-agnostic SQL intermediate language (IL). Frans is dropping the Skip() and Take() methods in favor of the API's pageNumber and pageSize values for paging, bypassing CROSS APPLY for SelectMany(), and implementing GroupJoin() and DefaultIfEmpty().

Frans' Developing Linq to LLBLGen Pro, part 9 post of November 23, 2007 provides the details of the trade-offs in implementing the preceding Standard Query Operators (SQOs) with the LLBLGen Pro API for multiple RDBMSs.

Neither the Entity Framework's ObjectContext or LINQ to SQL's DataContext is serializable, so you can't access the ObjectContext.ObjectStateManger's or the DataContext.Table<T>'s original values for an entity.

Alex James Joins the ADO.NET Team as Program Manager for Metadata

Alex James has moved from New Zealand to Seattle to join the ADO.NET Team as Program Manager for Entity Framework (EF) metadata. His initial Introductions blog post appeared on November 9, 2007 but the other ADO.NET EF folks didn't post an announcement, so I missed it. (Thanks to Julie Lerman for the heads-up).

Alex's posting frequency bodes well for his participation in the EF community.

Added: 11/23/2007

Danny Simmons on Serializing Entity Framework Object Graphs

Serializing Entity Framework (EF) object graphs to enable multi-tier data access scenarios incurs the same problems that caused Dinesh Kulkarni to admit that LINQ to SQL Has No "Out-of-the-Box Multi-Tier Story, as discussed in my post of October 18, 2007.

By default, the current EF CTP2 doesn't serialize associated entities. This omission requires that the client to make a method call for each set of instances required and then reconstruct the relationships locally. In this January 27, 2007 Entity Web Service: Returning Category thread Danny recommends creating a facade object to contain associated objects. The thread starter uploaded the alpha 0.1 version of a facade implementation to CodePlex.

Binary serialization for EF entities preserves associated entities but is limited to remoting or services on intranets because the payload won't pass through Internet firewalls. Binary serialization creates a deep clone of associated objects; that is, references to shared objects turn into individual copies of these objects. This results in what's called a "data explosion" and very large service/remoting payloads.

Return the entire object graph with 1:many and many:1 associations intact. Such graphs generate extremely large payloads and won't serialize with the default DataContractSerializer of Window Communication Foundation (WCF) because of cyclic references (also called cycles.)

Remove the 1:many associations to prevent conflicts when multiple operations return graphs that include duplicate objects. LINQ to SQL implements this option by not serializing any 1:many associations. This is required to prevent cyclic references.

Specify subgraphs on a method-by-method basis.

To serialize complete graphs with 1:many and many:1 (bidirectional) associations intact requires a rather complex workaround for the Windows Communication Framework's default DataContractSerializer or use of the NetDataContractSerializer.

Note that neither ObjectContext nor DataContext objects are serializable, which limits their identity managers and change trackers to in-process use.

Unless the ADO.NET team chooses one of the three options for EF v1, EF won't have an "out-of-the-box multi-tier story" either.

I'm working on a post with some examples and an analysis of a choice between options 2 and 3. Click here for a preview of the Serializing Object Graphs Without and With References post of November 21, 2007 (updated 11/23 and 11/25/2007) with downloadable files containing examples of serialized XML Infosets and schemas generated by the DataContractSerializer (DCS) and NetDataContractSerializer NDCS. Test with in-memory List<T> collections show that the NDCS serializes and deserializes bidirectional associations correctly.

InterLINQ Provider Alpha 0.2 Serializes Expression Trees

Pascal Schaefer and Manuel Bauer of the Swiss Software Solution Center have posted the Alpha 0.2 release of the InterLINQ library, including sample implementations for NHibernate, LINQ to SQL and LINQ to Objects, and a help file.

InterLINQ serializes the expression tree of a LINQ query so you can transport it across machine boundaries and execute it with a LINQ implementation on a server. It's up to you how to return the query result set to the client. LINQ to SQL lets you mark its partial classes as unidirectionally serializable, so classes with m:1 EntityRefs and 1:nEntitySets lose their EntityRefs because of the circular reference.

Tip: Right-click the InterLinq.chm file, choose properties, and click the Unblock button to read the help pages.

Danny Simmons: Entity Framework Beta 3 Due in a Few Weeks

We're working hard to wrap up the last round of testing and bug fixes on the EF and the EF designer. The plan is to release beta 3 of the EF/CTP 2 of the designer in a few weeks (think "early December").

Paul Stovell Explains SyncLINQ for Real and Derived Client State

In his Why SyncLINQ should matter to you post of November 15, 2007 and follow up Three categories of SyncLINQ queries article of November 16, Paul explains how derived state differs from real state in desktop client apps. He als demonstrates how SyncLINQ keeps derived state synchronized with real state as the data underlying real state changes.

Paul's in the process of coding the first SyncLINQ iteration. He writes:

In terms of right now, I have just implemented the first set of aggregate extensions, and I have a basic dependency system working. Over the coming week I’ll be working to implement an operator, and refactor much of the dependency system to support the overloads above.

His Introducing SyncLINQ post of October 16, 2007 includes a seven-minute video clip of SyncLINQ in action.

Note: There's still no library to download.

Mary Jo Foley: VS 2008 Rumored to be Available Monday 11/19/2007

eWeek's Daryl K. Taft quotes the MSDN Subscriptions WebLog post of November 16, 2007 in his Visual Studio 2008 Is Imminent post of November 17, 2007 as stating "Visual Studio 2008 is anticipated out early next week, with availability for subscribers." However, on November 18, 2007, the posts first sentence read "Visual Studio 2008 is anticipated out very soon, with availability for Subscribers." I assume the blog author edited the post without revision marks.

Hopefully, the LINQ to Objects, LINQ to XML and LINQ to DataSets teams will post Beta 2 to RTM changes, if any, soon.

Update 11/19/2007 0700 PST: The rumors were correct. Microsoft made the download available but the MSDN servers were throwing "Server Error in '/' Application. The file '/home.aspx' has not been pre-compiled, and cannot be requested." errors from the entry page for the download: https://msdn.one.microsoft.com/home.aspx?ApplicationID ..., which is not a good sign. Subsequently, the error messages changed to the more common "Server too busy."

If your DBA has set up Relationships from Hell as described in Danny's An Interesting Relationship Mapping Scenario post of November 17, 2007 there might be a solution that works with Entity Framework v1, but it's more likely that you'll need to model a containment relationship with v2 (or later).

The many:one relationships Danny describes are defined by foreign key and type column values. The foreign key value points to the primary key of a table that's selected from a group of tables by a value in a type column. Only one many:one relationship occurs at any instance.

Danny describes a containment relationship thusly:

The basic idea of this construct is that "child" entities may be related to one or more "parent" entities but only one parent at a time.

The relationship Danny describes doesn't fit with my understanding of a containment relationship, which is more in tune with that described in the Object Containment topic of class notes for a Database Systems and Structures class at Simon Fraser University.

LINQ to Entities Variables May be More Variable Than Expected

When a local variable is used in a lambda expression it behaves as a parameter, i.e. it’s value is re-evaluated each time the query is executed; when a variable is used in a static expression, it’s value is picked at compile-time, and then it’s used as a constant.

Zlatko provides sample code that demonstrates this problem.

Entity Framework Beta 3 for VS 2008 RTM will provide a CompiledQuery.Compile((ObjectContext, variable1, variable2 ...) => (QueryExpression)) method that results in a query in which all local variables used as constants retain their compile-time values during multiple query executions.

Zlatko's November 16, 2007 How to Parse an EntityDataReader post describes a class and methods that "'visit' an arbitrary EntityDataReader to obtain the metadata" for determining the built-in type. ("Visit" is appropriate because Zlatko's code emulates a simple implementation of the visitor pattern.)

The complexity of this process might convince you to take advantage of the object layer and, potentially, LINQ to Entities with complex entity graphs.

Added: 11/16/2007 16:30 PST

Rick Strahl Does C# 3.0 Anonymous Types

Rick's Anonymous Types in C# 3.0 post of November 15, 2007 explains that the anonymous types you create on the fly with the var keyword are immutable and have a scope that's limited to the currently executed method.

Rick didn't mention that anonymous types will infer field names. Writing var anon = new {DateTime.Now, SomeField=12};causes Intellisense to treat Now as the field name.

VB 9.0 also supports anonymous types using With syntax such as: Dim anon = New With {DateTime.Now, .SomeField = 12}

Ben Hall Unit Tests LINQ to SQL with MbUnit and TypeMock

Ben starts his How To Unit Test - Linq to SQL and Mocking post of November 15, 2007 by running tests on the DataContext with MbUnit from a ProductController, then adds a layer called LinqProductRepository between the the DataContext and ProductController, which he test with a live database connection.

He then sets up an interface and a MockRepository with RhinoMocks and runs tests against the ProductController with lists created by C# collection initializers.

Finally Ben tests TypeMock, which claims to be able to mock any object. The Mocking Linq - Preview post of August 19, 2007 by Eli Lopian, TypeMock's developer, shows how TypeMock can Mock an entire LINQ to SQL statement and return a fake Customers collection.

As usual, Ben provides the source code for this three mocking projects.

Added: 11/15/2007 17:00 PST

Update: 11/16/2007: Eric Hauser's Building a Generic DAO / Repository with LINQ post of November 8, 2007 contains code for an IRepository interface and its unit-testable implementation for the DataContext. The Repository class is a bit more filled out than Ben's.

IBM Announces "Blue Cloud," a Large-Scale Astoria Clone

Not to be outdone by Microsoft's Astoria and its RESTful "data in the cloud" capabilities, IBM announced from Armonk (NY) and Shanghai (China) on November 15, 2007 its "Blue Cloud" technology "will allow corporate data centers to operate more like the Internet by enabling computing across a distributed, globally accessible fabric of resources, rather than on local machines or remote server farms."

The press release names a single customer, the Vietnamese Ministry of Science and Technology, as having "this week announced a cloud computing project with IBM." Reading below the fold shows that the project is a simple Internet portal.

The way I read the tea leaves, corporate data centers presently are allowed to do what Blue Cloud purports to enable. I also question whether science and technology in Vietnam has reached a scale that requires giving "citizens, communities and government organizations" access to this "rich content source" by "a distributed, globally accessible fabric of resources." A 1U Dell server with ASP.NET and Astoria would probably satisfy their needs.

According to the press release, "IBM’s first Blue Cloud offerings are expected to be available to customers in the spring of 2008." I doubt that IBM will need to hire extra security folks to quell the crowds surging to sign up when Blue Cloud RTMs.

Added: 11/15/2007

Architectural Point of View for LINQ-Enabled Data Access Video

Architectural evangelist Bill Zack delivers an 11:47 Architectural Point of View (arcPOV) presentation on LINQ to SQL, LINQ to DataSets, and LINQ to Entities. Here's the Channel9 description:

Bill Zack presents an Architect Point of View (arcPOV) presentation on selecting the right LINQ model for your application. Bill covers 3 different flavors of LINQ; LINQ to SQL, LINQ to Datasets, and LINQ to Entities. Bill discusses how these flavors of LINQ can be used to access data in relational databases. He then covers the differences and the trade-offs involved in choosing between them.

Added: 11/15/2007

NLinq for .NET 2.0 Beta 2 Released

NLinq is an open-source (MIT-licensed) implementation of LINQ to Objects that runs under VS 2005 and .NET 2.0. NLinq accepts LINQ queries in a literal string, so you can emulate dynamic queries. It's not clear who needs this functionality but if anyone does, the source code is available from CodePlex.

Added: 11/14/2007

LINQ to Lucene 2.1 Query Version Released

LINQ to Lucene, an open-source LINQ implementation for the open-source Lucene.Net 2.0 full-text search engine reached release 4, where it's capable of querying Lucene indexes, on November 13, 2007. The source code is based on Visual Studio 2008 Beta 2, licensed under the Microsoft Permissive License (Ms-PL, no version number) and contains Unit Tests for the MSTest implementation that's included with VS 2008 Professional Edition.

According to Locksley, the project's coordinator:

The goal of LINQ to Lucene is to provide developers with the ability to enjoy full-text searching using a fast-proven search-engine within the .Net managed CLR. The plan is to get the LINQ to Lucene code-base pushed up to CodePlex for additional developers to generate feed-back and improve and enhance.

Note: Dare Obasanjo reported in his RSS Bandit v1.5.0.17 Released post of September 16, 2007 "a ton of stability issues related to our usage of Lucene for search engine for our full-text search" feature in the RSS Bandit reader. This is not to say that LINQ to Lucene is unstable but it's something to watch for.

Persisting business objects to SQL Server 2005 or later with object/relational mapping with LINQ to SQL can exact a performance penalty. Learn how to take maximum advantage of LINQ's strongly typed query capabilities without overtaxing the database server or losing front-end agility.

The story has tables of relative performance data for lazy versus eager loading of entities of varying sizes, with and without associated EntitySets. In all cases, DataSets beat LINQ to SQL times to load DataGridView controls via BindingSource components. The sample test harness code is available for downloading.

You'll notice that 1105 Media's Redmond Media Group (RMG) has updated the magazine's site design, which looks 20 years younger than it's FTPOnline predecessor.

Scott's detailed ASP.NET MVC Framework (Part 1) post of 11/13/2007 describes the ASP.NET team's forthcoming Model-View-Controller (MVC) Framework for Visual Studio 2008 and .NET 3.5. His sample project is the starting point for a simple e-commerce storefront site based on the Northwind Products table.

The post, which I estimate is at least 15 feet long, describes MVC architecture and its benefits, especially separation of concerns, and then continues with a detailed description of building the MVC Framework starting with the ASP.NET MVC Web Application template. The template adds Models, Views, and Controller folders to the main project and a ProjectNameTestProject folder for adding unit tests of the controller with MSTest. (VS 2008 Profession Edition includes the MSTest client).

Scott then derives a ProductsController class from the Framework's new System.Web.MVC.Controller class, which contains helper methods and added controller functionality. For this tutorial, Scott uses LINQ to SQL as the object/relational mapping (O/RM) tool and data access layer (DAL), although he says:

The ASP.NET MVC Framework enables you to use any data access pattern or framework you want in order to retrieve and manage your models. If you want to use ADO.NET DataSets/DataReaders (or abstractions built on top of them) you can. If you prefer to use an object relational mapper (ORM) like NHibernate, LLBLGen, WilsonORMapper, LINQ to SQL/LINQ to Entities you can absolutely use those as well.

Scott mentions that the new System.Web.IHttpRequest and System.Web.IHttpResponse interfaces will be easy to mock. It will be interesting to see how his team mocks LINQ to SQL's DataContext, especially for shopping cart updates.

Overall, it looks to me as if the ASP.NET team has come up with an excellent MVC framework starter edition, but it seems to me that it needs SubSonic's (or someone else's) scaffolding, migrations, and RDBMS-agnostic persistence layer to compete with Ruby on Rails.

What I see is a framework that might very nicely help me build a RESTful service. It does a nice job (and presumably a very flexible job) of routing URLs and query strings through to my classes and then we go get some data and invoke the view. In Scott's case, the view is an ASPX page producing HTML but I might equally want to invoke something to produce me some XML or JSON data.

I'm not sure whether the framework comes with a mechanism for routing through to ASMX or not but, either way, it looks like it would be easy to take this and make use of it in a service context.

Interesting idea.

Dot Net Rocks! Interviews Pablo Castro About Astoria

Show #289 pits the Technical Lead of the Entity Framework and Astoria against Carl Franklin and Richard Campbell in a 72-minute interview described as follows:

Pablo Castro from the SQL Server Product Group at Microsoft talks about his work with Astoria, an infrastructure for bringing web technologies and data sources together. Astoria uses the Entity Framework, which Pablo is also involved with.

The question of the week: Has Pablo had his picture taken since High School?

Added: 11/13/2007 17:00 PST

Surprise: Don Box Likes VB

Yes Steve, I've Tried (11/12/2007): Personally, my dream stack would be ubiquitous WS-Security/WS-Trust over HTTP GET and POST and tossing out WSDL in favor of doing direct XML programming against payloads from VB9 (or XQuery), but hey, I have unusual tastes.

Lambda in VB (11/11/2007): I'm starting to write more VB code lately. The XML support rocks.

More importantly, a huge amount of people who use the technology I work on see it through that lens, so I'd be an idiot not to walk in their shoes as much as possible.

Query syntax in VB (11/12/2007): "One other feature I'm liking in VB is that you can omit the trailing [S]elect clause in a query."

Another reason for Bart writing a LINQ to SharePoint implementation might be its current and future ubiquity. According to Joe Wilcox's report, Ballmer: Advertising Is the Future, on the Microsoft shareholders' meeting held today:

Gates spoke about future investments and made absolutely clear that SharePoint, with "over 100 million customers," would anchor Microsoft's server applications portfolio. He likened SharePoint to Office, asserting that SharePoint would similarly take a "central position as a key tool."

Beth Massi Fixes Her LINQ to XML VS Tip of the Day Browser App

In this brief Merging XML using Visual Basic 9 post of November 13, 2007, Beth discovers that MSDN blogs' Rss.xml files only contain the last 15 posts, so she adapts the LINQ to XML code to merge the current content with that stored on disk to maintain a complete archive.

Mike Taulty: LINQ's Not a Fit with SQL Server's SQLCLR

I'm not sure why one would want to write an SQLCLR stored proc, function, or whatever that uses LINQ to Something queries, but Mike says it can't be done in his LINQ and SQLCLR post of November 13, 2007.

Mike's attempt to load System.Core.dll in SQL Server failed SAFE verification and he knows of no plans to make LINQ run with SQL Server 2008's SQLCLR feature.

Added: 11/13/2007

Mike Taulty Draws a XAML Sine Wave with LINQ to XML

Mike was more successful in demonstrating that you can write XAML with LINQ to XML.

The dual Web role application has been running in Microsoft's South Central US (San Antonio) data center since September 2009. I believe it is the oldest continuously running Windows Azure application.

About Me

I'm a Windows Azure Insider, a retired Windows Azure MVP, the principal developer for OakLeaf Systems and the author of 30+ books on Microsoft software. The books have more than 1.25 million English copies in print and have been translated into 20+ languages.

Full disclosure: I make part of my livelihood by writing about Microsoft products in books and for magazines. I regularly receive free evaluation software from Microsoft and press credentials for Microsoft Tech•Ed and PDC. I'm also a member of the Microsoft Partner Network.