Topics

Featured in Development

Understandability is the concept that a system should be presented so that an engineer can easily comprehend it. The more understandable a system is, the easier it will be for engineers to change it in a predictable and safe manner. A system is understandable if it meets the following criteria: complete, concise, clear, and organized.

Featured in Architecture & Design

Sonali Sharma and Shriya Arora describe how Netflix solved a complex join of two high-volume event streams using Flink. They also talk about managing out of order events and processing late arriving data, exploring keyed state for maintaining large state, fault tolerance of a stateful application, strategies for failure recovery, data validation batch vs streaming, and more.

Featured in Culture & Methods

Tim Cochran presents research gathered from ThoughtWorks' varied clients and projects, and shows some of the metrics their teams have identified as guides to creating the platform and the culture for high performing teams.

Microsoft Answers “Top 10 Questions on Data”

Microsoft has answered what they call “Top Ten Questions on Data”, explaining what has happened or it is going to happen to Oslo, ADO.NET Data Services, WCF, LINQ to SQL, T-SQL and other technologies.

Oslo

According to Microsoft, the “Oslo” code name is no longer used for the group of technologies it represented, but “Microsoft is still committed to their development”. The current name is SQL Server Modeling CTP, and it will be incorporated into SQL Server because its affinity, especially Quadrant and Repository – now called SQL Server Modeling Services -, with SQL-related technologies.

Microsoft also explains the relationship between SQL Server Modeling and .NET: it makes it easier for developers to create model-driven applications.

ADO.NET Data Services and .NET RIA Services

ADO.NET Data Services has become WCF Data Services while .NET RIA Services now is WCF RIA Services. The purpose was to make WCF a “’one stop shop’ for building services and n-tier applications”, and ADO.NET Data Services and .NET RIA Services complement WCF in creating such applications.

LINQ to SQL

After noticing that LINQ to SQL’s development continued in .NET 4.0, Microsoft acknowledges that:

We do, however, expect that the bulk of our overall investment will be in the Entity Framework, as this framework is built around the Entity Data Model (EDM). EDM represents a key strategic direction for Microsoft that spans many of our products, including SQL Server, .NET, and Visual Studio.

The message is clear: LINQ to SQL is not to be considered when making plans for the future.

“M” Language

Microsoft has an ambiguous position on this. Firstly, they say M will be incorporated into SQL Server as the other Oslo technologies:

As for the code name “M” modeling language, it’s important to understand that SQL Server is a much broader product than just its core engine—it contains many other services and tools. The “M” language and its associated tools will become part of that collection and thus be available to software that uses the SQL Server product (in its broadest definition).

Later, they say:

Microsoft is also presently aligning “M” with the Entity Data Model as used by the Entity Framework and Data Services. This will result in one common data model with two alternate representations: CSDL, the current XML-based language for the Entity Data Model that’s best suited for interoperability, and “M”, which is better suited for programmers.

It is not very clear which way M will go. Perhaps, M will be included in SQL Server but it will be transformed in order to be aligned with the Entity Data Model.

“M” vs. T-SQL

Since M is not a released product, Microsoft advices on using T-SQL for now, but that is going to change in the future:

Once “M” ships, the goal is to use “M” to describe a problem domain in a higher-level of abstraction. Then, “M” can be translated into T-SQL, C# or other formats, enabling further optimizations in the native target runtime.

WCF Data Services

Microsoft emphasizes some of the new WCF Data Services features already present in .NET 4.0 and VS 2010:

Microsoft also explains what is Quadrant – “a tool for viewing, querying, and editing data in a SQL database with a variety of built-in viewers such as tree, list, table, and master/detail”, and what is the Open Data Protocol (OData), a topic covered by InfoQ in the past.

LINQ to SQL is not to be considered when making plans for the future.

Your message is awaiting moderation. Thank you for participating in the discussion.

I don't think that this message is so clear. L2S is a good LINQ query engine for SQL server with a leightweight ORM. If that's all you need, it's going to be easier to use than EF. That's its strength, so it wouldn't make any sense to cram more features into it.

Same message as the last 10 years

Your message is awaiting moderation. Thank you for participating in the discussion.

In my opinion, the message on the data strategy is still the same it has been all along.

The server is great and will get better. .NET is great and will get better. The data access strategies are great, but Microsoft will continue to change its mind so often as to make them a huge moving target, thus even if some are the best thing since sliced bread, you cannot use them because you can't even have the slightest expectation of what Microsoft will shoot at tomorrow.

Take datasets for example. The idea behind them is absolutely great, and while many don't realize it (because no one uses them right =P), the are in many way revolutionary. They were a new way of thinking at the time though, so obviously there's a bunch of design mistakes in them. It could be fixed, but will it? Nope. New data access strategy from the ground up, with marginal effort on datasets (LINQ to Dataset). 10 years of development on Datasets could have kept them current, removed their shortcoming, etc. R.I.P datasets.

All these techs will go the same way. So while i love the Entity Framework (well, since 4.0 anyway), LINQ to SQL, ADO.NET Data services, and all of them...I can't build a lasting solution on them.

Re: It seems MS will never recognize efforts from open source community.

Your message is awaiting moderation. Thank you for participating in the discussion.

to be fair, they're changing a bit. See: JQuery.

Entity Framework, IMO, is mostly a reaction to all the MS shops that will not use a data access solution if its not Microsoft branded. NHibernate doesn't help since the learning curve and iffy documentation really stings (we use NHibernate and I -love- it, but it has some serious issues). LLBLGEN doesn't have that issue, but it isn't free :)

NAnt doesn't have the following Ant does, so I can see where MSBuild comes from. But they're slowly pulling a "Dataset" stunt on that one: TFS Build 2010 is Windows Workflow Foundation based (it does use MSBuild, but its delegated to the back, like it is in Visual Studio, and a lot of people don't even realize its there...)

Re: LINQ to SQL is not to be considered when making plans for the future.

Your message is awaiting moderation. Thank you for participating in the discussion.

There are a lot of developers who will howl in rage at this suggestion.

There was a famous petition from the Beta developers of Linq to EF, that basically said "use is sparingly and with caution". There were a bunch of issues, including that it's canonical / Least-Common-Denominator approach to the "intermediate" query language (suits everthing from SQL server, to XML and beyond) resulted in unnecessary bad performance. It's still online.

Some developers become advocates for L2EF others abandon it early and some have even converted to L2EF then gone back to L2S for performance reasons.

I don't have an up-to-date re-analysis of everything since .NET 4 was released but this isn't clear cut at all. L2S got updated for .NET 4 so reports of it's death are greatly exaggerated.

Then there's things like Plinqo. In that (and other products) third parties have taken L2S and added functionality that MS didn't. It might cost a bit more but there's an ardent band of smart people out there supporting L2S. Some corporations depend on it. Death is unlikely any time soon!

Just give it a try with something like LinqPad to give you a quick, thoroughly satisfying development experience. You'll see why Matt Warren's creation might just be here to stay. (And then Mono is seeking to emulate his long and sophisticated processing pipeline too. Taking this great implementation to a wider audience.)

(Of course if you want performance you need profiling and quite likely T-SQL / M / Whatever. You can integrate that into the LINQy things without throwing it all away. Thereby taking a possible performance dog and turning it into a performant solution, if that's what you need!)

Lastly the fact that L2S only does SQL Server, is not intrinsic in the design. It's a political / funding decision. (Matt has good posts about how you can plug your own data provider into this sort of thing.)

Re: LINQ to SQL is not to be considered when making plans for the future.

Your message is awaiting moderation. Thank you for participating in the discussion.

I was not saying L2S would die, quite to the contrary: It has found it's spot, but it's not likely to aggregate many new features as that would put it in competition with EF. Which would definately be a bad thing, because then nobody would dare to bet their money on either one, assuming that eventually one will win and one will die.

If you don't like EF, fine, neither do I. There are good alternatives out there, ranging from DbLinq to NHibernate in the free world, plus a whole range of commercial ones.

The main problem of L2S is its dependence on MS SQL Server. The way things are going today, I'm surprised it hasn't been packaged with SQL server. Not that it would make any difference at that point.

Re: It seems MS will never recognize efforts from open source community.

Your message is awaiting moderation. Thank you for participating in the discussion.

MS have said that a reason why they provide alternatives to open source implementations is that many of their customers will not consider a technology at all unless Microsoft provides it. I work as a contract developer and I have certainly encountered this attitude at some client sites.