Is SQL on the way out?

0

Ok, so I will be the first to admit that, in language definition terms, SQL is horrible. Having written a code completion engine for it, I think I can say in a reasonably qualified form that the language specification leaves a lot to be desired. Lexing is OK (despite a few weird rules) but Parsing SQL is really hard.

So along comes LINQ. For all the language definition reasons, it's way ahead, as it would be, it's a much newer language.

But I have to say, I just don't like it. While the language definition is much better, I don't think that it reads nearly as well. Sure, for a simple 'let's get one row back from the DB' then it's perfectly understandable - but I would imagine that most people here tend to do a lot more than that while generating reports etc.

The other thing that scares me about it is the 'trendiness' factor. It's like you get kudos for using LINQ in whatever situation in some circles - and, in my experience, that has always lead a good technology to become a 'band-aid' solution which is used everywhere, no matter how inappropriate. Given how much I've learned about query optimisation through participation here (and being more of a 'lurker' on the main SSC site) I don't want the opportunity to exercise that knowledge to disappear - because, for the main part, efficient queries come from really understanding the data at hand, and I don't think it's possible for LINQ to have that level of understanding.

What are your thoughts? And apologies that this is a 'no-correct-answer' sort of question, accept for highest voted answer in 7 days...

For so many reasons, like Håkan mentions, SQL will not succumb to LINQ, or any ORM.

I'm not saying SQL won't change, but the fundamentals seem so firmly based and have stood the test of time, that it seems unlikely. What may change is the data model that gets used. Relational data isn't perfect by any stretch, but fits more scenarios than hierarchical data models, network data-models, eav models and whatever else has been tried over the last 30 years or so.

I accept that as a SQL professional, I am defensive about this, but to be honest every time another trendy alternative comes along, I just see it as an opportunity in-waiting, as surely a number of months/years down the line, the 'trendy' solution just can't cope and the data layer needs re-writing using good old SQL.

As long as we have structured data, there's going to be a need for a language to query against it. That's not to say that SQL in it's current form is the way to go, but some type of language that is capable of dealing with pulling data out of relations within structured data is needed, and so far, none of the alternatives is more workable than SQL, in fact, most are less workable. Both literally and figuratively, in terms of an alternative to SQL, I'm from Missouri... Show me.

Until I see a fully functional alternative, I'm going to stick with SQL. That means that all the problematic ORM tools are going to remain, to a small degree, a thorn in my side. They don't deal well with SQL or relationships. They are, in my opinion, an attempt to ignore SQL, not replace it or deal with it. That, unfortunately, is the worst possible approach. Better to deal with it badly than try to pretend it doesn't exist and then generate all the horrors that I've seen & heard about.

However, get me a better method of querying the database, and I'm all for it. There's no reason to be married to a language. Based on the last 60 years of computing history and my own 20 years within it, we will get a different language eventually. Languages, OS's and platforms come & go. Nothing is carved in stone. But, at this time, there's nothing viable as an alternative that I can see.

I have seen so many bad SQL statements written by developers and LINQ scares the hell out of me. I am afraid that with LINQ, even more developers will think that they can develop efficient database driven applications, and ignore the need for an experienced database developer, and in the end when the developers have left the building (consultants) the database performance will slowly degrades until the system is useless.

By then, when a DBA is involved trying to optimize the code, everything is build in C#, VB.NET or whatever, leaving the DBA without any options then add more hardware.

I think for small to medium sized databases,it is possible to write crappy code and the system still works (thanks to the hardware), but when it comes to large and huge databases it is not possible to solve the issues from crappy code with more hardware. The only solution is to modify the code.

However the note by dportas is interesting. Should we be looking at a way to possibly evolve to a new language that works better? We have seen "C" evolve to C++, C#, and now F# in some ways. Python is a very interesting environment to me as well, but in spite of all the work done on languages, there's still a decent amount of C around, and I wonder how much C++ code is actually C compiled in a new IDE.

From what I've seen in terms of how we work with data, SQL works pretty well, but it has a decent learning curve and requires someone to fundamentally be able to understand how SETs interact. That feels like the pointer issues of my early career where lots of people just could not understand a pointer.

If I had to say that SQL fails in one place, and I've seen this with the intellisense challenges, is that it doesn't fundamentally order the commands correctly. In my mind we ought to decide what tables we are using first, then get columns and relationships. So something like

I think that we possibly could also find some other ways to create set interactions, or perhaps even have FKs automatically joined with a keyword, could build cleaner code. The whole change to using a CTE in SQL, with the code at the top, to me, means it's fundamentally harder to read. That was, in some sense, a step backward by bolting something on rather than fundamentally re-working the language.

I think LINQ wasn't a bad idea, but the way it translates into SQL is what I find problematic.

Agreed - +1... And I know about the FROM bit first - I put a mode into my engine where you type SELECT tablename. and then it changes it to SELECT schema.tablename. FROM schema.tablename - was the only way I could think of to address that one...

MDX has had WITH clauses for some time now so I believe they added CTEs to SQL to create similar functionality. LINQ has the table/source first for precisely the reason you have indicated, to make intellisense work

Your last sentence is the reason Matt will not have to discard his SQL knowledge any time soon. Eg You can write sloppy C# code and it will still seem to run as fast as optimized code. You can add more layers of abstraction to a language and it will seem to perform admirably.

The same is not true for SQL, what may seem like modest change in logic or design of a query can literally alter the execution time from seconds to tens of minutes, or even hours. To formulate optimal SQL you need to have an intimate knowledge of the data, the schema, how users use it, the db platform and a deeper understanding of your problem domain, what you want to accomplish and how to translate that solution in a way that the db can carryout efficiently. LINQ knows little of this and must work with the lowest common denominator so it can be used with disparate data sources.

For this I am not a fan of Linq to SQL because I know my SQL knowledge can always outperform it, Linq to objects/xml is quite useful though. I think a great deal of improvement can still be made but translating what someone wants from a database into optimal SQL would be cost prohibitive and take too long to develop

Good question. SQL is surely a very poor platform for modern development needs. The fundamental language features of SQL DBMSs have improved very little in the last 30 years. Compare that to the major innovations in object-oriented languages over the same time period. Part of the problem is that the SQL model was fundamentally based on some flawed ideas and designed to work within the constraints of 1970s systems. The domination of the SQL DBMS market by three major corporations with vested interests in maintaining the status quo has also not helped.

I think the database management profession ought to be able to answer the question by saying "Yes, I hope so". We really ought to aspire to better things than SQL. I'm definitely not saying that LINQ is the right successor to SQL but at least it is one initiative that is showing some kind of way forward. The various products that go under the "NoSQL" banner are another example.

Unfortunately I think SQL database professionals have sometimes been a little too defensive about SQL and this has left them somewhat outside of the discussion rather than participating in it and putting forward alternatives. I think more data management professionals at the grassroots level need to recognise the need for new alternatives to SQL and contribute to the development of those future solutions. For instance, where are the relational-based alternatives to the SQL DBMS model?

A very thought provoking example, and I agree that we should be looking for something better than SQL. The problem is that nothing has emerged that is in general better than SQL. ORMs have their place, but it is currently niche. The same with the NoSQL technologies. They are certainly better for some things. But for many things right now there is no better solution than SQL and I have not yet heard of anything on the horizon that I expect will get there. I can think of some tweaks to sql I would like to see, but fundamentally I cannot come up with a better alternative myself.