The comment property on the node property is being refactored into a field. Let's pretend -- just because it's a great example not because we want to postpone it -- this happens during Drupal 9! Modules using $node->comment break but modules using $node->get('comment') don't. Modules running SQL queries against {node}.comment break but modules running entity queries with entity_query('node')->condition('comment', 1) do not. So here's the challenge to the community: if you want to start a serious discussion about backwards compatibility stop hacking and start using APIs. Now. Already Drupal 7 provides a lot that people deliberately code around. Don't do that. Get into the proper API-driven mindset so that the Drupal 8 ports will be written properly enabling a discussion whether we want those to run under Drupal 9. If you keep messing with the database directly and access raw data structures, there's not a chance of that.

It could also mean that Drupal people are so used to using raw queries and raw data structures that it doesn't occur to them to use a real API. Drupal has architecturally been very "naked" for a long time, which means sneezing causes an API break. (Irony: As I typed that, I actually sneezed!) That began to change a bit in Drupal 7. It's changing a great deal in Drupal 8. Hopefully by Drupal 9 the transition to for-reals APIs will be more or less complete, but time will tell.

But this is a cultural change, too, which is what chx is getting at. Backward compatibility requires a level of abstraction, and hiding implementation details even when they may be momentarily convenient. If we want backward compatibility, we need to get with the program. chx is dead right here.

That would be a cultural change about as big as writing Drupal 9 in Java. The problem is like that nobody uses the API because it sucks, and nobody will bother improving the API because nobody is using it anyway, and as long as there is a big influx of n00bs who don't care about backward compatibility, the stable API standpoint won't get to be a critical issue. As long as Drupal feels they are the winning team, why change it?

We are aware. Painfully so. We are providing a lot better APIs in Drupal 8 but that doesn't mean people should not begin to seriously think even in Drupal 7 "is there an API I could use" instead of thinking "What database tables do I want to mess with"

The discoverability of public fields is much better than that of member functions. dpm()/krumo and debuggers show raw data structures, not APIs.

Getting krumo to show the proper accessors could provide significant push into the right direction. Given a known and an unknown alternative, most people will choose the known one rather than searching for the unknown.

It's not that "people deliberately code around" APIs, they're just not familiar enough with them to prefer them over what just works...

And its much harder to write and understand than SQL. So in general, no, I don't want backwards compatibility (because that prevents us from fixing our mistakes), and if I have to use DBTNG, my answer is "definitively not".

Very often, I purposefully write an SQL query to directly access / modify the data I'm seeking, rather than going via the appropriate Drupal API function. And chances are, I do this for performance reasons - the fact is, that many parts of Drupal's API are really sloooow.

If I need to quickly get the first 1,000 "home phone" field values for my site's users, I'd rather code up a query directly on the field_data_field_home_phone DB table, than call user_load() within a loop. If I need to unpublish 500 nodes fast, I'd rather directly update the status field in my DB's node table, it will be a zillion times faster than repeatedly calling node_save().

I do a fair bit of bulk import / export work - so, hope you can understand that in this use case, going via the API is often not the most attractive option. Of course, anyone going down this path needs to understand the consequences - most importantly, that bypassing the API will also bypass hook invocations (but then again, sometimes that's actually preferable behaviour).