There are so many programmers out there who are also an expert at Query writing and Database design.

Should this be a core requirement to be an expert programmer or software engineer?

Though there are lots of similarities in the way queries and codes are developed, my personal opinion is, Queries seem to have a different Structure than Code and it can be tough to Master both simultaneously due to the different approaches.

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.
If this question can be reworded to fit the rules in the help center, please edit the question.

2

What do you mean by "structure"? If you're talking about semantics, than grasping any kind of a new semantics should not be a problem for an "expert". By definition. OTOH, only a few of developers are exposed to databases and query languages, the rest of us does not care at all.
–
SK-logicJun 8 '11 at 12:46

3

I think this is an erroneous assumption: "There are so many programmers out there who are also an expert at Query writing and Database design." There are relatively few programmers who are expert with these things: DBA != SE.
–
AshleyJun 8 '11 at 17:46

1

How difficult is it really to write database queries?
–
Diego DeberdtJun 27 '11 at 8:54

@CaptainShakespeare, really it can get quite difficult once you get past CRUD operations. Ty doing complex reporting sometime. And then look at performance tuning queries.
–
HLGEMFeb 6 '13 at 14:45

12 Answers
12

Whether or not database query writing should be a core requirement depends on the job, but relational databases are ubiquitous in current technology.

So, if I met a programmer that didn't know how to write database queries, I would expect one of two things:

They are generally inexperienced.

They are highly specialized in another field (e.g. embedded systems) and have never needed to learn it.

Database queries are fundamentally different from more standard programming languages. They are algebraic and intended to operate on relational data, while C# or Java are imperative and operate on disks, memory, user input, etc. Even functional languages like LISP or Haskell that are more algebraic in form are less oriented to relational data.

EDIT: As has been pointed out in the comments by me and others, there are some valid reasons why an experienced developer may not know database queries well:

Their team used ORM/NoSQL

Their team had DB programmers

The complexity of the application was in the business logic, and the DB queries were trivial

Their team apportioned the work such that some programmers didn't write queries

Though valid, these caveats are not convincing reasons why an experienced developer would not know database queries. Unless highly specialized, a programmer should be familiar with relational databases.

So, if someone did a non trivial project that used Database, he is expected to be acquainted with queries, right?
–
Shamim HafizJun 8 '11 at 12:07

3

@Shamim, I would expect this person to be moderately experienced with queries unless this person was a junior or entry level. Perhaps this person only has a few years experience and was sheltered on a highly specialized team?
–
maple_shaft♦Jun 8 '11 at 12:12

12

@Shamim I would probably expect so. They still could be a good programmer. Questions like this are very hard to answer, because they are so many caveats: maybe the team had a DB programmer; maybe the non-trivalness of the application was in the business logic, and the database queries were trivial; maybe they apportioned the work such that your programmer didn't work on the queries; etc.
–
Matthew RodatusJun 8 '11 at 12:13

4

My development team has dedicated PL/SQL programmers as part of the project. So while the .Net programmers can do simple queries they are there to review it and develop the more complex queries. Moreover with proliferation of ORM (and NOSQL) why do you think non-SQL developers must know complex queries.
–
PratikJun 8 '11 at 13:35

2

@Matthew Rodatus: I've worked at places that had libraries that managed queries, so it would be theoretically possible to work there without understanding simple SQL. I believe all developers were competent in it in practice.
–
David ThornleyJun 8 '11 at 15:29

Any software engineer should have a basic understanding of databases and how to store and retrieve data using SQL, at least to the level where they have an understanding of what this can be used for (and with that I would include an understanding of keys, views, stored procedures and triggers).

Not every software engineer needs to be an expert, and the level of expertise required really depends on the type of software they focus on. Embedded software, hardware drivers and operating systems rarely use SQL, but application software (be it web or desktop or service/daemon based) use databases all the time.

Fortunately, nowdays it is perfectly ok to do applications without any RDBMSes at all. Most tasks don't need them, or simply cannot be adequately mapped to a relational model. There are tons of non-relational storage options available.
–
SK-logicJun 8 '11 at 12:50

@SK-logic, What kinds of options do you feel make relational databases irrelevant? Datawarehousing is too specialized for analytics to be useful in a transactional system. And don't get me started on everything wrong with OODBMS.
–
maple_shaft♦Jun 8 '11 at 14:46

1

@maple_shaft, there are numerous specialised, domain-oriented storage solutions. RDBMSes tries to be generic, and fails badly in that. In some cases even ancient hierachical DBMSes are much better than relational.
–
SK-logicJun 8 '11 at 14:59

6

Not all desktop software uses databases, so be careful when you say that it happens all the time.
–
Anna Lear♦Jun 8 '11 at 15:19

There are some areas of expertise (embedded systems for example) where database knowledge is not needed. But most business applications use a database of some kind and if you don't thoroughly understand how to use it properly, you can create a performance mess that is extremely difficult to fix. Refactoring databases can be a complex and difficult process and many places opt not to fix the structural problems because of that difficulty and just dig themselves deeper into a hole. If you have database knowledge, design is much easier and far more likely to work well over time.

ORMs are not a substitute for getting database knowlege. Anyone who uses one without knowing the basics of database querying and design is doomed to havea poorly performing, badly designed database which will affect the long range abilty of your application to handle the load. ORMs in the hands of someone who knows what he or she is doing are fine; in the hands of people who can't be bothered to learn about databases, they are usually a disaster.

If I had a project with a database backend, the database specialist would be the second developer I would hire (after the intial application developer). Databases are generally not throwaways, that data will still be there in close to the same form 20 years later, it pays to have expertise in the beginning stages.

Projects often get into trouble because they don't hire these people until the database has 100,000,000 records and is running slowly. Or they blame the tool for being bad (no SQL Server is not slow if you design correctly) not their design incompetence.

+1 for mentioning that the presence of an ORM does not replace the need to know SQL (or the underlying factors in whatever type of db you're using).
–
RHSeegerJun 8 '11 at 14:38

4

+1 and I wish I could give you 100 more! I know that corellation != causation but it is more than apparent to me that the most effective application developers I have EVER worked with had a thorough understanding of how to write a standard SELECT query. I should be able to hand a good developer a data model and a "question" about the data and that person should eventually be able to write a query that "answers" my question.
–
maple_shaft♦Jun 8 '11 at 14:40

1

+1, totally agree. I dont buy the explanation that 'we use ORM' or 'we have dedicated programmers for that'. If someone is really experienced, they would have filled the role of db developer at one point. Thats what experience is.
–
GrandmasterBJun 8 '11 at 18:08

The politically correct answer: It depends. SQL knowledge has no value whatsoever if the developer never works with relational databases (and in this day and age of NoSQL applications, that's actually quite likely).

Second, when there's a DBA or full-time query writer (whatever the title is), then understanding is also of lesser importance.

It's only really important if the developer needs to be a jack-of-all-trades and there's a requirement in his projects for using a relational database (for example in old-fashioned web applications or connecting with existing databases)

My personal opinion: No. An experienced software developer should be able to learn a new skill (such as SQL) if and when they need to, not 'by default'. Flexibility and the ability to learn and understand is, imho, what differentiates a good developer from an okay one. The 'golden hammer' rule also applies - if you have a developer with extensive SQL knowledge, it's very likely this developer will pull out the tool he knows best - relational databases - to attempt to solve every problem, whilst it doesn't necessarily have to be the best solution. Of course, this also applies to NoSQL advocates, ;).

Picking the right tool for the right job is what an experienced programmer should know.

Computer programming (often shortened
to programming or coding) is the
process of designing, writing,
testing, debugging / troubleshooting,
and maintaining the source code of
computer programs. This source code is
written in a programming language. The
purpose of programming is to create a
program that exhibits a certain
desired behavior.

Database queries has their own languages , they could be designed, tested , debbuged and mantained. The purpouse of a database query is to allow you to obtain the information you need, the way you need it.

Database design is a particular type of design. It's not that hard to learn, but the typical database designer doesn't get that many opportunities to design a database.

The place I'm working now has the same database design that it had in 1970. We've moved the database from IDMS to DB2, but it's the same network database design. I have had the opportunity to create 5 new DB2 tables in the 9 years I've worked here.

I suspect that there are very few work places with a dedicated database designer. So, I'd conclude that database design is considered part of the repertoire of a senior analyst.

A good software engineer with a background in enterprise and business applications (EDIT: specifically in projects that utilize an RDBMS) should have expert knowledge of writing relational database queries in the standard format. Further they should be able to understand complex schema and propose schema designs of at least moderate complexity.

Extremely advanced or complicated schema design should be the realm of a data modeler or functional architect.

This doesn't mean that Database Programmers don't have a place either. Complex stored procedures, complex and efficient queries and database tier software design and architecture focused on the unique tools and offerings of a single database vendor (Eg. Oracle, MySQL, SQLServer, etc...) should be best left when possible to professional software engineers who have experience with these highly specialized and complicated offerrings.

The vast majority of business and enterprise systems however in my opinion don't justify the need for data modelers and specialized database programmers but I have worked on such projects before that GREATLY benefited from the knowledge and expertise that these people brought to the table.

-1: strongly disagree that a "good" software engineer should be an expert at relational database queries.
–
John SaundersJun 8 '11 at 13:43

3

Would you still disagree if I say that a good ENTERPRISE or BUSINESS application software engineer (as opposed to embedded systems, etc...) and if I said that this person should be an expert at STANDARD relational database queries (without fancy-pants vendor specifics like analytical queries and the like)? A thorough understanding of SQL SELECT statements, all types of joins, unions, intersects and merges, inline views, conditions, ordering and grouping result sets should be thoroughly understood and amply demonstrated by ANY software engineer that carries the labels that I specified above.
–
maple_shaft♦Jun 8 '11 at 14:27

4

Meh. I work at a large software company and we don't deal with any RDBMS whatsoever. My last job developing desktop software didn't require any SQL either. I'm not sure how you define enterprise and business applications, but it seems to me that your view on things is a bit narrow.
–
Anna Lear♦Jun 8 '11 at 15:16

2

I stand by what I said. Theoretically if the application is tiered and componentized well then there is no need however in my entire professional experience my team and I would have DROWNED if we weren't experts at SQL, even when we had a dedicated team for RDBMS design and development. Perhaps I am old fashioned or just have terrible luck in my career?
–
maple_shaft♦Jun 8 '11 at 15:31

3

@maple_shaft Yeah, it's not that the applications I've worked on were sufficiently componentized. It's that they didn't use any RDBMS, period. Different fields, I guess. The point is, you can't say that every business/enterprise developer must be good at SQL. It's simply not true. Be good at it if you use it. Don't worry too much if you don't until you need it, as with any other language or technology.
–
Anna Lear♦Jun 8 '11 at 15:50

I'm quite frankly amazed that so many of us think that every development revolves around a database, and an SQL database at that.

Others have mentioned the many ways in which we can avoid the nitty-gritty of SQL in our jobs, even when we are working (indirectly) with databases, but what about all the developers that write the firmware for the 101 electrical products the we each possess? What about the guys specializing in real-time monitoring?

I would suggest that the majority of today's developers will have SQL skills to varying degree, but it is far from being a barometer of their ability.

I don't think query writing should be a core requirement for programmers. Having said that, I believe that a programmer who can write queries and design databases would be more valuable to an organization.

However, if this programmer can only write "select * from tblxxxx" type queries I would not consider this programmer to be an expert. Likewise, if the database designed by this programmer places one-to-many relationships into one table instead of two tables then I would not consider this programmer to be an expert.

Here's how I explain this to non IT people. IT professionals specialize in certain areas similar to how carpenters, electricians and plumbers specialize in there respected fields. They tend to overlap some of the skills but are not experts in all areas. An electrician can do simple carpentry tasks with confidence but would not bode well trying to tackle complex structures.

Likewise, a programmer can and should know how to write or manipulate simple queries and database designs but not be expected to design a complex data structure.

I would expect a generalist developer to have at least an awareness of database technologies (relational or otherwise) and be able to discuss the pros and cons of using them. Otherwise, I'd be afraid all they know how to do is stuff data into flat files.