There's Hope for us all

Doug's Oracle Blog

I empathise with Dom Brook's recent blog post. We've all been there and I don't just mean being unable to write a technical post when emotions and opinions need to be dealt with first In fact, a day wll come soon (I'm full of promises, eh?) when I'll have a wee rant about the state of the industry myself. Isn't that what blogs are for sometimes?

However, what this post is really about is a comment that Tim Gorman sent to the Oak Table mailing list, where Dom's posting has been today's favourite topic. I believe he posted it on Oracle-L first. Although it doesn't reflect my personal experience in the slightest, there was something about what he had to say and the way he said it that rung very true to me and I wanted to post it here.

"Back in the 1980s, I worked for a company that had built some really cool applications in the area of travel reservations. Eventually, the travel providers (i.e. airlines, hotels, car rental agencies, etc) caught on to what we were doing and did it themselves, effectively putting us out of business overnight. So, it came time to sell the company off in pieces. We tried to sell the applications, but nobody wanted them -- they had their own, or could buy or build better. We sold the hardware and facilities, but for pennies on the dollar. Then, when we tried to sell the data, we hit the jackpot -- everybody wanted the data, and we were able to sell it over and over again, to multiple buyers.

I never forgot that lesson, and several years later traded being a programmer for being a DBA because (as Michael just said, below) I like working with data. Data, not programs, is the only thing that matters -- applications are transient and have no value except to acquire, manipulate, and display data. Data is the only thing with value. The long-term value of data is the reason I've moved toward data warehousing and business intelligence, too.

Data is important. Databases manage data. DBAs architect, configure, and manage databases. So, being a skilled database administrator will always be necessary as long as data exists. If the state of the art ceases advancing, then automation will finally catch up to extinguish the DBA role/job. But until then, being a DBA is a career.

...and that's my story, and I'm sticking to it, regardless of the latest flavor-of-the-month heralding the death of the database..."

Data Matters, but So Do&nbsp;Algorithms
I just read this post about the importance of data over applications. It reminded me of a passage from Eric Raymond, in The Cathedral and the Bazaar, where he paraphrased Fred Brooks:
Show me your code and conceal your data structures, and I shall cont...

For the kind of stuff I build at work, he's totally right. But I'd wager a significant sum that my databases at work will have less long-lasting value than Emacs.

Of course, Emacs manipulates data, like most applications. Then again, much of the data it manipulates is the sourcecode for other applications.

For an even purer counterexample, how about Mathematica. When you generate a 3-D graph of a complex equation, what data did you manipulate? Seems to me the answer is: none whatsoever. And I'm pretty sure my data is more transient than Mathematica as well.

And what about SecondLife? Is the data in there really important? Does it have any value whatsoever apart from the application?

I'd argue that in all three of your examples (Emacs, Mathematica, and Second Life), the argument can still be made that the data is more significant than the tool that manipulates it.

Emacs: without any source, text documents, etc. to edit, you'd have a really nice way to put characters on the screen and run some logic (I know, I know, you could probably implement an OS with Emacs, but you get the point)

Mathematica: Here again, if you're only interested in single session use, then the ability to save your work may not be of value. However, if you're like most people, you'll want to save the equations you're working with.

SecondLife: As with the previous examples, SecondLife by itself is a great virtual world simulator. However, if I had to recreate all the structures inside everytime I used the program, I'd get pretty frustrated in a hurry.

The point is that it is the data being manipulated by these programs that provides lasting value. It is the data that can be transferred between apps and re-used long after a particular data manipulator is gone.

Not a single text file I've edited with Emacs will be valuable to myself or anyone else ten years from now. Emacs has been a valuable program for decades already.

It would be easy to build a new text editor, but replacing Emacs would take quite a long time.

Working with Mathematica, in many cases at least, is not about analyzing data. It's about writing code. Not so different than writing Visual Basic code. If you want to stretch your definition so far that your own sourcecode counts as "data," then ok, but you're essentially saying nothing at that point.

Of course SecondLife uses data. But in the absence of the application, nobody would care about it.

And for an even more pure example, how about the real-time code running embedded devices, without saving any data whatsoever?

Typical database guy's view of the world. Still though, hard to fault him for his error in logic. http://www.nizkor.org/features/fallacies/hasty-generalization.html

It was probably fairly traumatic to watch the company you worked for quickly go out of business, and would probably lead most people to draw erroneous conclusions from life-changing events going on around you.

Data in a database is worthless in and of itself. It only becomes valuable to the business when an application is available to let the business users view and manipulate the data in specific ways. In the end, it doesn't matter if that data sits in an rdbms, in xml files, in flat files, comes from web services, whatever, the important thing is the application that exposes the data, often in new and interesting ways.

The data behind Google Maps was there for how long? It was how valuable to a few cartographers? The GMaps application is what counts, not the data.

The data behind Google Maps was there for how long? It was how valuable to a few cartographers? The GMaps application is what counts, not the data.

Really? The ability to quickly and easily display maps as a representation of geographic data is indeed neat - and mainstreaming that is way beyond time - but how is it used? For directions from place to place, to show where points of interest are, to highlight areas of interest. By doing what manipulating data. Imagine you had the maps but no co-ordinate references. Imagine those maps had no roads/towns etc displayed upon them. That api only works because of the data. It isn't a control system or similar it's just a neat way of visualising data. Same as excel - well cooler obviously unless you are an accountant - but still used for data visualisation.

Incidentally GIS systems were valuable to more than just a few cartogrophers - likely you drive on highways, use water systems and consume government services all of which have been provided in part thanks to GIS. You weren't a user, you likely were a consumer.

I'm going to have to side with author here. Applications are the conduit, the data is the prize.

Of course, insofar as the application is able to manage the data it is not insignificant. But it is the data that counts.
When the data is unique and cannot be re-created/recaptured and for some reason it gets lost, that is when we find ourselves up shite creek without a paddle.
An appilcation can be installed again, unique data is priceless to the point where if it vanishes so does the entity that was supposed to manage it.

It's even more true with personal data. Stuff you created that can be found nowhere else and can never be re-created. I have never worried about the availability of applications. Applications are a dime a dozen.
My data and that in my organization is everything. It does depend on the kind of data of course, something like Google Maps can always generate fresh images. The planet will always be there [or else we've got other problems]
You can't ask your 10-year old to become 2 once more and make that face again.
You can't run that unique test on the batch of user data again when the source data gets lost/corrupted.

It's data. It truly is.

All hail the database administrator for they are the keepers of the world. And, because I'm not exclusive or naturally hate mongering: a big, wet kiss for our valiant application providers when they manage to create a masterful user experience that allows us to think about the job we're doing and not the way we're supposed to fumble with the widgets.

I had a colleague who referred to agencies having 'cemetaries of data' (i.e. paper printouts in file cabinets) that are essentially worthless unless they are digitized.

While I respect the arguement that some applications make data useful, sometimes more useful than in their original form (e.g. geographic information systems, GIS), but the application itself can not operate in a vacuum.

However, I'd argue that it is the 'form' that the data comes in that is important. How much government data is online, but essentially useless because it in an archaic format or simply not accessible due to interface. The 'form' of data is now becoming intimately tied to applications in some sense, so their is less pure data and pure app, but a symbiosis of the two.

Tell an artist that his brushes have no value. Tell him his paint is worthless. Tell him the lenses in his glasses which he uses to see the canvas are unimportant. And then somehow claim his artwork is a masterpiece. See what kind of artwork the artist can make when you take away his paint!

Applications turn data into information. Without applications, databases are just unfilled bit buckets. Without databases applications create no value. The two coexist and are codependant.

Your assertion is better stated as data oftentimes being a costly or irreplaceable asset.

Many applications are commodity -- they provide the same value for the same purpose. Oracle and SQL Server -- commodity software. Word processors, spreadsheets are also commodity.

In lieu of software patents, applications can be recreated. Data rarely has this luxury.

But to say that applications have no value is simply a false statement.

But to say that applications have no value is simply a false statement.

I agree.

a) It wasn't my statement, I just thought it was thought-provoking.

b) It was a light-hearted piece meant to support a poor fed-up database bloke. I hadn't realised the strong feelings it would expose. I must confess my feelings on the subject aren't that strong one way or the other.

All it's really served is to illustrate the power of Reddit. Someone decided to take it on themselves to link to this (not my doing I might add) and before I know it, I have a comment storm!

We probably have a pretty similar outlook, actually. I get pretty fed up with application programmers who work with relational databases yet refuse to learn SQL well.

Or the various people who claim that relational databases are old legacy technologies, and want to build all the new stuff with some kind of primitive object persistence mechanism, with nowhere near the querying flexibility that SQL has.

When you ring a bell, what makes the sound? Is it the metal forged in the shape of the bell that makes it, or is it the air inside the bell vibrating?

The sound comes from both. As value comes from both the application and the data. Without a meaningful way of viewing and understanding the data, the data is worthless. Without meaningful data to drive it, even the coolest application is useless.

That's only true for data-centric applications. I've been doing computer programming for years (specifically 15 years), and have yet to end up working on any projects that even used a database.

Is the control system for a space-shuttle "all about the data?" No. It's about the expert system, encoded in the application, that flies the shuttle. Is a video game "all about the data?" No. It's about creating an interesting algorithmic framework that's fun to play with.

Is an operating system about the data? No, it's about creating a harness for other applications to run in. How about 3-d modeling software, my own current field? Arguably, the data there is the models that you're creating, but the algorithms for generating that data from raw thoughtstuff are absolutely critical to the process. The only data being processed exists solely in the mind of the designer, and its final form is intrinsically affected by the shape of the software.

In short, the opinion here is one of those biases that shows up when you confine yourself to a particular domain. If you've done nothing but pound nails your whole life, you're going to wonder why anyone needs tools other than a hammer. Don't get me wrong--hammers are terribly useful devices--but computing is more than data processing.

Can't believe no one beat on the obvious here - a video game is uninteresting without being played, shuttle won't fly long without input and output.

Kids were complaining this weekend they couldn't save games on Playstation 1 (I was too cheap to buy modules for). DS has them spoiled.

But I agree with the symbiosis crowd, data by itself is valueless - just ask NASA what's happening to all that Pioneer data. (Oh, guess I was out of date on that one: http://www.space.com/scienceastronomy/070327_scitues_pioneeranom.html )

Here's one of the Tom Kyte discussions, Christian: http://tkyte.blogspot.com/2007/10/it-all-about-data.html

The data (as stored in databases) is just for testing purposes. It is there to ensure that our programs are correct and are designed well - whether they do or don't have the clever little hacks we put in.

Data engenders itself by virtue of mere existence of humans (number of toilet rolls consumed, litres of petrol burned, etc.) but programs are brought into existence by a conscious effort of innovation, creativity & deadlines.

Look, code is data and data is code. It all depends on what you think of as "the machine." Dig the oneness. If you're after power, own the data. If you want beauty, breath code. If you want beauty and power, do both.

Whilst I agree that data can be important, the kinds of applications I have worked on involve a team of programmers building and refining an application over a number of years.

The application then becomes a repository of design decisions, requirements, innnovations, inventions, optimisations, etc etc. All largely undocumented, so that the application becomes a valuable resource in and of itself, in which is encoded the 'process'. Asked to recreate the application from scratch, most times, no-one could do it - its irreplaceable.

I remember to have read something similar on Thomas Kyte book where he states something like (i've not the book at hand now) "the software technologies came and go, but data remains" for example today java is very popular, but tomorrow? I would also refer to O'Reilly article (http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html?page=1) about web2 where states that data is important.

I think most of it is in the how you define data. Data for me (DBA dude) is everything that is molded in a "container", so stuff like collections, arrays, clusters, tables. In cache, memory, on hard disks. Coherence, Oracle, Hypersonic, database block, bittorrent pieces flying around. Especially in this internet century it will be more and more fuzzy regarding its definition

The thing totally missed from this debate between application and data is the logic, the model of the business or game or process that the application (or the database) is trying to replicate.

No one has defended that the database can in fact model the business or a subset of the business if done properly.

The whole reason for having a relational database is to have a relational model. As a DBA you get enlightened from tinkering and mucking around with parameters and settings, to adding indexes to finally realizing the design (the model) is core issue.

Before all the developers suggest this is going back to the waterfall method, rather than Agile, Iterative or Extreme or whatever.
When you are coding a application you have a goal in mind, even if you are building the app up in bits.
The goal is the model, the model is the goal. If the goal moves (updated), the model is updated.

Have Fun

Paul

Add Comment

Name

Email

Homepage

In reply to

Comment

Standard emoticons like :-) and ;-) are converted to images.

E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

Disclaimer

For the avoidance of any doubt, all views expressed here are my own and not those of past or current employers, clients, friends, Oracle Corporation, my Mum or, indeed, Flatcat. If you want to sue someone, I suggest you pick on Tigger, but I hope you have a good lawyer. Frankly, I doubt any of the former agree with my views or would want to be associated with them in any way.