James McGovern kindly linked to my screed against REST, but I think he misunderstood me when I talked about SAML. No problem... if I read as many blogs per day as he does, I'd do the same.

His quote was this:

Bex Huff provides an interesting perspective on REST within the ECM domain. His comment: you could "punt" and rely on wacky SAML, but that just seems to complicate things beyond necessity... seems as if folks in the ECM domain don't believe in the notion of SSO and would rather force complexity in other ways such as making folks log into different systems of course using different passwords, making enterprise administrators duplicate identity stores instead of leveraging an existing one such as Active Directory and so on.

Now... everybody I know in the ECM space cares about Single Sign On (SSO). In fact, Stellent/Oracle supports Active Directory and LDAP out of the box, a few minor tweaks gets you SSL certificates, plus we've made dozens of customizations for Site Minder, and custom/exotic SSO system. I even made an ANT script that could build a custom security integration with just about anything with a few lines of C++.

Trust me, we all know and love SSO.

The problem I have is more specific to SAML. I just don't like it. In fact, I hate SAML. Nothing personal, I just start out hating all technology. I have to. Otherwise, I find it difficult to discover its flaws. If I don't know the flaws, I can't effectively recommend when to use it. There is no silver bullet, and after working with computers for 20 years I've learned to distrust almost everything.

So, I started out hating SAML four or five years ago, when I first heard of it. Guess what? Thus far I've encountered no reason whatsoever to reduce my dislike.

Most of the cool stuff in identity management seems to be with OpenID and SXIP. SAML has been around forever, and who is using it? Its not saying "here's some useful technology," its saying "here's how things should be done." It feels like something from the peaks of the XML ivory tower that makes the claim (yet again) that the entire world would magically be better if we took all information and put <angle brackets> around it... Where's the evidence? Where's the proof?

I get why people are hot about Active Directory, SXIP, and OpenID... I just don't believe SAML has proven it deserves any hype. It might make somebody's job easier, but at what cost? I'm totally open to the possibility that I'm wrong, or that SAML 2.0 is a million times better... but I'll believe that when I see it.

Only a select few people get to play with Oracle's 11g database (beta) before November -- and I'm not one of them :-(

No matter... it looks like they are sneaking in some pretty awesome features, one of which appears to be something along the lines of a just-in-time query optimizer for screaming fast performance:

Iggy Fernandez, editor of the NoCOUG Journal, the official newsletter for the Northern California Oracle Users Group, praises what he says is 11g's unheralded "learning optimizer" feature.

"When relational databases replaced hierarchical and network databases in the 1980s, the promise was that programmers would no longer need to optimize their queries by hand," said Fernandez. While no database has "completely realized" that vision, 11g, says Fernandez, is a "great step -- the query optimizer simply learns from its mistakes -- in fact, it can stop a query that is already in progress and try a different approach!"

awesome... although if my guess is correct, this will only work for parameterized queries, and not SQL statements built on-the-fly.

I did also hear a fairly detailed rumor of some unbelievably cool performance features they have planned for a "future release", which may or may not be 11g. No, nobody inside Oracle leaked this to me... all parties are independent of big red. I just know a guy who knows a guy who owns the patent. And that's ALL I plan to say about it...

OK, I've been getting a lot of questions about how I solved the Idiot Test 4. Its a bit tricky, so I decided to put together all the answers and the passwords for the test in a walk-through format. This should help you beat (and cheat) the idiot test.

Update: Ryan Curtis, the creator of the Idiot Test, has made this the official guide... and clarifies that indeed, using this guide is cheating!

They are all fascinating and thought-provoking. The Ze Frank talk about creativity is a popular favorite, as are the ones from Hans Rosling about statistics and global poverty. But one that is an absolute must-see is the one by Dan Dennett on ants, terrorists, and the mighty power of memes:

Highly recommended... although I disagree a little with the parallels to Guns, Germs, and Steel. I particularly liked how he summed up the meaning of life and the secret of happiness in one sentence:

Find something more important than you are, and dedicate your life to it. -- Dan Dennett

Compelling... but I prefer my personal philosophy three times better, because it has three times as many sentences (inspired by Dawkins):

Whether you're an atheist or a believer, the meaning of life is the never-ending quest for purpose. We are the only mortals who see the world not as it is, but as it could be, and then create it. Thus, we're here to ask that ultimate question "Why am I here?" and never be long satisfied with our answers. -- Bex

Coming to terms with that has helped me sleep soundly at night... and occasionally in the afternoon. ;-)

More pragmatic advice from James McGovern, this time about top-down versus bottom-up IT initiatives. In other words, CEO mandate versus workers-know-best. Both are needed, but when is which one appropriate? My favorite quote:

One of the more interesting things is that a person debating top-down vs bottoms-up needs to take into consideration is historically which one has had better results. Of course, top-down has the advantage that even mediocre delivery will be declared success while bottoms-up mediocrity will probably result in throwing daggers.

Consideration? Sure, OK. That's the politically expedient thing to do for bottom-up... but it misses a vital point: is the technology something that people will absolutely love? If so, mediocre implementation will be forgiven, and the top-down criticism will eventually be won over. As I always say, its easier to get forgiveness than permission.

I was a part of multiple skunk works projects at Stellent, some of which got me into a teensy bit of trouble with executives. Some of these projects languished, but many of them took off like gangbusters because people loved them. In the words of Cathy Sierra, these systems helped people kick ass. In the words of Clay Shirky, they were systems where people took care of one another, and that more than anything else guaranteed their existence the next day.

James feels that Outsourcing, CMMi and/or PMI, and Identity Management are best done top down... I agree with everything but Outsourcing, unless its on a large scale. There's nothing wrong with giving your middle managers a budget to outsource. Heck, I see nothing wrong with giving every developer a $5,000k budget, and let them "outsource" some tricky code to another developer who can do it in half the time. Or buy a awesome monitor. As long as there's modest oversight, and your developers aren't idiots, there's very little harm that can be done.

James also says Agile, SOA, and Open Source are best done bottom-up... whereas I agree its easier from a bottom-up perspective to sneak-in cutting edge technology, I'm not sure if its best. This problem is by no means unique to the software world. Its the same age-old question: when should I let my employees take the initiative? The same rules apply across the board, whether the "initiative" is a software methodology, an innovative marketing plan, or sales incentives. The two most important questions are:

How does this decision affect the rest of the organization?

Do the people making this decision fully understand its risks and benefits?

No matter what you're doing, the answer is always the same: unless your employees are bound by a genuine shared purpose, bottom-up initiatives will do more harm than good! In a typical company, you'll be OK if your IT department sneaks in Agile techniques... but in a dysfunctional company, you run a significant risk of making things worse. This is bad, because in dysfunctional companies, bottom-up initiatives are sometimes the only way to get anything done.

Unless you have some folks at the bottom who have a 50,000 foot view of the entire company, or are genuinely dedicated to helping your customers kick ass, then bottom-up initiatives are just spinning wheels on systems that go nowhere. If your IT department wants to take the initiative, they better be keenly aware how their changes effect everybody else. They had better understand that all change has positive- and negative-effects, and keep track of them all. Finally, they had better have a plan to roll-back the changes, should the negatives dominate for too long.

This way, you're more likely to have bottom-up initiatives that kick ass! Also, you'll have a nice paper trail of successes to prove its value to your inevitable critics.

A resource-oriented interface to an enterprise content management repository may not be the best approach... especially when you need to do things like workflows, subscriptions, conversion, multiple taxonomies, or basic business process management. You need a service-oriented interface that focuses on the action, not the back-end implementation. See my anti REST rant for more info.

The response that search doesn't matter, and to just "Use Google" to find content is beyond glib... using Google means losing metadata. Instead, the interface should make it brain-dead easy to discover lists of items based on metadata, as well as "related content" in nice little buckets. If we're just going to kick metadata in the head, we might as well just use ZFS over iSCSI, and call it good.

I can't say for certain yet, but I'd suspect it would be tricky to embed multiple content items in one feed item, as well as binary data... Can APP be used to "chunk" files larger larger than 2GB? Since APP is so resource-based, what if I want a batch check-in of multiple resources for better performance? What about syndicating secure data out of the repository? Again, I sense danger...

For the record, its wasn't tough to get Stellent to output RSS feeds. That only took a few hours... What was a royal pain was discovering how rotten most RSS readers are, and trying to tweak the output just right so that everybody could consume it.

Switching to ATOM may be a tiny bit tougher because it supports more metadata... its an obvious replacement for RSS, but I think a few more pieces need to be added to the "ATOM Stack" before it could do as much as WebDAV. Search, specifically... and I'd push towards a service-oriented publishing model.

Regarding the security layer... you could "punt" and rely on wacky XCAML/SAML, but that just seems to complicate things beyond necessity... and that ain't good for anybody except security consultants. A simpler idea would be an ATOM and LDAP Mashup, and make every single resource identity aware. If done right, you can authenticate with the enterprise LDAP server, and authorize with the department's federated LDAP server. Seems pretty simple to me...

A door-to-door religious book seller was struck down by lightning in South Florida yesterday. This is despite the fact that it was a clear day, and hadn't even rained. He's recovering in a local hospital, but hasn't yet regained consciousness. Hopefully he'll be OK in a few days.

The phenomenon is called dry lightning... which means that the air is so dry that rain from higher elevations evaporates before hitting the ground. Thus, people don't know its raining, and don't take the normal precautions they would during a storm. Unfortunately, lightning doesn't evaporate...

The entire airline industry is facing serious management problems... union trouble, aging pilots, bankruptcy, canceled flights, and the like. Their software is woefully outdated, and the latest security laws are causing people to check twice as much baggage... which naturally leads to significantly more lost luggage...

Now, logistics ain't rocket science, but I am sympathetic that it's difficult to get right. However, there are dozens of industries that have made a significant profit by simply getting things from point A to point B reliably... so I'm mystified why the airline industries are struggling.

Ask around... it isn't easy being a Northwest Airlines customer... just talk to anybody who has to travel through Chicago. A lot of their pilots are nearing 60, which is bumping up against the mandatory retirement age. There are also strict rules about how many hours a pilot can fly in the month. Therefore, if you want to minimize chances of a canceled flight, book it no later than the 15th of the month.

You'd think that Northwest could have seen this coming five years ago and hired more pilots... sadly no.

Seriously, it could be a good move. FedEx already has all the software and a good chunk of the infrastructure. They have a brand name that screams "reliable," and relationships with airplane manufacturers. They just need a handful of the best and brightest execs from Northwest so they can learn the nuances of the travel industry... then they could be the 100% business infrastructure solution.

They'll print out your reports, ship a crate of them to your destination, and even pick you up from the airport in one of their stylish trucks! And if you can fit into their one-size-fits-all travel pod, they can guarantee delivery.

Like I said... not fully baked. Just the rantings of a frustrated customer.

James McGovern and other are back talking about Enterprise Content Management standards... his latest advice is to take a closer look at RSS or WebDAV as a standard. Others chimed in that there may be something there...

If I may offer some advice: HOLY CRAP, NO!

RSS is a nice model for consumption of streams of text, but it has many many problems. It doesn't have revisions. It can't handle large binary data streams. Even by jumping to ATOM and adding metadata, you still can't do searching, editing, or contribution for crying out loud. In a word, NO!

WebDAV is good for quick changes to web files on a shared filesystem... but authentication is a mess, it cannot handle dates properly, nor metadata-based search, nor metadata-based contribution without massive kludges. AIIM says that Records Management will be a huge factor for why people purchase ECM... and WebDAV just doesn't have the metadata muscle to keep up. In a word, NO!.

Also, I believe Laurence Hart completely misses the point of Billy Cripe's comment. Standards only enable business if they have sufficient features. The entire point of a standard is that you lose functionality by standardizing, but you gain flexibility. The question is, does that help more than it hurts? At present, it hurts more.

I totally disagree that a standard -- or anything else for that matter -- is inherently good. If they were, then everybody would stop whining about the lack of ECM standards, and freaking use one of the four existing standards. Stop worshiping technology for technology's sake, and make something useful.

The four existing standard are crap, and the next 4 will be as well... unless:

The analysts stop letting Microsoft get away with calling Sharepoint an ECM system,

The top 5 (or 10) ECM vendors get together and decide what a real ECM standard needs to be useful, and

All these niche repositories either write an interface, or go away.

Like I said, probably not before 2009. Those niche products are still pretty useful, even if they aren't ECM.

Alternatively, a large neutral company -- like BEA, Sun, or Sybase -- designs a "universal connector" for each specific ECM system. Several small firms have made these, but they were bought up and shut down by Documentum. IBM used to have a good one for WebSphere, but it also has languished because it means people could dump Content DB or FileNet whenever they wanted. Great for the WebSphere team, but bad for their ECM team.

And forgive me if I find Microsoft, EMC, and IBM to be completely disingenuous when calling for decent ECM standards. Those 3 companies either blocked decent open standards, or shut down universal connectors.

Many time vendors try to sell Business Process Management (BPM) along with Enterprise Content Management (ECM) as a means of helping companies get their information process under control. However, there's a huge disconnect in both the buying habits and implementation schemes for these two systems.

After many years, organizations finally admit that they need to get their content under control, and support ECM as enterprise infrastructure. They even feel good about using hosted solutions for their content management -- good news for 3 of my clients.

However, BPM is still stuck in the fiefdom stage. Very few implementations are company wide: they are highly departmental. Despite management believing BPM is highly useful, there is a strong problem with lack of ownership, which may be a big cause of the percieved lack of success.

Whereas individual departmental groups will find BPM highly useful... but without enterprise-wide owners, you'll be unable to get enterprise-wide information process under control.

UPDATE: Several commenters below claim that this technology never existed. Also, the man who invented it -- John Pereless -- took money from investors, and never gave them anything in return. I cannot speak to either claim... but I usually urge caution when it comes to products "too good to be true."

Called the Hawk-10, it uses an array of finely tuned microwaves to precisely break the hydrogen bonds in the long hydrocarbon molecules of solid plastics. This breakdown yields simpler, smaller hydrocarbon chains, such as oil and natural gas.

In their tests, you could grind up one steel-belted tire to yield a gallon of diesel fuel, two pounds of natural gas, two pounds of steel, and a bit of carbon ash. It can easily strip insulation off of wires, leaving scrap metal behind. Its value in the automotive recycling field is obvious... its less obvious if it makes sense to do this for all recycled plastics.

No word yet on the toxicity of the byproducts, but its possible that with a bit of tuning they could alter the molecular structure of the plastics more carefully.

Look at the mathematical puzzle below... in it you should be able to move one single matchstick to create a mathematically correct statement with Roman Numerals:

Single slanted sticks are not allowed: there must be 2 slanted stick in a V to represent 5, and a single non-slanted stick to represent a 1. You have 3 minutes... GO! The original blog article has the answer.

Next, try this one:

A bit tougher, huh? In studies, only 43% of people could solve the second puzzle in under 3 minutes. However, 80% of people with lateral prefrontal damage to their brain could solve it! Mental patents get all the fun...

In theory, that part of the brain is important for determining how to solve a problem, which is called cognitive guidance. It's essential for doing ordinary math quickly, but it hinders problem solving when you don't have a rigid framework to help you. In other words, a healthy brain makes it difficult to "think outside the box".

Heh...

Thus, some brain damage patients are better at letting their minds wander and solve problems in any way that seems to work for them... which on occasion helps them see things that other people do not. In other cases, it makes it extremely difficult for them to solve problems within a framework with apparently arbitrary rules. I bet Kafka would have nailed the second puzzle...

Over the weekend, James McGovern complained about the lack of Enterprise Content Management (ECM) standards. He was brutally criticising one EMC Documentum blogger for his general lack of enthusiasm on the subject. Many customers don't care one iota about the standards, other than as a "checkbox feature" to be fully buzzword compliant. However, McGovern saw it as ECM's responsibility to push customers in the right direction.

Now, I can understand why an enterprise architect would be screaming for an ECM standard. After all, its their job to make sure their portal server can easily interface with the half dozen repositories in the organization. Its not unusual for a large company to have many ECM systems -- due to mergers, acquisitions, or departments that hate each other -- and it would be nice to have one standard way to interface with them all.

However, I totally disagree that a useful ECM standard will be developed any time in the near term. Why? One simple reason... there are already four separate ECM standards, none of which are much used.

First, there was ODMA, which some used for content management. Then BEA came up with the Service Provider Interface. Then came WebDAV, who's biggest supporter was Microsoft. Then the Java folks chimed in with JSR170. Now, we are awaiting the fifth: JSR283. Guess what? They all suck.

Why??? My opinion is that its because there are 40 different organizations who claim to be "Enterprise Content Management," and they all have a different definition of what that means. Some have limited metadata, some have extensive metadata. Some use a file and folder structure, others realize that an ECM system with findable content must have multidimensional metadata taxonomies which render a folder structure obsolete. Some have workflows and business process management, others barely have revisioning. Some can render a Word document into thumbnails on the fly, others barely support files. Some have compound documents, others don't.

Thus, any standard will be forced to be the lowest common denominator between all 40 systems. Any customer limiting their enterprise to just those basic ECM services will be horribly disappointed at the lack of features. I know of one major customer who was extremely gung-ho about "standard" ECM interfaces, until they realized that the standards lacked vitally important features. Now they never use them. Other customers were more extreme: one actually banned WebDAV across the entire enterprise.

A true ECM system is vastly more interesting than the search/edit/save model that standards bodies would have you believe... A fully-featured "standard" ECM interface won't look anything like SQL. Databases are about structure, ECMs are about semantic and context as well.

Now, there is some value in a highly simplified interface to an ECM repository, but we must all acknowledge that this is the goal, and design a highly simple interface that everyone agrees is barely useful. We should stop wrapping everything in rigid and obtuse XML like WebDAV, or in over-engineered and under-useful standards like JSR-everything. One reader suggested a ReSTful API to simplify things. Now, I'm not a ReST fanboy, but I see the merits of a resource-based interface like ReST for "dumb" content access... as long as the hype acknowledges it's uselessness, it won't cause much harm.

Until the market defines the minimum requirements of an ECM system, there's not much point in making a beefy interface. Almost everybody agrees that such a definition would leave out Sharepoint because of its wretched metadata engine... everybody, that is, except Microsoft... if the analysts continue to support Microsoft's claims that Sharepoint is ECM, then a decent standard will never get traction, because there's no way Sharepoint could support it. Jeez, Sharepoint barely supports WebDAV... what are the odds it will support JSR283???

So until Sharepoint gets its act together, some more consolidation happens in the market, and some of these niche players stop calling themselves ECM, I don't foresee efforts towards a decent standard ever bearing fruit. That might happen by 2009, but don't hold your breath.

Morville shares research and anecdotes from business, history, library science, anthropology, and neurobiology in his quest for the perfect system where everything in the world is instinctively easy to locate. Can we ever achieve ambient findability? And what would the world look like in such a place? What are the social and political ramifications of findability? Will it be big brother, or will the very concept of unquestionable authority wither and die?

Recent manifestations such as Google, Wikipedia, and blogger watchdogs suggest the latter is more likely...

Ironically, the more information we have, the less likely anybody is to use it. Obtaining information is very painful, even if the data is easy to find. The relatively unknown Mooers law states:

An information retrieval system will tend to NOT be used whenever it is more painful and troublesome for a customer to have information than for him not to have it.

Meaning, if I have a problem, I can either look up the answer, or ask somebody for help. If I ask somebody, then they might do all my work for me, which is good for me. However, if I look up the answer online, then I have to read it, understand it, and implement the solution myself. Not only must I confront my own ignorance, but its a lot more work. Stupid Google.

Along the same lines, it's insufficient for information merely to be available and findable... it must also be believable, useful, and tailored to the audience so its easy to absorb. That's the top-to-bottom challenge, and very few people understand it. This book doesn't give much practical advice about absorbability, but it covers findability needs and existing technology quite well. The rest is up to you.