From Business As Usual to Knowledge-Driven Architecture – Part I

[Editor’s Note: This week, we welcome Yefim “Jeff” Zhuk of Sallie Mae as he presents a series on Knowledge-Driven Architecture. This series follows up the author’s presentation at the recent international 2011 Semantic Technology Conference San Francisco and further expands on the subject of integrated software and knowledge engineering, originally described by Mr. Zhuk in the book “Integration-ready Architecture and Design.”]

Business and technical people don’t always understand each other. (That might be an understatement.)

Translation from business to technology is called the development process.

“Cooking” an application involves several translation layers and teams:

We are still jumping through the same hoops after we have developed multiple services and promoted service-oriented architecture (SOA), after we’ve invested in the enterprise service bus, business modeling tools, and more…

Don’t take me wrong. I am for SOA. SOA saves budgets, and for bigger companies the savings are bigger.

But to fully benefit from SOA, we need to give subject matter experts the keys to Business Architecture.

This long shot will further the convergence of Business and Technology and will prevent the usual “lost in translation” and “too-late-for-market” effects.

IT is not about infrastructure

Doing “more with less” and continuing “rightsizing” (shareholders like these exercises) adds more pressure for IT.

More than 50% of IT budgets go to “keeping lights on” – maintaining the infrastructure. Inherited from the past and obtained through the mergers and acquisitions, multiple systems from many vendors need a good eye and a strong arm 24×7. Each new project adds more: more databases with data tables, more components and complex connectivity.

At the same time, each application is focusing on specific pieces of information to display data for a specific audience under specific rules to answer specific business questions.

What am I trying to say with the last long sentence? The real focus is on information, not infrastructure! It just happened that we set up Information Technology to manage information with complex infrastructure.

Focus on information

What is the alternative? Let me come back to the food analogy: “what do you do if you are hungry and cooking is not your sport?” You go to places where food is prepared for you. You go out or even to your kitchen, if someone else in your household can easily play all the necessary roles to make dinner.

Can we apply this situation to application development? Yes, we can! Just imagine that the complete information related to your company is already collected in a single database and ready for digestion. A single service that creates a query for this database can understand your language and translate your request into a proper query or product order that is also an informational transaction in the same database. And if you don’t know the terms, the service will initiate a conversation to help you. For example, a Business Analyst (BA) writes a line of requirements: “application starts with login” and the service would reply “Do you mean the Authentication Service?” BA would confirm and the service would ask “What Roles and Privileges do you have in mind for your users?” This friendly dialog will lead to building a working application on-the-fly.

How close is this picture to reality? A NoSQL database, like Triple Store [1], is reality. Triple Store can be downloaded for free and available from multiple vendors. The benefit of this type of storage is in its simplicity. Triple Store does not require data modeling and new tables. Think about an unlimited size single table with three columns. Any database administrator would tell us that this is not efficient. This might not be efficient for a single specific task, but this is extremely productive for multiple tasks dealing with terabytes of corporate information. For example, the DoD plans to transition from relational databases to NoSQL storage and re-allocate a significant part of its future IT budget from maintenance to development [2].

The conversational service was described in the book on integrated software and knowledge engineering “Integration-ready Architecture and Design” [3]. Keep reading and check for the Conversational Semantic Decision Support (CSDS) section; this article expands on that subject. The trickiest part is collecting complete information related to a company and its business.

Information: Structured, Unstructured and “Tribal Knowledge”

What is so tricky about collecting complete information related to a company and its business?

There are two major parts of electronically stored information: structured data and unstructured data.

Structured data are very formal. They are defined in terms so precise that not only people but even computers can understand these data. Databases, Business and Data Models, Services and XML Files are structured and understood by computers.

Unstructured data are documents and communications artifacts, like taped messages and video clips that make sense to people. The knowledge captured in unstructured data is hardly available to computer systems. We’d like to change this. We’d like to use this knowledge to make computers a bit smarter. Semantic technologies help us to better organize unstructured information and create a conceptual model (ontology) of the knowledge that is represented there. I promise to expand on semantic technologies a bit later.

The biggest portion of information that is used daily in business routine has never been captured. It is so-called “Tribal Knowledge”. My conservative estimate of the ratio between structured, unstructured and “tribal” knowledge is 10%, 20% and 70%. For many companies, more realistic numbers are 5%, 15% and 80%.

Informational gaps or uncaptured “tribal knowledge” is not a problem for a single subject matter expert (SME).

The problem is more visible when several SMEs participate in a complex process.

We cover informational gaps in every field with multiple meetings and phone calls.

Constant (and expensive!) re-discovery is our “normal” process.

It becomes abnormal after we have invested in SOA and would like to link business vision to services. We feel the growing dissonance between advanced architecture and old knowledge culture.

Director of Enterprise Architecture, Yefim leads Information Architecture at Sallie Mae. In the past he consulted government agencies and corporations in SOA and knowledge engineering, shared his expertise at Java One, Wireless One and Boeing Conferences. Cambridge University Press published his book “Integration-Ready Architecture and Design”. In the book and several patents he described a new field of Integrated Software and Knowledge Engineering and Knowledge-Driven Architecture. Hobby: mountaineering and guitar.

About the author

Chief Architect at ITS, Inc., Yefim (Jeff) Zhuk, worked as Director of Enterprise Architecture at Sallie Mae, consulted Boeing and other corporate and government agencies in SOA and knowledge engineering, shared his expertise at Java One, Semantic Tech, and Boeing Conferences. In his books, patents and publications Yefim described a new field of Integrated Software and Knowledge Engineering. The methods and prototypes described him as Business Architecture Sandbox for Enterprise (BASE) and Conversational Semantic Decision Support place new technology seeds in the current business ground, helping transitioning to rule-based applications and Semantic Cloud Architecture.