Oracle and Microsoft Join the MITI Metadata Bridge Love In

I don’t think Oracle or Microsoft enjoyed being the only vendors to move backwards in the last Gartner Magic Quadrant for Data Integration tools. Both vendors may look for help and join in the metadata sharing love in.

Over the last 18 months Oracle and Microsoft have been working with metadata broker vendor MITI to deliver a set of metadata bridges for exchanging metadata with various Data Integration, Business Intelligence and Modelling tools. The biggest beneficiary of these brokers are Microsoft and Oracle’s competitors.

MITI Background

But first some background. A company called Meta Integration Technologies Inc – MITI – makes metadata bridges and brokers that exports the metadata of popular enterprise tools to a common metamodel that can the be imported into other tools.

Incorporated in 1997, Meta Integration Technology, Inc. (MITI) is a privately held company (100% employee owned) based at the heart of Silicon Valley (Mountain View) in California. MITI is the leading metadata component provider to major database, data integration, business intelligence, repository, and modeling tool vendors. MITI has established itself as the " Switzerland of Metadata " as expressed by then-Giga Information Group Analyst Lou Agosta in a 2002 IdeaByte article.

MITI has been profitable since its inception in 1997. MITI started as a metadata solution provider to large corporations and government agencies largely through consulting. MITI rapidly focused as the leading metadata component provider to the largest data integration, business intelligence and database vendors which represents 80% of its revenues today.

MITI is a 100% employee owned company. Every employee participates in the stock option plan. We want every employee to share in MITI's financial success because we believe employee ownership is critical to our long term success.

IBM uses them in Cognos, Information Server and Rational Data Architect. Informatica uses them for PowerCenter. Business Objects, SAS, ErWin and SAP MDM also use them. There were two notable holdouts on metadata – Microsoft and Oracle had very few tools on the list.

A metadata bridge or metadata broker attaches itself to the application layer of a product and sucks out metadata that describes the components being built by that product. So for BI tools like Business Objects and Cognos it sucks out the metadata layer of entities and attributes, plus the reports that use that metadata. For ETL tools like DataStage and PowerCenter it sucks out job names and some of the tables and columns they process. When you use these brokers to import metadata they can help a metadata reporting tool detect data lineage across products – finding where an ETL job loads a table, what database schema that table is in and what BI tool defines it as a metadata source and the BI reports that use it.

Here is a picture from the MITI website:

It’s important to note that most DI, BI and modelling vendors will benefit from the left side of this diagram – easy exchange of metadata. Only vendors like IBM and Informatica with metadata reporting tools will benefit from the right hand side with tools that can do the metadata stitching and data lineage across products. MITI also supply a metadata management tool that shows a data lineage like this one:

There are three types of MITI partners:

- Those who help build MITI bridges, Oracle and Microsoft fall into this category.

- Those who help build MITI bridges and bundle bridges with their own products. These are known as OEM partners – Original Equipment Manufacturer. IBM fall into this category.

- Those who help build MITI bridges, bundle bridges and bundle additional metadata management or mapping tools. Informatica are in this category.

With the way premium vendors are joining the MITI partner list they are more like the United Nations of metadata rather than the Switzerland of metadata. They have a lot of contracts now with major vendors that have made these vendors dependant on MITI brokers. It makes me wonder what this 100% employee owned company would think of an acquisition offer from a major IT vendor for over $100 million and how the United Nations of metadata would operation under the ownership of a sovereign metadata nation.

Without MITI

Before release of the Information Server IBM-Ascential tried writing their own metadata brokers. Waiting for a vendor to build in house import bridges is like waiting for the latest Hollywood movie to come to free to air television. When most of my clients were using ErWin 7 I only had access to ErWin 4 brokers and there were only a handful of brokers to choose from. MITI are like pay per view television – they bring you a wider range of brokers more quickly. You still have to wait for them to build and certify brokers for the latest product versions but because they are supplying these brokers to dozens of software partners and it’s their core business they do it more quickly and professionally.

Life without MITI makes the data integration vendors less competitive as they waste valuable time and money working on these tricky brokers when they should be out building cool metadata management tools.

MITI and Microsoft

MITI have expanded the Microsoft bridges since 2007 for all the SQL Server 2005 add on tools. MITI CEO Christian Bremeau is at the SQLPass Summit this week presenting on what the new bridges mean to Microsoft users:

Microsoft's data integration and business intelligence (DI and BI) suite of tools based upon the SQL Server platform (Integration, Analysis, and Reporting Services, or SSIS, SSAS, and SSRS) comprise a rich opportunity for metadata integration with other popular modeling tools. In particular, data modeling tools, such as CA ERwin Data Modeler, may contain and/or maintain the metadata required for source and/or target DI specifications in SSIS, SSAS relational databases. Other design tools, like Kalido, maintain warehouse and BI designs which may be forward engineered into SSAS. In addition, with multiple producers and consumers of metadata there is a great need for multi-vendor metadata management, as well as end-to-end lineage and impact analysis. This session will explore the means by which these other vendors are able to exchange relational, OLAP and reporting design metadata with SSIS, SSAS and SSRS. In addition, the session will show end-to-end lineage and impact analysis for complete IM solutions based upon a collection of the products. Finally, the metadata management implications of such a heterogeneous environment will be discussed.

Microsoft still don’t have an OEM arrangement with MITI so if a Microsoft customer wants these tools they have to go to MITI and buy them – which is a bit of a problem if your average Joe the Plumber hasn’t hear of MITI and MITI don’t have the kind of massive sales teams that enterprise vendors have. Remember 80% of MITI revenue comes from partners, they don’t get many walk in customers. So I expect most of the people using the Microsoft bridges will be using them with products that compete against Microsoft like IBM, Informatica and Sybase.

Microsoft has a whitepaper on SQL Server 2005 metadata that goes into the disjointed metadata offerings. Microsoft are trying their hand at metadata – they have extended properties on SQL Server tables that can capture additional metadata about those tables and columns. It’s a poor place to put business metadata because business users can’t find it if they don’t know what the physical column names mean.

Microsoft added some free metadata tools that use the SSIS API such as the DepencyViewer. You start to see data lineage and diagrams but it only seems to work on SSIS metadata and cannot see the metadata from other SQL Server tools or from metadata imported via MITI bridges.

So with the current metadata strategy Microsoft are not about to advance on the data integration quadrant. They need to turn the fledgling SSIS metadata tools into a proper metadata offering and bundle the MITI bridges for larger SSIS sites so they can get at decent metadata.

MITI and Oracle

In one of my IBM IOD posts I said that Oracle were a MITI black hole – few products worked with MITI bridges and they were invisible to the data lineage reporting and modelling tools. MITI CEO Christian Bremeau sent me an email correcting me:

So the current Information Server is running version 5.x bridges but the version 6.x bridges that have additional Oracle and Microsoft support are coming to the Information Server soon.

In fact MITI and Oracle signed a partnership in 2004 so that MITI metadata management tools could be used with Oracle Warehouse Builder. There are two problems with this partnership – 1) it has taken a few years to build the bridges for all the Oracle data integration tools. 2) No one at Oracle seems to know about this partnership.

The incorporation of MITI's Meta Integration® Model Bridge (MIMB) technology with Oracle Warehouse Builder provides users with greater control of their business data integration processes. MIMB enables the movement of metadata across numerous environments such as databases and data warehouses as well as BI, design, and extract, transfer and load (ETL) tools. Metadata is defined as data about data and protects data investment by setting the stage for the data re-use. This partnership enables:

Synchronization of data models across different data repositories and BI tools

Elimination of manual re-coding of data models throughout organizations

Simplification of the maintenance of metadata and data models

According to the partnership FAQ:

1.1 What is the Oracle Warehouse Builder metadata integration solutionStarting from the Warehouse Builder version 9.2 it is possible to import metadata from a number of third party tools by using the Metaintegration Technology Inc. (MITI, www.metaintegration.net) technology, now seamlessly integrated into OWB. It is now possible to import metadata from a variety of third party tools into OWB by using the MITI bridges. Warehouse Builder users can now reuse data model designs developed by using a number of third party tools.

The main problem is that MITI bridges are not bundled with OWB:

3.1 How to obtain the MITI bridges?The MITI bridges can be purchased and downloaded from the MITI web site (www.metaintegration.net ). A free trial version of all the bridges is also available. The trial version has all the functionalities available in the full version but limits the number of metadata objects (tables, views etc.) that can be moved to 10.

When I ask Oracle sales about data lineage and impact analysis I get blank stares. Maybe that's because MITI no longer sell bridges directly to customers, they only sell through partners. Oracle don't sell it, they recommend it. They are recommending something you can no longer buy! Despite acquiring Sunopsis and signing a deal with Trillium for data quality and have MDM products in the stables and helping to build MITI bridges they still don't offer metadata exchange between products.*

Are MITI bridges even bundled in the Oracle Data Integration Suite? It should be.

MITI and IBM

In my next MITI blog post I will look at how the bridges are being used by IBM and Informatica – but I’ve rambled on enough and there is only so much information about metadata that the human brain can absorb in one session, and I passed that threshold a few paragraphs ago. For now I’ll just say that IBM offers all the MITI bridges to Information Server customers without any additional licensing as they have the Metadata Workbench – a metadata reporting tool that can stitch up the metadata between tools and they will be major beneficiaries of the new Oracle and Microsoft bridges.

Both Microsoft and Oracle could improve their metadata plight by bundling more MITI products but hey don't seem to be aiming for the cross enterprise market of the major data integration vendors.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

Vincent McBurney is an IBM Information Champion for Information Integration.

Related White Papers

5 Comments

UnknownNov 20, 2008

ReachIT is a premier IT training and development centre located in Hyderabad, India. We provide excellent quality ONLINE / IN-CLASS trainings on almost all latest software courses.

We at ReachIT guarantee you excellent Quality trainings. We have the excellent team of corporate trainers who take up the classes as per your convenience. Our trainers are well experienced and are working with the leading MNC'S .They have expertise in their respective fields.

* I updated one of the Oracle paragraphs after getting some additional information about the bridges. Oracle were recommending the bridges in 2004 but in 2008 don't seem to offer MITI bridges in any way to customers. They haven't replaced them with an alternative and it seems to be a backward step for Oracle data integration.

Thanks again for Vincent's article to clear some of the fog in metadata solution battle field.

Each BI and ETL vendors claim that they have the universal metadata solution, or they provide the centralized metadata repository which can serve as the hub for the whole enterprise. Every year, the customers are excited about the marketing slides from SAS, Cognos, Informatica, Oracle, etc., and then expect to build their comprehensive metadata center based the vendor's tool. But the reality is:

These vendors (including IBM) only dumped the MITI libraries files into their ETL/BI tool directory, and create a metadata repository focusing on the vendor's own core metadata structure

There will be an web application on top of such repository to render all kinds of metadata reports. But little detail of the imported metadata (from its competitors' tools) can be explored here, because none of the names above really care that much about the competitor's metadata structure.

CWM or CWMI is often mentioned in those vendor slides, but the real implementation is far from satisfaction. As Vincent found, some feature claims were not fulfilled after many years :-)

Even Informatica and DataStage's metadata components are not even quite useful until second-half of 2008.

SAP has a pretty good metadata/knowledge management implementation(in my opinion), which glues the business and technical metadata very well. But it's hard to imagine that people import other vendors' metadata into SAP and let it serve as the metadata portal

The storage of such metadata repository can be popular RDBMS or even proprietary in-memory database. If the customers want to leverage the metadata to generate code/document, conduct further impact analysis, mash up with other application, etc., then they will have to learn the vendor's SDK for the metadata component in addition to MITI SDK, or reverse engineer the repository data model.

BI/ETL tools' metadata is a gold mine, but due to the vendor disparity/isolation, a lot of effort has to be dedicated to integrate the metadata across vendors (even among different components of a single vendor).

There are pure metadata vendors which are independent to ETL/BI vendors, such as:

Adaptive Repository http://www.adaptive.com

ASG / Rochade http://www.rochade.com

InfoLibrarian http://www.infolibcorp.com

But these folks are very expensive, and they always fall behind the upgrade pace of the ETL/BI vendors's metadata structure.

In the coming 2~3 years, I hope to see:

Multiple metadata repositories co-exist within a single IT infrastructure

A cross-vendor mash-up & reference application is running as the metadata portal

High-level lineage can be rendered by the portal

Detailed lineage and report are rendered within the vendor's own metadata component, while the drill-down path is provided & maintained at the portal level

Thanks for the great comment Eric. I agree with your wish list. Metadata vendors need to become better at delivering data lineage and semantic metadata into the corporate portal.

My wish list for 2009:
Informatica and DataStage make the metadata reporting tools free and open it up wide to enterprise search tools, BI tools and wikis. This will give these metadata repositories a better chance of surviving and being relevant to business users with flow on benefits and sales of the other products in the suite (profiling, etl and data quality).

I think the enterprise search box will be the main place people go for information and if the metadata repository is a walled garden that requires a special login/URL/license then it will fail. I've seen million dollar metadata implementations that get beaten for popularity by a free glossary search tool because the glossary search tool is on the intranet home page for all users and the metadata repository is not.

Disclaimer: Blog contents express the viewpoints of their independent authors and
are not reviewed for correctness or accuracy by
Toolbox for IT. Any opinions, comments, solutions or other commentary
expressed by blog authors are not endorsed or recommended by
Toolbox for IT
or any vendor. If you feel a blog entry is inappropriate,
click here to notify
Toolbox for IT.

Vincent McBurney is an IBM Champion for Information Integration and has been blogging for many years on InfoSphere software and ...
more

Vincent McBurney is an IBM Champion for Information Integration and has been blogging for many years on InfoSphere software and competitors in Information Management, Governance, Data Integration and Data Warehousing.
less

Receive the latest blog posts:

Share Your Perspective

Share your professional knowledge and experience with peers. Start a blog on Toolbox for IT today!