Category Archives: Graphitization

[Since first writing this article in January 2018, I’ve concluded that Ethereum is not capable of being a platform for Tokenizing Every Little Thing. Ethereum is a one-trick pony x 1500 when it comes to creating large-scale decentralized applications (i.e. Ethereum/Solidity smart contracts are best for creating single, simple entities like alt-coins). Checkout slide 56 of this presentation: NEO Blockchain Vancouver 20180315 Meetup. The NEO Blockchain and NEO Smart Economy is the best available 3rd generation distributed application platform on the planet and improving every day. Michael Herman, March 17, 2018]

Move beyond digitalization of the enterprise to graphitization of the enterprise.

For 2018 and beyond, the challenge is simpler but more difficult:

Tokenize Every Little Thing (ELT)

To provide more context, let me first quote from the introductory paragraphs of the #Graphitization article.

Here’s a great diagram that explains this concept [graphitization]. (click on the diagram to enlarge it)

Figure 1. The New Model of IT

Graphitization of not only all of your corporate information assets across all of your constituencies and stakeholders – at the data, application entity, and business object level – but also the graphitization of all of the interconnections between every business process, application system, infrastructure component, cloud service, vendor/service provider, and business role that uses, manages, or stores corporate information (Crossing the EA Chasm: Automating Enterprise Architecture Modeling #2).

Use graphitization to make your existing corporate information more available, more usable, and more informative. Graphitization enables you to “Keep Calm and Have IT Your Way“.

What is #Graphitization?

#Graphitization is a data science and enterprise architecture-inspired framework and process model for modeling, ingesting, organizing, analyzing, and visualizing any domain of endeavor by using graphs – networks of connected objects and relationships with each object and relationship annotated with additional descriptive information (metadata).

Why #Tokenization?

Given the burgeoning preoccupation of the world’s business, finance, government, and technology sectors with blockchain technologies, cryptocurrencies, and token-this and token-that, the buzzword for 2018 will be #Tokenization …the creation of tokens (multiple versions of tokens) to represent every thing on the planet …Every Little Thing (ELT).

While individuals, startups and larger organizations are trying to dream up the next big, one-off, token or crytocurrency, why not just admit that, “in the end”, everything will be represented by a token?

Why try to knock these off one at a time (e.g. Bitcoins, Ethers, altcoins, CryptoKitties, letters of credit, auctions, escrow agreements, electronic health records (EHR), electronic medical records (EMR), etc.) when the ultimate goal to to create a universal interconnected graph of ELT (Every Little Thing) in the universe?

Why #graphitize the enterprise when you can #tokenize the universe?

What is #Tokenization?

Let’s get a little computer-sciency for just a minute. A common task to to take an input stream (a string of characters, a stream of data, a data file or database table), analysis it, and convert it into a collection or sequence of higher-level tokens for further analysis (a process that can be applied recursively). Here’s an explanation from Wikipedia…

In computer science, lexical analysis, lexing or tokenization is the process of converting a sequence of characters (such as in a computer program or web page) into a sequence of tokens (strings with an assigned and thus identified meaning). A program that performs lexical analysis may be termed a lexer, tokenizer, or scanner… [https://en.wikipedia.org/wiki/Lexical_analysis]

Coming up for air… Why not represent ELT that happens in the universe as a stream of blockchain transactions?

the events in your life?

everything that occurs during a Presidential election?

the 24-hour cycle of one day changing into the next?

the activity-by-activity and task-by-task execution of a business process?

a stream of events from your Internet-of-Things (IoT) enabled car, toaster or refrigerator?

Jim Gray and TerraServer

In one of his several presentations on Scalable Computing (circa 1999), Jim Gray (relational database pioneer and Turing Award winner) describes the TerraServer project in the following way:

[Users navigate] an ‘almost seamless’ image of earth.

SkyServer was a similar project quarterbacked by Gray:

TerraServer allowed access to newly-available satellite imagery with resolution of 1.5 meters/pixel. SkyServer, a collaboration with Alexander Szalay and his colleagues at Johns Hopkins, allowed access to astronomical data from the Sloan Digital Sky Survey. SkyServer led to additional work with astronomical data, … [https://amturing.acm.org/award_winners/gray_3649936.cfm]

Tokenize Every Little Thing

With the advent of blockchain technologies (in particular, the Ethereum extensible blockchain platform), why can’t we embark on a grander mission to tokenize Every Little Thing? …and including all token-pair relationships (TPRs).

What will it take?

What needs to change in the Ethereum blockchain platform? Will Ethereum be able to scale to support modeling, ingesting, organizing, analyzing, and visualizing of Every Little Thing (ELT)?

Gordon Bell, MyLifeBits MSR Project (early 2000’s). I remember Jim Gray telling this story but I had trouble finding a proper reference because I thought it was Gray’s story instead of Bell’s. I now know better but I’ve already finished the above article. A Wikipedia MyLifeBits reference can be found here. YouTube videos can be found here, here, and others over here. Channel 9 videos: Part 1 and Part 2. Computerworld article (2008). Business Inside article (2016).

Michael Herman is an expert when it comes to the mathematical modeling, analysis, and visualization of almost everything:

Large enterprise organizations,

Commercial, global-scale, cloud services platforms,

Organization principles and belief systems,

Human platforms,

Aircraft engines, and

Muscle cars.

Michael is the inventor of the #Graphitization Continous Transformation Model – a closed-closed loop feedback process for the ingestion, modeling, analysis, visualization, systems optimization, and life cycle management of any type of strategy, system, asset, architecture, or process.

Figure 1. #Graphitization Continuous Transformation Model

A key concept of #Graphitization is the implementation of Transformative Changes that result in positive increases in business value in the system being modeled.

#Graphitization

What is #Graphitization?

#Graphitization is a data science and enterprise architecture framework and process model for modeling, ingesting, organizing, analyzing, and visualizing any domain of endeavor by using graphs – networks of connected objects and relationships with each object and relationship annotated with additional descriptive information (metadata).

The primary applications of #Graphitization are:

System optimization,

Systems life cycle management, and

Transformative Change in resulting in positive increases in business value for the system being studied.

A system is defined as any collection of strategies, system components, assets, architectures or processes.

Description

SerentityData Graph is an open source, entity-relationship modeling, serialization, and graph data visualization and analysis solution that supports the development of traditional full-stack, blockchain-based smart contract, and Neo4j graph database applications.

SerentityData features complete life-cycle integration with Neo4j for on-chain and off-chain graph data creation, migration, visualization, and analysis. Live code walk-throughs and demonstrations will enable you to begin using SerenityData and Neo4j immediately. Github: https://github.com/mwherman2000/serentitydata-compiler

4. Automated service composition of cloud services-based data systems

Call the solution “Expedia for Microsoft Azure/AWS/SFDC/…” or whatever you prefer, today’s commercial cloud services platforms are still a pain in the ass to use for creating non-trivial applications. Left, right, and center you have to hand-code a myriad of worker processes simply to reformat and pass data around.

5. Large collaborative ecosystems: employee groups, business partners, social networks

Project “Boston” is named after some potential business partners and the embryo for the idea coming from my months as a founding Groove Networks business partner (including many of my most important relationships that I still maintain today).

6. Large ecosystems of competing or competitive business organizations

Modeling of large ecosystems of competing/competitive business organizations is a straightforward #Graphitization use case.

7. Organization principles and belief systems

On the surface, the #Graphitization of principle and belief-based frameworks is pretty straightforward but this is because the basic #Graphitization serves as the substrate for many advanced data ingestion, analysis, and visualization projects.

Below are the results of the #Graphitization of two principle and belief-based frameworks:

References

11. Parallelspace ModelMate

Parallelspace ModelMate is an approach (platform and language framework) for creating domain specific languages (DSLs) for enterprise architecture. It is realized using #Graphitization and the ArchiMate enterprise architecture modeling language.

References

12. Internet of Things (IoT)

IoT is an interesting beast. It is a reference to an application service for processing raw events from a device or dynamically generated events from a software system. IoT also defines a conceptual software and data flow architecture that can also be used for the dynamic creating and maintenance of complex systems such as large enterprise architectures.

Appendix A – Amazon Leadership Principles (and Subprinciples) contains an ArchiMate enterprise architecture model that depicts the (and then decomposes) the 14 Amazon Leadership Principles into multiple levels of subprinciples. Scroll down to the bottom of this article to check it out.
NOTE: The underlining in Appendix A attempts to highlight the individual Subprinciples and Relationships found in the text description of each of the 14 Principles.

The first real section Amazon Leadership Principles, Core Entities, and Relationships presents a new innovative way to learn, remember, understand, and apply the Amazon Leadership Principles as highly visual web (or mesh or graph) of principles, concrete entities, abstract entities, and relationships.

The last section (just before Appendix A), entitled Personal Leadership Principle Maps, depicts how the experiences and accomplishments of one person’s career (mine) can be (formally) mapped the Amazon Leadership Principles.

Let’s start the journey. If you’re not familiar with the Principles, start by reading:

Appendix B – Amazon Leadership Principles; then

Appendix A – Amazon Leadership Principles (and Subprinciples)

All of the figures in this article represent different graphitized views of the Amazon Leadership Principles (click here) …all built from a single underlying graph model (which, in total, is referred to as the #Graphitization of the Amazon Leadership Principles).

The existence, enablement, creation and/or execution of each group of relationships gives rise to (or realizes) one or more of the 14 Principles (and/or their Subprinciples). When these realization relationships are added to the Core Entities depicted in Figure 1, Figure 2., the “Complete Model”, is the result. (Click to enlarge.)

To simplify the understanding of the model, 14 new views were created – one for each of the 14 Principles – each overlayed on the original Core Model (Figure 1). Figure 3 is an example drawn from one of these 14 views: Principle 1. Customer Obsession.

So far, we’ve addressed the “what” of the Amazon Leadership Principles depicted as a #Graphitization model projected as a number of different views.

In the next section, the Amazon Leadership Principles are used as a framework for cataloging one’s lifetime experiences and accomplishments. Personal Leadership Principle Maps is an Amazon Leadership Principles application – it’s the Amazon Leadership Principles put into action.

Personal Leadership Principle Maps

Have you been living an Amazon Leadership Principled career/faith/life?

Figure 5. is a copy of my Personal Leadership Principle Map (PLPM).

ArchiMate Assessment entities are used to model specific experiences and accomplishments.

ArchiMate Outcome entities are used to model specific evidence, learnings, or proof that one has been able to apply the specific principle in their career, faith and/or life.

In my case, for Principle 7. Insist on the Highest Standards, I have specific experiences related to the recent Toronto Salesforce 2017 Tour, working at Parallelspace Corporation, the IBM Canada Toronto Software Lab, and at Microsoft.

Specific evidence includes:

Parallelspace trust framework (Relationships-Reputation-Trust)

Working as an ISO-9000 Quality Analyst and a certified Quality Assurance Auditor

A concept I call focusing on the success of an Individual Individual

Various and diverse experiences working for Microsoft as a full-time employee (blue badge) and as a Microsoft partner

Next Steps for Iteration 2

Possible next steps include:

Federation of Personal Leadership Principle Maps – at the Employee Team, business unit, or Organization level to discover the aggregates collective experiences and accomplishments for the purpose of rebalancing hiring objectives (Principle Gap Analysis), accumulating customer as well as competitive intelligence, etc. to support Customer Obsession, Ownership, Invent and Simplify, etc. goals and objectives. Identifying the best sources of experiences and accomplishments for specific Principles based on a Team’s or Organization’s previous roles, education, or training.

Use of both the Core Model and the Complete Model as well as the Federate Personal Leadership Principle Maps to create a graph database repository to real-time query analysis and visualization (e.g. using the Neo4j graph database).

Appendix B – Amazon Leadership Principles

The underlining attempts to highlight the individual Subprinciples and Relationships found in the text description of each of the 14 Principles.

Leadership Principles

Our Leadership Principles aren’t just a pretty inspirational wall hanging. These Principles work hard, just like we do. Amazonians use them, every day, whether they’re discussing ideas for new projects, deciding on the best solution for a customer’s problem, or interviewing candidates. It’s just one of the things that make Amazon peculiar.

Customer Obsession (1)

Leaders start with the customer and work backward. They work vigorously to earn and keep customer trust. Although leaders pay attention to competitors, they obsess over customers.

Ownership (2)

Leaders are owners. They think long term and don’t sacrifice long-term value for short-term results. They act on behalf of the entire company, beyond just their own team. They never say “that’s not my job”.

Invent and Simplify (3)

Leaders expect and require innovation and invention from their teams and always find ways to simplify. They are externally aware, look for new ideas from everywhere, and are not limited by “not invented here”. As we do new things, we accept that we may be misunderstood for long periods of time.

Are Right, A Lot (4)

Leaders are right a lot. They have strong judgment and good instincts. They seek diverse perspectives and work to disconfirm their beliefs.

Learn and Be Curious (5)

Leaders are never done learning and always seek to improve themselves. They are curious about new possibilities and act to explore them.

Hire and Develop the Best (6)

Leaders raise the performance bar with every hire and promotion. They recognize exceptional talent and willingly move them throughout the organization. Leaders develop leaders and take seriously their role in coaching others. We work on behalf of our people to invent mechanisms for development like Career Choice.

Insist on the Highest Standards (7)

Leaders have relentlessly high standards – many people may think these standards are unreasonably high. Leaders are continually raising the bar and driving their teams to deliver high-qualityproducts, services, and processes. Leaders ensure that defects do not get sent down the line and that problems are fixed so they stay fixed.

Think Big (8)

Thinking small is a self-fulfilling prophecy. Leaders create and communicate a bold direction that inspires results. They think differently and look around corners for ways to serve customers.

Bias for Action (9)

Speed matters in business. Many decisions and actions are reversible and do not need extensive study. We value calculated risk taking.

Frugality (10)

Accomplish more with less. Constraints breedresourcefulness, self-sufficiency, and invention. There are no extra points for growing headcount, budget size or fixed expense.

Earn Trust (11)

Leaders listen attentively, speak candidly, and treat others respectfully. They are vocally self-critical, even when doing so is awkward or embarrassing. Leaders do not believe their or their team’s body odor smells of perfume. They benchmark themselves and their teams against the best.

Dive Deep (12)

Leaders operate at all levels, stay connected to the details, audit frequently, and are skeptical when metrics and anecdote differ. No task is beneath them.

Have Backbone; Disagree and Commit (13)

Leaders are obligated to respectfully challenge decisions when they disagree, even when doing so is uncomfortable or exhausting. Leaders have conviction and are tenacious. They do not compromise for the sake of social cohesion. Once a decision is determined, they commit wholly.

Deliver Results (14)

Leaders focus on the key inputs for their business and deliver them with the right quality and in a timely fashion. Despite setbacks, they rise to the occasion and never settle.

This article documents the current iteration of the ModelMate project, Iteration 2, whose goal is to create an improved queryable repository for a corrected version of relationship matrix (more correct relative to the Appendix B tables in the current ArchiMate 3.0 Specification).

Background

Over the past few months, I’ve written several articles commenting on the current state of the ArchiMate language for Enterprise Architecture including:

The recent effort, called the ModelMate project, is a focused effort to create a more broadly applicable, usable, useful, ArchiMate-based, extensible language environment for enterprise architecture as described in these 4 articles:

The current scenario is highlighted by the following points taken from the above references:

“[People should be] encouraged to try to model these examples for yourself: to start learning how to “think in ArchiMate” as your second or third written language.” The way the ArchiMate language is currently designed and, more importantly, described makes this difficult.

No machine-readable version of the tables are available for external validation for correctness

The tables contain errors in the approximately 11,000 relations that are represented in the tables. Is is estimated that there are few hundred to a few thousand errors present in the current ArchiMate 3.0 tables

The tables contain all possible (valid) relations but do not differentiate between Core relations and Derived (non-Core) relations.

All three issues are critical for the ArchiMate 3.0 Specification and these tables to be trusted and more generally useful.

In addition, the Derived Relation Derivation Algorithm has never been published by The Open Group. Attempts to create an alternative algorithm have highlighted that the text of the ArchiMate 3.0 Specification is neither consistent nor complete when it comes to identifying the set of Core Relations and a correct and complete Derviation Algorithm.

Lastly, when dealing with 1000+ Core Relations and several thousand Derived Relations (8000-9000 or more), it’s difficult to analyze and visualize what the ArchiMate 3.0 relationship matrix looks like in total, or when subdivided by Domain (Layer) or Aspect, or when focused on a specific element prototype (e.g. Node).

Solution Overview

The goal of this solution is to publish a very detailed, rich, unnormalized version of the latest and greatest ArchiMate 3.0 relationship matrix in multiple formats; including:

CSV text file

Microsoft Excel workbook

Microsoft Access database

Neo4j Cypher Query Language (CQL) queryable graph database file

When loaded into Microsoft Excel, the CSV and Microsoft Excel workbook format files appear as shown in Figure 1 (below).

Figure 1. ModelMate Master Datasets: Excel 2016 and CSV File Formats

The Microsoft Excel (and CSV) format file can also be used with the Microsoft Excel Web App (Figure 2) and the Microsoft Excel format can be used to create custom SharePoint lists (Figure 3).

Figure 6 is an example of the output from a single line CQL query run against the ArchiMate 3.0 graph database (implemented using Neo4j). If you look closely at the CQL statement at the top of this screen shot (click Figure 6. to enlarge it), you’ll see that it is selecting all of the relationships across all of the element prototypes in the Technology/Infrastructure domain of the ArchiMate 3.0 metamodel that connect to the Node element prototype.

Figure 6. ModelMate Master Datasets: Graph Mining Analysis Sample

File Downloads

You can download the files referred to in this article from the GitHub repository. Click here to download the ModelMate Master Datasets files.

In addition, there is a Neo4j Cypher Query Language (CQL) file available for download that will ingest all of the element prototypes and relations into a graph database using a single Neo4j shell invocation. From the Windows Powershell or Windows Command Prompt, use:

Lastly, there is Microsoft Access 2016 database version of the CSV file that is available for download if you prefer using Microsoft Access SQL queries or graphical SQL queries.

Solution Details

Below is a copy of the workflow and dataflow used to create the Parallelspace ModelMate Master Datasets. It’s not as messy as it looks – it’s true mashup and a valuable one at that. It’s primarily the result of the truly ad-hoc collaboration between 3 enterprise architecture professionals with an interesting mix of diverse goals (Gerben Wierda, Ed Roberts and myself); each of us with our own set of preferred development technologies and goals (with Excel being the greatest common denominator (GCD)).

Figure 7. ArchiMate 3.0 Relationship Matrix Resolution Process

The numbered steps in Figure 7. are explained below:

Data Sources. There are many sources of information about the ArchiMate relationship matrix in addition to the Appendix B tables in the ArchiMate 3.0 Specification. The list in Figure 7. is a fairly complete. Key data sources include the GitHub Archi repository for the most widely used ArchiMate modeling tool for enterprise architecture and Gerben Wierda’s multiple ArchiMate resources publishing under the Mastering ArchiMate brand.

“MA Core Set” Spreadsheet. Wierda worked to consolidate various data sources from Step 1 above to create the “MA Core Set” Mastering ArchiMate relationship matrix (plus a number of other relationship matrices that Wierda used for comparative analysis and troubleshooting purposes). The “MA Core Set” represents the “seed” or Core Set of (non-derived) ArchiMate relations. Wierda created this Core Set over several iterations reviewing the word-for-word text of the Specification, the inheritance diagrams, as well as incorporating his extensive practical knowledge and experience documenting ArchiMate in the book entitled Mastering ArchiMate – Edition II.

The “MA Core Set” tab in the AllowedRelationsArchiMate30VBA-public.xlsm Excel spreadsheet also includes additional columns that are reserved for calculating and storing an intermediate 3-column, reverse-transposed version of the relationship matrix (Step 3 below).

CreatePrologList() Visual Basic for Applications (VBA) Macro: This macro is used to perform the actual reverse-transposition of the “MA Core Set” relationship matrix into the 3-column format which including a column for storing the relation(source,target) 3-tuple formatted data (in Prolog format). The 2-D relationship matrix is the input to the macro (along with some additional master data tables that are part of the VBA code). The 3-tuples are the essential output of the VBA macro (stored “in-place” in the first 3 columns of the spreadsheet).

CoreSet.prolog File. To proceed through to the next step of the workflow, the Prolog format data is copied from the spreadsheet and pasted into a plain text file called CoreSet.prolog, for example (or any other filename you would like to use).

Derivaton.py Python Script and outfile.csv. The Derivation.py script contains is the “magic sauce”. Written by Wierda, Derivation.py reads the CoreSet.prolog file and executes a complex and detailed algorithm to expand the Core Set of ArchiMate relations read from the CoreSet.prolog file into a number of alternative output formats, including CSV and Prolog formats.

To support the ModelMate project, a version of Derivation.py was modified to output a number of additional CSV columns (outfile.csv). Columns:

SourceElement

TargetElement

Relation

RelativeStrength

IsInputRelation

StandardVersion

ScriptVersion

Outfile.xml File. Steps 6 and 7 are part of a sequence of activities that were used to create a relationships.xml file that is compatible with the relationship configuration requirements of the Archi modeling tool. This process, originally implemented by Ed Roberts, owner of Dallas-based Elparazim, uses Excel to load the outfile.csv save it out as an outfile.xml file.

For Step 7, Ed Roberts wrote an XSL Transform script that when applied to the outfile.xml file creates the Archi-compatible relationship.xml that is used by the Archi model to automatically configure the element-element relations supported in a given version of Archi (e.g. Archi 4.0).

Steps 8-10 mark an alternative data flow created to support the needs of the ModelMate Master Datasets project.

In Step 8, the contents of the ModelMate-compatible modified CSV output from Step 5 (outfile.csv) is copy-and-pasted into the Parallelspace_ModelMate_MasterDatasets_CoreAndDerivedNN.xlsx Excel workbook (where NN is a version number).

A matrix of automated Excel functions in the Complete spreadsheet merge the elements and relations master data attributes from the Elements and Relations spreadsheet with the data from the Derived spreadsheet to compute the corresponding column values in the Complete spreadsheet. Think of the Complete spreadsheet as a super-unnormalized version of the relationship matrix. The InInputRelation column values indicate whether a specific relation (and it’s companion source and target elements) are Core relations or Derived relations.

Derived spreadsheet – copy-and-pasted version of the outfile.csv from Step 4. The “input” spreadsheet.
Columns:

SourceElement

TargetElement

Relation

RelativeStrength

IsInputRelation

StandardVersion

ScriptVersion

Complete spreadsheet – leverages the master data in the Elements and Relations tabs to expand the columns in the Derived spreadsheet to include additional metadata property columns for the source and target elements as well as the relations. The “output” spreadsheet that will be saved as a CSV file in Step. 9.
Columns:

In Step 9, columns 4-32 of the Complete spreadsheet are saved as a separate CSV format file (using the same versioned file name as the parent workbook but with a suffix of .csv).

Also considered part of Step 9, the CSV file is imported into an empty Microsoft Access database. The datatype of the InInputRelation is changed to be a Yes/No (boolean) field. The database file is given the same name as the CSV file but with a suffix of .accdb.

This section documents the results of the following use cases (queries against the Neo4j graph model):

All Business domain source and target element prototypes and all related Core and Derived relationships

All Core relationships where the source element prototype is from the Business domain

All Core relationships where the source and target element prototypes are from the Business domain

All Application domain source and target element prototypes and all related Core and Derived relationships

All Core relationships where the source and target element prototypes are from the Application domain

All Core and Derived relationships where the source and target element prototypes are from the Technology/Infrastructure domain

All Core relationships where the source and target element prototypes are from the Technology domain

All Core relationships where the source and target element prototypes are from the Technology domain and are identical to each other

All Core relationships where the source and target element prototypes are from the Technology domain and are different from each other (non-self referencing)

All Core and Derived relationships where the source and target element prototypes are from the Technology domain and are different from each other (non-self referencing)

All Core and Derived relationships where the source and target element prototypes are from the Technology domain and connected to the Node element prototype

All Core relationships where the source and target element prototypes belong to the Passive Structure aspect

All Core relationships where the source and target element prototypes belong to the Active Structure aspect

All Core relationships where the source and target element prototypes belong to the Behavior aspect

Use Case Results

Click on any of the figures to enlarge them in a separate browser tab.

Business Domain Use Case Results

Use Case 1: All Business domain source and target element prototypes and all related Core and Derived relationshipsUse Cases

Figure 8. is the result of an ad-hoc CQL query against all element prototypes in the Business domain; more specifically, where both the source and target element prototypes are in the Business domain.

Figure 8. All Business domain source and target element prototypes and all related Core and Derived relationships

Use Case 2: All Core relationships where the source element prototype is from the Business domain

Figure 9. is the result of an ad-hoc CQL query against all Core relationships where the source element prototype is from the Business domain.

Figure 9. All Core relationships where the source element prototype is from the Business domain

Use Case 3: All Core relationships where the source and target element prototypes are from the Business domain

Figure 10. is the result of an ad-hoc CQL query against all Core relationships where both the source and target element prototypes are from the Business domain.

Figure 10. All Core relationships where the source element prototype is from the Business domain

Application Domain Use Case Results

Use Case 4: All Application domain source and target element prototypes and all related Core and Derived relationships

Figure 11. is the result of an ad-hoc CQL query against all element prototypes in the Application domain; more specifically, where both the source and target element prototypes are in the Application domain.

Use Case 5: All Core relationships where the source and target element prototypes are from the Application domain

Figure 12. is the result of an ad-hoc CQL query against all Core and Derived relationships where the source element prototype is from the Application domain.

Figure 12. All Core relationships where the source and target element prototypes are from the Application domain

Technology Domain Use Case Results

Use Case 6: All Core and Derived relationships where the source and target element prototypes are from the Technology/Infrastructure domain

Figure 13. is the result of an ad-hoc CQL query against all Core relationships where both the source and target element prototypes are from the Technology/Infrastructure domain.

Figure 13. All Core and Derived relationships where the source and target element prototypes are from the Technology/Infrastructure domain

Use Case 7: All Core relationships where the source and target element prototypes are from the Technology domain

Figure 14. is the result of an ad-hoc CQL query against all Core relationships where both the source and target element prototypes are from the Technology domain.

Figure 14. All Core relationships where the source and target element prototypes are from the Technology domain

Use Case 8: All Core relationships where the source and target element prototypes are from the Technology domain and are identical to each other

Figure 15. is the result of an ad-hoc CQL query against all Core relationships where both the source and target element prototypes are from the Technology domain and are identical to each other.

Figure 15. All Core relationships where the source and target element prototypes are from the Technology domain and identical to each other

Use Case 9: All Core relationships where the source and target element prototypes are from the Technology domain and are different from each other (non-self referencing)

Figure 16. is the result of an ad-hoc CQL query against all Core relationships where both the source and target element prototypes are from the Technology domain and are different from each other (non-self referencing).

Figure 16. All Core relationships where the source and target element prototypes are from the Technology domain and different from each other (non-self referencing)

Use Case 10: All Core and Derived relationships where the source and target element prototypes are from the Technology domain and are different from each other (non-self referencing)

Figure 17. is the result of an ad-hoc CQL query against all Core and Derived relationships where both the source and target element prototypes are from the Technology domain and are different from each other (non-self referencing).

Figure 17. All Core and Derived relationships where the source and target element prototypes are from the Technology domain and are different from each other (non-self referencing)

Use Case 11: All Core and Derived relationships where the source and target element prototypes are from the Technology domain and connected to the Node element prototype

Figure 18. is the result of an ad-hoc CQL query against all Core and Derived relationships where both the source and target element prototypes are from the Technology domain and are connected to the Node element prototype.

Figure 18. All Core and Derived relationships where the source and target element prototypes are from the Technology domain and connected to the Node element prototype

NOTE: You are encouraged to try to model these examples for yourself: to start learning how to “think in ArchiMate” as your second or third written language. Archi is a great free tool for learning the ArchiMate language. You can download the Archi .archimate file containing the model used for this article from here. You can download the latest version of the Archi 4.0 modeling tool from here (which includes full support for the ArchiMate 3.0 language).

ArchiMate 3.0 is used as the baseline enterprise architecture modeling language for this discussion; especially the new Grouping element.

The Proposals

There are 2 new proposals described in this article: one more generic and one more specific.

Proposal 1: A new (general) approach for visually presenting answers to the question “How do I model X in ArchiMate?” using a metamodel-level reference model modeling strategy

Proposal 2: A specific approach (reference model) for modeling a Company and its Locations, Markets, Employees, Investors, etc. and their Roles.

The second proposal is an example or use case for the former.

Proposal 1: Modeling of Best Practice Modeling Patterns

Proposal 1 is illustrated in Figure 1 and Figure 2. These figures illustrate a general approach for modeling and visually presenting answers to the question “How do I model X in ArchiMate?”.

Rather than provide simple, less-informative, textual answers such as “use Business Collaborations to model Companies” or in ArchiMate 3.0, “use Groupings to model Companies”, why not:

Leverage Specialization relationships to model, name, and visually illustrate, in these examples, alternative representations of a Company element

From a presentation perspective, place the new best practices modeling pattern on the left – side-by-side – with the portion of the applicable elements of the base-level ArchiMate metamodel on the right

as illustrated in Figure 1 and Figure 2.

NOTE: Proposal 1 is illustrated with 2 examples. The merits of the individual examples are discussed below in Proposal 2. The comparison of these 2 examples is not part of Proposal 1.

Figure 1. Metamodel-level Reference Model for a Company using Business Collaboration

Figure 2. Metamodel-level Reference Model for a Company using Grouping

Proposal 2: Specific approach (reference model) for modeling a Company

Proposal 2 asks the question: Of the 2 options presented above (or any additional alternative options), which option represents a best practice reference model for modeling a Company and its Locations, Markets, Employees, Investors, etc. and their Roles.

The only tangible difference between the modeling strategy in Figure 1 vs. Figure 2 is:

The first step in the ArchiMate Graphitization Project was to consolidate and make consistent these 4 data sources; or rather, attempt to consolidate and make them consistent. The Appendix B Tables are not very usable (i.e. not machine-readable in their current form) and hence, can only be used as a secondary, passive reference. These tables are also known to have some bugs which are expected to be fixed in the next release of the ArchiMate Specification.

The most reliable, most usable, and most complete data source is the Archi 4.0 project’s relationships.xml file; defining 11,000+ relationships. This file’s only drawback is that it includes all relationships – both Derived as well as the Core Subset – and does not have any tags or other markings to identify the Core relations; regardless, this file proved to be exceedingly valuable. The Appendix B Tables suffer from this same problem: no tagging to help identify the Core relationships.

Gerben Wierda’s ArchiMate 3.0 metamodel PDF is the best and internationally renown reference for the most commonly used ArchiMate relationships (what I think of as the ArciMate Core Subset). Gerben has done an excellent job of documenting these for ArchiMate 2.1, and now, ArchiMate 3.0. Using his PDF data, we were able to troubleshoot a number of missing as well as extra relationships across all 4 data sources [Thank you Gerben].

Colin Coate’s ARKWRT is also an excellent tool. I learned about Colin’s tool too late into my project cycle to use it; regardless, Colin has also been very helpful based on his deep knowledge of the ArchiMate specifications.

For this iteration of the ArchiMate Graphitization Project, I combined the relationship.xml data with Gerben’s PDF data. I was careful to tag each element and relation with its data source(s). The Appendix B Tables were used as a secondary reference for verification purposes.

Graphitizing the ArchiMate Relationship Table

I won’t go into detail about all the steps but I do want to emphasize that this was a very straightforward process once I had some good data – as in any Data Science project. The relationships.xml file is well formed and opened nicely in Excel 2016 (as well as earlier versions of Excel no doubt). It was simple to save it out as a CSV file (after some preformatting to create namespaces for the AchiMate element and relationship names). Here’s a sample that resembles what the final CSV file looked like before ingesting it into Neo4j graph database (click the figure below to enlarge it).

Ingesting the CSV file, creating the graph nodes, labeling them based on ArchiMate element type (i.e. ModelMate Verb), and finally, creating the relationships were also pretty easy to accomplish using the Cypher language. Here’s the actual query:

There were a couple of additional steps such as assigning weights (strengths) to each of the relationships based on the relationship type, tagging relations with a data source code, etc. that were part of the overall process.

Recreating a Rendering of Gerben’s PDF

The biggest challenge for the project was to see how easy it would be to recreate something that looked similar to Gerben’s ArchiMate 3.0 metamodel PDF. Here’s a copy of the actual Cypher query. It’s not optimized but only takes a few seconds to run.

The highlighted clauses encode the margin notes on page 1 of Gerben’s PDF. The unhighlighted clauses at the bottom (that refer to Strength) reduce the multiple relations-per-element-pair returned by the first half of the query to a single relation per distinct element-pair (based on the relative weight or strength of the relationships for each pair of elements).

The initial graph looked like this:

Because all of the elements are connected together with relations that act like springs, it took only a couple minutes to manually produce this layout (very similar to page 1 of Gerben’s PDF):

More importantly, I can now answer questions like:

Show me how Nodes are connected to Products
using only Core Subset relationships
using 2 hops
in the ArchiMate 3.0 metamodel

Move beyond digitalization of the enterprise to graphitization of the enterprise. Here’s a great diagram that explains this concept. (click on the diagram to enlarge it)

Figure 1. The New Model of IT

Graphitization of not only all of your corporate information assets across all of your constituencies and stakeholders – at the data, application entity, and business object level – but also the graphitization of all of the interconnections between every business process, application system, infrastructure component, cloud service, vendor/service provider, and business role that uses, manages, or stores corporate information (Crossing the EA Chasm: Automating Enterprise Architecture Modeling #2).

Use graphitization to make your existing corporate information more available, more usable, and more informative. Graphitization enables you to “Keep Calm and Have IT Your Way“.

What is #Graphitization?

#Graphitization is a data science and enterprise architecture-inspired framework and process model for modeling, ingesting, organizing, analyzing, and visualizing any domain of endeavor by using graphs – networks of connected objects and relationships with each object and relationship annotated with additional descriptive information (metadata).

The primary applications of #Graphitization are:

System optimization,

Systems life cycle management, and

Transformative Change in resulting in positive increases in business value for the system being studied.

A system is defined as any collection of strategies, system components, assets, architectures or processes.

Using #Graphitization

Use graphitization of your organization to help close both the Enterprise Architecture Chasm and the Operational Data Chasm. See below.

This article is the third in a series on #Graphitization. Click here to explore the other articles in this series.

Iteration 2 is a small iteration that had a goal of improved key phrase-based exploration and visualization of The Principles of Ray Dalio. This iteration builds on the ModelMate model of The Principles described earlier in this series: #Graphitization of Ray Dalio’s Principles: Iteration 1 and represents a significant improvement in terms of understanding which principles are realized by specific combinations of key phases.

Iteration 2 uses the same query used in Iteration 1. This time, the Linkurious graph visualization app is used to display the subgraph of all Topics, Principles, Subprinciples, Commentary, Questions, etc. directly or indirectly related to the key phrases “radically” and “transparent”. This concept is represented by the following simple query: