About Tony Sarris – N2Semantics

I am the founder and principal of N2Semantics. I provide consulting services to companies developing and applying semantic technologies, including artificial intelligence and cognitive computing.
At N2Semantics we believe the future of apps is to make them smarter -- more aware of the world in which they operate to serve the needs and interests of their users.
Our mission is to foster the development of semantic technologies and to help companies determine the best approaches and technologies for adding semantic knowledge or meaning to their applications.

What Happens When a Martian Makes a Martini? (Photo credit: Wikipedia)

In my last blog post, I stated I felt expert systems were an important forerunner of today’s emerging digital personal assistants and any other software technologies that include an element of ‘agency’ — acting on behalf of others, in this case the humans who invoke them. For someone or something to act on your behalf effectively, they need to understand many specific things about the particular domain they are tasked with working in, along with some general knowledge of the type that cuts horizontally across many vertical domains, and of course they need to know some things about you.

Chuck Dement, the late founder of Ontek Corporation and one of the smartest people I’ve met, used to say that teaching software to understand and execute the everyday tasks that humans do was like teaching a Martian visiting here on Earth how to make a martini. His favorite Martian, George the Gasbag, like the empty shell of a computer program, didn’t know anything about our world or how it works, let alone the specifics of making a martini. Forgetting for a moment George’s physical limitations due to being a gasbag, imagine trying to explain to him (or to encode in software) the process of martini-making — starting with basically no existing knowledge.

First, George has to know something about the laws of physics. He doesn’t need to understand the full quantum model (does anyone actually understand it?), but he does need to be aware of some of the more practical aspects of physics from the standpoint of how it applies to everyday life on the surface of Earth. Much of martini-making involves combining liquid substances. Liquid substances need to be confined in a container of some sort, preferably a non-porous one. The container has to maintain a [relatively] stable and upright position during much of the process. The container holds certain quantities of the liquids. For a martini to be a martini and to taste ‘right’ to its human consumers, the liquids have to be particular substances. Their chemical properties have to meet certain criteria to be suitable (and legal) for use. The quantities of the liquid have to measured in relative proportions to one another. The total combined quantity shouldn’t (or at least needn’t) exceed the total quantity that the container can hold.

You need some ice, which involves another substance — water — its liquid form having been transformed into a solid at a certain temperature. If you are making the martini indoors in most cases or outside when the temperature is warm, the process of producing ice from water requires special devices to create the required temperature conditions within some fixed space. And so on and so forth. You can pull on any of those threads and dive into the subject. Think of having a conversation with a 4 or 5 year-old child and answering all the “Why?” and “How?” questions.

Of course there are at least two major different processes that can be used to mix the liquids along with the ice. They involve different motions — stirring the liquid within the container versus shaking the container (after putting a lid or similar enclosure on the previously ‘open part’ of the container to keep the liquid from flying out). The latter begs the question: is the open ‘part’ of the container really even a part of it, or the absence of some part?

There are allowable variations in the substances (ingredients), both in terms of kinds and specific brands (gin versus vodka, Beefeater versus Tanqueray for gin). Both the process and the ingredients often come down to the specific preferences of the intended individual consumer (take James Bond, for example), but may also be influenced by availability, business criteria such as price or terms of supplier contracts, and whether the consumer has already consumed several martinis or similar alcoholic beverages within some relatively fixed timeframe (don’t forget here to factor in the person’s gender, body size, previous night’s sleep, history of alcohol consumption, altitude, etc.). The main point here is simply if they’ve had several such drinks, their preferences may be more flexible than for the first one or two!

Whew!!! All that just to make a martini? That’s all to illustrate that encoding knowledge for everyday tasks is non-trivial. No one ever said developing intelligent agent software would be simple. But as previously mentioned, George doesn’t need to know everything about every aspect of the domains involved in martini-making. Going overboard is a sure recipe for failure. Knowing where to draw the line is the key and so a healthy serving of pragmatism is recommended. A place to start is I think even getting in the ballpark of knowledge about everyday things and applying that approximate knowledge to practical application uses. Since you don’t always know beforehand how much knowledge you need, I’m a fan of the generative approach to semantic technologies (see my related blog post on approaches to semantic technologies). The generative approach allows agility and flexibility in the production of that knowledge, as well as providing ways to tailor it for individual differences.

And speaking of individual differences: how will George recognize when I’m ready for him to make me a martini? What are the triggers and any prerequisite conditions (like being of legal drinking age in the geo-location where the drink is being made and consumed). Well, I could always ask George (or my personal, robotically-enabled, martini-making software assistant), but I trust that he knows me well enough to recognize that telling look that says, “I could sure use a drink, my friend,…especially after all the knowledge I had to encode to enable you to make one.”

One of my mentors from years ago — who was one of the smartest people I’ve known — had a great deal of admiration for personal assistants. The human kind I mean. Through him, I gained an appreciation for them, too. By personal assistants I mean anyone who directly helps someone else do their job or perform a similar sort of function.

A personal assistant, like other employees, works for their boss. You can say that all employees are extensions of their boss and broadly-speaking they help their boss achieve the goals of her/his organization. But most employees have specialized jobs to do that apply to particular aspects of the organization they are part of. A personal assistant is different in that their job is to help their boss do his/her job more efficiently or effectively. They take on tasks that their boss can’t do due to time or other constraints, doesn’t want to do, finds value in having someone help with, etc. They remove roadblocks, take care of details, keep things organized and stop things falling through the cracks, and sometimes even act as roadblocks themselves — preventing other people from distracting their boss or wasting her/his time. In numerous ways they augment the value and productivity of their boss and make his/her life a little bit easier.

A good personal assistant is a natural, almost seamless extension of their employer. They have a deep understanding of the organization, its goals, resources and constraints, and the nature of the tasks to be done. They understand their employer’s role in the organization, her/his motivations and his/her opinions of other people inside the organization and externally in partner, customer and competitive organizations. They know who can be trusted and who to ‘go to’ for specific needs. They are perhaps their boss’ most trusted employee, a confidant.

Really good personal assistants anticipate their boss’ next requests and are prepared to respond to those as soon as the boss gives even an indication. They know the set of possible options/next steps and the likely decision criteria that would trigger one versus another. When something unexpected comes up, they know to alert their boss. They are honest and open with their boss (but not necessarily fully open with others, as they need to know when to protect their boss and her/his interests). They don’t necessarily need to be told explicitly when to do something or what to do. They can often read between the lines or recognize triggers for actions they should be taking. They can be entrusted to represent their boss and to make appropriate decisions on their boss’ behalf, when needed (and at the same time not overstep the boundaries of their authority and responsibility).

So how does a personal assistant become capable of contributing so effectively to the execution of tasks — some more mundane, some more important — that their boss otherwise would be doing? It’s all about context. A personal assistant knows the background for tasks and decisions. They know the history. They have strong domain knowledge. They know their boss’ preferences. They know how ‘outsiders’ (anyone but them and their boss) are likely to behave in certain situations, who to trust and who to keep an eye on. Some of this information they have been told explicitly by their boss. Some of it they have picked up just from working closely with their boss for some time. They need to know the limits of their own skills and knowledge and who to bring into the process (often on their boss’ behalf) to achieve the goal. They know where and when to ‘ping’ their boss and get additional direction, clarification or confirmation.

All of this is relevant to my interest in semantics for the following reason: if we want to develop virtual personal assistants — intelligent software agents — they will need similar knowledge and capabilities. That will have to come from somewhere, and it seems unlikely or at least inefficient if all that knowledge has to be hand-curated and all that logic has to be hand-coded. Some of it at least will need to be machine- learned/generated and then validated, based on existing knowledge and through learning loops (i.e., by trial and error and/or explicit feedback loops).

Think about the so-called personal digital assistants that exist today. Do they have these capabilities and knowledge? Not the ones I’m familiar with. They do a decent job of interpreting my explicit voice commands for a relatively fixed set of tasks, such as search, creating a meeting event or calling someone from my contact list. More complex tasks, if handled at all, involve a constant series of back and forth questions between the personal assistant and me. Human personal assistants might do this early in a task to establish the framing criteria, but part of their productivity-enhancing abilities lies in the fact that they can do many things autonomously or semi-autonomously (knowing when to come back to me with questions, or to discuss options or problems). Can you imagine if they behaved like their digital counterparts do today, simply echoing back much of your input and making you connect most of the dots and orchestrate most of the workflows through repeated questions derived from some sort of decision tree? Those aren’t true personal assistants, those are at best ‘helper apps’. But you have to start somewhere I guess.

We have a long way to go to get to where we have real virtual personal assistants. I think the basic technology components exist for the most part today to at least get started. But there is much work to do to put them together in the right way and create apps that do even a little bit of what Corporal ‘Radar’ O’Reilly did for the Colonel on M*A*S*H [Note: if you don’t know that reference, by all means Google it! He’s the kind of personal assistant I want to have in automated form].

This is my introductory blog. It’s hard to describe what I’m trying to do with N2Semantics without giving you a little background on who I am and what I’ve done so far in my career. I would say I’m a computer scientist or technologist, but not because I particularly like computers or information technology. I like what you can do with such technologies, or even more importantly, what they can do for you!

I spent the early part of my career designing and developing applications, primarily in the fields of product definition (engineering), development (manufacturing) and delivery (packaging, distribution, transportation). I ended up focusing on the representational aspects of such systems — their architecture, logic and data structures. I did a lot of enterprise information modeling, data and process modeling, system and database design, and software development and implementation (and support — let’s not forget support!). I found a passion in the challenges of reflecting the real world inside computer systems. That led me to gaining knowledge and experience in the field of knowledge representation and doing some of the early, pioneering work in conceptual models or conceptual schemas — what came to be known as ‘ontologies’.

Representing human knowledge in a form that computers can make use — and actually enabling them to make use of it — became a life-long pursuit. I’ve been pursuing it for over 20 years and I’m still pursuing it. Along the way, there have been lots of successes and also many failures. If you never make a mistake, you’re probably not pushing the edge of discovery. It’s from those failures that you learn (hopefully!) and they become the basis for progress and success. I’ll try to talk about some of the failures in future blog posts, as well as some successes.

I feel optimistic about pursuing intelligent systems today — more optimistic than I have ever been in my career. I feel the required technology components exist today, at least in sufficient form to put to productive, practical use. And that’s what I want to do. I’m not interested in doing fundamental research. I want to work with providers of leading-edge, innovative technologies and business people with real application ideas and challenges for which those technologies provide enabling solutions. While many technologies comprise pieces of this puzzle, in particular I focus on semantic technologies. I want to help companies use semantic technologies — along with mobile devices, content sources, social media, et al — to create intelligent software agents. In future posts I’ll talk about some of the [productive] ways software agents might assist us in going about our day-to-day lives. That’s what interests me and that’s what’s behind my starting up N2Semantics. The journey into semantic applications is going to be a fun, but challenging journey. Join me on that journey by following this blog.