Archive for the ‘data mining’ Category

Since we began the process of building applications using our AI engine, we have been focused on working with ideas or concepts. With BrainDocs we built intelligent agents to find and score similarity for ideas in paragraphs, but still fell short of the vision we have for our solution. Missing was an intuitive and visual UI to explore content interactively using multiple concepts and metadata (like dates, locations, etc). We want to give our users the power to create a rich and personal context to power through their research. What do I call this?

Some Google research led me to a great visualization and blog by David McCandless on the Taxonomy of Ideas. While the words in his viz are attributes of ideas, not the ideas themselves, it got me thinking in different ways about the problem.

If you substitute an idea (product or problem) in David’s matrix and add the dimension of time, you create a useful framework. If the idea above was “car”, then the top right might be Tesla and bottom left a Yugo (remember those?). Narrow the definition to “electric car” or generalize to “eco-friendly personal transportation” and the matrix changes. But insert an unsolved problem and now you have trouble applying the attributes. You also arrive at an innovator’s dilemma (not the seminal book by Clayton Christensen), the challenge of researching something that hasn’t been labeled and categorized yet.

Ideas begin in someone’s head. With research, debate, and engineering, they become products. Products have labels and categories that facilitate communication, search and commerce. The challenge for idea search on future problems is that the opposite occurs: products are not yet ideas and the problems they solve may not have been defined yet. If I may, Donald Rumsfeld nailed the problem with this famous quote:

“There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.”

And if it’s an unknown unknown, it certainly hasn’t been labeled yet so how do you search for it? Our CEO Walt Diggelmann used to say it this way, “ai-one gives you an answer to a question, you did not know that you have to ask….! “

Innovators work in this whitespace.

If you could build and combine different intelligent (idea) agents for problems as easily as you test different combinations of words in a search box, you could drive an interactive and spontaneous exploration of ideas. In some ways this is the gift of our intelligence. New ideas and innovation are in great part combinatorial, collaborative and stimulated by bringing together seemingly unrelated knowledge to find new solutions.

Instead of pumping everything into your brain (or an AI) and hoping the ideas pop out, we want to give you the ability to mix combinations of brains, add goals and constraints and see what you can create. Matt Ridley termed this “ideas having sex”. This is our goal for Topic-Mapper (not the sex part).

So what better place to apply this approach than to the exploration of space? NASA already created a “taxonomy of ideas” for the missions of the next few decades. In my next blog I’ll describe the demo we’re working on for the grandest of the grand challenges, human space exploration.

If the real life Tony Stark and technology golden boy, Elon Musk, is worried that AI is an existential threat to humanity, are we doomed? Can mere mortals do anything about this when the issue is cloaked in dozens of buzzwords and the primary voices on the subject are evangelists with 180 IQs from Singularity University? Fortunately, you can get smart and challenge them without a degree in AI from MIT.

“Smarter Than Us – The Rise of Machine Intelligence” by Stuart Armstrong can also be downloaded at iTunes.

“It will sharpen your focus to see AI from a different view. The book does not provide a manual for Friendly AI, but its shows the problems and it points to the 3 critical things needed. We are evaluating the best way for ai-one to participate in the years ahead.” Walt Diggelmann, CEO ai-one.

In Chapter 11 Armstrong recommends we take an active role in the future development and deployment of AI, AGI and ASI. The developments are coming; the challenge is to make sure AI plays a positive role for everyone. A short summary:

“That’s Where You Come In . . .

There are three things needed—three little things that will make an AI future bright and full of meaning and joy, rather than dark, dismal, and empty. They are research, funds, and awareness.

Research is the most obvious.
A tremendous amount of good research has been accomplished by a very small number of people over the course of the last few years—but so much more remains to be done. And every step we take toward safe AI highlights just how long the road will be and how much more we need to know, to analyze, to test, and to implement.

Moreover, it’s a race. Plans for safe AI must be developed before the first dangerous AI is created.
The software industry is worth many billions of dollars, and much effort (and government/defense money) is being devoted to new AI technologies. Plans to slow down this rate of development seem unrealistic. So we have to race toward the distant destination of safe AI and get there fast, outrunning the progress of the computer industry.

Funds are the magical ingredient that will make all of this needed research.
In applied philosophy, ethics, AI itself, and implementing all these results—a reality. Consider donating to the Machine Intelligence Research Institute (MIRI), the Future of Humanity Institute (FHI), or the Center for the Study of Existential Risk (CSER). These organizations are focused on the right research problems. Additional researchers are ready for hire. Projects are sitting on the drawing board. All they lack is the necessary funding. How long can we afford to postpone these research efforts before time runs out? “

About Stuart: “After a misspent youth doing mathematical and medical research, Stuart Armstrong was blown away by the idea that people would actually pay him to work on the most important problems facing humanity. He hasn’t looked back since, and has been focusing mainly on existential risk, anthropic probability, AI, decision theory, moral uncertainty, and long-term space exploration. He also walks the dog a lot, and was recently involved in the coproduction of the strange intelligent agent that is a human baby.”

Since ai-one is a part of this industry and one of the many companies moving the field forward, there will be many more posts on the different issues confronting AI. We will try to keep you updated and hope you’ll join the conversation on Google+, Facebook, Twitter or LinkedIn. AI is already pervasive and developments toward AGI can be a force for tremendous good. Do we think you should worry? Yes, we think it’s better to lose some sleep now so we don’t lose more than that later.

In the sensationally titled Forbes post, Tech 2015: Deep Learning And Machine Intelligence Will Eat The World, author Anthony Wing Kosner surveys the impact of deep learning technology in 2015. This is nothing new for those in the field of AI. His post reflects the recent increase in coverage artificial intelligence (AI) technologies and companies are getting in business and mainstream media. As a core technology vendor in AI for over ten years, it’s a welcome change in perspective and attitude.

We are pleased to see ai-one correctly positioned as a core technology vendor in the Machine Intelligence Landscape chart featured in the article. The chart, created by Shivon Zilis, investor at BloombergBETA, is well done and should be incorporated into the research of anyone seriously tracking this space.

Especially significant is Zilis’ focus on “companies that will change the world of work” since these are companies applying AI technologies to innovation and productivity challenges across the public and private sectors. The resulting solutions will provide real value through the combination of domain expertise (experts and data) and innovative application development.

This investment thesis is supported by the work of Erik Brynjolfsson and Andrew McAfee in their book “The Second Machine Age”, a thorough discussion of value creation (and disruption) by the forces of innovation that is digital, exponential and combinatorial. The impact of these technologies will change the economics of every industry over years if not decades to come. Progress and returns will be uneven in their impact on industry, regional and demographic sectors. While deep learning is early in Gartner’s Hype Cycle, it is clear that the market value of machine learning companies and data science talent are climbing fast.

This need for data scientists is growing but the business impact of AI may be limited in the near future by the lack of traditional developers who can apply them. Jeff Hawkins of Numenta has spoken out on this issue and we agree. It is a fundamentally different way to create an application for “ordinary humans” and until the “killer app” Hawkin’s speaks about is created, it will be hard to attract enough developers to invest time learning new AI tools. As the chart shows, there are many technologies competing for their time. Developers can’t build applications with buzzwords and one size fits all APIs or collections of open source algorithms. Technology vendors have a lot of work to do in this respect.

Returning to Kosner’s post, what exactly is deep learning and how is it different from machine learning/artificial intelligence? According to Wikipedia,

Deep learning is a class of machine learning training algorithms that use many layers of nonlinear processing units for feature extraction and transformation. The algorithms may be supervised or unsupervised and applications include pattern recognition and statistical classification.

are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher level features are derived from lower level features to form a hierarchical representation.

are part of the broader machine learning field of learning representations of data.

learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts.

form a new field with the goal of moving toward artificial intelligence. The different levels of representation help make sense of data such as images, sounds and texts.

These definitions have in common (1) multiple layers of nonlinear processing units and (2) the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features.

While in the 4th bullet this is termed a new field moving toward artificial intelligence, it is generally considered to be part of the larger field of AI already. Deep learning and machine intelligence is not the same as human intelligence. Artificial intelligence in this definition above and in the popular press usually refers to Artificial General Intelligence (AGI). AGI and the next evolution, Artificial Super Intelligence (ASI) are the forms of AI that Stephen Hawking and Elon Musk are worried about.

This is powerful stuff no question, but as an investor, user or application developer in 2015 look for the right combination of technology, data, domain expertise, and application talent applied to a compelling (valuable) problem in order to create a disruptive innovation (value). This is where the money is over the new five years and this is our focus at ai-one.

Robert Scoble and Shel Israel’s latest book, Age of Context, is a survey of the contributions across the globe to the forces influencing technology and our lives today. The five forces are mobile, social media, data, sensors and location. Scoble calls these the five forces of context and harnessed, they are the future of computing.

Pete Mortensen also addressed context in his brilliant May 2013 article in Fast Company “The Future of Technology Isn’t Mobile, It’s Contextual.” So why is context so important (and difficult)? First, context is fundamental to our ability to understand the text we’re reading and the world we live in. In semantics, there is the meaning of the words in the sentence, the context of the page, chapter, book and prior works or conversations, but also the context the reader’s education and experience add to the understanding. As a computing problem, this is the domain of text analytics.

Second, if you broaden the discussion as Mortensen does to personal intelligent agents (Siri, Google Now), the bigger challenge is complexity. Inability to understand context has always made it difficult for computers and people to work together. People and the language we use to describe our world is complex, not mathematical, You can’t be reduced to a formula or rule set, no matter how much data is crunched. Mortensen argues (and we agree) that the five forces are finally giving computers the foundational information needed to understand “your context” and that context is expressed in four data graphs. These data graphs are

Social (friends, family and colleagues),

Interest (likes & purchases),

Behavior (what you do & where) and

Personal (beliefs & values).

While Google Glass might be the poster child of a contextual UX, ai-one has the technology to power these experiences by extracting Mortensen’s graphs from the volumes of complex data generated by each of us through our use of digital devices and interaction with increasing numbers of sensors known as the Internet of Things (IoT). The Nathan API is already being used to process and store unstructured text and deliver a representation of that knowledge in the form of a graph. This approach is being used today in our BrainDocs product for eDiscovery and compliance.

In Age of Context, ai-one is pleased to be recognized as a new technology addressing the demands of these new types of data. The data and the applications that use them are no longer stored in silos where only domain experts can access them. With Nathan the data space learns from the content, delivering a more relevant contextual response to applications in real time with user interfaces that are multi-sensory, human and intuitive.

We provide developers this new capability in a RESTful API. In addition to extracting graphs from user data, they can build biologically inspired intelligent agents they can train and embed in intelligent architectures. Our new Nathan is enriched with NLP in a new Python middleware that allows us to reach more OEM developers. Running in the cloud and integrated with big data sources and ecosystems of existing APIs and applications, developers can quickly create and test new applications or add intelligence to old ones.

For end users, the Analyst Toolbox (BrainBrowser and BrainDocs) demonstrates the value proposition of our new form of artificial intelligence and shows developers how Nathan can be used with other technologies to solve language problems. While we will continue to roll out new features to this SaaS offering for researchers, marketers, government and compliance professionals, the APIs driving the applications will be available to developers.

Mortensen closes, “Within a decade, contextual computing will be the dominant paradigm in technology.” But how? That’s where ai-one delivers. In coming posts we will discuss some of the intelligent architectures built with the Nathan API.

We are pleased to announce the availability of the following publication from prestigious ETH University in Zurich. This book will be a valuable resource to developers, data scientists, search and knowledge management educators and practitioners trying to deal with the massive amounts of information in both public and private data sources. We are proud to have our contribution to the field acknowledged in this way.

ai-one was invited to contribute as co-author to a chapter in this technical book.

In the anthology readers will find very different conceptual and technological methods for modeling and digital representation of knowledge for knowledge organizations (universities, research institutes and educational institutions), and companies based on practical examples presented in a synopsis. Both basic models of the organization of knowledge and technical implementations are discussed including their limitations and difficulties in practice. In particular the areas of knowledge representation and the semantic web are explored. Best practice examples and successful application scenarios provide the reader with a knowledge repository and a guide for the implementation of their own projects. The following topics are covered in the articles:

hypertext-based knowledge management

digital optimization of the proven analog technology of the list box

innovative knowledge organization using social media

search process visualization for digital libraries

semantic events and visualization of knowledge

ontological mind maps and knowledge maps

intelligent semantic knowledge processing systems

fundamentals of computer-based knowledge organization and integration

The book also includes coding medical diagnoses, contributions to the automated creation of records management models, business fundamentals of computer-aided knowledge organization and integration, the concept of mega regions to support of search processes and the management of print publications in libraries.

What if your computer could find ideas in documents? Building on the idea of fingerprinting documents, ai-one helped develop ai-BrainDocs – a tool to mine large sets of documents to find ideas using intelligent agents. This solves a big problem for knowledge workers: How to find ideas in documents that are missed by traditional keyword search tools (such as Google, Lucine, Solr, FAST, etc.).

Customers Struggle with Unstructured Text

Almost every organization struggles to find value in “big data” – especially ideas buried within unstructured text. Often a very limited set of vocabulary can be used to express very different ideas. Lawyers are particularly talented at this: They can use 100 unique words to express thousands of ideas by simply changing the ordering and frequencies of the words.

Lawyers are not the only ones that need to find ideas inside documents. Other use cases include finding and classifying complaints, identifying concepts within social media feeds such as Twitter or Facebook and mining PubMed find related research articles. Recently, we have had several healthcare companies contact us to mine electronic health records (EHR) data to find information that is buried within doctors notes so they can predict adverse reactions, find co-morbidity risks and detect fraud.

The common denominator for all these uses cases is simple: How to find “what matters most” in documents? They need a way to find these ideas fast enough to keep pace with the growth in documents. Given that information is growing at almost 20% per year – this means that a very big problem now will be enormous next year.

Problems with Current Approaches

We’ve heard numerous stories from customers who were frustrated at the cost, complexity and expertise required to implement solutions to enable machines to read and understand the meaning of free-form text. Often these solutions use latent semantic indexing (LSI) and latent Dirichlet allocation (LDA). In one case, a customer spent more than two years trying to combine LSI with a Microsoft FAST Enterprise search appliance running on SharePoint. It failed because they were searching a high-volume of legal documents with very low variability. They were searching legal contracts to find paragraphs that included a very specific legal concept that could be expressed with many different combinations of words. Keyword search failed because the legal concept used commonly used words. LSI and LDA failed because the systems required a very large training set — often involving hundreds of documents. Even after reducing the specificity requirements, LSI and LDA still failed because they could not find the legal ideas at the paragraph level.

Inspiration

We found inspiration in the complaints we heard from customers: What if we could build an “intelligent agent” that could read documents like a person? We thought of the agent as an entry-level staff person who could be taught with a few examples then highlight paragraphs that were similar to (but not exactly like) the teaching examples.

Solution: Building Intelligent Agents

For several months, we have been developing prototypes of intelligent agents to mine unstructured text to find meaning. We built a Java application that combine ai-one’s machine learning API with natural language processing (OpenNLP) and NoSQL databases (MongoDB). Our approach generates an “ai-Fingerprint” that is a representational model of a document using keywords and association words. The “ai-Fingerprint” is similar to a graph G[V,E] where G is the knowledge representation, V (vertices) are keywords, and E (edges) are associations. This can also be thought of as a topic model.

The ai-Fingerprint can be generated for almost any size text – from sentences to entire libraries of documents. As you might expect, the “intelligence” (or richness) of the ai-Fingerprint is proportional to the size of text it represents. Very sparse text (such as a tweet) has very little meaning. Large texts, such as legal documents, are very rich. This approach to topic modelling is precise — even without training or using external ontologies.

[NOTE: We are experimenting with using ontologies (such as OWL and RDF) as a way to enrich ai-Fingerprints with more intelligence. We are eager to find customers who want to build prototypes using this approach.]

The Secret Sauce

The magic is that ai-one’s API automatically detects keywords and associations – so it learns faster, with fewer documents and provides a more precise solution than mainstream machine learning methods using latent semantic analysis. Moreover, using ai-one’s approach makes it relatively easy for almost any developer to build intelligent agents.

How to Build Intelligent Agents?

To build an intelligent agent, we first had to consider how a human reads and understands a document.

The Human Perspective

Human are very good at detecting ideas – regardless of the words used to express them. As mentioned above, lawyers can express dozens of completely different legal concepts with a vocabulary of just a few hundred words. Humans can recognize the subtle differences of two paragraphs by how a lawyer uses words – both in meaning (semantics) and structure (syntax). Part of the cleverness of a lawyer is finding ways to combine as few words as possible to express a very precise idea to accomplish a specific legal or business objective. In legal documents, each new idea is almost always expressed in a paragraph. So two paragraphs might have the exact same words but express completely different ideas.

To find these ideas, a person (or computer) must detect the patterns of word use – similar to the finding a pattern in a signal. For example, as a child I knew I was in trouble when my mother called me by my first and last name – the combination of these words created a “signal” that was different than when she just used my first name. Similarly, a legal concept has a different meaning if two words occur together, such as “written consent” than if it only uses the word “consent.”

The (Conventional) Machine Learning Perspective

It’s almost impossible to program a computer to find such “faint signals” within a large number of documents. To do so would require a computer to be programmed to find all possible combinations of words for a given idea to search and match.

Machine learning technologies enable computers to identify features within the data to detect patterns. The computer “learns” by recognizing the combinations of features as patterns.

[There are many forms of machine learning – so I will keep focused only on those related to our text analytics problem.]

Natural Language Processing

One of the most important forms of machine learning for text analytics is natural language processing (NLP). NLP tools are very good at codifying the rules of language for computers to detect linguistic features – such as parts of speech, named entities, etc.

However (at the time of this writing), most NLP systems can’t detect patterns unless they are explicitly programmed or trained to do so. Linguistic patterns are very domain specific. The language used in medicine is different than what is used in law, etc. Thus, NLP is not easily generalized. NLP only works in specific situations where there is predictable syntax, semantics and context. IBM Watson can play Jeopardy! but has had tremendous problems finding commercial applications in marketing or medical records processing. Very few organizations have the budget or expertise to train NLP systems. They are left to either buy an off-the-shelf solution (such as StoredIQ ) or hire a team of PhDs to modify one of the open-source NLP tools. Good luck.

Latent Analysis Techniques

Tools such as latent semantic analysis (LSA), latent semantic indexing (LSI) and latent Dirichlet allocation (LDA) are all capable of detecting patterns within language. However, they require tremendous expertise to implement and often require large numbers of training documents. LSA and LSI are computationally expensive because they must recalculate the relationships between features each time they are given something new to learn. Thus, learning the meaning of the 1,001th document requires a calculation across the 1,000 previously learned documents. LSA uses a statistical approach called single variable decomposition to isolate keywords. Unlike LSA, ai-one’s technology also detects the association words that give a keyword context.

Similar to our ai-Fingerprint approach, LDA uses a graphical model for topic discovery. However, it takes tremendous skill to develop applications using LDA. Even when implemented, it requires the user to make informed guesses about the nature of the text. Unlike LDA, ai-one’s technology can be learned in a few hours. It requires no supervision or human interaction. It simply detects the inherent semantic value of text – regardless of language.

Our First Intelligent Agent Prototype: ai-BrainDocs

It took our team about a month to build the initial version of ai-BrainDocs. Our team used ai-one’s keyword and association commands to generate a graph for each document. This graph goes into MongoDB as a JSON object that represents the knowledge (content) of each document.
Next we created an easy way to build intelligent agents. We simply provide the API with examples of concepts we want to find. This training set can be very short. For one type of legal contracts, it only took 4 examples of text for the intelligent agent to achieve 90% accuracy in finding similar concepts.

Unlike solutions that use LSI, LDA and other technologies, the intelligent agents in ai-BrainDocs finds ideas at the paragraph level. This is a huge advantage when looking at large documents – such as medical research or SEC filings.

Next we built an interface that allows the end-user to control the intelligent agents by setting thresholds for sensitivity and determining how many paragraphs to scan at a time.

Our first customers are now testing ai-BrainDocs – and so far they love it. We expect to learn a lot as more people use the tool for different purposes. We are looking forward to developing ways for intelligent agents to interact – just like people – by comparing what they find within documents. We are finding that it is best for each agent to specialize in a specific subject. So finding ways for agents to compare their results using Boolean operators enables them to find similarities and differences between documents.

One thing is clear: Intelligent agents are ideal for mining unstructured text to find small ideas hidden in big data.

New Partnership Targets Creation of Social Media Intelligence Tools

Press Release

New tools will enable machine learning of twitter feeds

La Jolla CA | Zurich | Berlin February 16 2012 – ai-one inc. and Gnostech Inc. announced a partnership today to build new machine learning applications for the US government and military. The deal brings together two small firms that are well known for developing cutting-edge technologies. Gnostech specializes in simulation and modeling, Command Control Communications Computers and Intelligence Surveillance and Reconnaissance (C4ISR) systems and security engineering and Information Assurance (IA) applications. The partnership with ai-one provides Gnostech with access to technology that enables computers to learn the meaning and context of data in a way that is similar to humans. Called “biologically inspired intelligence” the technology is a new form of machine learning that is particularly useful for understanding complex, unstructured information – such as conversations in social media.

In the past month, the US government has issued six requests for companies to create solutions to help better understand Twitter, Facebook and other social media sources. These broad area announcements (BAAs) are formal requests from the Government to invite companies to provide turn-key solutions. With more than 800 million people actively using Facebook and more than 100 million Twitter users, governments and intelligence agencies know that they need better ways to mine this data to get real-time information to protect national security.“

We now have more than 40 partners worldwide that are experimenting with our technology – but only 3 that specialize in US government applications,” said Tom Marsh, President of ai-one. “Gnostech is local, technically driven and well positioned to develop rapid prototypes using our technology.”

About Gnostech, Since 1981, Gnostech has provided technical and engineering services to the Department of Defense (DOD) and Department of Homeland Security (DHS). Gnostech has a proven reputation for engineering efficiency, systems innovation, and dedicated customer service.

Gnostech Inc. began as an engineering and consulting company in Warminster, PA with expertise in GPS simulations and software, initially supporting the US Navy at the Naval Air Development Center (NADC) in Warminster, PA. Today, Gnostech has grown from a few people to about 50 employees with a satellite office in San Diego, CA and engineering support staff in Norfolk, VA, Morristown, NJ and Philadelphia, PA. Gnostech’s technical expertise expands upon our GPS experience and extends into Mission Planning, Network Engineering, Information Assurance and Security Engineering. www.gnostech.com