An Introduction to WEKA - Machine Learning in Java

DZone's Guide to

An Introduction to WEKA - Machine Learning in Java

WEKA (Waikato Environment for Knowledge Analysis) is an open source library for machine learning, bundling lots of techniques from Support Vector Machines to C4.5 Decision Trees in a single Java package.

WEKA (Waikato Environment for Knowledge Analysis) is an open source library for machine learning, bundling lots of techniques from Support Vector Machines to C4.5 Decision Trees in a single Java package.

My examples in this article will be based on binary classification, but what I say is also valid for regression and in many cases for unsupervised learning.

Why and when would you use a library?

I'm not a fan of integrating libraries and frameworks just because they exist; but machine learning is something where you have to rely on a library if you're using codified algorithms as they're implemented more efficiently than what you and I can possibly code in an afternoon. Efficiency means a lot in machine learning as supervised learning is one of the few programs that is really CPU-bound and can't be optimized further with I/O improvements. There's also the fact that you have to trade off a bit of the clarity for your object model when pursuing efficiency, so it's better to leave this bit of technical debt to the library than to pick it up by yourself.

Correctness is also a big deal: you can't be sure you have perfectly implemented the C4.5 algorithm for building decision trees just after reading the original paper twice. An open source library contains all the tweaks that are not explained in the scientific literature.

That said, Weka has a modern architecture, and polymorphism is very much used in the abstraction of different techniques; for example, many classifiers are available as separate objects and you gain the ability of swapping out different models with literally two lines of code:

There is a context where you wouldn't use a library: when studying new variations of an algorithm, as they are seen as configurable black boxes in Weka and other tools. You're not going to improve on one of this algorithms via subclassing, and it may not also be the case to prototype in a general-purpose language like Java.

API

When you have a set of samples, the first thing to do is to define attributes; the columns of your table. Given a list of strings as their names:

Weka is also meant to be used from the command line (that's why the options are formatted as switches in an array of string); so these objects, such as the instance, instance set and the classifier, are easily displayable by calling the toString() method on them. Machine learning results aren't always easily interpretable, but you can usually see if something has gone wrong by inspecting the results on stdout.

Isolation

Isolation from the library code is still important, however. Maybe you will have one or two naive or novel implementation to compare with what Weka does, or have to populate it from multiple data sources.

Here are some ideas I used while isolating all Weka dependencies into one Java package.

I built my own InstanceSet that I can add my data types to (like JSONObjects, or domain objects): it creates a data structure for Weka internally and never exposes it.

The same goes for Instance: my own class that wraps the Weka one and only exposes it via methods like populate(Instances wekaSet).

The InstanceSet has a method like buildClassifier(Classifier c) that returns my own type, again wrapping the weka one:

interface Classifier {
boolean classify(Instance i);
}

class WekaClassifier implements Classifier { ... }

There are several reasons for all this wrapping:

avoid a direct dependency: you can always create classifiers by yourself for testing purposes or comparing Weka results to a baseline or a model thought up by a human. Just implement Classifier.

You can add all the metadata you need to the Instance class wrapping Weka's own instance. It is kind of a Value object.

You can talk your language (e.g. booleans for classifying between true and false) instead of using doubles for everything like Weka does internally.

Generalizing, for what regards the InstanceSet, you can put on him more responsibilities than just being a data container. In my case it remembers the mapping from domain object to Instance, or it can calculate the error of a Classifier.

Conclusions

Weka is a powerful tool, and you should definitely delegate to it the responsibility of correctly implementing standard machine learning algorithms. However, don't let it creep inside every line of code of your application: it is possible to isolate it in a single package and swap it out when necessary.

TrueSight is an AIOps platform, powered by machine learning and analytics, that elevates IT operations to address multi-cloud complexity and the speed of digital transformation.