Musings about grad school and personal fulfillment.

RWeka Hacks: Applying Supervised Filters to Test Sets

As a researcher, analyzing data is one of my core responsibilities. While there are many tookits, libraries and languages dedicated to statistical analysis, I generally find myself using R. R does mostly what I want. But, sometimes it doesn’t.

For example, I have a pretty specific workflow for building performant classifiers (especially the kind that are atheoretic, so their performance need not be explained). Maybe one day I’ll write a post about that, but it generally involves supervised discretization [1], feature subset selection [2], followed by model selection [3]. Weka is a machine learning tookit that makes all of these steps easy. R probably has some combination of libraries that does all of these things independently, but I have yet to find a franken-workflow that makes this process as painless as just using Weka. Weka exposes a Java library, though, and I find using Java for vector processing is cumbersome.

Luckily, you can talk to Weka in R with the RWeka package [3]. The RWeka package is great. It uses rJava to create all of the links required to interchange data frames into ArrayLists and what not. But, it’s one of those many libraries-on-the-fringe of conventional development that is only half built. It exposes some of the functionality of Weka to R through a nice interface, and a framework to provide the rest of the functionality, but if you need anything other than the typical uses of Weka then you’ll have to do some reverse engineering.

At this point, I’ve done a lot of that. So, I thought I’d share a series of posts where I talk about how I implemented some the functionality of Weka that I would like in R but that is not supported by the prepackaged RWeka interface. This first post is about applying a supervised discretization filter that is fitted on a training set but then applied to a test set.

Details

I’ll keep this short. This is a function I modified from the RWeka source code that takes in a train set, a test set and the name of a supervised attribute filter [4]. The function will train the filter on the train set and transform the test set accordingly. Similarly, it will also apply the trained filter on the test set and transform the test set accordingly.

# Takes in a train and test model frame along with the name
# of a supervised Weka filter. Trains the filter on the training
# model frame, applies the fitted filter to the test model frame
# and returns the transformed train and test frames.
rWeka.apply.filter

Usage:

# train is any data frame, test is any other data frame of the same form as train.
rWeka.apply.filter(train, test, "weka.filters.supervised.attribute.Discretize")

While I've only tested with the Discretize attribute filter (because that's the only supervised attribute filter I typically use), this function should theoretically work for any other supervised attribute filter as well. It should also be easy to generalize this function to more than just a train and test set. I would just accept a list argument and lapply the the .apply.filter helper function.

Unfortunately, in order to get this function to work, you'll also need to copy over a couple of RWeka's internal functions that are not exposed by simply installing the package. To do so, you'll have to download the RWeka source. Once you do, open up readers.R and copy over the following two functions:

That's about it. This one was pretty simple, but hopefully it's helpful!

Footnotes:

[1] Supervised discretization is just the process of transforming continuous variables into discrete variable by binning them along cut-points that best distinguish between classes. I generally use Fayyad-Irani's entropy minimization approach.

[2] Feature subset selection is the process of selecting a performant subset of features for building a classifier. It's important because it helps avoid overfitting your model to training data, particularly when you do not have a lot of training data. If I have few features, I'll use the Wrapper approach that tests every possible subset of features. If not, I'll generally use correlation feature selection.

[3] This is less systematic. I generally have some intuition as to which models will work best based on the shape and properties of my data, but it's mostly just trial and error and parameter tuning.

[4] In making this function, I referenced both the Weka javadocs and the RWeka internal function for using filters (RWeka_use_filter in filters.R). I basically just generalized the existing function to apply a filter to multiple datasets and return a list.

4 thoughts on “RWeka Hacks: Applying Supervised Filters to Test Sets”

Hello,
I have come across your blog while while i was looking for answer about the topic you presented here. But still, I can’t apply my RWeka::Discretize() function to a test function. I would like some help from you. Especially on the two functions read_data_into_weka() and read_Instances_from_Weka(). How do you manage to get them recognize by your RWeka package.

You should copy over the two functions (read_data_into_weka() and read_Instances_from_Weka()) from the RWeka source code so that your local code has access to those functions. RWeka has access to those functions by default. Have you copied over the functions?

Hi sauvik,
Thank you for your reply.
Yes i copied them over into R.
While i can use those functions into my R local code, I am having another trouble in this part of your code : # .jcall(filter, “Z”, “setInputFormat”, input$train)
R is throwing an exception saying: # weka.core.UnassignedClassException:
weka.filters.supervised.attribute.Discretize: Class attribute not set!

In Rweka::Discretize(), you have to set a fomula, let’s say # Discretize(y~.) #. I was wondering why in your function we don’t have that kind of setting? Why isn’t there a response variable from which we apply the filter to the other variables in train set?