Abstract

Today's potential users of machine learning technology are faced with the non-trivial problem of choosing, from the large, ever-increasing number of available tools, the one most appropriate for their particular task. To assist the often non-initiated users, it is desirable that this model selection process be automated. Using experience from base level learning, researchers have proposed meta-learning as a possible solution. Historically, predictive accuracy has been the de facto criterion, with most work in meta-learning focusing on the discovery of rules that match applications to models based on accuracy only. Although predictive accuracy is clearly an important criterion, it is also the case that there are a number of other criteria that could, and often ought to, be considered when learning about model selection. This paper presents a number of such criteria and discusses the impact they have on meta-level approaches to model selection.