Search form

You are here

Supervised learning

Definition : Supervised learning

Supervised learning is a Machine Learning (ML) technique for approximating the input/output behavior of complex systems. The task of the supervised learner is to predict the output behavior of a system for any set of input values, after an initial trainig phase.

Background

In supervised learning, there are two factors we can control for optimal generalization: sample points and models. A sample point corresponds to a query to the "oracle", and a model refers to, for example, the type and number of basis functions used for learning. Model SelectionModel selection is one of the central research topics in supervised learning. It is important to determine the complexity of learning machines (models) appropriately for optimal generalization performance. If the model is too simple, it is not flexible enough to represent the learning target function and therefore the generalization performance can not be improved even if a large amount of training data is employed for learning. On the other hand, if the model is too complex, it is capable of representing the target function, but such a complex model is heavily affected by the noise in the training data. Therefore, in practical situations with a rather small number of training samples, complex models do not give better generalization performance. Active Learning The problem of selecting sample points is called active learning (also known as sample selection, sequential design, or optimal experimental design (OED)). If the training data set is designed appropriately, one can obtain better generalization capability even with a small amount of training data. [ref]