A PAC-Bayesian Approach for Domain Adaptation with Specialization to Linear Classifiers

Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):738-746, 2013.

Abstract

We provide a first PAC-Bayesian analysis for domain adaptation (DA) which arises when the learning and test distributions differ. It relies on a novel distribution pseudodistance based on a disagreement averaging. Using this measure, we derive a PAC-Bayesian DA bound for the stochastic Gibbs classifier. This bound has the advantage of being directly optimizable for any hypothesis space. We specialize it to linear classifiers, and design a learning algorithm which shows interesting results on a synthetic problem and on a popular sentiment annotation task. This opens the door to tackling DA tasks by making use of all the PAC-Bayesian tools.

Related Material

@InProceedings{pmlr-v28-germain13,
title = {A PAC-Bayesian Approach for Domain Adaptation with Specialization to Linear Classifiers},
author = {Pascal Germain and Amaury Habrard and François Laviolette and Emilie Morvant},
booktitle = {Proceedings of the 30th International Conference on Machine Learning},
pages = {738--746},
year = {2013},
editor = {Sanjoy Dasgupta and David McAllester},
volume = {28},
number = {3},
series = {Proceedings of Machine Learning Research},
address = {Atlanta, Georgia, USA},
month = {17--19 Jun},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v28/germain13.pdf},
url = {http://proceedings.mlr.press/v28/germain13.html},
abstract = {We provide a first PAC-Bayesian analysis for domain adaptation (DA) which arises when the learning and test distributions differ. It relies on a novel distribution pseudodistance based on a disagreement averaging. Using this measure, we derive a PAC-Bayesian DA bound for the stochastic Gibbs classifier. This bound has the advantage of being directly optimizable for any hypothesis space. We specialize it to linear classifiers, and design a learning algorithm which shows interesting results on a synthetic problem and on a popular sentiment annotation task. This opens the door to tackling DA tasks by making use of all the PAC-Bayesian tools. }
}

%0 Conference Paper
%T A PAC-Bayesian Approach for Domain Adaptation with Specialization to Linear Classifiers
%A Pascal Germain
%A Amaury Habrard
%A François Laviolette
%A Emilie Morvant
%B Proceedings of the 30th International Conference on Machine Learning
%C Proceedings of Machine Learning Research
%D 2013
%E Sanjoy Dasgupta
%E David McAllester
%F pmlr-v28-germain13
%I PMLR
%J Proceedings of Machine Learning Research
%P 738--746
%U http://proceedings.mlr.press
%V 28
%N 3
%W PMLR
%X We provide a first PAC-Bayesian analysis for domain adaptation (DA) which arises when the learning and test distributions differ. It relies on a novel distribution pseudodistance based on a disagreement averaging. Using this measure, we derive a PAC-Bayesian DA bound for the stochastic Gibbs classifier. This bound has the advantage of being directly optimizable for any hypothesis space. We specialize it to linear classifiers, and design a learning algorithm which shows interesting results on a synthetic problem and on a popular sentiment annotation task. This opens the door to tackling DA tasks by making use of all the PAC-Bayesian tools.

TY - CPAPER
TI - A PAC-Bayesian Approach for Domain Adaptation with Specialization to Linear Classifiers
AU - Pascal Germain
AU - Amaury Habrard
AU - François Laviolette
AU - Emilie Morvant
BT - Proceedings of the 30th International Conference on Machine Learning
PY - 2013/02/13
DA - 2013/02/13
ED - Sanjoy Dasgupta
ED - David McAllester
ID - pmlr-v28-germain13
PB - PMLR
SP - 738
DP - PMLR
EP - 746
L1 - http://proceedings.mlr.press/v28/germain13.pdf
UR - http://proceedings.mlr.press/v28/germain13.html
AB - We provide a first PAC-Bayesian analysis for domain adaptation (DA) which arises when the learning and test distributions differ. It relies on a novel distribution pseudodistance based on a disagreement averaging. Using this measure, we derive a PAC-Bayesian DA bound for the stochastic Gibbs classifier. This bound has the advantage of being directly optimizable for any hypothesis space. We specialize it to linear classifiers, and design a learning algorithm which shows interesting results on a synthetic problem and on a popular sentiment annotation task. This opens the door to tackling DA tasks by making use of all the PAC-Bayesian tools.
ER -