BibTeX

Bookmark

OpenURL

Abstract

This paper presents a co-regularization based approach to semi-supervised domain adaptation. Our proposed approach (EA++) builds on the notion of augmented space (introduced in EASYADAPT (EA) [1]) and harnesses unlabeled data in target domain to further assist the transfer of information from source to target. This semi-supervised approach to domain adaptation is extremely simple to implement and can be applied as a pre-processing step to any supervised learner. Our theoretical analysis (in terms of Rademacher complexity) of EA and EA++ show that the hypothesis class of EA++ has lower complexity (compared to EA) and hence results in tighter generalization bounds. Experimental results on sentiment analysis tasks reinforce our theoretical findings and demonstrate the efficacy of the proposed method when compared to EA as well as few other representative baseline approaches. 1

Citations

.... We have formulated EA and EA++ in terms of co-regularization, an idea that originated in the context of multiview learning [13, 19]. Our proposed formulation also bears resemblance to existing work =-=[20]-=- in multiview semi-supervised (SSL) literature which has been studied extensively in [21, 22, 23]. The difference being, while in multiview SSL one would try to make the different hypotheses learned f...

...bstract This paper presents a co-regularization based approach to semi-supervised domain adaptation. Our proposed approach (EA++) builds on the notion of augmented space (introduced in EASYADAPT (EA) =-=[1]-=-) and harnesses unlabeled data in target domain to further assist the transfer of information from source to target. This semi-supervised approach to domain adaptation is extremely simple to implement...

...to online multi-domain setting in [4]. Prior work on semi-supervised approaches to domain adaptation also exists in literature. Extraction of specific features from the available dataset was proposed =-=[5, 6]-=- to facilitate the task of domain adaptation. Co-adaptation [7], a combination of co-training and domain adaptation, can also be considered as a semi-supervised approach to domain adaptation. A semi-s...

...and target unlabeled samples. 5 Experiments We follow experimental setups similar to [1] but report our empirical results for the task of sentiment classification using the SENTIMENT data provided by =-=[15]-=-. The task of sentiment classification is a binary classification task which corresponds to classifying a review as positive or negative for user reviews of eight product types (apparel, books, DVD, e...

...e for user reviews of eight product types (apparel, books, DVD, electronics, kitchen, music, video, and other) collected from amazon.com. We quantify the domain divergences in terms of the A-distance =-=[16]-=- which is computed [17] from finite samples of source and target domain using the proxy A-distance [16]. For our experiments, we consider the following domain-pairs: (a) DVD→BOOKS (proxy A-distance=0....

... scenarios. In this section, we extend EA to semi-supervised settings while maintaining the desirable classifier-agnostic property. 23.1 Motivation In multi-view approach to semi-supervised learning =-=[12]-=-, different hypotheses are learned using different views of the dataset. Thereafter, unlabeled data is utilized to co-regularize these learned hypotheses by making them agree on unlabeled samples. In ...

... t,ft) and νt := ǫt(h ∗ t,ft), where fs and ft are the source and target labeling functions, and h ∗ t is the optimal target hypothesis in target hypothesis class. It also uses dH∆H(Ds, Dt)− distance =-=[14]-=-, which is defined as sup h1,h2∈H 2|ǫs(h1, h2) − ǫt(h1, h2)|. The dH∆H−distance measures the distance between two distribution using a hypothesis class-specific distance measure. If the two domains ar...

...n the context of multiview learning [13, 19]. Our proposed formulation also bears resemblance to existing work [20] in multiview semi-supervised (SSL) literature which has been studied extensively in =-=[21, 22, 23]-=-. The difference being, while in multiview SSL one would try to make the different hypotheses learned from different views agree on unlabeled data, in semi-supervised domain adaptation the aim is to m...

...7], a combination of co-training and domain adaptation, can also be considered as a semi-supervised approach to domain adaptation. A semi-supervised EM algorithm for domain adaptation was proposed in =-=[8]-=-. Similar to graph based semi-supervised approaches, a label propagation method was proposed [9] to facilitate domain adaptation. Domain Adaptation Machine (DAM) [10] is a semi-supervised extension of...

... domain adaptation was proposed in [8]. Similar to graph based semi-supervised approaches, a label propagation method was proposed [9] to facilitate domain adaptation. Domain Adaptation Machine (DAM) =-=[10]-=- is a semi-supervised extension of SVMs for domain adaptation and presents extensive empirical results. Nevertheless, in almost all of the above cases, the proposed methods either use specifics of the...

.... We assume that the loss L(y,h.x) is bounded by 1 for the zero hypothesis h = 0. This is true for many popular loss functions including square loss and hinge loss when y ∈ {−1, +1}. One possible way =-=[13]-=- of defining the hypotheses classes is to substitute trivial hypothesesh1 = h2 = 0 in both the cost functions which makes all regularizers and co-regularizers equal to zero and thus bounds the cost fu...

...ies to both fully supervised and semi-supervised domain adaptation settings. We have formulated EA and EA++ in terms of co-regularization, an idea that originated in the context of multiview learning =-=[13, 19]-=-. Our proposed formulation also bears resemblance to existing work [20] in multiview semi-supervised (SSL) literature which has been studied extensively in [21, 22, 23]. The difference being, while in...

...n the context of multiview learning [13, 19]. Our proposed formulation also bears resemblance to existing work [20] in multiview semi-supervised (SSL) literature which has been studied extensively in =-=[21, 22, 23]-=-. The difference being, while in multiview SSL one would try to make the different hypotheses learned from different views agree on unlabeled data, in semi-supervised domain adaptation the aim is to m...

...sed approach to domain adaptation. A semi-supervised EM algorithm for domain adaptation was proposed in [8]. Similar to graph based semi-supervised approaches, a label propagation method was proposed =-=[9]-=- to facilitate domain adaptation. Domain Adaptation Machine (DAM) [10] is a semi-supervised extension of SVMs for domain adaptation and presents extensive empirical results. Nevertheless, in almost al...

...-pairs: (a) DVD→BOOKS (proxy A-distance=0.7616) and, (b) KITCHEN→APPAREL (proxy A-distance=0.0459). As in [1], we use an averaged perceptron classifier from the Megam framework (implementation due to =-=[18]-=-) for all the aforementioned tasks. The training sample size varies from 1k to 16k. In all cases, the amount of unlabeled target data is equal to the total amount of labeled source and target data. We...

...n the context of multiview learning [13, 19]. Our proposed formulation also bears resemblance to existing work [20] in multiview semi-supervised (SSL) literature which has been studied extensively in =-=[21, 22, 23]-=-. The difference being, while in multiview SSL one would try to make the different hypotheses learned from different views agree on unlabeled data, in semi-supervised domain adaptation the aim is to m...

...lds relative improvements in the range of 14.08% − 39.29% over EA for different number of sample points experimented with. Similar trends were observed for other tasks and datasets (refer Figure 3 of =-=[2]-=-). 6 Conclusions We proposed a semi-supervised extension to an existing domain adaptation technique (EA). Our approach EA++, leverages unlabeled data to improve the performance of EA. With this extens...

...ed approaches to domain adaptation also exists in literature. Extraction of specific features from the available dataset was proposed [5, 6] to facilitate the task of domain adaptation. Co-adaptation =-=[7]-=-, a combination of co-training and domain adaptation, can also be considered as a semi-supervised approach to domain adaptation. A semi-supervised EM algorithm for domain adaptation was proposed in [8...

...ean. SVMs were used as the base classifiers and the algorithm was formulated in the standard SVM dual optimization setting. Subsequently, this framework was extended to online multi-domain setting in =-=[4]-=-. Prior work on semi-supervised approaches to domain adaptation also exists in literature. Extraction of specific features from the available dataset was proposed [5, 6] to facilitate the task of doma...

...to online multi-domain setting in [4]. Prior work on semi-supervised approaches to domain adaptation also exists in literature. Extraction of specific features from the available dataset was proposed =-=[5, 6]-=- to facilitate the task of domain adaptation. Co-adaptation [7], a combination of co-training and domain adaptation, can also be considered as a semi-supervised approach to domain adaptation. A semi-s...

... generalization bounds also apply to the approach proposed in [3] for domain adaptation setting, where we are only concerned with the error on target domain. The closest to our work is a recent paper =-=[11]-=- that theoretically analyzes EASYADAPT. Their paper investigates the necessity to combine supervised and unsupervised domain adaptation (which the authors refer to as labeled and unlabeled adaptation ...