Example-based Hypernetworks for Multi-source Adaptation to Unseen Domains
Tomer Volk, Eyal Ben-David, Ohad Amosy, Gal Chechik, Roi Reichart · 2023
As Natural Language Processing (NLP) algorithms continually achieve new milestones, outof-distribution generalization remains a significant challenge.This paper addresses the issue of multi-source adaptation for unfamiliar domains: We leverage labeled data from multiple source domains to generalize to unknown target domains at training.Our innovative framework employs example-based Hypernetwork adaptation: a T5 encoder-decoder initially generates a unique signature from an input example, embedding it within the source domains' semantic space.This signature is subsequently utilized by a Hypernetwork to generate the task classifier's weights.In an advanced version, the signature also enriches the input example's representation.We evaluated our method across two tasks-sentiment classification and natural language inference-in 29 adaptation scenarios, where it outpaced established algorithms.We also compare our finetuned architecture to few-shot GPT-3, demonstrating its effectiveness in essential use cases.To our knowledge, this marks the first application of Hypernetworks to the adaptation for unknown domains 1 .