Cross-Task Generalization Abilities of Large Language Models
Qinyuan Ye · 2024
Humans can learn a new language task efficiently with only few examples, by leveraging their knowledge and experience obtained when learning prior tasks.Enabling similar crosstask generalization abilities in NLP systems is fundamental for approaching the goal of general intelligence and expanding the reach of language technology in the future.In this thesis proposal, I will present my work on (1) benchmarking cross-task generalization abilities with diverse NLP tasks; (2) developing model architectures for improving cross-task generalization abilities; (3) analyzing and predicting the generalization landscape of current state-of-theart large language models.Additionally, I will outline future research directions, along with preliminary thoughts on addressing them.