Do Text-to-Text Multi-Task Learners Suffer from Task Conflict?
David M. Mueller, Nicholas Andrews, Mark H. Dredze · 2022
Traditional multi-task learning architectures learn a single model across multiple tasks through a shared encoder followed by taskspecific decoders.Learning these models often requires specialized training algorithms that address task-conflict in the shared parameter updates, which otherwise can lead to negative transfer.A new type of multi-task learning within NLP homogenizes multi-task architectures as a shared encoder and language model decoder, which does surprisingly well across a range of diverse tasks (Raffel et al., 2020).Does this new architecture suffer from taskconflicts that require specialized training algorithms?We study how certain factors in the shift towards text-to-text models affects multitask conflict and negative transfer, finding that both directional conflict and transfer are surprisingly constant across architectures.