![]() Moreover, task scaling consistently improves performance at various model scales. With an extremely large number of training tasks, the model size has little impact on performance. Moreover, we present a prompting method that incorporates a genetic algorithm to automatically search for the best prompt for unseen tasks, along with a few other improvements.Įmpirically, ZeroPrompt substantially improves both the efficiency and the performance of zero-shot learning across a variety of academic and production datasets.ġ Introduction Figure 1: Task scaling vs. ![]() Our results show that task scaling can substantially improve training efficiency by 30 times in FLOPs. ![]() This leads to a crucial discovery that task scaling can be an efficient alternative to model scaling i.e., the model size has little impact on performance with an extremely large number of tasks. While previous models are trained on only a few dozen tasks, we scale to 1,000 tasks for the first time using real-world data. We propose a multitask pretraining approach ZeroPrompt for zero-shot generalization, focusing on task scaling and zero-shot prompting. ![]() Abstract † † ∗ Equal contribution † † † Corresponding author
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |