T5_(language_model)
T5 (language model)
Series of large language models
T5 (Text-to-Text Transfer Transformer) is a series of large language models developed by Google AI. Introduced in 2019,[1] T5 models are trained on a massive dataset of text and code using a text-to-text framework. The T5 models are capable of performing the text-based tasks that they were pretrained for. They can also be finetuned to perform other tasks.They have been employed in various applications, including chatbots, machine translation systems, text summarization tools, code generation, and robotics.
Like the original Transformer model,[2] T5 models are encoder-decoder Transformers, where the encoder processes the input text, and the decoder generates the output text.
It was updated by T5X in 2022 to use JAX.[3] In 2024, T5X was updated to Pile-T5 by training the same architecture on an improved dataset (The Pile).[4]