五年，终于等来Transformers v5

Core Insights - The article discusses the release of Transformers v5.0.0rc0, marking a significant evolution in the AI infrastructure library after a five-year development cycle from v4 to v5 [3] - The update highlights the growth of the Transformers library, with daily downloads increasing from 20,000 to over 3 million and total installations surpassing 1.2 billion since the v4 release in November 2020 [3] - The new version focuses on four key dimensions: simplicity, transition from fine-tuning to pre-training, interoperability with high-performance inference engines, and making quantization a core feature [3] Simplification - The primary focus of the team is on simplicity, aiming for a clean and clear integration of models, which will enhance standardization, versatility, and community support [5][6] - The library has adopted a modular design approach, facilitating easier maintenance and faster integration, while promoting collaboration within the community [10] Model Updates - Transformers serves as a toolbox for model architectures, with the goal of including all the latest models and becoming the trusted source for model definitions [7] - Over the past five years, an average of 1-3 new models has been added weekly [8] Model Conversion Tools - Hugging Face is developing tools to identify similarities between new models and existing architectures, aiming to automate the model conversion process into the Transformers format [13][14] Training Enhancements - The v5 version emphasizes support for pre-training, with redesigned model initialization and broader compatibility with optimization operators [20] - Hugging Face continues to collaborate with fine-tuning tools in the Python ecosystem and is ensuring compatibility with tools in the JAX ecosystem [21] Inference Improvements - Inference is a key area of optimization in v5, with updates including dedicated kernels, cleaner default settings, new APIs, and enhanced support for inference engines [22][25] - The goal is not to replace specialized inference engines but to achieve compatibility with them [25] Local Deployment - The team collaborates with popular inference engines to ensure that models added to Transformers are immediately available and can leverage the advantages of these engines [27] - Hugging Face is also working on local inference capabilities, allowing models to run directly on devices, with expanding support for multimodal models [28] Quantization - Quantization is becoming a standard in modern model development, with many state-of-the-art models being released in low-precision formats such as 8-bit and 4-bit [29]