“The preeminent book for the preeminent transformers library—a model of clarity!” —Jeremy Howard, cofounder of fast.ai and professor at University of Queensland
“A wonderfully clear and incisive guide to modern NLP’s most essential library. Recommended!” —Christopher Manning, Thomas M. Siebel Professor in Machine Learning, Stanford University
Buy the book on Amazon Read the book online at O’Reilly Download the book’s code
Since their introduction in 2017, transformers have quickly become the dominant architecture for achieving state-of-the-art results on a variety of natural language processing tasks. If you’re a data scientist or coder, this practical book shows you how to train and scale these large models using Hugging Face Transformers, a Python-based deep learning library.
Transformers have been used to write realistic news stories, improve Google Search queries, and even create chatbots that tell corny jokes. In this guide, authors Lewis Tunstall, Leandro von Werra, and Thomas Wolf use a hands-on approach to teach you how transformers work and how to integrate them in your applications. You’ll quickly learn a variety of tasks they can help you solve.
- Build, debug, and optimize transformer models for core NLP tasks, such as text classification, named entity recognition, and question answering
- Learn how transformers can be used for cross-lingual transfer learning
- Apply transformers in real-world scenarios where labeled data is scarce
- Make transformer models efficient for deployment using techniques such as distillation, pruning, and quantization
- Train transformers from scratch and learn how to scale to multiple GPUs and distributed environments
News 🗞️
January 31, 2022
Lewis will be joining Abhishek Thakur to talk about the book and various techniques you can use to optimize Transformer models for production environments. We’ll also be giving away 5 copies of the book – join the event here!
June 17, 2022
Due to the popularity of the book, O’Reilly has decided to print it in full color from now on in the revised edition 🥳. Thank you to everyone who helped make this happen!
June 29, 2022
Lewis will be presenting at Munich NLP to talk about the book and various techniques you can use to optimize Transformer models for production environments. We’ll also be giving away 3 electronic copies of the book – join the event here!
Contact
For questions, comments, or requests to interview the authors, please send an email to contact@transformersbook.com.
To submit errata or errors in the book, please do so via the O’Reilly platform.