2-Using_Transformers-0-Introduction

中英文对照学习，效果更佳！
原课程链接：https://huggingface.co/course/chapter2/1?fw=pt

Introduction

引言

Ask a Question

问一个问题

As you saw in [Chapter 1], Transformer models are usually very large. With millions to tens of billions of parameters, training and deploying these models is a complicated undertaking. Furthermore, with new models being released on a near-daily basis and each having its own implementation, trying them all out is no easy task.

正如你在第一章中看到的，Transformer的模型通常都很大。在数百万到数百亿个参数的情况下，培训和部署这些模型是一项复杂的任务。此外，由于几乎每天都有新的型号发布，而且每个型号都有自己的实现，尝试所有这些型号并不是一件容易的事情。

The 🤗 Transformers library was created to solve this problem. Its goal is to provide a single API through which any Transformer model can be loaded, trained, and saved. The library’s main features are:

🤗Transformer程序库就是为了解决这个问题而创建的。它的目标是提供一个单一的API，通过该API可以加载、训练和保存任何Transformer模型。该图书馆的主要特点包括：

  • Ease of use: Downloading, loading, and using a state-of-the-art NLP model for inference can be done in just two lines of code.
  • Flexibility: At their core, all models are simple PyTorch nn.Module or TensorFlow tf.keras.Model classes and can be handled like any other models in their respective machine learning (ML) frameworks.
  • Simplicity: Hardly any abstractions are made across the library. The “All in one file” is a core concept: a model’s forward pass is entirely defined in a single file, so that the code itself is understandable and hackable.

This last feature makes 🤗 Transformers quite different from other ML libraries. The models are not built on modules
that are shared across files; instead, each model has its own layers. In addition to making the models more approachable and understandable, this allows you to easily experiment on one model without affecting others.

易于使用：下载、下载和使用最先进的NLP模型进行推理只需两行代码即可完成。灵活性：所有模型的核心都是简单的PyTorchnn.Module或TensorFlowtf.keras.Model类，并且可以像它们各自的机器学习(ML)框架中的任何其他模型一样进行处理。简单性：几乎没有跨库进行任何抽象。“所有在一个文件中”是一个核心概念：模型的前向传递完全定义在单个文件中，因此代码本身是可理解和可破解的。这最后一个特性使🤗Transformer与其他ML库有很大的不同。这些模型不是在跨文件共享的模块上构建的；相反，每个模型都有自己的层。除了使模型更易于接近和理解外，这还允许您轻松地在一个模型上进行实验，而不会影响其他模型。

This chapter will begin with an end-to-end example where we use a model and a tokenizer together to replicate the pipeline() function introduced in [Chapter 1]. Next, we’ll discuss the model API: we’ll dive into the model and configuration classes, and show you how to load a model and how it processes numerical inputs to output predictions.

本章将从一个端到端的示例开始，在该示例中，我们将一起使用模型和记号赋值器来复制第一章中介绍的`Pipeline()‘函数。接下来，我们将讨论模型API：我们将深入研究模型和配置类，并向您展示如何加载模型以及它如何处理数值输入以输出预测。

Then we’ll look at the tokenizer API, which is the other main component of the pipeline() function. Tokenizers take care of the first and last processing steps, handling the conversion from text to numerical inputs for the neural network, and the conversion back to text when it is needed. Finally, we’ll show you how to handle sending multiple sentences through a model in a prepared batch, then wrap it all up with a closer look at the high-level tokenizer() function.

然后，我们将查看记号赋值器API，它是Pipeline()函数的另一个主要组件。令牌化器负责第一个也是最后一个处理步骤，处理从文本到神经网络的数字输入的转换，并在需要时转换回文本。最后，我们将向您展示如何在准备好的批处理中通过一个模型发送多个语句，然后通过更仔细地研究高级的`tokenizer()‘函数来总结所有这些内容。

⚠️ In order to benefit from all features available with the Model Hub and 🤗 Transformers, we recommend creating an account.

⚠️为了从Model Hub和🤗Transformers提供的所有功能中获益，我们建议创建一个帐户。