Loading topic...

Hugging Face Transformers Library Complete Python Guide

Hugging Face Transformers

Learn the Hugging Face Transformers library with Python. Master AutoModel, AutoTokenizer, pipelines, model loading, saving, the Model Hub, and NLP tasks.

Module 10 — Hugging Face Transformers Library

Introduction

The Hugging Face Transformers library is the most widely used open-source framework for working with Transformer models. It provides a unified API for thousands of pretrained models across Natural Language Processing (NLP), Computer Vision (CV), Audio, and Multimodal AI.

Instead of implementing every Transformer architecture from scratch, the library allows you to load pretrained models with just a few lines of code.

The library supports models such as:

BERT
RoBERTa
GPT-2
GPT-Neo
LLaMA
T5
BART
ViT
CLIP
Whisper
Qwen
Gemma
DeepSeek

In this module, you'll learn:

Installation
AutoModel
AutoTokenizer
AutoProcessor
Pipelines
Config
Model Loading
Saving Models
Hugging Face Model Hub
Build Text Classification
Translation
Summarization

1. Installation

Install the required libraries using pip.

bash
1pip install transformers

Install PyTorch (if not already installed)

bash
1pip install torch

Install additional utilities

bash
1pip install datasets
2pip install accelerate
3pip install sentencepiece
4pip install safetensors

Verify installation

python
1import transformers
2
3print(transformers.__version__)

2. AutoModel

AutoModel automatically loads the correct model architecture based on the model name.

Instead of manually importing model classes, the library determines which architecture to instantiate.

Example

python
1from transformers import AutoModel
2
3model = AutoModel.from_pretrained(
4    "bert-base-uncased"
5)
6
7print(type(model))

Output

text
1BertModel

Advantages

One API for many architectures
Automatically selects the correct class
Supports thousands of models

3. AutoTokenizer

Neural networks cannot process raw text.

A tokenizer converts text into token IDs.

Example

python
1from transformers import AutoTokenizer
2
3tokenizer = AutoTokenizer.from_pretrained(
4    "bert-base-uncased"
5)
6
7text = "Transformers are amazing."
8
9tokens = tokenizer(text)
10
11print(tokens)

Typical output

text
1{
2 'input_ids': [...],
3 'attention_mask': [...],
4 'token_type_ids': [...]
5}

Return tensors

python
1encoded = tokenizer(
2    text,
3    return_tensors="pt"
4)
5
6print(encoded["input_ids"].shape)

4. AutoProcessor

Some models require more than a tokenizer.

For example

Images
Audio
Video
Multimodal inputs

AutoProcessor combines the required preprocessing steps into one interface.

Supported tasks

Vision
OCR
Speech
Vision-Language Models (VLMs)

Example

python
1from transformers import AutoProcessor
2
3processor = AutoProcessor.from_pretrained(
4    "google/vit-base-patch16-224"
5)
6
7print(type(processor))

For models like CLIP or Whisper, the processor handles multiple input modalities.

5. Pipelines

Pipelines provide the easiest way to use pretrained models.

Architecture

text
1Input
2
3↓
4
5Tokenizer / Processor
6
7↓
8
9Model
10
11↓
12
13Postprocessing
14
15↓
16
17Prediction

Sentiment Analysis

python
1from transformers import pipeline
2
3classifier = pipeline(
4    "sentiment-analysis"
5)
6
7result = classifier(
8    "Transformers are amazing!"
9)
10
11print(result)

Fill Mask

python
1from transformers import pipeline
2
3fill = pipeline(
4    "fill-mask"
5)
6
7result = fill(
8    "Paris is the <mask> of France."
9)
10
11print(result)

Question Answering

python
1qa = pipeline(
2    "question-answering"
3)
4
5result = qa(
6    question="What is AI?",
7    context="Artificial Intelligence is the simulation of human intelligence."
8)
9
10print(result)

Pipelines are ideal for quick experiments and prototyping.

6. Config

Every Hugging Face model contains a configuration object.

The configuration stores model hyperparameters, such as:

Hidden size
Number of layers
Attention heads
Vocabulary size
Activation function

Example

python
1from transformers import AutoConfig
2
3config = AutoConfig.from_pretrained(
4    "bert-base-uncased"
5)
6
7print(config)

Access individual fields

python
1print(config.hidden_size)
2
3print(config.num_attention_heads)
4
5print(config.num_hidden_layers)

7. Model Loading

Load pretrained weights from the Hugging Face Hub.

Example

python
1from transformers import AutoModel
2
3model = AutoModel.from_pretrained(
4    "bert-base-uncased"
5)

Load a sequence classification model

python
1from transformers import AutoModelForSequenceClassification
2
3model = AutoModelForSequenceClassification.from_pretrained(
4    "distilbert-base-uncased",
5    num_labels=2
6)

Load a causal language model

python
1from transformers import AutoModelForCausalLM
2
3model = AutoModelForCausalLM.from_pretrained(
4    "gpt2"
5)

The AutoModelFor... classes automatically attach task-specific heads.

8. Saving Models

After fine-tuning a model, save both the model weights and tokenizer.

Save

python
1model.save_pretrained(
2    "./saved_model"
3)
4
5tokenizer.save_pretrained(
6    "./saved_model"
7)

Reload later

python
1from transformers import AutoModel
2from transformers import AutoTokenizer
3
4model = AutoModel.from_pretrained(
5    "./saved_model"
6)
7
8tokenizer = AutoTokenizer.from_pretrained(
9    "./saved_model"
10)

This preserves the model weights, configuration, and tokenizer vocabulary.

9. Hugging Face Model Hub

The Hugging Face Model Hub hosts hundreds of thousands of pretrained models.

You can:

Search models
Download checkpoints
Upload your own models
Share fine-tuned models
Browse datasets
Access model documentation

Typical model identifiers

text
1bert-base-uncased
2
3distilbert-base-uncased
4
5facebook/bart-large-cnn
6
7google/flan-t5-base
8
9openai/clip-vit-base-patch32

Benefits

Open-source community
Versioned models
Easy sharing
Reproducible research

Practice 1 — Text Classification

python
1from transformers import pipeline
2
3classifier = pipeline(
4    task="text-classification",
5    model="distilbert-base-uncased-finetuned-sst-2-english"
6)
7
8texts = [
9    "This course is excellent.",
10    "The service was terrible."
11]
12
13results = classifier(texts)
14
15for text, result in zip(texts, results):
16    print(text)
17    print(result)
18    print("-" * 40)

What You'll Learn

Load a pretrained sentiment analysis model
Perform text classification
Interpret prediction labels and confidence scores

Practice 2 — Translation

python
1from transformers import pipeline
2
3translator = pipeline(
4    task="translation",
5    model="google-t5/t5-small"
6)
7
8text = "Machine learning is transforming the world."
9
10result = translator(text)
11
12print(result)

What You'll Learn

Load a pretrained translation model
Translate text between languages
Understand sequence-to-sequence inference

Practice 3 — Summarization

python
1from transformers import pipeline
2
3summarizer = pipeline(
4    task="summarization",
5    model="facebook/bart-large-cnn"
6)
7
8article = """
9Transformers have become the foundation of modern
10natural language processing. They enable parallel
11processing of sequences and use self-attention
12to capture long-range dependencies efficiently.
13"""
14
15summary = summarizer(
16    article,
17    max_length=40,
18    min_length=15,
19    do_sample=False
20)
21
22print(summary[0]["summary_text"])

What You'll Learn

Load a pretrained summarization model
Generate concise summaries from long text
Configure generation parameters such as maximum length

Mini Project — Unified Inference Script

The following example demonstrates a reusable pattern for loading a tokenizer and model.

python
1from transformers import (
2    AutoTokenizer,
3    AutoModelForSequenceClassification
4)
5import torch
6
7model_name = "distilbert-base-uncased-finetuned-sst-2-english"
8
9tokenizer = AutoTokenizer.from_pretrained(model_name)
10
11model = AutoModelForSequenceClassification.from_pretrained(model_name)
12
13text = "Learning Transformers is exciting."
14
15inputs = tokenizer(
16    text,
17    return_tensors="pt"
18)
19
20with torch.no_grad():
21    outputs = model(**inputs)
22
23prediction = torch.argmax(outputs.logits, dim=-1)
24
25print("Predicted Label:", prediction.item())

Module Summary

After completing this module, you will be able to:

Install and configure the Hugging Face Transformers ecosystem.
Load pretrained models using AutoModel.
Tokenize text with AutoTokenizer.
Process multimodal inputs using AutoProcessor.
Use high-level pipelines for common AI tasks.
Inspect model configurations with AutoConfig.
Load task-specific models using AutoModelFor... classes.
Save and reload fine-tuned models and tokenizers.
Navigate and use models from the Hugging Face Model Hub.
Build practical applications for text classification, translation, and summarization.

Next Module: Module 11 – Hugging Face Datasets & Tokenizers, where you'll learn to load datasets, create custom datasets, preprocess text, batch tokenization, dynamic padding, data collators, and build efficient data pipelines for Transformer training.

Loading topic...

Hugging Face Transformers

Learn the Hugging Face Transformers library with Python. Master AutoModel, AutoTokenizer, pipelines, model loading, saving, the Model Hub, and NLP tasks.

Module 10 — Hugging Face Transformers Library

Introduction

Instead of implementing every Transformer architecture from scratch, the library allows you to load pretrained models with just a few lines of code.

The library supports models such as:

BERT
RoBERTa
GPT-2
GPT-Neo
LLaMA
T5
BART
ViT
CLIP
Whisper
Qwen
Gemma
DeepSeek

In this module, you'll learn:

Installation
AutoModel
AutoTokenizer
AutoProcessor
Pipelines
Config
Model Loading
Saving Models
Hugging Face Model Hub
Build Text Classification
Translation
Summarization

1. Installation

Install the required libraries using pip.

bash
1pip install transformers

Install PyTorch (if not already installed)

bash
1pip install torch

Install additional utilities

bash
1pip install datasets
2pip install accelerate
3pip install sentencepiece
4pip install safetensors

Verify installation

python
1import transformers
2
3print(transformers.__version__)

2. AutoModel

AutoModel automatically loads the correct model architecture based on the model name.

Instead of manually importing model classes, the library determines which architecture to instantiate.

Example

python
1from transformers import AutoModel
2
3model = AutoModel.from_pretrained(
4    "bert-base-uncased"
5)
6
7print(type(model))

Output

text
1BertModel

Advantages

One API for many architectures
Automatically selects the correct class
Supports thousands of models

3. AutoTokenizer

Neural networks cannot process raw text.

A tokenizer converts text into token IDs.

Example

python
1from transformers import AutoTokenizer
2
3tokenizer = AutoTokenizer.from_pretrained(
4    "bert-base-uncased"
5)
6
7text = "Transformers are amazing."
8
9tokens = tokenizer(text)
10
11print(tokens)

Typical output

text
1{
2 'input_ids': [...],
3 'attention_mask': [...],
4 'token_type_ids': [...]
5}

Return tensors

python
1encoded = tokenizer(
2    text,
3    return_tensors="pt"
4)
5
6print(encoded["input_ids"].shape)

4. AutoProcessor

Some models require more than a tokenizer.

For example

Images
Audio
Video
Multimodal inputs

AutoProcessor combines the required preprocessing steps into one interface.

Supported tasks

Vision
OCR
Speech
Vision-Language Models (VLMs)

Example

python
1from transformers import AutoProcessor
2
3processor = AutoProcessor.from_pretrained(
4    "google/vit-base-patch16-224"
5)
6
7print(type(processor))

For models like CLIP or Whisper, the processor handles multiple input modalities.

5. Pipelines

Pipelines provide the easiest way to use pretrained models.

Architecture

text
1Input
2
3↓
4
5Tokenizer / Processor
6
7↓
8
9Model
10
11↓
12
13Postprocessing
14
15↓
16
17Prediction

Sentiment Analysis

python
1from transformers import pipeline
2
3classifier = pipeline(
4    "sentiment-analysis"
5)
6
7result = classifier(
8    "Transformers are amazing!"
9)
10
11print(result)

Fill Mask

python
1from transformers import pipeline
2
3fill = pipeline(
4    "fill-mask"
5)
6
7result = fill(
8    "Paris is the <mask> of France."
9)
10
11print(result)

Question Answering

python
1qa = pipeline(
2    "question-answering"
3)
4
5result = qa(
6    question="What is AI?",
7    context="Artificial Intelligence is the simulation of human intelligence."
8)
9
10print(result)

Pipelines are ideal for quick experiments and prototyping.

6. Config

Every Hugging Face model contains a configuration object.

The configuration stores model hyperparameters, such as:

Hidden size
Number of layers
Attention heads
Vocabulary size
Activation function

Example

python
1from transformers import AutoConfig
2
3config = AutoConfig.from_pretrained(
4    "bert-base-uncased"
5)
6
7print(config)

Access individual fields

python
1print(config.hidden_size)
2
3print(config.num_attention_heads)
4
5print(config.num_hidden_layers)

7. Model Loading

Load pretrained weights from the Hugging Face Hub.

Example

python
1from transformers import AutoModel
2
3model = AutoModel.from_pretrained(
4    "bert-base-uncased"
5)

Load a sequence classification model

python
1from transformers import AutoModelForSequenceClassification
2
3model = AutoModelForSequenceClassification.from_pretrained(
4    "distilbert-base-uncased",
5    num_labels=2
6)

Load a causal language model

python
1from transformers import AutoModelForCausalLM
2
3model = AutoModelForCausalLM.from_pretrained(
4    "gpt2"
5)

The AutoModelFor... classes automatically attach task-specific heads.

8. Saving Models

After fine-tuning a model, save both the model weights and tokenizer.

Save

python
1model.save_pretrained(
2    "./saved_model"
3)
4
5tokenizer.save_pretrained(
6    "./saved_model"
7)

Reload later

python
1from transformers import AutoModel
2from transformers import AutoTokenizer
3
4model = AutoModel.from_pretrained(
5    "./saved_model"
6)
7
8tokenizer = AutoTokenizer.from_pretrained(
9    "./saved_model"
10)

This preserves the model weights, configuration, and tokenizer vocabulary.

9. Hugging Face Model Hub

The Hugging Face Model Hub hosts hundreds of thousands of pretrained models.

You can:

Search models
Download checkpoints
Upload your own models
Share fine-tuned models
Browse datasets
Access model documentation

Typical model identifiers

text
1bert-base-uncased
2
3distilbert-base-uncased
4
5facebook/bart-large-cnn
6
7google/flan-t5-base
8
9openai/clip-vit-base-patch32

Benefits

Open-source community
Versioned models
Easy sharing
Reproducible research

Practice 1 — Text Classification

python
1from transformers import pipeline
2
3classifier = pipeline(
4    task="text-classification",
5    model="distilbert-base-uncased-finetuned-sst-2-english"
6)
7
8texts = [
9    "This course is excellent.",
10    "The service was terrible."
11]
12
13results = classifier(texts)
14
15for text, result in zip(texts, results):
16    print(text)
17    print(result)
18    print("-" * 40)

What You'll Learn

Load a pretrained sentiment analysis model
Perform text classification
Interpret prediction labels and confidence scores

Practice 2 — Translation

python
1from transformers import pipeline
2
3translator = pipeline(
4    task="translation",
5    model="google-t5/t5-small"
6)
7
8text = "Machine learning is transforming the world."
9
10result = translator(text)
11
12print(result)

What You'll Learn

Load a pretrained translation model
Translate text between languages
Understand sequence-to-sequence inference

Practice 3 — Summarization

python
1from transformers import pipeline
2
3summarizer = pipeline(
4    task="summarization",
5    model="facebook/bart-large-cnn"
6)
7
8article = """
9Transformers have become the foundation of modern
10natural language processing. They enable parallel
11processing of sequences and use self-attention
12to capture long-range dependencies efficiently.
13"""
14
15summary = summarizer(
16    article,
17    max_length=40,
18    min_length=15,
19    do_sample=False
20)
21
22print(summary[0]["summary_text"])

What You'll Learn

Load a pretrained summarization model
Generate concise summaries from long text
Configure generation parameters such as maximum length

Mini Project — Unified Inference Script

The following example demonstrates a reusable pattern for loading a tokenizer and model.

python
1from transformers import (
2    AutoTokenizer,
3    AutoModelForSequenceClassification
4)
5import torch
6
7model_name = "distilbert-base-uncased-finetuned-sst-2-english"
8
9tokenizer = AutoTokenizer.from_pretrained(model_name)
10
11model = AutoModelForSequenceClassification.from_pretrained(model_name)
12
13text = "Learning Transformers is exciting."
14
15inputs = tokenizer(
16    text,
17    return_tensors="pt"
18)
19
20with torch.no_grad():
21    outputs = model(**inputs)
22
23prediction = torch.argmax(outputs.logits, dim=-1)
24
25print("Predicted Label:", prediction.item())

Module Summary

After completing this module, you will be able to:

Install and configure the Hugging Face Transformers ecosystem.
Load pretrained models using AutoModel.
Tokenize text with AutoTokenizer.
Process multimodal inputs using AutoProcessor.
Use high-level pipelines for common AI tasks.
Inspect model configurations with AutoConfig.
Load task-specific models using AutoModelFor... classes.
Save and reload fine-tuned models and tokenizers.
Navigate and use models from the Hugging Face Model Hub.
Build practical applications for text classification, translation, and summarization.