Module 20 — Fine-Tuning Transformers

Introduction

Pretrained Transformer models already contain a vast amount of linguistic and domain knowledge learned from massive datasets. However, to perform well on a specific task—such as sentiment analysis, document classification, medical question answering, or customer support—they often need additional training on task-specific data.

This process is called fine-tuning.

Modern fine-tuning techniques range from updating all model parameters to training only a small subset of parameters while keeping the original model frozen. These efficient methods reduce memory usage, training time, and hardware requirements.

Applications include:

• Text Classification
• Named Entity Recognition (NER)
• Question Answering
• Machine Translation
• Chatbots
• Domain-Specific LLMs
• Code Generation
• Vision and Speech Models

In this module, you'll learn:

• Full Fine-Tuning
• Transfer Learning
• PEFT (Parameter-Efficient Fine-Tuning)
• LoRA
• QLoRA
• Adapters
• Prefix Tuning
• Prompt Tuning
• BitFit
• IA³
• Fine-tune BERT
• Fine-tune LLaMA using LoRA

Fine-Tuning Workflow

textCopy
1        Pretrained Model
2               │
3     Task-Specific Dataset
4               │
5      Fine-Tuning Strategy
6               │
7      Model Optimization
8               │
9      Fine-Tuned Model
10               │
11        Deployment

1. Full Fine-Tuning

What is Full Fine-Tuning?

In full fine-tuning, every trainable parameter in the model is updated.

Architecture

textCopy
1Pretrained Model
2
3↓
4
5Update All Parameters
6
7↓
8
9Task-Specific Model

Advantages

• Highest flexibility
• Maximum adaptation to new tasks
• Often best performance when sufficient data and compute are available

Disadvantages

• High GPU memory usage
• Long training time
• Large optimizer state
• Expensive for very large LLMs

Suitable For

• BERT
• RoBERTa
• DistilBERT
• T5
• Smaller LLMs

2. Transfer Learning

What is Transfer Learning?

Transfer learning reuses knowledge learned during pretraining and adapts it to a downstream task.

Pipeline

textCopy
1Large Corpus
2
3↓
4
5Pretraining
6
7↓
8
9Pretrained Model
10
11↓
12
13Fine-Tuning
14
15↓
16
17Specific Task

Benefits

• Faster convergence
• Less labeled data
• Better generalization
• Reduced training cost

3. PEFT (Parameter-Efficient Fine-Tuning)

What is PEFT?

PEFT trains only a small number of additional parameters while freezing the original model weights.

Architecture

textCopy
1Frozen Model
2
3↓
4
5Small Trainable Modules
6
7↓
8
9Task-Specific Output

Advantages

• Low memory usage
• Fast training
• Small checkpoints
• Easy deployment
• Multiple task adapters for one base model

Common PEFT methods

• LoRA
• QLoRA
• Adapters
• Prefix Tuning
• Prompt Tuning
• BitFit
• IA³

4. LoRA (Low-Rank Adaptation)

What is LoRA?

LoRA freezes the pretrained weights and injects low-rank trainable matrices into selected linear layers (commonly the attention projections).

Instead of updating the full weight matrix W, LoRA learns two smaller matrices A and B.

Concept

textCopy
1Original Weight (Frozen)
2
3↓
4
5Low-Rank Matrix A
6
7↓
8
9Low-Rank Matrix B
10
11↓
12
13Updated Output

Advantages

• Very low GPU memory usage
• Fast training
• Small checkpoints
• Widely supported

Common targets

• Query projection
• Key projection
• Value projection
• Output projection

5. QLoRA

What is QLoRA?

QLoRA combines 4-bit quantization with LoRA.

Pipeline

textCopy
1Pretrained Model
2
3↓
4
54-bit Quantization
6
7↓
8
9LoRA Layers
10
11↓
12
13Fine-Tuning

Advantages

• Fine-tune very large LLMs on a single GPU
• Reduced VRAM usage
• Maintains strong performance
• Lower storage requirements

6. Adapters

What are Adapters?

Adapters insert small neural network modules between Transformer layers while leaving the original model frozen.

Architecture

textCopy
1Transformer Layer
2
3↓
4
5Adapter
6
7↓
8
9Transformer Layer

Advantages

• Lightweight
• Modular
• Multiple task-specific adapters
• Easy to swap between tasks

7. Prefix Tuning

What is Prefix Tuning?

Prefix Tuning learns a set of trainable vectors that are prepended to the attention mechanism.

Architecture

textCopy
1Learned Prefix
2
3↓
4
5Transformer
6
7↓
8
9Task Output

Advantages

• Few trainable parameters
• Efficient training
• Useful for generation tasks

8. Prompt Tuning

What is Prompt Tuning?

Prompt Tuning learns continuous prompt embeddings instead of modifying the model weights.

Architecture

textCopy
1Learned Prompt Embeddings
2
3↓
4
5Frozen Model
6
7↓
8
9Prediction

Advantages

• Extremely lightweight
• Efficient for large language models
• Easy to store and share

9. BitFit

What is BitFit?

BitFit updates only the bias parameters in the model while freezing all other weights.

Pipeline

textCopy
1Model
2
3↓
4
5Freeze Weights
6
7↓
8
9Train Bias Terms
10
11↓
12
13Prediction

Advantages

• Very few trainable parameters
• Fast optimization
• Strong baseline for lightweight adaptation

10. IA³ (Infused Adapter by Inhibiting and Amplifying Inner Activations)

What is IA³?

IA³ introduces learnable scaling vectors that modulate internal activations without changing the original weight matrices.

Architecture

textCopy
1Frozen Layer
2
3↓
4
5Scaling Vectors
6
7↓
8
9Modified Activations
10
11↓
12
13Output

Advantages

• Very parameter efficient
• Low memory footprint
• Competitive performance on many tasks

Comparison of Fine-Tuning Methods

Practice 1 — Fine-Tune BERT

The following example fine-tunes BERT for text classification using the Hugging Face Trainer API.

pythonCopy
1from datasets import load_dataset
2from transformers import (
3    AutoTokenizer,
4    AutoModelForSequenceClassification,
5    Trainer,
6    TrainingArguments
7)
8
9dataset = load_dataset("imdb")
10
11tokenizer = AutoTokenizer.from_pretrained(
12    "bert-base-uncased"
13)
14
15def tokenize(batch):
16    return tokenizer(
17        batch["text"],
18        truncation=True,
19        padding="max_length",
20        max_length=256
21    )
22
23dataset = dataset.map(tokenize, batched=True)
24
25model = AutoModelForSequenceClassification.from_pretrained(
26    "bert-base-uncased",
27    num_labels=2
28)
29
30training_args = TrainingArguments(
31    output_dir="./bert-imdb",
32    learning_rate=2e-5,
33    per_device_train_batch_size=16,
34    per_device_eval_batch_size=16,
35    num_train_epochs=3,
36    evaluation_strategy="epoch",
37    save_strategy="epoch",
38    weight_decay=0.01
39)
40
41trainer = Trainer(
42    model=model,
43    args=training_args,
44    train_dataset=dataset["train"],
45    eval_dataset=dataset["test"]
46)
47
48trainer.train()

What You'll Learn

• Load a pretrained BERT model.
• Tokenize a text classification dataset.
• Configure the Hugging Face Trainer.
• Fine-tune and evaluate BERT.

Practice 2 — Fine-Tune LLaMA using LoRA

This example uses the PEFT library to apply LoRA to a LLaMA-style causal language model.

pythonCopy
1from datasets import load_dataset
2from transformers import (
3    AutoTokenizer,
4    AutoModelForCausalLM,
5    TrainingArguments,
6    Trainer
7)
8from peft import (
9    LoraConfig,
10    get_peft_model,
11    TaskType
12)
13
14model_name = "meta-llama/Llama-3.2-1B"
15
16tokenizer = AutoTokenizer.from_pretrained(model_name)
17
18model = AutoModelForCausalLM.from_pretrained(model_name)
19
20lora_config = LoraConfig(
21    task_type=TaskType.CAUSAL_LM,
22    r=16,
23    lora_alpha=32,
24    lora_dropout=0.05,
25    target_modules=[
26        "q_proj",
27        "k_proj",
28        "v_proj",
29        "o_proj"
30    ]
31)
32
33model = get_peft_model(
34    model,
35    lora_config
36)
37
38dataset = load_dataset(
39    "Abirate/english_quotes"
40)
41
42def preprocess(example):
43    tokens = tokenizer(
44        example["quote"],
45        truncation=True,
46        max_length=128
47    )
48    tokens["labels"] = tokens["input_ids"].copy()
49    return tokens
50
51dataset = dataset.map(preprocess)
52
53training_args = TrainingArguments(
54    output_dir="./llama-lora",
55    learning_rate=2e-4,
56    num_train_epochs=3,
57    per_device_train_batch_size=4,
58    save_strategy="epoch",
59    logging_steps=20
60)
61
62trainer = Trainer(
63    model=model,
64    args=training_args,
65    train_dataset=dataset["train"]
66)
67
68trainer.train()
69
70model.save_pretrained("./llama-lora")

What You'll Learn

• Load a pretrained LLaMA model.
• Configure LoRA with PEFT.
• Fine-tune only the adapter parameters.
• Save reusable LoRA adapter weights.

Choosing the Right Fine-Tuning Method

Best Practices

• Start with a pretrained model close to your target domain.
• Use PEFT methods for models with billions of parameters.
• Monitor training and validation metrics to avoid overfitting.
• Save checkpoints regularly.
• Use mixed precision (FP16/BF16) when supported.
• Apply quantization (e.g., QLoRA) for memory-constrained hardware.
• Evaluate on held-out datasets before deployment.

Module Summary

After completing this module, you will be able to:

• Explain the difference between full fine-tuning and parameter-efficient fine-tuning.
• Understand how transfer learning enables efficient adaptation of pretrained Transformers.
• Apply PEFT techniques such as LoRA, QLoRA, Adapters, Prefix Tuning, Prompt Tuning, BitFit, and IA³.
• Choose the appropriate fine-tuning strategy based on model size, hardware, and task requirements.
• Fine-tune BERT for downstream NLP tasks using the Hugging Face Trainer.
• Fine-tune LLaMA-style models using LoRA and the PEFT library.
• Build efficient, domain-specific Transformer models while minimizing compute and memory usage.

Next Module: Module 21 – Transformer Deployment & Optimization, where you'll learn model quantization, pruning, ONNX, TorchScript, TensorRT, vLLM, Text Generation Inference (TGI), serving with FastAPI, and deploying Transformer models in production.