Module 11: Dataset & DataLoader

What You Will Learn

In this module, you will learn:

• What is a Dataset?
• What is a DataLoader?
• TensorDataset
• Custom Dataset
• Batch Size
• Shuffle
• Sampler
• num_workers
• pin_memory
• collate_fn
• Building a Custom Image Dataset

What is a Dataset?

A Dataset is a collection of training data.

It stores:

• Features (Inputs)
• Labels (Targets)

Every Deep Learning project starts with a dataset.

Example:

textCopy
1Image            Label
2Dog.jpg            0
3Cat.jpg            1
4Bird.jpg           2

or

textCopy
1Age    Salary
2
320      20000
425      35000
530      50000

Why Do We Need Dataset?

Without Dataset

textCopy
1Images
2
3↓
4
5Neural Network

Impossible to organize data.

With Dataset

textCopy
1Dataset
2
3↓
4
5DataLoader
6
7↓
8
9Mini Batches
10
11↓
12
13Neural Network

Everything becomes organized and efficient.

Dataset in PyTorch

PyTorch provides

pythonCopy
1torch.utils.data.Dataset

It is a base class.

Every custom dataset inherits from it.

pythonCopy
1from torch.utils.data import Dataset

Dataset Structure

Every Dataset must implement three methods.

pythonCopy
1class MyDataset(Dataset):
2
3    def __len__(self):
4        ...
5
6    def __getitem__(self, index):
7        ...

len()

Returns total number of samples.

Example

pythonCopy
1def __len__(self):
2    return len(self.data)

getitem()

Returns one sample.

Example

pythonCopy
1def __getitem__(self, index):
2
3    image = self.images[index]
4
5    label = self.labels[index]
6
7    return image, label

Example Dataset

pythonCopy
1from torch.utils.data import Dataset
2
3class NumberDataset(Dataset):
4
5    def __init__(self):
6
7        self.data = [1,2,3,4,5]
8
9    def __len__(self):
10
11        return len(self.data)
12
13    def __getitem__(self,index):
14
15        return self.data[index]

Usage

pythonCopy
1dataset = NumberDataset()
2
3print(len(dataset))
4
5print(dataset[2])

Output

textCopy
15
2
33

TensorDataset

PyTorch provides a ready-made dataset called

pythonCopy
1TensorDataset

Useful when your data is already stored in tensors.

Example

pythonCopy
1import torch
2from torch.utils.data import TensorDataset
3
4X = torch.randn(100,10)
5
6y = torch.randint(0,2,(100,))
7
8dataset = TensorDataset(X,y)
9
10print(len(dataset))

Output

textCopy
1100

Access Sample

pythonCopy
1sample = dataset[0]
2
3print(sample)

Output

textCopy
1(tensor(...), tensor(...))

What is DataLoader?

Dataset stores data.

DataLoader loads data efficiently.

Responsibilities

• Batching
• Shuffling
• Parallel Loading
• Memory Optimization

Pipeline

textCopy
1Dataset
2
3↓
4
5DataLoader
6
7↓
8
9Mini Batches
10
11↓
12
13Model

Creating DataLoader

pythonCopy
1from torch.utils.data import DataLoader
2
3loader = DataLoader(
4
5    dataset,
6
7    batch_size=32,
8
9    shuffle=True
10)

Iterate Over DataLoader

pythonCopy
1for batch in loader:
2
3    X,y = batch
4
5    print(X.shape)
6
7    print(y.shape)

Output

textCopy
1torch.Size([32,10])
2
3torch.Size([32])

Batch Size

Batch Size determines

How many samples are processed together.

Example

Dataset

textCopy
1100 Images

Batch Size

textCopy
110

Result

textCopy
110 batches

Example

pythonCopy
1loader = DataLoader(
2
3    dataset,
4
5    batch_size=16
6)

Common Batch Sizes

Shuffle

Shuffle randomizes data order.

Without Shuffle

textCopy
1Cat
2
3Cat
4
5Cat
6
7Dog
8
9Dog
10
11Dog

Model may learn bias.

With Shuffle

textCopy
1Dog
2
3Cat
4
5Dog
6
7Bird
8
9Cat
10
11Dog

Better learning.

Example

pythonCopy
1loader = DataLoader(
2
3    dataset,
4
5    shuffle=True
6)

Sampler

Sampler controls

Which samples are loaded.

Instead of

textCopy
11
2
32
4
53
6
74

It can load

textCopy
15
2
38
4
51
6
710

SequentialSampler

pythonCopy
1from torch.utils.data import SequentialSampler
2
3sampler = SequentialSampler(dataset)
4
5loader = DataLoader(
6
7    dataset,
8
9    sampler=sampler
10)

RandomSampler

pythonCopy
1from torch.utils.data import RandomSampler
2
3sampler = RandomSampler(dataset)
4
5loader = DataLoader(
6
7    dataset,
8
9    sampler=sampler
10)

WeightedRandomSampler

Useful for imbalanced datasets.

pythonCopy
1from torch.utils.data import WeightedRandomSampler
2import torch
3
4weights = torch.tensor([0.1,0.2,0.7])
5
6sampler = WeightedRandomSampler(
7
8    weights,
9
10    num_samples=3
11)

num_workers

DataLoader can load data using multiple CPU processes.

textCopy
1Main Process
2
3↓
4
5Worker 1
6
7Worker 2
8
9Worker 3
10
11Worker 4

Example

pythonCopy
1loader = DataLoader(
2
3    dataset,
4
5    batch_size=32,
6
7    num_workers=4
8)

Typical Values

pin_memory

Transfers data to GPU faster.

Useful only when using CUDA.

Example

pythonCopy
1loader = DataLoader(
2
3    dataset,
4
5    pin_memory=True
6)

GPU Example

pythonCopy
1device = torch.device("cuda")
2
3for X,y in loader:
4
5    X = X.to(device,non_blocking=True)
6
7    y = y.to(device,non_blocking=True)

collate_fn

DataLoader normally stacks samples automatically.

Sometimes data has different sizes.

Example

Sentence 1

textCopy
1I love AI

Sentence 2

textCopy
1PyTorch is awesome for Deep Learning

Lengths differ.

Need custom batching.

Example

pythonCopy
1def custom_collate(batch):
2
3    return batch
4
5loader = DataLoader(
6
7    dataset,
8
9    collate_fn=custom_collate
10)

Complete Custom Dataset

pythonCopy
1from torch.utils.data import Dataset
2import torch
3
4class StudentDataset(Dataset):
5
6    def __init__(self):
7
8        self.features = torch.randn(100,5)
9
10        self.labels = torch.randint(0,2,(100,))
11
12    def __len__(self):
13
14        return len(self.features)
15
16    def __getitem__(self,index):
17
18        return self.features[index],self.labels[index]
19
20
21dataset = StudentDataset()
22
23loader = DataLoader(
24
25    dataset,
26
27    batch_size=16,
28
29    shuffle=True
30)
31
32for X,y in loader:
33
34    print(X.shape,y.shape)
35
36    break

Output

textCopy
1torch.Size([16,5])
2
3torch.Size([16])

Practice Project

Custom Image Dataset

Folder Structure

textCopy
1cats_vs_dogs/
2
3│
4
5├── cats/
6
7│      cat1.jpg
8
9│      cat2.jpg
10
11│
12
13└── dogs/
14
15       dog1.jpg
16
17       dog2.jpg

Dataset Class

pythonCopy
1import os
2from PIL import Image
3from torch.utils.data import Dataset
4from torchvision import transforms
5
6class CatsDogsDataset(Dataset):
7
8    def __init__(self, root_dir, transform=None):
9        self.root_dir = root_dir
10        self.transform = transform
11        self.samples = []
12
13        classes = ["cats", "dogs"]
14
15        for label, class_name in enumerate(classes):
16            class_path = os.path.join(root_dir, class_name)
17
18            for image_name in os.listdir(class_path):
19                image_path = os.path.join(class_path, image_name)
20
21                self.samples.append(
22                    (image_path, label)
23                )
24
25    def __len__(self):
26        return len(self.samples)
27
28    def __getitem__(self, index):
29
30        image_path, label = self.samples[index]
31
32        image = Image.open(image_path).convert("RGB")
33
34        if self.transform:
35            image = self.transform(image)
36
37        return image, label

Transforms

pythonCopy
1transform = transforms.Compose([
2
3    transforms.Resize((224,224)),
4
5    transforms.ToTensor()
6])

Create Dataset

pythonCopy
1dataset = CatsDogsDataset(
2
3    root_dir="cats_vs_dogs",
4
5    transform=transform
6)

Create DataLoader

pythonCopy
1loader = DataLoader(
2
3    dataset,
4
5    batch_size=8,
6
7    shuffle=True,
8
9    num_workers=2,
10
11    pin_memory=True
12)

Training Loop

pythonCopy
1device = torch.device(
2    "cuda" if torch.cuda.is_available() else "cpu"
3)
4
5for images, labels in loader:
6
7    images = images.to(device, non_blocking=True)
8
9    labels = labels.to(device, non_blocking=True)
10
11    print(images.shape)
12
13    print(labels)
14
15    break

Output

textCopy
1torch.Size([8,3,224,224])

Dataset vs DataLoader

DataLoader Parameters

Best Practices

• Create a custom Dataset for your own data format.
• Use TensorDataset when data is already in tensors.
• Use shuffle=True during training but disable it during validation and testing.
• Choose a batch_size that fits your available GPU memory.
• Increase num_workers on Linux or servers to speed up data loading.
• Enable pin_memory=True when training on CUDA GPUs.
• Use a custom collate_fn for variable-length data such as text, audio, or object detection annotations.
• Apply data preprocessing and augmentation using torchvision.transforms inside the dataset.

Module Summary

In this module, you learned:

• ✅ What a Dataset is and how it organizes training data.
• ✅ How to create custom datasets by inheriting from torch.utils.data.Dataset.
• ✅ How to use TensorDataset for tensor-based data.
• ✅ How DataLoader creates mini-batches and efficiently feeds data to a model.
• ✅ The purpose of batch_size, shuffle, sampler, num_workers, pin_memory, and collate_fn.
• ✅ How to build a complete Custom Image Dataset using folders, PIL, and torchvision.transforms.
• ✅ Best practices for creating efficient and scalable PyTorch data pipelines.

In the next module, you'll learn how to preprocess and augment data using TorchVision Transforms, enabling your models to generalize better by applying operations such as resizing, normalization, random flipping, cropping, and color augmentation.