CPU and GPU

PyTorch CPU vs GPU and CUDA: Complete Beginner Tutorial

Topics Covered

• CPU vs GPU for deep learning
• What is a CPU?
• What is a GPU?
• CPU and GPU architecture
• Parallel processing
• Why GPUs accelerate deep learning
• NVIDIA CUDA
• CUDA architecture and execution
• CUDA and PyTorch
• CUDA-enabled PyTorch installation
• Checking CUDA availability
• Detecting NVIDIA GPUs
• Checking CUDA and cuDNN versions
• GPU compute capability
• Verifying GPU acceleration
• Common CUDA and PyTorch GPU errors
• Practical GPU verification exercises

Learning Objectives

After completing this module, you will be able to:

• Explain the difference between CPU and GPU computing.
• Understand why GPUs are widely used for deep learning.
• Explain the role of NVIDIA CUDA in PyTorch.
• Check whether CUDA is available on a system.
• Identify the installed GPU using PyTorch.
• Check the CUDA version used by a PyTorch installation.
• Check GPU compute capability.
• Verify that PyTorch can execute operations on an NVIDIA GPU.
• Diagnose common PyTorch CUDA configuration problems.
• Write basic device-aware PyTorch programs.

Introduction to CPU and GPU Computing

Modern deep learning involves a very large number of mathematical operations.

A neural network may repeatedly perform:

• Matrix multiplication
• Tensor operations
• Convolution
• Attention calculations
• Activation functions
• Gradient computation
• Backpropagation
• Weight updates

For small models and experiments, a CPU can perform these operations effectively. However, training larger neural networks can require billions or even trillions of numerical operations.

GPUs are particularly effective for these workloads because many operations can be executed in parallel.

A simplified deep learning workflow looks like this:

textCopy
1Training Dataset
2       ↓
3Data Loading
4       ↓
5Tensor Operations
6       ↓
7Forward Pass
8       ↓
9Loss Calculation
10       ↓
11Backward Pass
12       ↓
13Gradient Computation
14       ↓
15Parameter Update
16       ↓
17Repeat

Many of these operations can be accelerated using a GPU.

What Is a CPU?

A CPU, or Central Processing Unit, is the general-purpose processor of a computer.

The CPU is responsible for executing operating-system instructions, applications, system logic, and many general-purpose computations.

Typical CPU workloads include:

• Operating systems
• Web browsers
• Compilers
• File management
• Database operations
• Application logic
• Sequential algorithms
• System processes

Modern CPUs contain multiple powerful cores.

A simplified representation is:

textCopy
1CPU
2├── Core 0
3├── Core 1
4├── Core 2
5├── Core 3
6└── ...

Each core is designed to handle relatively complex instructions and control-flow-heavy workloads efficiently.

CPU Strengths

CPUs are particularly useful for:

• General-purpose computing
• Sequential algorithms
• Operating systems
• Branch-heavy programs
• Application logic
• Data preprocessing
• Small machine learning experiments

A CPU is not necessarily "slow." It is optimized for a different type of workload than a GPU.

What Is a GPU?

GPU stands for Graphics Processing Unit.

Modern GPUs are highly parallel processors that can execute large numbers of similar numerical operations simultaneously.

A simplified representation is:

textCopy
1GPU
2├── Processing Unit
3├── Processing Unit
4├── Processing Unit
5├── Processing Unit
6├── Processing Unit
7└── ...

A GPU is particularly effective when the same operation needs to be performed across many pieces of data.

This property makes GPUs extremely useful for:

• Deep learning
• Matrix multiplication
• Tensor operations
• Computer vision
• Scientific computing
• Large-scale numerical workloads
• Transformer models

CPU vs GPU

The exact performance difference depends on the CPU, GPU, model, tensor size, memory transfer overhead, and software implementation.

Why GPUs Are Used for Deep Learning

Neural networks contain large numbers of numerical operations.

For example, a matrix multiplication can be represented as:

textCopy
1A × B = C

If the matrices are large, millions of individual multiplication and addition operations may be required.

A GPU can process many independent operations simultaneously.

This is called parallel processing.

Sequential Processing vs Parallel Processing

A simplified CPU workload can be visualized as:

textCopy
1Task 1
2  ↓
3Task 2
4  ↓
5Task 3
6  ↓
7Task 4

A highly parallel workload can be visualized as:

textCopy
1Task 1 ─┐
2Task 2 ─┤
3Task 3 ─┤──→ Execute in Parallel
4Task 4 ─┘

This is one of the fundamental reasons GPUs are so effective for deep learning.

Example: Matrix Multiplication

Consider two large matrices:

textCopy
1A = 10000 × 10000
2
3B = 10000 × 10000

The resulting matrix contains:

textCopy
110000 × 10000

elements.

Computing these values involves a huge number of arithmetic operations.

A GPU can distribute many of these operations across its parallel processing hardware.

Actual performance depends on the hardware and implementation, so it is better to avoid assuming a fixed CPU-versus-GPU execution time.

What Is CUDA?

CUDA stands for Compute Unified Device Architecture.

CUDA is NVIDIA's platform and programming model for general-purpose GPU computing.

It allows software applications to execute computational workloads on compatible NVIDIA GPUs.

The simplified software stack is:

textCopy
1PyTorch
2   ↓
3CUDA-enabled libraries
4   ↓
5CUDA Runtime
6   ↓
7NVIDIA Driver
8   ↓
9NVIDIA GPU

PyTorch uses this ecosystem to execute supported tensor operations on NVIDIA GPUs.

CUDA and PyTorch

PyTorch provides CUDA support through the torch.cuda module.

For example:

pythonCopy
1import torch
2
3print(torch.cuda.is_available())

If CUDA is correctly available, the result may be:

textCopy
1True

Otherwise:

textCopy
1False

However, CUDA availability is not determined only by whether an NVIDIA GPU physically exists.

The installed PyTorch build, NVIDIA driver, GPU compatibility, and runtime configuration also matter.

CUDA Architecture

A simplified GPU execution model contains:

textCopy
1CPU
2 ↓
3CUDA Runtime
4 ↓
5GPU
6 ├── Streaming Multiprocessors
7 │   ├── CUDA Cores
8 │   ├── Registers
9 │   └── Shared Memory
10 │
11 └── Global GPU Memory

Modern NVIDIA GPU architectures are more complex than this simplified diagram, but this model is useful for understanding the basic concept.

The GPU contains many execution resources that allow thousands of threads to participate in parallel computation.

CUDA Cores

CUDA cores are execution units inside NVIDIA GPU hardware.

A GPU can contain many CUDA cores distributed across multiple Streaming Multiprocessors.

However, comparing GPUs simply by CUDA-core count is not sufficient to determine performance.

Other factors include:

• GPU architecture
• Clock frequency
• Memory bandwidth
• Cache
• Tensor Cores
• Precision
• Software libraries
• Power limits
• Workload characteristics

Tensor Cores

Modern NVIDIA GPUs may also contain Tensor Cores.

Tensor Cores are specialized hardware designed to accelerate certain matrix operations used heavily in artificial intelligence and deep learning.

They are especially useful for workloads involving supported lower-precision formats such as:

• FP16
• BF16
• TF32
• FP8 on supported hardware

For example, deep learning workloads can use optimized matrix operations through libraries such as cuBLAS and cuDNN.

CUDA Libraries Used by Deep Learning

The CUDA ecosystem includes optimized libraries for different workloads.

Important examples include:

• cuBLAS — optimized linear algebra operations
• cuDNN — deep neural network operations
• NCCL — multi-GPU and distributed communication
• CUDA Runtime — GPU execution and management

PyTorch uses these components where appropriate to accelerate operations.

Installing CUDA-Enabled PyTorch

The recommended installation should be selected according to the PyTorch version and supported CUDA build.

A generic CPU installation might look like:

bashCopy
1pip install torch torchvision torchaudio

For an NVIDIA CUDA-enabled PyTorch build, an installation command may look like:

bashCopy
1pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

The exact command should be generated from the PyTorch installation selector for your operating system, Python version, and desired CUDA build.

Do not assume that the CUDA toolkit version installed separately on the system must exactly match the CUDA version shown by:

pythonCopy
1torch.version.cuda

PyTorch wheels can ship with their own CUDA runtime components.

Verify the PyTorch Installation

After installing PyTorch, check the version:

pythonCopy
1import torch
2
3print(torch.__version__)

Example:

textCopy
12.x.x

The exact version depends on the installed PyTorch release.

Check CUDA Availability

Use:

pythonCopy
1import torch
2
3available = torch.cuda.is_available()
4
5print("CUDA Available:", available)

Example:

textCopy
1CUDA Available: True

If the result is:

textCopy
1False

PyTorch currently cannot use CUDA.

Check the CUDA Version Used by PyTorch

Use:

pythonCopy
1import torch
2
3print(torch.version.cuda)

Example:

textCopy
112.8

This reports the CUDA version associated with the installed PyTorch build.

It is different from checking the system toolkit using a command such as:

bashCopy
1nvcc --version

These values do not necessarily have to be identical.

Check the Number of GPUs

pythonCopy
1import torch
2
3print(torch.cuda.device_count())

Example:

textCopy
11

If the system has multiple supported GPUs, the result could be:

textCopy
12

or another number.

Get the Current GPU Device

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4    print(torch.cuda.current_device())

Example:

textCopy
10

GPU indexing starts at zero.

For two GPUs:

textCopy
1GPU 0
2GPU 1

Get the GPU Name

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4    print(torch.cuda.get_device_name(0))

Example:

textCopy
1NVIDIA GeForce RTX 4060 Laptop GPU

The exact name depends on your hardware.

Get GPU Compute Capability

PyTorch can report the compute capability:

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4    capability = torch.cuda.get_device_capability(0)
5    print(capability)

Example:

textCopy
1(8, 9)

The values represent the major and minor compute capability versions.

Compute capability is useful when determining hardware support for particular CUDA features and kernels.

Get Detailed GPU Properties

PyTorch provides:

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4
5    properties = torch.cuda.get_device_properties(0)
6
7    print("Name:", properties.name)
8    print("Total Memory:", properties.total_memory)
9    print("Multiprocessors:", properties.multi_processor_count)
10    print("Compute Capability:",
11          properties.major,
12          properties.minor)

This provides additional information about the GPU.

Convert GPU Memory to GB

GPU memory is reported in bytes.

You can convert it to gigabytes:

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4
5    memory = torch.cuda.get_device_properties(0).total_memory
6
7    memory_gb = memory / (1024 ** 3)
8
9    print(f"GPU Memory: {memory_gb:.2f} GB")

Example:

textCopy
1GPU Memory: 8.00 GB

The actual reported amount may differ slightly from the marketed capacity.

Check cuDNN

cuDNN is NVIDIA's library for accelerating deep neural network operations.

PyTorch exposes cuDNN information through:

pythonCopy
1import torch
2
3print("cuDNN Enabled:",
4      torch.backends.cudnn.enabled)

Example:

textCopy
1cuDNN Enabled: True

Check cuDNN Version

pythonCopy
1import torch
2
3print(torch.backends.cudnn.version())

Example:

textCopy
191002

The exact version depends on the PyTorch build and environment.

Complete GPU Diagnostic Program

The following program provides a useful first diagnostic when setting up PyTorch for GPU computing:

pythonCopy
1import torch
2
3print("=" * 50)
4print("PyTorch GPU Diagnostic")
5print("=" * 50)
6
7print("PyTorch Version:", torch.__version__)
8print("CUDA Available:", torch.cuda.is_available())
9print("PyTorch CUDA Version:", torch.version.cuda)
10print("cuDNN Enabled:", torch.backends.cudnn.enabled)
11print("cuDNN Version:", torch.backends.cudnn.version())
12
13if torch.cuda.is_available():
14
15    count = torch.cuda.device_count()
16
17    print("GPU Count:", count)
18
19    for i in range(count):
20
21        properties = torch.cuda.get_device_properties(i)
22
23        print(f"\nGPU {i}")
24        print("Name:", properties.name)
25        print("Memory:",
26              f"{properties.total_memory / (1024 ** 3):.2f} GB")
27        print("Compute Capability:",
28              f"{properties.major}.{properties.minor}")
29        print("Multiprocessors:",
30              properties.multi_processor_count)
31
32else:
33
34    print("\nCUDA is not available.")
35    print("PyTorch is currently running in CPU mode.")

This is more useful than checking only:

pythonCopy
1torch.cuda.is_available()

because it provides additional diagnostic information.

Verify GPU Tensor Creation

Checking CUDA availability is useful, but it is also important to verify that PyTorch can actually create tensors on the GPU.

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4
5    x = torch.tensor(
6        [1, 2, 3],
7        device="cuda"
8    )
9
10    print(x)
11    print("Device:", x.device)

Example:

textCopy
1tensor([1, 2, 3], device='cuda:0')
2Device: cuda:0

This confirms that the tensor resides on the GPU.

CPU Tensor vs GPU Tensor

By default:

pythonCopy
1import torch
2
3x = torch.tensor([1, 2, 3])
4
5print(x.device)

Output:

textCopy
1cpu

To create it directly on the GPU:

pythonCopy
1if torch.cuda.is_available():
2
3    x = torch.tensor(
4        [1, 2, 3],
5        device="cuda"
6    )
7
8    print(x.device)

Output:

textCopy
1cuda:0

Moving a Tensor to the GPU

You can move a tensor using .to():

pythonCopy
1import torch
2
3x = torch.tensor([1, 2, 3])
4
5if torch.cuda.is_available():
6
7    x = x.to("cuda")
8
9    print(x.device)

This concept will be explored in detail in Part 2.

Simple GPU Computation

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4
5    device = torch.device("cuda")
6
7    a = torch.randn(2000, 2000, device=device)
8    b = torch.randn(2000, 2000, device=device)
9
10    c = a @ b
11
12    print("Device:", c.device)
13    print("Shape:", c.shape)

Example:

textCopy
1Device: cuda:0
2Shape: torch.Size([2000, 2000])

The matrix multiplication is performed on the GPU.

CPU and GPU Tensors Cannot Be Arbitrarily Mixed

Consider:

pythonCopy
1import torch
2
3cpu_tensor = torch.tensor([1, 2, 3])
4
5if torch.cuda.is_available():
6
7    gpu_tensor = torch.tensor(
8        [4, 5, 6],
9        device="cuda"
10    )
11
12    result = cpu_tensor + gpu_tensor

This causes a device mismatch because the tensors are on different devices.

The tensors involved in an operation generally need to be placed on compatible devices.

Correct:

pythonCopy
1if torch.cuda.is_available():
2
3    cpu_tensor = torch.tensor([1, 2, 3])
4
5    gpu_tensor = torch.tensor(
6        [4, 5, 6],
7        device="cuda"
8    )
9
10    cpu_tensor = cpu_tensor.to("cuda")
11
12    result = cpu_tensor + gpu_tensor
13
14    print(result)

Understanding CUDA Device Names

PyTorch uses device strings such as:

textCopy
1cpu
2cuda
3cuda:0
4cuda:1

For example:

pythonCopy
1device = torch.device("cuda:0")

means the first CUDA device.

For a second GPU:

pythonCopy
1device = torch.device("cuda:1")

Part 2 will cover device management in greater detail.

Practical Example: GPU Verification Script

A small verification script can be written as:

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4
5    print("GPU Ready")
6
7    device = torch.device("cuda")
8
9    x = torch.randn(1000, 1000, device=device)
10    y = torch.randn(1000, 1000, device=device)
11
12    z = x @ y
13
14    print("Device:", z.device)
15    print("Result Shape:", z.shape)
16
17else:
18
19    print("CPU Mode")

Expected GPU output:

textCopy
1GPU Ready
2Device: cuda:0
3Result Shape: torch.Size([1000, 1000])

Listing All Available GPUs

Use:

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4
5    count = torch.cuda.device_count()
6
7    for i in range(count):
8
9        print(
10            f"GPU {i}:",
11            torch.cuda.get_device_name(i)
12        )
13
14else:
15
16    print("No CUDA GPU available.")

Example:

textCopy
1GPU 0: NVIDIA GeForce RTX 4060 Laptop GPU

Common CUDA and PyTorch GPU Problems

CUDA Availability Returns False

Check:

pythonCopy
1import torch
2
3print(torch.cuda.is_available())
4print(torch.version.cuda)

Then check the NVIDIA driver from the operating system.

On Linux:

bashCopy
1nvidia-smi

If nvidia-smi cannot communicate with the NVIDIA driver, PyTorch CUDA support will not work correctly.

CPU-Only PyTorch Is Installed

A CPU-only PyTorch installation cannot execute CUDA tensor operations.

Check:

pythonCopy
1import torch
2
3print(torch.version.cuda)

If it returns:

textCopy
1None

the installed PyTorch build may be CPU-only.

NVIDIA Driver Problem

Check:

bashCopy
1nvidia-smi

This should display information about the NVIDIA GPU and driver.

Typical information includes:

textCopy
1Driver Version
2CUDA Version
3GPU Name
4GPU Memory
5GPU Utilization

The CUDA version displayed by nvidia-smi should not be interpreted as necessarily identical to the CUDA runtime bundled with your PyTorch installation.

CUDA Toolkit Confusion

A common misconception is:

textCopy
1System CUDA version
2=
3PyTorch CUDA version

This is not always true.

For example:

pythonCopy
1torch.version.cuda

reports the CUDA version associated with the PyTorch build.

Meanwhile:

bashCopy
1nvcc --version

reports the installed CUDA compiler toolkit version, if nvcc is installed.

These serve different purposes.

CUDA Out-of-Memory Error

A common error during deep learning is:

textCopy
1CUDA out of memory

This occurs when the GPU does not have enough available memory for an operation.

Possible solutions include:

• Reduce batch size.
• Reduce input resolution.
• Use mixed precision when appropriate.
• Reduce model size.
• Delete unnecessary GPU tensors.
• Use gradient accumulation.
• Check for tensors accidentally retained in memory.
• Monitor GPU memory usage.

For example:

pythonCopy
1print(torch.cuda.memory_allocated() / (1024 ** 2))

This displays allocated GPU memory in MiB.

Check GPU Memory Usage

PyTorch provides several memory utilities:

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4
5    allocated = torch.cuda.memory_allocated()
6    reserved = torch.cuda.memory_reserved()
7
8    print(
9        "Allocated:",
10        allocated / (1024 ** 2),
11        "MiB"
12    )
13
14    print(
15        "Reserved:",
16        reserved / (1024 ** 2),
17        "MiB"
18    )

Understanding allocated versus reserved memory becomes important when optimizing deep learning workloads.

Clear Unused CUDA Memory

PyTorch provides:

pythonCopy
1torch.cuda.empty_cache()

Example:

pythonCopy
1import torch
2
3if torch.cuda.is_available():
4
5    torch.cuda.empty_cache()

This releases unused cached memory held by PyTorch's caching allocator.

It does not magically increase the physical VRAM capacity of the GPU and does not free memory that is still actively referenced by tensors.

CPU and GPU Data Transfer

Moving data between CPU and GPU has a cost.

For example:

pythonCopy
1x = torch.randn(1000, 1000)
2
3x_gpu = x.to("cuda")

The tensor is transferred from CPU memory to GPU memory.

Moving it back:

pythonCopy
1x_cpu = x_gpu.to("cpu")

For high-performance training, unnecessary CPU-GPU transfers should be minimized.

Why GPU Memory Matters

A GPU may have:

textCopy
16 GB VRAM
28 GB VRAM
312 GB VRAM
416 GB VRAM
524 GB VRAM
648 GB VRAM

The amount of available VRAM affects the size of:

• Models
• Batch sizes
• Input tensors
• Activations
• Gradients
• Optimizer states

For example, training a large Transformer model can require substantially more memory than running a small neural network.

Deep Learning GPU Workflow

A simplified training workflow looks like:

textCopy
1Dataset
2   ↓
3CPU RAM
4   ↓
5DataLoader
6   ↓
7GPU VRAM
8   ↓
9Model
10   ↓
11Forward Pass
12   ↓
13Loss
14   ↓
15Backward Pass
16   ↓
17Gradients
18   ↓
19Optimizer
20   ↓
21Updated Model

Efficient data loading and minimizing unnecessary CPU-GPU transfers can have a significant effect on training performance.

Real-World Applications of PyTorch GPU Acceleration

GPU acceleration is commonly used for:

• CNN training
• Image classification
• Object detection
• Semantic segmentation
• Natural language processing
• Transformer training
• Large language models
• Generative AI
• Diffusion models
• Recommendation systems
• Reinforcement learning
• Scientific machine learning
• Computer vision

Common Mistakes

Incorrectly Assuming Every PyTorch Tensor Uses the GPU

This:

pythonCopy
1x = torch.randn(10)

creates a CPU tensor by default.

Check:

pythonCopy
1print(x.device)

Output:

textCopy
1cpu

Forgetting to Move the Model to the GPU

If the model is on CPU:

pythonCopy
1model = model.to("cpu")

and the input is on CUDA:

pythonCopy
1x = x.to("cuda")

the operation will fail because the model and input are on different devices.

The model and input should normally be placed on compatible devices.

Mixing CPU and CUDA Tensors

Incorrect:

pythonCopy
1a = torch.randn(10)
2
3b = torch.randn(
4    10,
5    device="cuda"
6)
7
8c = a + b

Correct:

pythonCopy
1a = torch.randn(10).to("cuda")
2
3b = torch.randn(
4    10,
5    device="cuda"
6)
7
8c = a + b

Assuming CUDA Means NVIDIA Driver Version

CUDA is an NVIDIA GPU computing platform.

The NVIDIA driver, CUDA runtime, CUDA toolkit, and PyTorch CUDA build are related but are not identical concepts.

Assuming AMD GPUs Use CUDA

CUDA is NVIDIA-specific.

AMD GPU acceleration for PyTorch can use supported ROCm configurations instead of CUDA.

Best Practices for PyTorch GPU Computing

• Check torch.cuda.is_available() before using CUDA.
• Check the installed PyTorch CUDA version.
• Verify the NVIDIA driver with nvidia-smi.
• Inspect GPU memory before large training jobs.
• Keep tensors and models on compatible devices.
• Avoid unnecessary CPU-GPU data transfers.
• Choose the appropriate PyTorch build for your environment.
• Reduce batch size when encountering GPU memory errors.
• Use mixed precision when appropriate for the workload and hardware.
• Monitor GPU utilization and memory during training.
• Do not assume that the system CUDA toolkit version must equal torch.version.cuda.

Practice Exercises

Exercise 1: GPU Detection

Write a Python program that prints:

• PyTorch version
• CUDA availability
• PyTorch CUDA version
• Number of GPUs
• GPU name
• GPU compute capability

Exercise 2: GPU Ready Check

Write a program that prints:

textCopy
1GPU Ready

when CUDA is available.

Otherwise print:

textCopy
1CPU Mode

Exercise 3: List All GPUs

Write a program that displays the name of every available GPU.

Hint:

pythonCopy
1torch.cuda.device_count()

Exercise 4: Create a GPU Tensor

Create a random tensor of shape:

textCopy
11000 × 1000

on the GPU.

Then print:

• Tensor shape
• Tensor device
• Tensor data type

Exercise 5: GPU Matrix Multiplication

Create two matrices on the GPU:

textCopy
12000 × 2000

Multiply them using:

pythonCopy
1@ 

Then print the result shape and device.

Exercise 6: GPU Memory

Create a large tensor on the GPU and inspect:

pythonCopy
1torch.cuda.memory_allocated()

and:

pythonCopy
1torch.cuda.memory_reserved()

Exercise 7: Device Comparison

Create one tensor on the CPU and another on the GPU.

Attempt to add them together.

Observe the device mismatch error.

Then move both tensors to the same device and perform the operation successfully.

Interview Questions

Q1. What is the difference between CPU and GPU?

A CPU contains a smaller number of powerful cores designed for general-purpose and sequential workloads, while a GPU contains many parallel execution resources optimized for highly parallel numerical workloads.

Q2. Why are GPUs useful for deep learning?

Deep learning relies heavily on operations such as matrix multiplication, convolution, and tensor transformations. These operations can often be parallelized efficiently on GPUs.

Q3. What is CUDA?

CUDA is NVIDIA's parallel computing platform and programming model that enables software applications to execute general-purpose computations on compatible NVIDIA GPUs.

Q4. How do you check CUDA availability in PyTorch?

Use:

pythonCopy
1torch.cuda.is_available()

Q5. How do you find the GPU name?

Use:

pythonCopy
1torch.cuda.get_device_name(0)

Q6. How do you find the number of GPUs?

Use:

pythonCopy
1torch.cuda.device_count()

Q7. What does torch.version.cuda return?

It reports the CUDA version associated with the installed PyTorch build.

Q8. What is cuDNN?

cuDNN is NVIDIA's GPU-accelerated library for deep neural network operations. PyTorch can use cuDNN for supported operations such as convolutions.

Q9. Why can CUDA be unavailable even when an NVIDIA GPU exists?

Possible causes include:

• Incompatible or missing NVIDIA drivers
• CPU-only PyTorch installation
• Unsupported GPU or software combination
• Incorrect environment configuration
• Driver/runtime compatibility problems

Q10. What is CUDA out-of-memory?

A CUDA out-of-memory error occurs when an operation requires more GPU memory than is currently available.

Common solutions include reducing batch size, reducing model/input size, using appropriate precision, and improving memory management.

Module Summary

In this module, you learned the fundamentals of CPU and GPU computing in PyTorch.

You learned:

• What a CPU is and where it is commonly used.
• What a GPU is and why it excels at parallel computation.
• The differences between CPU and GPU architecture.
• Why GPUs are important for deep learning.
• What NVIDIA CUDA is.
• How CUDA connects PyTorch with NVIDIA GPU hardware.
• The role of CUDA libraries such as cuDNN and cuBLAS.
• How to install a CUDA-enabled PyTorch build.
• How to check CUDA availability.
• How to identify available GPUs.
• How to retrieve GPU memory and compute capability.
• How to check the CUDA and cuDNN versions.
• How to create tensors directly on a GPU.
• How to verify actual GPU tensor computation.
• How to identify common CUDA configuration and memory problems.
• Why unnecessary CPU-GPU transfers should be avoided.

These concepts provide the foundation for GPU-accelerated deep learning with PyTorch.

Next Module: Module 5 — CPU and GPU Device Management

In Part 2, you will learn how to manage CPU and GPU devices in real PyTorch applications.

You will learn:

• torch.device
• CPU and CUDA devices
• tensor.to()
• tensor.cuda()
• model.to()
• Moving models between CPU and GPU
• Device-aware programming
• Device-independent PyTorch code
• Checking tensor devices
• Checking model devices
• GPU memory management
• CPU-GPU data transfer
• Multiple GPU basics
• cuda:0, cuda:1, and additional devices
• Practical device management projects
• GPU debugging techniques

Understanding device management is essential before building GPU-accelerated neural networks, CNNs, Transformer models, and large-scale PyTorch training pipelines.