Tensor Indexing & Slicing

Advanced Tensor Indexing, Boolean Masking, where(), gather(), and Tensor Assignment

Topics Covered

• Boolean masking in PyTorch
• Conditional tensor selection
• Advanced and fancy tensor indexing
• torch.where()
• torch.nonzero()
• torch.index_select()
• torch.gather()
• Tensor value assignment
• Masked tensor modification
• Practical indexing for deep learning
• Indexing techniques for NLP, computer vision, and Transformers

Learning Objectives

After completing this module, you will be able to:

• Filter PyTorch tensors using Boolean masks.
• Combine multiple tensor conditions.
• Perform advanced tensor indexing with index tensors.
• Select tensor values conditionally using torch.where().
• Find the locations of matching elements with torch.nonzero().
• Select rows and columns using torch.index_select().
• Understand how torch.gather() works.
• Modify tensor values using indexing and Boolean masks.
• Apply tensor indexing to image, NLP, classification, and Transformer workflows.
• Avoid common indexing and shape-related errors in PyTorch.

Introduction to Advanced Tensor Indexing

In the previous part, you learned how to access tensor elements using positive indexing, negative indexing, slicing, and multidimensional indexing.

In practical deep learning applications, however, we often need more than simply accessing an element.

For example, a machine learning pipeline may need to:

• Select only valid samples from a batch.
• Remove padding tokens from a sequence.
• Select predictions belonging to specific classes.
• Replace invalid values.
• Extract specific rows from a dataset.
• Select particular channels from an image.
• Retrieve values corresponding to target class indices.
• Create attention masks for Transformer models.

PyTorch provides several tensor indexing operations for these tasks.

A useful way to organize them is:

textCopy
1PyTorch Tensor Indexing
2        │
3        ├── Boolean Masking
4        │
5        ├── Advanced Indexing
6        │
7        ├── torch.where()
8        │
9        ├── torch.nonzero()
10        │
11        ├── torch.index_select()
12        │
13        ├── torch.gather()
14        │
15        └── Tensor Assignment

Understanding these operations is important because tensor indexing appears throughout neural networks, data preprocessing, computer vision, NLP, and Transformer architectures.

Boolean Masking in PyTorch

A Boolean mask is a tensor containing True and False values.

The mask has a logical relationship with the tensor being indexed.

For example:

textCopy
1Tensor:
2
310  20  30  40  50
4
5Condition: x > 25
6
7False False True True True

The True positions are selected.

Boolean masking is especially useful for filtering tensor data based on conditions.

Creating a Boolean Mask

pythonCopy
1import torch
2
3x = torch.tensor([10, 20, 30, 40, 50])
4
5mask = x > 25
6
7print(mask)

Output:

textCopy
1tensor([False, False,  True,  True,  True])

The expression:

pythonCopy
1x > 25

does not immediately return the values greater than 25.

Instead, it creates a Boolean tensor describing which elements satisfy the condition.

Selecting Values Using a Boolean Mask

The Boolean mask can then be used for indexing:

pythonCopy
1print(x[mask])

Output:

textCopy
1tensor([30, 40, 50])

The same operation can be written more compactly:

pythonCopy
1print(x[x > 25])

Output:

textCopy
1tensor([30, 40, 50])

This is one of the most commonly used PyTorch tensor filtering techniques.

Filtering Tensor Values

Suppose we have model scores:

pythonCopy
1scores = torch.tensor([0.2, 0.8, 0.4, 0.95, 0.1])
2
3high_scores = scores[scores > 0.7]
4
5print(high_scores)

Output:

textCopy
1tensor([0.8000, 0.9500])

This technique can be used to filter predictions, confidence scores, measurements, or other numerical data.

Common Comparison Operators

Example:

pythonCopy
1x = torch.tensor([5, 10, 15, 20, 25])
2
3print(x[x >= 15])

Output:

textCopy
1tensor([15, 20, 25])

Combining Boolean Conditions

Multiple conditions can be combined using PyTorch's element-wise logical operators.

Use:

• & for AND
• | for OR
• ~ for NOT

Parentheses should be used around individual conditions.

AND Condition

Select values greater than 10 and less than 25:

pythonCopy
1import torch
2
3x = torch.tensor([5, 10, 15, 20, 25, 30])
4
5mask = (x > 10) & (x < 25)
6
7print(x[mask])

Output:

textCopy
1tensor([15, 20])

The expression:

pythonCopy
1(x > 10) & (x < 25)

means:

textCopy
1greater than 10
2        AND
3less than 25

OR Condition

Select values below 10 or above 20:

pythonCopy
1mask = (x < 10) | (x > 20)
2
3print(x[mask])

Output:

textCopy
1tensor([ 5, 25, 30])

NOT Condition

The ~ operator reverses Boolean values.

pythonCopy
1mask = ~(x > 15)
2
3print(x[mask])

Output:

textCopy
1tensor([ 5, 10, 15])

The expression:

pythonCopy
1~(x > 15)

means:

textCopy
1NOT greater than 15

Boolean Masking on 2D Tensors

Boolean masking can also be applied to matrices.

pythonCopy
1import torch
2
3A = torch.tensor([
4    [5, 10],
5    [15, 20]
6])
7
8result = A[A > 10]
9
10print(result)

Output:

textCopy
1tensor([15, 20])

An important detail is that Boolean indexing returns the selected elements as a one-dimensional tensor.

It does not preserve the original matrix shape.

For example:

textCopy
1Original:
2
35   10
415  20
5
6Mask:
7
8False False
9True  True
10
11Result:
12
1315 20

The result has shape:

textCopy
1torch.Size([2])

Advanced or Fancy Indexing

Advanced indexing allows you to select elements using another tensor containing indices.

For example:

pythonCopy
1import torch
2
3x = torch.tensor([10, 20, 30, 40, 50])
4
5indices = torch.tensor([0, 2, 4])
6
7result = x[indices]
8
9print(result)

Output:

textCopy
1tensor([10, 30, 50])

The index tensor:

pythonCopy
1[0, 2, 4]

means:

textCopy
1x[0] → 10
2x[2] → 30
3x[4] → 50

Duplicate Indices

Advanced indexing can contain duplicate indices.

pythonCopy
1indices = torch.tensor([1, 1, 3])
2
3print(x[indices])

Output:

textCopy
1tensor([20, 20, 40])

The same element can therefore be selected multiple times.

Advanced Indexing with a Matrix

Consider:

pythonCopy
1A = torch.tensor([
2    [1, 2],
3    [3, 4],
4    [5, 6]
5])

Select rows 0 and 2:

pythonCopy
1rows = torch.tensor([0, 2])
2
3result = A[rows]
4
5print(result)

Output:

textCopy
1tensor([
2    [1, 2],
3    [5, 6]
4])

Selecting Columns with Advanced Indexing

To select columns 0 and 1:

pythonCopy
1columns = torch.tensor([0, 1])
2
3result = A[:, columns]
4
5print(result)

Output:

textCopy
1tensor([
2    [1, 2],
3    [3, 4],
4    [5, 6]
5])

To select only column 1:

pythonCopy
1columns = torch.tensor([1])
2
3result = A[:, columns]
4
5print(result)

Output:

textCopy
1tensor([
2    [2],
3    [4],
4    [6]
5])

Notice that:

pythonCopy
1A[:, 1]

returns a one-dimensional tensor, while:

pythonCopy
1A[:, [1]]

preserves the column dimension.

This distinction is important when preparing tensors for neural network operations.

Conditional Selection with torch.where()

torch.where() is used to select between two values according to a condition.

General syntax:

pythonCopy
1torch.where(condition, input, other)

Conceptually:

textCopy
1condition = True
2        ↓
3     input
4
5condition = False
6        ↓
7     other

Basic torch.where() Example

pythonCopy
1import torch
2
3x = torch.tensor([5, 15, 25])
4
5result = torch.where(x > 10, x, torch.tensor(0))
6
7print(result)

Output:

textCopy
1tensor([ 0, 15, 25])

For each element:

textCopy
15  → condition False → 0
215 → condition True  → 15
325 → condition True  → 25

Replace Negative Values

A common data preprocessing task is replacing negative values with zero.

pythonCopy
1x = torch.tensor([-2, 5, -1, 7])
2
3result = torch.where(x < 0, torch.tensor(0), x)
4
5print(result)

Output:

textCopy
1tensor([0, 5, 0, 7])

Conditional Clipping

Suppose values above 10 should be replaced with 10:

pythonCopy
1x = torch.tensor([5, 10, 15, 20])
2
3result = torch.where(x > 10, torch.tensor(10), x)
4
5print(result)

Output:

textCopy
1tensor([ 5, 10, 10, 10])

Conditional Label Transformation

Suppose values greater than or equal to 50 should be classified as 1, while smaller values become 0:

pythonCopy
1marks = torch.tensor([35, 60, 48, 90, 72])
2
3labels = torch.where(
4    marks >= 50,
5    torch.tensor(1),
6    torch.tensor(0)
7)
8
9print(labels)

Output:

textCopy
1tensor([0, 1, 0, 1, 1])

This demonstrates how tensor operations can be used to create labels or binary conditions.

Finding Positions with torch.nonzero()

torch.nonzero() returns the indices of elements that are non-zero.

Example:

pythonCopy
1import torch
2
3x = torch.tensor([0, 2, 0, 5, 7])
4
5indices = torch.nonzero(x)
6
7print(indices)

Output:

textCopy
1tensor([
2    [1],
3    [3],
4    [4]
5])

The result means:

textCopy
1Index 1 → value 2
2Index 3 → value 5
3Index 4 → value 7

Finding Indices That Match a Condition

torch.nonzero() can also be used with a Boolean expression.

pythonCopy
1x = torch.tensor([4, 8, 15, 16])
2
3indices = torch.nonzero(x > 10)
4
5print(indices)

Output:

textCopy
1tensor([
2    [2],
3    [3]
4])

The values satisfying the condition are:

textCopy
115
216

Their indices are:

textCopy
12
23

For a one-dimensional tensor, you can also use:

pythonCopy
1indices = torch.nonzero(x > 10, as_tuple=True)[0]
2
3print(indices)

Output:

textCopy
1tensor([2, 3])

Using as_tuple=True can be convenient when the indices will be used directly for indexing.

torch.index_select()

torch.index_select() selects elements along a specified dimension using an index tensor.

Syntax:

pythonCopy
1torch.index_select(input, dim, index)

The important parameters are:

• input: source tensor
• dim: dimension along which selection occurs
• index: tensor containing indices

Selecting Rows with index_select()

pythonCopy
1import torch
2
3A = torch.tensor([
4    [10, 20],
5    [30, 40],
6    [50, 60]
7])
8
9rows = torch.tensor([0, 2])
10
11result = torch.index_select(A, dim=0, index=rows)
12
13print(result)

Output:

textCopy
1tensor([
2    [10, 20],
3    [50, 60]
4])

Here:

pythonCopy
1dim=0

means that rows are being selected.

Selecting Columns with index_select()

pythonCopy
1columns = torch.tensor([1])
2
3result = torch.index_select(A, dim=1, index=columns)
4
5print(result)

Output:

textCopy
1tensor([
2    [20],
3    [40],
4    [60]
5])

Here:

pythonCopy
1dim=1

means that columns are being selected.

index_select() vs Advanced Indexing

These two approaches can often accomplish similar tasks.

Advanced indexing:

pythonCopy
1A[rows]

Explicit index selection:

pythonCopy
1torch.index_select(A, 0, rows)

index_select() is particularly useful when you want to explicitly specify the dimension being indexed.

Understanding torch.gather()

torch.gather() is one of the most important advanced tensor indexing operations in PyTorch.

It selects values from a tensor using an index tensor along a specified dimension.

Syntax:

pythonCopy
1torch.gather(input, dim, index)

Unlike selecting complete rows or columns, gather() allows you to select different positions for different rows or batches.

This makes it particularly useful in:

• Classification
• NLP
• Transformer models
• Attention mechanisms
• Reinforcement learning
• Sequence processing
• Selecting action values
• Extracting token-specific values

Basic torch.gather() Example

Consider:

pythonCopy
1import torch
2
3A = torch.tensor([
4    [10, 20],
5    [30, 40]
6])
7
8index = torch.tensor([
9    [0, 1],
10    [1, 0]
11])
12
13result = torch.gather(A, dim=1, index=index)
14
15print(result)

Output:

textCopy
1tensor([
2    [10, 20],
3    [40, 30]
4])

Let's understand the operation.

For the first row:

textCopy
1A[0] = [10, 20]
2
3indices = [0, 1]
4
5→ [10, 20]

For the second row:

textCopy
1A[1] = [30, 40]
2
3indices = [1, 0]
4
5→ [40, 30]

Therefore:

textCopy
1Input:
2
310 20
430 40
5
6Index:
7
80  1
91  0
10
11Result:
12
1310 20
1440 30

gather() with Classification Scores

Suppose a neural network produces class scores:

pythonCopy
1scores = torch.tensor([
2    [0.1, 0.7, 0.2],
3    [0.8, 0.1, 0.1],
4    [0.2, 0.3, 0.5]
5])

There are three samples and three classes.

Suppose the correct class for each sample is:

pythonCopy
1targets = torch.tensor([1, 0, 2])

We can retrieve the score corresponding to each target class.

First, reshape the targets:

pythonCopy
1target_indices = targets.unsqueeze(1)
2
3print(target_indices)

Output:

textCopy
1tensor([
2    [1],
3    [0],
4    [2]
5])

Now gather the corresponding scores:

pythonCopy
1target_scores = torch.gather(
2    scores,
3    dim=1,
4    index=target_indices
5)
6
7print(target_scores)

Output:

textCopy
1tensor([
2    [0.7000],
3    [0.8000],
4    [0.5000]
5])

The operation selected:

textCopy
1Sample 1 → class 1 → 0.7
2Sample 2 → class 0 → 0.8
3Sample 3 → class 2 → 0.5

This pattern is extremely useful for understanding how class-specific values can be extracted from batched model outputs.

Note that CrossEntropyLoss is implemented using optimized internal operations and should not be described as simply being a wrapper around gather(). However, class-index-based selection is an important concept for understanding many classification and loss calculations.

gather() in Transformer Workflows

Transformer models frequently work with tensors containing dimensions such as:

textCopy
1(batch_size, sequence_length, hidden_size)

For example:

textCopy
1(4, 128, 768)

A model may need to select values associated with particular token positions.

gather() can perform this type of batched selection without manually looping over every sample.

This is one reason understanding torch.gather() is valuable for advanced PyTorch and Transformer development.

Modifying Tensor Values

PyTorch tensors can be modified using indexing and assignment.

For example:

pythonCopy
1import torch
2
3x = torch.tensor([10, 20, 30])
4
5x[1] = 99
6
7print(x)

Output:

textCopy
1tensor([10, 99, 30])

The element at index 1 was replaced.

Modifying Multiple Elements

pythonCopy
1x = torch.tensor([10, 20, 30, 40])
2
3x[1:3] = torch.tensor([200, 300])
4
5print(x)

Output:

textCopy
1tensor([10, 200, 300, 40])

The selected slice was replaced.

Modifying an Entire Row

pythonCopy
1A = torch.tensor([
2    [1, 2],
3    [3, 4]
4])
5
6A[0] = torch.tensor([9, 9])
7
8print(A)

Output:

textCopy
1tensor([
2    [9, 9],
3    [3, 4]
4])

Modifying a Column

pythonCopy
1A[:, 1] = 0
2
3print(A)

Output:

textCopy
1tensor([
2    [9, 0],
3    [3, 0]
4])

Modifying Values with Boolean Masks

Boolean masking can be combined with assignment.

pythonCopy
1x = torch.tensor([5, 10, 15, 20])
2
3x[x > 10] = 100
4
5print(x)

Output:

textCopy
1tensor([5, 10, 100, 100])

This is useful for tasks such as:

• Removing invalid values
• Clipping data
• Replacing outliers
• Creating masks
• Preparing training data

torch.where() vs Boolean Assignment

Both techniques can modify values, but they serve slightly different purposes.

Using Boolean assignment:

pythonCopy
1x[x < 0] = 0

This directly modifies the existing tensor.

Using torch.where():

pythonCopy
1x = torch.where(x < 0, torch.tensor(0), x)

This creates a result tensor based on the condition.

A simple rule is:

textCopy
1Need direct modification?
2        ↓
3Boolean assignment
4
5Need conditional result?
6        ↓
7torch.where()

When working with tensors that participate in gradient computation, be careful with in-place modifications because they can interfere with Autograd's ability to track values needed for backward computation.

Practical Example: Filtering Positive Values

pythonCopy
1import torch
2
3x = torch.tensor([-5, -2, 0, 3, 8, 10])
4
5positive_values = x[x > 0]
6
7print(positive_values)

Output:

textCopy
1tensor([3, 8, 10])

Practical Example: Removing Padding Tokens

Natural language processing models often use a special padding token.

For example:

pythonCopy
1PAD_ID = 0
2
3tokens = torch.tensor([
4    12, 45, 78, 0, 0
5])

To remove padding from this one-dimensional sequence:

pythonCopy
1valid_tokens = tokens[tokens != PAD_ID]
2
3print(valid_tokens)

Output:

textCopy
1tensor([12, 45, 78])

In real Transformer pipelines, padding is often retained and represented using an attention mask rather than physically removed. Nevertheless, Boolean indexing is useful when filtering standalone sequences or preprocessing data.

Practical Example: Create an Attention Mask

Suppose 0 represents padding:

pythonCopy
1input_ids = torch.tensor([
2    [12, 45, 78, 0, 0],
3    [34, 56, 0, 0, 0]
4])

Create a mask:

pythonCopy
1attention_mask = (input_ids != 0)
2
3print(attention_mask)

Output:

textCopy
1tensor([
2    [ True,  True,  True, False, False],
3    [ True,  True, False, False, False]
4])

Convert it to integer values if required by a particular API:

pythonCopy
1attention_mask = (input_ids != 0).long()
2
3print(attention_mask)

Output:

textCopy
1tensor([
2    [1, 1, 1, 0, 0],
3    [1, 1, 0, 0, 0]
4])

This demonstrates a common tensor indexing and masking pattern used in NLP and Transformer workflows.

Practical Example: Select Specific Image Channels

An RGB image in channel-first PyTorch format commonly has:

textCopy
1(C, H, W)

For example:

pythonCopy
1image = torch.randn(3, 224, 224)

The three channels are:

textCopy
10 → Red
21 → Green
32 → Blue

Select the red channel:

pythonCopy
1red_channel = image[0]
2
3print(red_channel.shape)

Output:

textCopy
1torch.Size([224, 224])

Select red and blue channels:

pythonCopy
1red_blue = image[[0, 2]]
2
3print(red_blue.shape)

Output:

textCopy
1torch.Size([2, 224, 224])

This type of tensor indexing is useful in computer vision preprocessing.

Practical Example: Select Samples from a Batch

Suppose a batch contains eight samples:

pythonCopy
1batch = torch.randn(8, 3, 64, 64)

Select samples 0, 3, and 6:

pythonCopy
1indices = torch.tensor([0, 3, 6])
2
3selected = batch[indices]
4
5print(selected.shape)

Output:

textCopy
1torch.Size([3, 3, 64, 64])

The batch size changed from:

textCopy
18

to:

textCopy
13

while the image dimensions remained unchanged.

Common Mistakes in PyTorch Tensor Indexing

Using Python and Instead of &

Incorrect:

pythonCopy
1mask = (x > 5) and (x < 20)

For element-wise tensor conditions, use:

pythonCopy
1mask = (x > 5) & (x < 20)

Forgetting Parentheses

Incorrect:

pythonCopy
1mask = x > 5 & x < 20

Correct:

pythonCopy
1mask = (x > 5) & (x < 20)

Parentheses make the individual Boolean expressions explicit and avoid Python operator-precedence problems.

Confusing Boolean Masking with Shape-Preserving Selection

This:

pythonCopy
1A[A > 5]

returns selected elements as a one-dimensional tensor.

It does not preserve the original matrix structure.

If you need to preserve the original shape while replacing values, use an operation such as:

pythonCopy
1torch.where()

For example:

pythonCopy
1A = torch.tensor([
2    [1, 6],
3    [8, 3]
4])
5
6result = torch.where(A > 5, A, torch.tensor(0))
7
8print(result)

Output:

textCopy
1tensor([
2    [0, 6],
3    [8, 0]
4])

Confusing argmax() with gather()

argmax() returns the position of the largest value.

Example:

pythonCopy
1x = torch.tensor([10, 50, 30])
2
3print(torch.argmax(x))

Output:

textCopy
1tensor(1)

gather() retrieves values according to explicit indices.

They solve different problems.

Shape Mismatch with gather()

The index tensor used by gather() must have an appropriate shape relative to the input tensor and selected dimension.

Always inspect:

pythonCopy
1print("Input:", input.shape)
2print("Index:", index.shape)

before debugging a complicated gather() operation.

Modifying a Tensor That Requires Gradients

Be careful with operations such as:

pythonCopy
1x[0] = 10

when x participates in a computation graph.

In-place modifications can cause Autograd errors or overwrite values required for gradient computation.

For model training code, prefer out-of-place operations when appropriate.

Boolean Masking vs Advanced Indexing

Choosing the Right Tensor Indexing Operation

A useful decision process is:

textCopy
1Do I need to filter by a condition?
2        │
3        └── Yes → Boolean Masking
4
5Do I need conditional replacement?
6        │
7        └── Yes → torch.where()
8
9Do I need the positions of matching elements?
10        │
11        └── Yes → torch.nonzero()
12
13Do I need specific rows or columns?
14        │
15        └── Yes → index_select()
16
17Do I need different indices for different rows/batches?
18        │
19        └── Yes → gather()
20
21Do I need arbitrary explicit positions?
22        │
23        └── Yes → Advanced Indexing

Best Practices for PyTorch Indexing

• Always check tensor shapes before performing multidimensional indexing.
• Use &, |, and ~ for element-wise Boolean conditions.
• Put parentheses around individual Boolean expressions.
• Use Boolean masks for readable tensor filtering.
• Use torch.where() for conditional tensor transformations.
• Use torch.nonzero() when the actual indices are required.
• Use index_select() when selecting complete positions along a specific dimension.
• Learn torch.gather() carefully because it is widely useful in batched model operations.
• Be careful with in-place tensor modifications when Autograd is enabled.
• Use explicit dimension arguments when working with multidimensional tensors.
• Test indexing operations with small tensors before applying them to large model inputs.

Practice Exercises

Beginner Exercise: Boolean Masking

Create:

pythonCopy
1import torch
2
3x = torch.tensor([5, 10, 15, 20, 25, 30])

Perform the following operations:

1. Select values greater than 15.
2. Select values less than or equal to 20.
3. Select values between 10 and 25.
4. Replace values greater than 20 with 0.
5. Count how many values are greater than 10.

Hint:

pythonCopy
1torch.count_nonzero(x > 10)

Intermediate Exercise: Advanced Indexing

Create:

pythonCopy
1A = torch.tensor([
2    [11, 12, 13],
3    [21, 22, 23],
4    [31, 32, 33]
5])

Perform:

1. Select rows 0 and 2.
2. Select columns 0 and 2.
3. Select the values 11, 22, and 33.
4. Replace all values greater than 20 with 99.
5. Find the indices of values greater than 25.

Advanced Exercise: Classification Scores

Create:

pythonCopy
1scores = torch.tensor([
2    [0.1, 0.7, 0.2],
3    [0.8, 0.1, 0.1],
4    [0.2, 0.3, 0.5]
5])
6
7targets = torch.tensor([1, 0, 2])

Use torch.gather() to retrieve the score associated with the correct class for each sample.

Expected result:

textCopy
1tensor([
2    [0.7],
3    [0.8],
4    [0.5]
5])

Advanced Exercise: Transformer-Style Masking

Create:

pythonCopy
1input_ids = torch.tensor([
2    [12, 45, 78, 0, 0],
3    [34, 56, 67, 89, 0]
4])

Tasks:

1. Identify padding tokens.
2. Create an attention mask.
3. Convert the Boolean mask into 0 and 1.
4. Count valid tokens in each sequence.
5. Explain why padding is normally masked rather than simply deleted when processing a batch.

Mini Project: Tensor Filtering Toolkit

Build a small utility that demonstrates multiple PyTorch indexing operations.

pythonCopy
1import torch
2
3
4def tensor_filtering_toolkit(x):
5    print("Original Tensor:")
6    print(x)
7
8    print("\nValues greater than 10:")
9    print(x[x > 10])
10
11    print("\nValues less than or equal to 20:")
12    print(x[x <= 20])
13
14    print("\nIndices of values greater than 10:")
15    print(torch.nonzero(x > 10, as_tuple=True)[0])
16
17    print("\nValues replaced when greater than 20:")
18    print(torch.where(x > 20, torch.tensor(0), x))
19
20
21x = torch.tensor([5, 10, 15, 20, 25, 30])
22
23tensor_filtering_toolkit(x)

This small project combines:

textCopy
1Boolean Masking
2       ↓
3Conditional Selection
4       ↓
5Nonzero Index Extraction
6       ↓
7Conditional Replacement

It is a useful exercise for understanding how multiple PyTorch tensor indexing operations work together.

Module Summary

In this module, you learned how to perform advanced tensor indexing and slicing operations in PyTorch.

You learned:

• How Boolean masks filter tensor values.
• How to combine multiple conditions using &, |, and ~.
• How advanced indexing selects arbitrary tensor positions.
• How torch.where() performs conditional selection and replacement.
• How torch.nonzero() finds indices satisfying a condition.
• How torch.index_select() selects rows and columns along a specified dimension.
• How torch.gather() performs index-based selection across tensor dimensions.
• How tensor assignment modifies individual elements, rows, columns, and masked values.
• How tensor indexing is applied to image batches, NLP sequences, classification outputs, and Transformer workflows.
• Why tensor shapes and dimensions are critical when using advanced indexing.

These techniques form an important part of PyTorch tensor manipulation and are used throughout data preprocessing, neural network training, computer vision, natural language processing, and Transformer architectures.

SEO Keywords

Primary keywords:

• PyTorch tensor indexing
• PyTorch tensor slicing
• PyTorch Boolean masking
• PyTorch advanced indexing
• PyTorch tensor operations
• PyTorch tensor manipulation
• PyTorch torch.where
• PyTorch torch.gather
• PyTorch torch.index_select
• PyTorch torch.nonzero

Related keywords:

• PyTorch indexing tutorial
• PyTorch Boolean mask example
• PyTorch advanced indexing tutorial
• PyTorch tensor filtering
• PyTorch conditional selection
• PyTorch tensor assignment
• PyTorch tensor masking
• PyTorch gather example
• PyTorch index_select example
• PyTorch nonzero example
• PyTorch tensor indexing examples
• PyTorch tensor manipulation tutorial
• PyTorch deep learning tutorial
• PyTorch tensor operations tutorial
• PyTorch NLP tensor indexing
• PyTorch Transformer tensor operations
• PyTorch attention mask
• PyTorch classification tensor indexing
• PyTorch computer vision tensors
• PyTorch batch tensor indexing

Next Module

In Part 3, you will move from basic and advanced indexing into more specialized tensor selection and modification techniques, including:

• Tensor assignment
• Scatter operations
• scatter_()
• scatter_add_()
• masked_fill()
• masked_fill_()
• masked_select()
• torch.take()
• Diagonal extraction
• Advanced masking
• Attention-mask construction
• Practical indexing patterns for deep learning pipelines

These operations will help you understand how modern PyTorch models manipulate tensor data efficiently before, during, and after neural network computation.