PyTorch Tensor Operations: Arithmetic, Broadcasting, Matrix Multiplication & Dot Product
Tensor operations are the mathematical foundation of PyTorch deep learning. Neural networks continuously perform operations such as addition, multiplication, matrix multiplication, normalization, activation functions, and reductions on tensors.
In the previous module, you learned how to create PyTorch tensors, inspect their shapes and data types, and initialize tensors using functions such as torch.zeros(), torch.ones(), torch.rand(), and torch.randn().
Now you will learn how to perform PyTorch tensor operations and understand how these operations are used inside neural networks.
This module focuses on:
- Tensor arithmetic
- Element-wise operations
- Mathematical functions
- Broadcasting
- Matrix multiplication
- Dot products
- Differences between
*,@,torch.matmul(), andtorch.dot() - Practical tensor operations used in deep learning
Learning Objectives
After completing this module, you will be able to:
- Perform arithmetic operations on PyTorch tensors.
- Understand element-wise tensor operations.
- Apply mathematical functions to tensors.
- Understand scalar-tensor operations.
- Explain PyTorch broadcasting.
- Determine whether two tensor shapes are broadcast-compatible.
- Perform matrix multiplication using
@andtorch.matmul(). - Calculate vector dot products.
- Distinguish element-wise multiplication from matrix multiplication.
- Understand why matrix multiplication is important in neural networks.
- Debug common tensor shape errors.
Why Tensor Operations Matter in Deep Learning
A neural network is essentially a sequence of mathematical operations.
Consider a simple neural-network layer:
1Input 2 │ 3 ▼ 4Matrix Multiplication 5 │ 6 ▼ 7Add Bias 8 │ 9 ▼ 10Activation Function 11 │ 12 ▼ 13Output
Mathematically, a common layer can be represented as:
1y = Wx + b
where:
xis the input tensor.Wrepresents learnable weights.brepresents the bias.yis the output.
PyTorch performs these calculations using tensor operations.
This means understanding tensor operations is essential before learning:
- Neural networks
- Convolutional neural networks
- Transformers
- Attention mechanisms
- Embeddings
- Large language models
- Computer vision
- Generative AI
Tensor Operations Overview
The major operations covered in this module are:
1Tensor Operations 2│ 3├── Arithmetic Operations 4│ ├── Addition 5│ ├── Subtraction 6│ ├── Multiplication 7│ ├── Division 8│ └── Power 9│ 10├── Element-wise Operations 11│ ├── sqrt 12│ ├── exp 13│ ├── log 14│ ├── abs 15│ └── trigonometric functions 16│ 17├── Broadcasting 18│ 19├── Matrix Multiplication 20│ 21└── Dot Product
PyTorch Arithmetic Operations
PyTorch supports standard arithmetic operations directly on tensors.
Common operations include:
| Operation | Python Operator | PyTorch Function |
|---|---|---|
| Addition | + | torch.add() |
| Subtraction | - | torch.sub() |
| Multiplication | * | torch.mul() |
| Division | / | torch.div() |
| Power | ** | torch.pow() |
| Remainder | % | torch.remainder() |
Tensor Addition
Two tensors with compatible shapes can be added element by element.
1import torch 2 3a = torch.tensor([1, 2, 3]) 4b = torch.tensor([4, 5, 6]) 5 6c = a + b 7 8print(c)
Output:
1tensor([5, 7, 9])
The calculation is:
1[1, 2, 3] 2+ 3[4, 5, 6] 4----------- 5[5, 7, 9]
Each element is added to the corresponding element.
Using torch.add()
The same operation can be written using the PyTorch function:
1import torch 2 3a = torch.tensor([10, 20]) 4b = torch.tensor([5, 3]) 5 6c = torch.add(a, b) 7 8print(c)
Output:
1tensor([15, 23])
For everyday PyTorch code, the + operator is often more readable:
1c = a + b
Tensor Subtraction
Subtraction is also performed element by element.
1import torch 2 3a = torch.tensor([10, 20, 30]) 4b = torch.tensor([2, 5, 10]) 5 6result = a - b 7 8print(result)
Output:
1tensor([ 8, 15, 20])
Calculation:
110 - 2 = 8 220 - 5 = 15 330 - 10 = 20
Tensor Multiplication
The * operator performs element-wise multiplication.
1import torch 2 3a = torch.tensor([2, 3, 4]) 4b = torch.tensor([5, 6, 7]) 5 6result = a * b 7 8print(result)
Output:
1tensor([10, 18, 28])
Calculation:
12 × 5 = 10 23 × 6 = 18 34 × 7 = 28
This is an important concept:
1a * b
does not perform matrix multiplication.
It performs element-wise multiplication.
Tensor Division
The / operator performs element-wise division.
1import torch 2 3a = torch.tensor([10, 20, 30]) 4b = torch.tensor([2, 4, 5]) 5 6result = a / b 7 8print(result)
Output:
1tensor([5., 5., 6.])
The result is floating point because regular tensor division produces floating-point results.
Floor Division
PyTorch also supports floor division:
1import torch 2 3a = torch.tensor([10, 20, 31]) 4b = torch.tensor([3, 4, 5]) 5 6result = a // b 7 8print(result)
Output:
1tensor([3, 5, 6])
For example:
110 // 3 = 3 220 // 4 = 5 331 // 5 = 6
Power Operation
Use ** to raise tensor values to a power.
1import torch 2 3x = torch.tensor([2, 3, 4]) 4 5result = x ** 2 6 7print(result)
Output:
1tensor([ 4, 9, 16])
You can also use:
1result = torch.pow(x, 2)
Modulus Operation
The % operator returns the remainder.
1import torch 2 3a = torch.tensor([10, 15, 20]) 4b = torch.tensor([3, 4, 6]) 5 6result = a % b 7 8print(result)
Output:
1tensor([1, 3, 2])
For example:
110 ÷ 3 → remainder 1 215 ÷ 4 → remainder 3 320 ÷ 6 → remainder 2
Arithmetic With Scalars
PyTorch can perform operations between tensors and scalar values.
For example:
1import torch 2 3x = torch.tensor([1, 2, 3]) 4 5print(x + 10)
Output:
1tensor([11, 12, 13])
Conceptually:
1[1, 2, 3] + 10 2 3↓ 4 5[1, 2, 3] 6+ 7[10, 10, 10] 8 9↓ 10 11[11, 12, 13]
This behavior is an example of broadcasting.
Scalar Multiplication
1import torch 2 3x = torch.tensor([2, 3, 4]) 4 5print(x * 5)
Output:
1tensor([10, 15, 20])
Conceptually:
1[2, 3, 4] × 5 2 3↓ 4 5[2, 3, 4] 6× 7[5, 5, 5] 8 9↓ 10 11[10, 15, 20]
Element-Wise Operations
An element-wise operation applies an operation independently to corresponding tensor elements.
For example:
1import torch 2 3a = torch.tensor([1, 2, 3]) 4b = torch.tensor([4, 5, 6]) 5 6result = a + b 7 8print(result)
Each position is processed independently:
1Position 0 → 1 + 4 = 5 2Position 1 → 2 + 5 = 7 3Position 2 → 3 + 6 = 9
Element-wise operations are heavily used throughout neural networks.
Element-Wise Square
Use torch.square():
1import torch 2 3x = torch.tensor([2, 4, 6]) 4 5result = torch.square(x) 6 7print(result)
Output:
1tensor([ 4, 16, 36])
Equivalent code:
1result = x ** 2
Square Root
Use torch.sqrt():
1import torch 2 3x = torch.tensor([4.0, 9.0, 16.0]) 4 5result = torch.sqrt(x) 6 7print(result)
Output:
1tensor([2., 3., 4.])
Exponential Function
Use torch.exp() to calculate the exponential of each element.
1import torch 2 3x = torch.tensor([1.0, 2.0, 3.0]) 4 5result = torch.exp(x) 6 7print(result)
Approximate output:
1tensor([ 2.7183, 7.3891, 20.0855])
The operation is applied independently:
1exp([x₁, x₂, x₃]) 2 3↓ 4 5[exp(x₁), exp(x₂), exp(x₃)]
The exponential function is important in machine learning, including probability-related calculations and functions such as softmax.
Logarithm
Use torch.log() for the natural logarithm.
1import torch 2 3x = torch.tensor([1.0, 2.0, 4.0]) 4 5result = torch.log(x) 6 7print(result)
Approximate output:
1tensor([0.0000, 0.6931, 1.3863])
Absolute Value
Use torch.abs():
1import torch 2 3x = torch.tensor([-5.0, 3.0, -7.0]) 4 5result = torch.abs(x) 6 7print(result)
Output:
1tensor([5., 3., 7.])
Trigonometric Functions
PyTorch also provides functions such as:
1torch.sin() 2torch.cos() 3torch.tan()
Example:
1import torch 2 3angles = torch.tensor([ 4 0.0, 5 torch.pi / 2 6]) 7 8print(torch.sin(angles)) 9print(torch.cos(angles))
Approximate output:
1tensor([0., 1.]) 2 3tensor([ 1., -0.])
These functions are useful in mathematical, scientific, and specialized machine-learning applications.
Broadcasting in PyTorch
Broadcasting allows PyTorch to perform operations between tensors with compatible but different shapes.
Consider:
1import torch 2 3a = torch.tensor([1, 2, 3]) 4b = torch.tensor(5) 5 6print(a + b)
Output:
1tensor([6, 7, 8])
The scalar value is logically applied to every element.
Conceptually:
1[1, 2, 3] 2+ 3 5 4 5↓ 6 7[1, 2, 3] 8+ 9[5, 5, 5] 10 11↓ 12 13[6, 7, 8]
PyTorch does not need to explicitly create a physical copy of the scalar for this operation.
Why Broadcasting Is Important
Broadcasting makes tensor code concise and efficient.
For example, suppose every feature in a matrix needs to be shifted by a different value.
Instead of manually creating repeated rows, you can write:
1result = matrix + bias
This pattern appears frequently in:
- Neural-network layers
- Bias addition
- Normalization
- Image processing
- Feature scaling
- Attention mechanisms
Broadcasting Rules
PyTorch follows broadcasting rules based on comparing tensor dimensions from right to left.
Two dimensions are compatible when:
- They are equal, or
- One of them is
1.
Missing leading dimensions are treated as having size 1.
Consider:
1(4, 3) 2(1, 3)
Compare from the right:
13 = 3 ✓ 24 vs 1 ✓
Therefore the shapes are compatible.
Broadcasting Example
Consider:
1import torch 2 3A = torch.tensor([ 4 [1], 5 [2], 6 [3] 7]) 8 9B = torch.tensor([ 10 10, 11 20, 12 30 13]) 14 15result = A + B 16 17print(result)
Output:
1tensor([ 2 [11, 21, 31], 3 [12, 22, 32], 4 [13, 23, 33] 5])
Shapes:
1A → (3, 1) 2B → (3,)
PyTorch treats the second tensor as compatible with:
1(1, 3)
and the resulting broadcast shape is:
1(3, 3)
Conceptually:
1A: 2 31 42 53 6 7Broadcast to: 8 91 1 1 102 2 2 113 3 3 12 13 14B: 15 1610 20 30 17 18Broadcast to: 19 2010 20 30 2110 20 30 2210 20 30 23 24 25Result: 26 2711 21 31 2812 22 32 2913 23 33
Broadcasting Example With a Matrix and Row Vector
A common pattern is:
1import torch 2 3matrix = torch.tensor([ 4 [1, 2, 3], 5 [4, 5, 6] 6]) 7 8bias = torch.tensor([10, 20, 30]) 9 10result = matrix + bias 11 12print(result)
Output:
1tensor([ 2 [11, 22, 33], 3 [14, 25, 36] 4])
Shapes:
1matrix → (2, 3) 2bias → (3,)
The bias is applied to every row.
This is conceptually similar to adding a bias vector in a neural-network layer.
Broadcasting Shape Example
The following shapes are compatible:
1(2, 3) 2(3,)
because (3,) can be treated as (1, 3).
Also:
1(4, 3, 5) 2(5,)
is compatible.
The second tensor can be treated as:
1(1, 1, 5)
and broadcast across the first two dimensions.
Invalid Broadcasting
Consider:
1import torch 2 3a = torch.randn(2, 3) 4b = torch.randn(4, 3) 5 6result = a + b
This fails because:
1(2, 3) 2(4, 3)
Compare from the right:
13 = 3 ✓ 22 vs 4 ✗
Neither dimension is equal and neither is 1.
Therefore the tensors cannot be broadcast together.
A Useful Broadcasting Method
When debugging broadcasting, write the shapes underneath each other and compare from the right:
1A: 2 3 2B: 4 3 3 ↑ ↑ 4 │ └── 3 = 3 ✓ 5 └───── 2 ≠ 4 ✗
This simple technique can help identify many shape errors.
Matrix Multiplication
Matrix multiplication is fundamentally different from element-wise multiplication.
For matrices:
1A → (m, n) 2B → (n, p)
the result has shape:
1(m, p)
The inner dimensions must match:
1(m, n) × (n, p) 2 ↑ 3 match
Matrix Multiplication Example
Consider:
1import torch 2 3A = torch.tensor([ 4 [1, 2], 5 [3, 4] 6]) 7 8B = torch.tensor([ 9 [5, 6], 10 [7, 8] 11]) 12 13C = A @ B 14 15print(C)
Output:
1tensor([ 2 [19, 22], 3 [43, 50] 4])
How Matrix Multiplication Works
The first output value is calculated using the first row of A and the first column of B:
1(1 × 5) + (2 × 7) 2 3= 5 + 14 4 5= 19
The second output value:
1(1 × 6) + (2 × 8) 2 3= 6 + 16 4 5= 22
The complete result is:
1[1 2] × [5 6] = [19 22] 2[3 4] [7 8] [43 50]
Matrix Multiplication With @
Python provides the @ operator specifically for matrix multiplication.
1C = A @ B
This is often the clearest syntax when writing PyTorch models.
Matrix Multiplication With torch.matmul()
You can also write:
1C = torch.matmul(A, B)
For these matrices, both approaches produce the same result:
1C1 = A @ B 2C2 = torch.matmul(A, B) 3 4print(torch.equal(C1, C2))
Output:
1True
Element-Wise Multiplication vs Matrix Multiplication
This distinction is extremely important.
Consider:
1import torch 2 3A = torch.tensor([ 4 [1, 2], 5 [3, 4] 6]) 7 8B = torch.tensor([ 9 [5, 6], 10 [7, 8] 11]) 12 13print("Element-wise:") 14print(A * B) 15 16print("Matrix multiplication:") 17print(A @ B)
Output:
1Element-wise: 2tensor([ 3 [ 5, 12], 4 [21, 32] 5]) 6 7Matrix multiplication: 8tensor([ 9 [19, 22], 10 [43, 50] 11])
Why are they different?
Element-Wise Multiplication
11 × 5 = 5 22 × 6 = 12 33 × 7 = 21 44 × 8 = 32
Result:
1[ 5 12] 2[21 32]
Matrix Multiplication
Each output value is calculated using a row and a column:
11×5 + 2×7 = 19 21×6 + 2×8 = 22 3 43×5 + 4×7 = 43 53×6 + 4×8 = 50
Result:
1[19 22] 2[43 50]
Important Difference
Remember:
1A * B
means:
Element-wise multiplication.
While:
1A @ B
means:
Matrix multiplication.
This distinction is one of the most important concepts in PyTorch.
Matrix Multiplication Shape Rules
Suppose:
1A → (2, 3) 2B → (3, 4)
Then:
1A @ B
is valid.
The result is:
1(2, 4)
because:
1(2, 3) × (3, 4) 2 ↓ 3 result 4 ↓ 5 (2, 4)
But:
1(2, 3) × (4, 2)
is invalid because:
13 ≠ 4
The inner dimensions do not match.
Matrix Multiplication in Neural Networks
Matrix multiplication is one of the most important operations in deep learning.
A fully connected layer can be represented conceptually as:
1Output = Input × Weights + Bias
For example:
1Input 2(1 × 3) 3 4× 5 6Weights 7(3 × 2) 8 9= 10 11Output 12(1 × 2)
This is why understanding matrix multiplication is essential before studying neural-network layers.
Dot Product
The dot product operates on two vectors of the same length.
For vectors:
1a = [a₁, a₂, a₃] 2 3b = [b₁, b₂, b₃]
the dot product is:
1a · b = a₁b₁ + a₂b₂ + a₃b₃
The result is a scalar.
PyTorch Dot Product Example
1import torch 2 3a = torch.tensor([1, 2, 3]) 4b = torch.tensor([4, 5, 6]) 5 6result = torch.dot(a, b) 7 8print(result)
Output:
1tensor(32)
Calculation:
1(1 × 4) + (2 × 5) + (3 × 6) 2 3= 4 + 10 + 18 4 5= 32
Dot Product vs Matrix Multiplication
For two one-dimensional vectors:
1a = torch.tensor([1, 2, 3]) 2b = torch.tensor([4, 5, 6])
you can use:
1torch.dot(a, b)
which returns a scalar.
You can also use:
1a @ b
for two 1D tensors, which performs the corresponding vector dot product.
However, torch.dot() is specifically designed for 1D tensors, while torch.matmul() and @ support broader tensor shapes according to PyTorch's matrix-multiplication rules.
Dot Product in Machine Learning
Dot products are fundamental to many machine-learning concepts.
They appear in:
- Linear models
- Neural-network layers
- Similarity calculations
- Embeddings
- Attention mechanisms
- Transformer architectures
For example, similarity between two vectors can be measured using their dot product.
This becomes particularly important when studying vector embeddings and attention mechanisms.
Comparison of Multiplication Operations
| Operation | Syntax | Main Purpose |
|---|---|---|
| Element-wise multiplication | A * B | Multiply corresponding elements |
| Matrix multiplication | A @ B | Matrix/vector multiplication |
| Matrix multiplication | torch.matmul(A, B) | General matrix multiplication |
| Dot product | torch.dot(a, b) | Dot product of two 1D tensors |
A useful rule is:
1* → element by element 2@ → matrix multiplication 3matmul → matrix multiplication rules 4dot → 1D vector dot product
In-Place Tensor Operations
PyTorch provides in-place operations that modify the existing tensor.
Many in-place methods end with _.
For example:
1import torch 2 3x = torch.tensor([1, 2, 3]) 4 5x.add_(5) 6 7print(x)
Output:
1tensor([6, 7, 8])
Other examples include:
1x.sub_(2) 2x.mul_(3) 3x.div_(2)
In-place operations can reduce memory usage, but they must be used carefully when tensors participate in Autograd computations.
For beginner code, prefer regular out-of-place operations unless there is a clear reason to use an in-place operation.
Tensor Operations With GPU
Tensor operations can run on a CUDA device when the tensors are placed on the GPU.
1import torch 2 3device = torch.device( 4 "cuda" if torch.cuda.is_available() else "cpu" 5) 6 7a = torch.tensor( 8 [1.0, 2.0, 3.0], 9 device=device 10) 11 12b = torch.tensor( 13 [4.0, 5.0, 6.0], 14 device=device 15) 16 17result = a + b 18 19print(result) 20print(result.device)
Possible output:
1tensor([5., 7., 9.], device='cuda:0') 2cuda:0
The important rule is that tensors involved in the same operation generally need to be on compatible devices.
Practical Example: Feature Scaling
Suppose a model receives:
1import torch 2 3features = torch.tensor([ 4 [10.0, 20.0, 30.0], 5 [15.0, 25.0, 35.0] 6])
You can scale the features using broadcasting:
1scale = torch.tensor([10.0, 5.0, 2.0]) 2 3scaled = features / scale 4 5print(scaled)
The shape is:
1features → (2, 3) 2scale → (3,)
Broadcasting allows the operation to be performed across every row.
Practical Example: Neural-Network Layer
Consider a single input:
1import torch 2 3x = torch.tensor([ 4 [1.0, 2.0, 3.0] 5])
Suppose the layer has:
1weights = torch.tensor([ 2 [1.0, 2.0], 3 [3.0, 4.0], 4 [5.0, 6.0] 5]) 6 7bias = torch.tensor([ 8 [0.5, 1.0] 9])
Perform the linear transformation:
1output = x @ weights + bias 2 3print(output)
The shapes are:
1x → (1, 3) 2weights → (3, 2) 3bias → (1, 2) 4 5output → (1, 2)
This simple example demonstrates three fundamental PyTorch concepts together:
1Matrix Multiplication 2 + 3Broadcasting 4 ↓ 5Neural Network Output
Common Tensor Operation Mistakes
Mistake 1: Confusing * and @
Incorrect assumption:
1A * B
performs matrix multiplication.
It does not.
Use:
1A @ B
or:
1torch.matmul(A, B)
for matrix multiplication.
Mistake 2: Ignoring Tensor Shapes
Before performing matrix multiplication, inspect:
1print(A.shape) 2print(B.shape)
For:
1A → (m, n) 2B → (n, p)
the multiplication is valid.
Mistake 3: Assuming Every Different Shape Can Broadcast
Broadcasting has specific rules.
For example:
1(2, 3) 2(4, 3)
cannot be broadcast together.
Mistake 4: Using torch.dot() for Arbitrary Tensors
torch.dot() is intended for one-dimensional tensors.
For more general matrix or higher-dimensional multiplication, use:
1torch.matmul()
or:
1@
Mistake 5: Forgetting Device Compatibility
This can cause errors:
1Tensor A → CPU 2Tensor B → CUDA
Move tensors to compatible devices before performing operations.
Practice Exercises
Exercise 1: Basic Arithmetic
Create:
1a = torch.tensor([10, 20, 30]) 2b = torch.tensor([2, 4, 5])
Calculate:
- Addition
- Subtraction
- Element-wise multiplication
- Division
- Modulus
Exercise 2: Mathematical Functions
Create:
1x = torch.tensor([1.0, 4.0, 9.0, 16.0])
Calculate:
- Square root
- Square
- Exponential
- Natural logarithm
Exercise 3: Broadcasting
Create:
1A = torch.tensor([ 2 [1, 2, 3], 3 [4, 5, 6] 4]) 5 6B = torch.tensor([10, 20, 30])
Use broadcasting to calculate:
1A + B
Determine the shape of the output.
Exercise 4: Broadcasting Challenge
Determine whether the following pairs are compatible:
1(3, 4) and (4,) 2(2, 3, 4) and (4,) 3(2, 3) and (2, 1) 4(2, 3) and (4, 3) 5(5, 1, 4) and (3, 4)
For every pair, explain why it is valid or invalid.
Exercise 5: Matrix Multiplication
Create:
1A = torch.randn(2, 3) 2B = torch.randn(3, 4)
Calculate:
1C = A @ B
Then print:
1print(A.shape) 2print(B.shape) 3print(C.shape)
Verify that:
1(2, 3) × (3, 4) = (2, 4)
Exercise 6: Element-Wise vs Matrix Multiplication
Create two 2 × 2 matrices.
Calculate:
1A * B
and:
1A @ B
Compare the results and explain why they are different.
Exercise 7: Dot Product
Create:
1a = torch.tensor([2, 4, 6]) 2b = torch.tensor([1, 3, 5])
Calculate the dot product using:
1torch.dot(a, b)
Verify the result manually.
Exercise 8: Neural-Network Calculation
Create:
1x = torch.tensor([[1.0, 2.0, 3.0]]) 2 3weights = torch.tensor([ 4 [1.0, 2.0], 5 [3.0, 4.0], 6 [5.0, 6.0] 7]) 8 9bias = torch.tensor([[0.5, 1.0]])
Calculate:
1output = x @ weights + bias
Then print:
1print(output) 2print(output.shape)
Explain how matrix multiplication and broadcasting were used.
Quick Reference
Addition
1a + b
Subtraction
1a - b
Element-Wise Multiplication
1a * b
Division
1a / b
Power
1a ** 2
Square Root
1torch.sqrt(a)
Exponential
1torch.exp(a)
Logarithm
1torch.log(a)
Absolute Value
1torch.abs(a)
Broadcasting
1a + scalar
or:
1matrix + vector
when the shapes are broadcast-compatible.
Matrix Multiplication
1A @ B
or:
1torch.matmul(A, B)
Dot Product
1torch.dot(a, b)
Key Takeaways
The most important concepts from this module are:
- PyTorch performs mathematical computations primarily through tensors.
- Arithmetic operators such as
+,-,*, and/can operate directly on tensors. *performs element-wise multiplication.@performs matrix multiplication.torch.matmul()provides general matrix-multiplication behavior for supported tensor dimensions.torch.dot()calculates the dot product of two 1D tensors.- Broadcasting allows compatible tensors with different shapes to participate in operations.
- Broadcasting compares dimensions from right to left.
- Two dimensions are compatible when they are equal or one of them is
1. - Matrix multiplication requires compatible inner dimensions.
- Matrix multiplication is fundamental to neural-network layers.
- Tensor shapes should always be checked when debugging operation errors.
- In-place operations modify the original tensor and should be used carefully with Autograd.
A useful mental model is:
1Tensor 2 │ 3 ├── Arithmetic 4 │ ├── + 5 │ ├── - 6 │ ├── * 7 │ └── / 8 │ 9 ├── Element-wise Functions 10 │ ├── sqrt 11 │ ├── exp 12 │ ├── log 13 │ └── abs 14 │ 15 ├── Broadcasting 16 │ └── Compatible Shapes 17 │ 18 ├── Matrix Multiplication 19 │ └── @ / matmul 20 │ 21 └── Dot Product 22 └── dot
Next Module
In Tensor Operations Part 2, you will build on these concepts by learning how to reduce, combine, reshape, and split tensors.
The next topics include:
- Tensor reduction operations
sum()mean()min()andmax()argmin()andargmax()- Tensor indexing
- Tensor slicing
- Reshaping
- Flattening
view()reshape()squeeze()unsqueeze()cat()stack()split()- Tensor dimension management
These operations are essential for preparing data and connecting tensors correctly between different layers of a deep learning model.