Module 9: Loss Functions in PyTorch

Loss functions are one of the most important components of a neural network training pipeline. They measure the difference between a model's prediction and the expected target and provide the signal used by backpropagation to calculate parameter gradients.

In PyTorch, loss functions are primarily available through torch.nn and are typically used together with an optimizer and autograd.

What You Will Learn

In this module, you will learn:

• What a loss function is
• Why loss functions are important in deep learning
• How loss functions work with backpropagation
• Regression loss functions
• Classification loss functions
• MSELoss
• L1Loss
• HuberLoss
• CrossEntropyLoss
• BCELoss
• BCEWithLogitsLoss
• NLLLoss
• Logits and probabilities
• Reduction modes
• Class imbalance and weighted losses
• How to choose the correct PyTorch loss function
• Common mistakes and best practices
• Practical PyTorch training examples

Learning Objectives

After completing this module, you will be able to:

• Explain the purpose of a loss function.
• Understand how loss connects the forward and backward passes.
• Select an appropriate loss function for regression and classification.
• Use PyTorch loss functions correctly.
• Understand the difference between logits and probabilities.
• Explain why BCEWithLogitsLoss is generally preferred over Sigmoid + BCELoss.
• Use CrossEntropyLoss correctly without applying Softmax beforehand.
• Handle loss values during a training loop.
• Understand gradient computation through a loss function.

What Is a Loss Function?

A loss function measures how different a model's prediction is from the target value.

For example, suppose a model predicts a house price:

textCopy
1Actual Price:     ₹50,00,000
2Predicted Price:  ₹48,00,000

The prediction is incorrect, so the loss function converts this error into a numerical value.

The training process can be represented as:

textCopy
1Input
2  │
3  ▼
4Neural Network
5  │
6  ▼
7Prediction
8  │
9  ▼
10Loss Function
11  │
12  ▼
13Loss Value
14  │
15  ▼
16Backward Pass
17  │
18  ▼
19Gradients
20  │
21  ▼
22Optimizer
23  │
24  ▼
25Updated Parameters

The objective of training is generally to minimize the loss.

Loss vs Error

Loss and error are related but are not necessarily the same thing.

An error may simply describe the difference:

textCopy
1error = prediction - target

A loss function applies a mathematical transformation to that difference.

For example, MSE uses the squared error:

textCopy
1loss = (prediction - target)²

This allows the training algorithm to optimize model parameters using gradients.

Why Are Loss Functions Important?

A neural network does not automatically know whether its prediction is good or bad.

Consider:

textCopy
1Target:     1
2Prediction: 0.2

The model needs a numerical signal describing how far the prediction is from the target.

The loss provides that signal.

During training:

textCopy
1Prediction
2     │
3     ▼
4   Loss
5     │
6     ▼
7backward()
8     │
9     ▼
10Gradients
11     │
12     ▼
13Optimizer
14     │
15     ▼
16New Parameters

This process is repeated over many batches and epochs.

Loss Function and Backpropagation

A loss function works together with PyTorch Autograd.

Example:

pythonCopy
1import torch
2
3x = torch.tensor(2.0, requires_grad=True)
4
5prediction = x ** 2
6target = torch.tensor(5.0)
7
8loss = (prediction - target) ** 2
9
10loss.backward()
11
12print("Prediction:", prediction.item())
13print("Loss:", loss.item())
14print("Gradient:", x.grad.item())

The important relationship is:

textCopy
1Model Parameters
2       │
3       ▼
4   Prediction
5       │
6       ▼
7      Loss
8       │
9       ▼
10   backward()
11       │
12       ▼
13Parameter Gradients

The optimizer then uses these gradients to update the parameters.

Main Categories of Loss Functions

Loss functions can be grouped according to the machine learning problem.

Choosing the correct loss function is important because the model output format and target format must match the loss function.

Regression Loss Functions

Regression predicts continuous numerical values.

Examples include:

• House prices
• Temperature
• Sales
• Revenue
• Sensor measurements
• Age prediction
• Demand forecasting

1. Mean Squared Error Loss

PyTorch provides MSE through:

pythonCopy
1nn.MSELoss()

The mathematical definition is:

where:

• (y_i) is the target
• (\hat{y}_i) is the prediction
• (N) is the number of elements

Because the error is squared, large errors receive significantly greater penalties.

MSELoss Example

pythonCopy
1import torch
2import torch.nn as nn
3
4loss_fn = nn.MSELoss()
5
6prediction = torch.tensor([2.5, 3.2, 4.8])
7target = torch.tensor([3.0, 3.0, 5.0])
8
9loss = loss_fn(prediction, target)
10
11print("MSE Loss:", loss.item())

The squared errors are:

textCopy
1(2.5 - 3.0)² = 0.25
2(3.2 - 3.0)² = 0.04
3(4.8 - 5.0)² = 0.04

Therefore:

textCopy
1MSE = (0.25 + 0.04 + 0.04) / 3
2    = 0.11

When Should You Use MSELoss?

MSE is a good choice when:

• Large errors should be strongly penalized.
• The target is continuous.
• The dataset does not contain problematic outliers.
• A smooth squared-error objective is appropriate.

MSE and Outliers

Suppose most errors are small:

textCopy
11
22
31
42

but one prediction has an error of:

textCopy
120

MSE squares the large error:

textCopy
120² = 400

Therefore, a few extreme errors can dominate the loss.

This is one reason robust alternatives such as Huber loss are useful.

2. L1 Loss

PyTorch provides Mean Absolute Error through:

pythonCopy
1nn.L1Loss()

The formula is:

Unlike MSE, it does not square the error.

L1Loss Example

pythonCopy
1import torch
2import torch.nn as nn
3
4loss_fn = nn.L1Loss()
5
6prediction = torch.tensor([2.5, 3.2, 4.8])
7target = torch.tensor([3.0, 3.0, 5.0])
8
9loss = loss_fn(prediction, target)
10
11print("L1 Loss:", loss.item())

The absolute errors are:

textCopy
10.5
20.2
30.2

Therefore:

textCopy
1L1 = (0.5 + 0.2 + 0.2) / 3
2   = 0.3

Advantages of L1 Loss

• More robust to outliers than MSE.
• Easy to interpret.
• Useful when absolute error is important.

MSE vs L1

Suppose:

textCopy
1Prediction = 100
2Target     = 150

Absolute error:

textCopy
1|100 - 150| = 50

L1 loss:

textCopy
150

Squared error:

textCopy
1(100 - 150)² = 2500

Therefore MSE places much greater emphasis on large errors.

3. Huber Loss

Huber loss combines characteristics of MSE and L1 loss.

PyTorch provides:

pythonCopy
1nn.HuberLoss()

For an error (e), Huber loss behaves approximately like:

textCopy
1Small error
2    ↓
3Squared error behavior
4
5Large error
6    ↓
7Absolute error behavior

With delta = 1:

HuberLoss Example

pythonCopy
1import torch
2import torch.nn as nn
3
4loss_fn = nn.HuberLoss(delta=1.0)
5
6prediction = torch.tensor([2.5, 3.2, 4.8])
7target = torch.tensor([3.0, 3.0, 5.0])
8
9loss = loss_fn(prediction, target)
10
11print("Huber Loss:", loss.item())

Why Use Huber Loss?

Huber loss is useful when:

• The dataset contains outliers.
• You want smoother optimization than pure L1.
• MSE is too sensitive to extreme errors.
• Stable regression training is important.

Regression Loss Comparison

Classification Loss Functions

Classification predicts categories.

Examples:

textCopy
1Spam / Not Spam
2Cat / Dog
3Positive / Negative
4Class A / Class B / Class C

Classification can be divided into:

• Binary classification
• Multi-class classification
• Multi-label classification

Binary Classification

Binary classification has two possible outcomes.

Example:

textCopy
10 → Not Spam
21 → Spam

A common PyTorch choice is:

pythonCopy
1nn.BCEWithLogitsLoss()

Multi-Class Classification

Multi-class classification selects one class from multiple possible classes.

Example:

textCopy
10 → Cat
21 → Dog
32 → Bird

A common PyTorch choice is:

pythonCopy
1nn.CrossEntropyLoss()

Multi-Label Classification

Multi-label classification allows multiple classes to be true simultaneously.

For example, an image could contain:

textCopy
1Car     → 1
2Person  → 1
3Dog     → 0
4Tree    → 1

A common choice is:

pythonCopy
1nn.BCEWithLogitsLoss()

4. CrossEntropyLoss

CrossEntropyLoss is one of the most important PyTorch loss functions for multi-class classification.

pythonCopy
1loss_fn = nn.CrossEntropyLoss()

It expects unnormalized logits from the model.

That means you normally should not add:

pythonCopy
1nn.Softmax()

before CrossEntropyLoss.

Correct Architecture

pythonCopy
1model = nn.Sequential(
2    nn.Linear(128, 10)
3)
4
5criterion = nn.CrossEntropyLoss()

The final linear layer produces:

textCopy
110 logits

CrossEntropyLoss internally applies the appropriate log-softmax operation and negative log-likelihood calculation.

CrossEntropyLoss Example

pythonCopy
1import torch
2import torch.nn as nn
3
4loss_fn = nn.CrossEntropyLoss()
5
6logits = torch.tensor([
7    [2.0, 1.0, 0.1]
8])
9
10target = torch.tensor([0])
11
12loss = loss_fn(logits, target)
13
14print("Loss:", loss.item())

The target:

textCopy
10

means that class index 0 is the correct class.

Understanding Logits

Suppose the model produces:

textCopy
1[2.0, 1.0, 0.1]

These are logits, not probabilities.

They can:

• Be negative.
• Be greater than 1.
• Have any real-valued magnitude.

Softmax converts logits into probabilities.

Conceptually:

textCopy
1Logits
2  │
3  ▼
4Softmax
5  │
6  ▼
7Probabilities

But when training with CrossEntropyLoss, you normally provide the logits directly:

textCopy
1Model
2  │
3  ▼
4Logits
5  │
6  ▼
7CrossEntropyLoss

CrossEntropyLoss with a Batch

pythonCopy
1import torch
2import torch.nn as nn
3
4loss_fn = nn.CrossEntropyLoss()
5
6logits = torch.tensor([
7    [2.0, 0.5, 0.1],
8    [0.2, 2.5, 0.3],
9    [0.1, 0.4, 3.0]
10])
11
12targets = torch.tensor([0, 1, 2])
13
14loss = loss_fn(logits, targets)
15
16print(loss.item())

Here:

textCopy
1Batch size = 3
2Number of classes = 3

The target tensor contains one class index per sample.

Important CrossEntropyLoss Rule

For standard multi-class classification:

textCopy
1Input:
2(N, C)
3
4Target:
5(N,)

where:

• N = batch size
• C = number of classes

Example:

textCopy
1Logits:
2(32, 10)
3
4Targets:
5(32,)

Do not convert the target into one-hot encoding for the usual CrossEntropyLoss setup.

5. Binary Cross Entropy

PyTorch provides:

pythonCopy
1nn.BCELoss()

BCELoss expects probabilities.

Therefore, the model output must normally be between 0 and 1.

A traditional architecture is:

pythonCopy
1model = nn.Sequential(
2    nn.Linear(20, 1),
3    nn.Sigmoid()
4)

Then:

pythonCopy
1criterion = nn.BCELoss()

BCELoss Example

pythonCopy
1import torch
2import torch.nn as nn
3
4loss_fn = nn.BCELoss()
5
6probabilities = torch.tensor([
7    0.9,
8    0.2,
9    0.8
10])
11
12targets = torch.tensor([
13    1.0,
14    0.0,
15    1.0
16])
17
18loss = loss_fn(probabilities, targets)
19
20print("BCE Loss:", loss.item())

Why BCELoss Is Usually Not the First Choice

The problem is that you must explicitly apply sigmoid:

textCopy
1Logits
2  ↓
3Sigmoid
4  ↓
5Probability
6  ↓
7BCELoss

This can be less numerically stable than combining the operations.

That is why PyTorch commonly recommends using:

pythonCopy
1nn.BCEWithLogitsLoss()

instead.

6. BCEWithLogitsLoss

BCEWithLogitsLoss combines sigmoid activation and binary cross entropy into one numerically stable operation.

pythonCopy
1nn.BCEWithLogitsLoss()

The model should output raw logits.

Correct:

textCopy
1Model
2  ↓
3Logits
4  ↓
5BCEWithLogitsLoss

Incorrect:

textCopy
1Model
2  ↓
3Sigmoid
4  ↓
5BCEWithLogitsLoss

BCEWithLogitsLoss Example

pythonCopy
1import torch
2import torch.nn as nn
3
4loss_fn = nn.BCEWithLogitsLoss()
5
6logits = torch.tensor([
7    2.5,
8    -1.3,
9    0.7
10])
11
12targets = torch.tensor([
13    1.0,
14    0.0,
15    1.0
16])
17
18loss = loss_fn(logits, targets)
19
20print("Loss:", loss.item())

Binary Classification Model

pythonCopy
1import torch.nn as nn
2
3model = nn.Sequential(
4    nn.Linear(20, 1)
5)
6
7criterion = nn.BCEWithLogitsLoss()

During inference, if probabilities are required:

pythonCopy
1probabilities = torch.sigmoid(logits)

For example:

pythonCopy
1logits = model(inputs)
2
3probabilities = torch.sigmoid(logits)
4
5predictions = (probabilities >= 0.5).float()

BCE vs BCEWithLogitsLoss

7. Negative Log Likelihood Loss

PyTorch provides:

pythonCopy
1nn.NLLLoss()

NLL loss expects log probabilities as input.

It is commonly used with:

pythonCopy
1nn.LogSoftmax()

Architecture:

textCopy
1Linear
2  ↓
3LogSoftmax
4  ↓
5NLLLoss

NLLLoss Example

pythonCopy
1import torch
2import torch.nn as nn
3
4logits = torch.tensor([
5    [2.0, 1.0, 0.1]
6])
7
8log_probabilities = torch.log_softmax(
9    logits,
10    dim=1
11)
12
13target = torch.tensor([0])
14
15loss_fn = nn.NLLLoss()
16
17loss = loss_fn(log_probabilities, target)
18
19print("NLL Loss:", loss.item())

NLLLoss Model

pythonCopy
1model = nn.Sequential(
2    nn.Linear(128, 10),
3    nn.LogSoftmax(dim=1)
4)
5
6criterion = nn.NLLLoss()

For most new multi-class models, using:

pythonCopy
1nn.CrossEntropyLoss()

directly on logits is simpler.

CrossEntropyLoss vs NLLLoss

Conceptually:

textCopy
1CrossEntropyLoss(logits)

is equivalent to applying:

textCopy
1LogSoftmax
2    ↓
3NLLLoss

Understanding Reduction

PyTorch loss functions often support a reduction argument.

Common values are:

pythonCopy
1"mean"
2"sum"
3"none"

The default is usually:

pythonCopy
1reduction="mean"

Mean Reduction

pythonCopy
1loss_fn = nn.MSELoss(reduction="mean")

The individual losses are averaged.

Sum Reduction

pythonCopy
1loss_fn = nn.MSELoss(reduction="sum")

The individual losses are added together.

No Reduction

pythonCopy
1loss_fn = nn.MSELoss(reduction="none")

This returns individual loss values.

Example:

pythonCopy
1import torch
2import torch.nn as nn
3
4loss_fn = nn.MSELoss(reduction="none")
5
6prediction = torch.tensor([2.0, 4.0, 6.0])
7target = torch.tensor([1.0, 5.0, 4.0])
8
9loss = loss_fn(prediction, target)
10
11print(loss)

Output:

textCopy
1tensor([1., 1., 4.])

This is useful when you need per-sample or per-element loss values.

Weighted Loss Functions

Class imbalance is common in real-world datasets.

Suppose a dataset contains:

textCopy
1Class 0 → 95%
2Class 1 → 5%

A model could achieve high accuracy by mostly predicting Class 0.

Loss weighting can help make mistakes on minority classes more important.

Weighted Cross Entropy

pythonCopy
1import torch
2import torch.nn as nn
3
4weights = torch.tensor([
5    1.0,
6    5.0,
7    2.0
8])
9
10criterion = nn.CrossEntropyLoss(
11    weight=weights
12)

The weight corresponds to each class index.

Loss Function and the Training Loop

A loss function becomes especially useful inside a training loop.

Example:

pythonCopy
1import torch
2import torch.nn as nn
3
4model = nn.Linear(10, 1)
5
6criterion = nn.MSELoss()
7
8optimizer = torch.optim.SGD(
9    model.parameters(),
10    lr=0.01
11)
12
13inputs = torch.randn(32, 10)
14targets = torch.randn(32, 1)
15
16optimizer.zero_grad()
17
18outputs = model(inputs)
19
20loss = criterion(outputs, targets)
21
22loss.backward()
23
24optimizer.step()
25
26print("Loss:", loss.item())

The important sequence is:

textCopy
1optimizer.zero_grad()
2        ↓
3model(inputs)
4        ↓
5loss = criterion(outputs, targets)
6        ↓
7loss.backward()
8        ↓
9optimizer.step()

This sequence is repeated during training.

Complete Binary Classification Example

pythonCopy
1import torch
2import torch.nn as nn
3
4model = nn.Sequential(
5    nn.Linear(5, 1)
6)
7
8criterion = nn.BCEWithLogitsLoss()
9
10inputs = torch.randn(8, 5)
11
12targets = torch.randint(
13    0,
14    2,
15    (8, 1)
16).float()
17
18outputs = model(inputs)
19
20loss = criterion(outputs, targets)
21
22print("Output shape:", outputs.shape)
23print("Target shape:", targets.shape)
24print("Loss:", loss.item())

The model produces:

textCopy
18 samples × 1 logit

and the target has:

textCopy
18 samples × 1 target

Complete Multi-Class Classification Example

pythonCopy
1import torch
2import torch.nn as nn
3
4model = nn.Sequential(
5    nn.Linear(20, 5)
6)
7
8criterion = nn.CrossEntropyLoss()
9
10inputs = torch.randn(16, 20)
11
12targets = torch.randint(
13    0,
14    5,
15    (16,)
16)
17
18outputs = model(inputs)
19
20loss = criterion(outputs, targets)
21
22print("Output shape:", outputs.shape)
23print("Target shape:", targets.shape)
24print("Loss:", loss.item())

Here:

textCopy
1Batch size = 16
2Classes = 5

The model outputs:

textCopy
1(16, 5)

while the target contains:

textCopy
1(16,)

class indices.

Complete Regression Example

pythonCopy
1import torch
2import torch.nn as nn
3
4model = nn.Linear(4, 1)
5
6criterion = nn.MSELoss()
7
8inputs = torch.randn(10, 4)
9targets = torch.randn(10, 1)
10
11outputs = model(inputs)
12
13loss = criterion(outputs, targets)
14
15print("Output shape:", outputs.shape)
16print("Target shape:", targets.shape)
17print("Loss:", loss.item())

Practical Example: Full Training Step

The following example combines the model, loss function, backpropagation, and optimizer.

pythonCopy
1import torch
2import torch.nn as nn
3
4model = nn.Linear(10, 1)
5
6criterion = nn.MSELoss()
7
8optimizer = torch.optim.Adam(
9    model.parameters(),
10    lr=0.001
11)
12
13inputs = torch.randn(32, 10)
14targets = torch.randn(32, 1)
15
16for epoch in range(10):
17
18    optimizer.zero_grad()
19
20    outputs = model(inputs)
21
22    loss = criterion(outputs, targets)
23
24    loss.backward()
25
26    optimizer.step()
27
28    print(
29        f"Epoch {epoch + 1}: "
30        f"Loss = {loss.item():.4f}"
31    )

This demonstrates the complete optimization cycle:

textCopy
1Input
2  ↓
3Model
4  ↓
5Prediction
6  ↓
7Loss
8  ↓
9Backward
10  ↓
11Gradients
12  ↓
13Optimizer
14  ↓
15Updated Model

Choosing the Right Loss Function

Loss Function Decision Guide

textCopy
1What is your task?
2       │
3       ├── Regression
4       │      │
5       │      ├── Normal errors → MSELoss
6       │      │
7       │      ├── Outliers → HuberLoss
8       │      │
9       │      └── Absolute error → L1Loss
10       │
11       └── Classification
12              │
13              ├── Binary → BCEWithLogitsLoss
14              │
15              ├── Multi-class → CrossEntropyLoss
16              │
17              └── Multi-label → BCEWithLogitsLoss

Common Mistakes

Applying Softmax Before CrossEntropyLoss

Incorrect:

pythonCopy
1model = nn.Sequential(
2    nn.Linear(128, 10),
3    nn.Softmax(dim=1)
4)
5
6criterion = nn.CrossEntropyLoss()

Correct:

pythonCopy
1model = nn.Sequential(
2    nn.Linear(128, 10)
3)
4
5criterion = nn.CrossEntropyLoss()

CrossEntropyLoss expects logits and internally handles the log-softmax portion of the computation.

Applying Sigmoid Before BCEWithLogitsLoss

Incorrect:

pythonCopy
1model = nn.Sequential(
2    nn.Linear(20, 1),
3    nn.Sigmoid()
4)
5
6criterion = nn.BCEWithLogitsLoss()

Correct:

pythonCopy
1model = nn.Sequential(
2    nn.Linear(20, 1)
3)
4
5criterion = nn.BCEWithLogitsLoss()

If you need probabilities for inference:

pythonCopy
1probabilities = torch.sigmoid(logits)

Using Incorrect Target Types

For CrossEntropyLoss, class targets are normally integer class indices:

pythonCopy
1targets = torch.tensor([0, 2, 1])

Use an integer tensor such as torch.long.

For BCEWithLogitsLoss, targets generally use floating-point values:

pythonCopy
1targets = torch.tensor([1.0, 0.0, 1.0])

Using the Wrong Target Shape

For binary classification with output:

textCopy
1(batch_size, 1)

the target should normally have the same shape:

textCopy
1(batch_size, 1)

For multi-class classification:

textCopy
1Logits:  (batch_size, classes)
2Target:  (batch_size)

Comparing Loss Values From Different Loss Functions

A numerical loss value only has meaning within the context of its particular loss definition.

For example:

textCopy
1MSE = 0.5
2CrossEntropy = 0.5

does not mean the two models are equally good.

Different loss functions measure different quantities.

Best Practices

• Select the loss function according to the actual learning objective.
• Use MSELoss for standard continuous regression when squared errors are appropriate.
• Consider HuberLoss when outliers can strongly affect regression.
• Use BCEWithLogitsLoss for most binary and multi-label classification problems.
• Use CrossEntropyLoss for standard multi-class classification.
• Do not apply Softmax before CrossEntropyLoss during training.
• Do not apply Sigmoid before BCEWithLogitsLoss.
• Verify model output and target shapes.
• Use the correct target data type.
• Call optimizer.zero_grad() before the backward pass in the standard training loop.
• Monitor loss across batches and epochs.
• Do not compare raw loss numbers across unrelated loss functions.
• Use class weighting when appropriate for imbalanced classification datasets.

Practice Exercises

Exercise 1: MSE Loss

Create:

pythonCopy
1prediction = torch.tensor([10.0, 20.0, 30.0])
2target = torch.tensor([12.0, 18.0, 29.0])

Calculate:

• MSE loss
• L1 loss
• Huber loss

Compare the results.

Exercise 2: Binary Classification

Create a binary classifier:

pythonCopy
1model = nn.Linear(10, 1)

Use:

pythonCopy
1nn.BCEWithLogitsLoss()

Generate a batch of 16 random inputs and binary targets.

Calculate the loss.

Exercise 3: Multi-Class Classification

Create a classifier with:

textCopy
1Input features = 20
2Classes = 4

Use:

pythonCopy
1nn.CrossEntropyLoss()

Generate a batch of 32 samples and calculate the loss.

Exercise 4: Compare BCE and BCEWithLogitsLoss

Implement both:

pythonCopy
1nn.BCELoss()

and:

pythonCopy
1nn.BCEWithLogitsLoss()

Compare their usage and explain why BCEWithLogitsLoss is generally preferred for training.

Exercise 5: Training Loop

Create a linear regression model and train it for 100 epochs.

Your training loop should contain:

pythonCopy
1optimizer.zero_grad()
2outputs = model(inputs)
3loss = criterion(outputs, targets)
4loss.backward()
5optimizer.step()

Print the loss every 10 epochs.

Exercise 6: Inspect Individual Losses

Use:

pythonCopy
1reduction="none"

with MSELoss.

Print the individual loss for every sample and calculate the mean manually.

Module Summary

In this module, you learned:

• What a loss function is and why it is essential for neural network training.
• How loss connects model predictions with backpropagation.
• How MSELoss, L1Loss, and HuberLoss are used for regression.
• How CrossEntropyLoss is used for multi-class classification.
• How BCELoss works with probability outputs.
• Why BCEWithLogitsLoss is generally preferred for binary classification.
• How NLLLoss works with LogSoftmax.
• The difference between logits and probabilities.
• How reduction="mean", "sum", and "none" affect loss computation.
• How class weighting can help with imbalanced datasets.
• How loss functions fit into a complete PyTorch training loop.
• How to select a loss function based on the machine learning task.

Key Takeaways

The most important combinations to remember are:

textCopy
1Regression
2    ↓
3MSELoss / L1Loss / HuberLoss

textCopy
1Binary Classification
2    ↓
3Raw Logits
4    ↓
5BCEWithLogitsLoss

textCopy
1Multi-Class Classification
2    ↓
3Raw Logits
4    ↓
5CrossEntropyLoss

textCopy
1LogSoftmax Output
2    ↓
3NLLLoss

A correct understanding of loss functions, logits, targets, gradients, and optimization is essential before moving deeper into PyTorch model training.