Django Middleware & Performance: Optimize Your App

Module 13: Django Middleware, Signals & Performance

🎯 What You Will Learn in This Module

By the end of this tutorial, you will:

• Understand the request/response middleware pipeline and how to customize it
• Build custom middleware for timing, logging, and API versioning
• Use Django signals to trigger actions when models change
• Master query optimization to eliminate the N+1 problem
• Implement low-level and per-view caching to speed up your application
• Use select_related, prefetch_related, annotate, and bulk_create
• Set up Redis caching for production-grade performance

📖 1. What is Middleware?

Middleware is a framework of hooks into Django's request/response processing. It's a lightweight, low-level plugin system for globally altering Django's input or output.

Think of middleware as airport security checkpoints. Every request (incoming) and every response (outgoing) must pass through each checkpoint in order.

Request → [CORS] → [Security] → [Session] → [Common] → [CSRF] → [Auth] → View
                                                                    ↑
Response ← [Security] ← [Session] ← [Common] ← [Messages] ← [XFrame] ←

Django processes middleware in the order defined in settings.py during the request phase, and in reverse order during the response phase.

🛠️ 2. Building Custom Middleware

A middleware class needs two things: an __init__ method that accepts get_response, and a __call__ method that processes the request.

Request Timing Middleware

pythonCopy
1# core/middleware.py
2
3import time
4import logging
5from django.http import JsonResponse
6
7logger = logging.getLogger('django')
8
9
10class RequestTimingMiddleware:
11    """
12    Measures how long each request takes and logs it.
13    Adds an X-Request-Duration header to every response.
14    """
15    
16    def __init__(self, get_response):
17        # get_response is the next middleware or view in the chain
18        self.get_response = get_response
19    
20    def __call__(self, request):
21        # Code executed BEFORE the view (request phase)
22        start_time = time.time()
23        
24        # Pass the request to the next middleware/view
25        response = self.get_response(request)
26        
27        # Code executed AFTER the view (response phase)
28        duration = time.time() - start_time
29        
30        # Add custom header to the response
31        response['X-Request-Duration'] = f'{duration:.3f}s'
32        
33        # Log slow requests for monitoring
34        logger.info(f'{request.method} {request.path} - {duration:.3f}s')
35        
36        return response

API Version Middleware

pythonCopy
1# core/middleware.py
2
3class APIVersionMiddleware:
4    """
5    Reads the X-API-Version header from requests and attaches it
6    to the request object for views to access.
7    """
8    
9    def __init__(self, get_response):
10        self.get_response = get_response
11    
12    def __call__(self, request):
13        # Extract API version from header, default to 'v1'
14        version = request.headers.get('X-API-Version', 'v1')
15        
16        # Attach to request so views can use request.api_version
17        request.api_version = version
18        
19        return self.get_response(request)

Registering Middleware

Add your custom middleware to settings.py. Order matters!

pythonCopy
1# myproject/settings.py
2
3MIDDLEWARE = [
4    'core.middleware.RequestTimingMiddleware',      # Custom: timing
5    'corsheaders.middleware.CorsMiddleware',        # Must be near top
6    'django.middleware.security.SecurityMiddleware',
7    'whitenoise.middleware.WhiteNoiseMiddleware',
8    'django.contrib.sessions.middleware.SessionMiddleware',
9    'django.middleware.common.CommonMiddleware',
10    'django.middleware.csrf.CsrfViewMiddleware',
11    'django.contrib.auth.middleware.AuthenticationMiddleware',
12    'django.contrib.messages.middleware.MessageMiddleware',
13    'django.middleware.clickjacking.XFrameOptionsMiddleware',
14    'core.middleware.APIVersionMiddleware',         # Custom: versioning
15]

⚠️ Warning: Your custom timing middleware is at the top so it captures the total request time including all other middleware. The API version middleware is at the bottom (before the view) because it doesn't need to wrap around other middleware.

📡 3. Django Signals

Signals allow certain senders to notify a set of receivers that some action has taken place. They're especially useful when you need to trigger side effects (send an email, create a profile, clear a cache) whenever a model is saved or deleted.

Built-in Signals

Creating User Profiles Automatically

pythonCopy
1# users/signals.py
2
3from django.db.models.signals import post_save
4from django.dispatch import receiver
5from django.core.mail import send_mail
6from django.conf import settings
7
8from .models import CustomUser, Profile
9from blog.models import Post
10
11
12@receiver(post_save, sender=CustomUser)
13def create_user_profile(sender, instance, created, **kwargs):
14    """
15    Automatically creates a Profile and sends a welcome email
16    whenever a new user is created.
17    """
18    if created:
19        # Create the user's profile
20        Profile.objects.create(user=instance)
21        
22        # Send welcome email
23        send_mail(
24            subject='Welcome to Our Platform',
25            message=f'Hi {instance.first_name}, thanks for joining our community!',
26            from_email=settings.DEFAULT_FROM_EMAIL,
27            recipient_list=[instance.email],
28            fail_silently=False,
29        )
30
31
32@receiver(post_save, sender=CustomUser)
33def save_user_profile(sender, instance, **kwargs):
34    """
35    Ensures the profile is saved whenever the user is saved.
36    """
37    instance.profile.save()
38
39
40@receiver(post_save, sender=Post)
41def notify_subscribers(sender, instance, created, **kwargs):
42    """
43    Trigger notifications when a new post is published.
44    """
45    if created and instance.status == 'published':
46        # Implementation for notification system would go here
47        # e.g., send push notifications, emails to subscribers
48        pass

Connecting Signals in AppConfig

Signals won't work unless Django knows where to find them. Update your app's apps.py:

pythonCopy
1# users/apps.py
2
3from django.apps import AppConfig
4
5
6class UsersConfig(AppConfig):
7    default_auto_field = 'django.db.models.BigAutoField'
8    name = 'users'
9    
10    def ready(self):
11        """
12        Import signals when the app is ready.
13        This ensures signals are registered at startup.
14        """
15        import users.signals  # noqa: F401

💡 The ready() method is called once when Django starts. Importing signals here ensures they are connected before any requests are handled.

⚡ 4. Query Optimization: Defeating the N+1 Problem

The N+1 query problem is the most common performance killer in Django. It happens when you fetch N objects, then make N additional queries to access related data.

The N+1 Problem Explained

pythonCopy
1# THE BAD WAY (N+1 queries)
2posts = Post.objects.all()          # 1 query to fetch all posts
3for post in posts:
4    print(post.author.username)     # N queries (one per post!)
5# Total: N + 1 queries

If you have 100 posts, this executes 101 database queries. In production, this destroys performance.

The Fix: select_related

pythonCopy
1# THE GOOD WAY (1 query)
2posts = Post.objects.select_related('author', 'category').all()
3for post in posts:
4    print(post.author.username)     # No extra queries!
5# Total: 1 query

select_related performs a SQL JOIN and fetches related ForeignKey/OneToOne data in the same query.

When to use: ForeignKey, OneToOneField

The Fix: prefetch_related

pythonCopy
1# THE GOOD WAY (2 queries)
2posts = Post.objects.prefetch_related('tags', 'comments').all()
3for post in posts:
4    for tag in post.tags.all():
5        print(tag.name)             # Already fetched!
6# Total: 2 queries (one for posts, one for tags)

prefetch_related performs a separate query for the related data and joins it in Python. It's necessary for ManyToManyField and reverse ForeignKey relationships.

When to use: ManyToManyField, reverse ForeignKey

Other Query Optimization Techniques

pythonCopy
1from django.db.models import Count, F
2
3# annotate: Add calculated fields without extra queries
4posts = Post.objects.annotate(comment_count=Count('comments'))
5
6# only: Fetch only specific fields (good for list views)
7posts = Post.objects.only('title', 'slug', 'published_at')
8
9# defer: Skip heavy fields (good when you don't need content)
10posts = Post.objects.defer('content')  # Excludes large text fields
11
12# exists: Fast boolean check (no object instantiation)
13if Post.objects.filter(status='draft').exists():
14    pass
15
16# count: Fast counting (database does it, not Python)
17count = Post.objects.filter(status='published').count()
18
19# bulk_create: Insert many objects in a single query
20Post.objects.bulk_create([
21    Post(title=f'Post {i}') for i in range(100)
22])
23
24# update: Update many records without fetching them
25Post.objects.filter(status='draft').update(status='archived')
26
27# F() expressions: Database-level operations without race conditions
28Post.objects.filter(pk=post.pk).update(view_count=F('view_count') + 1)

Optimization Cheat Sheet

💾 5. Caching in Django

Caching stores expensive-to-compute results so they don't need to be recalculated on every request.

Per-View Caching

pythonCopy
1from django.views.decorators.cache import cache_page
2from django.utils.decorators import method_decorator
3
4
5# Cache this view for 15 minutes (900 seconds)
6@cache_page(60 * 15)
7def post_list(request):
8    posts = Post.objects.all()
9    return render(request, 'blog/post_list.html', {'posts': posts})
10
11
12# For Class-Based Views
13@method_decorator(cache_page(60 * 15), name='get')
14class PostListView(ListView):
15    model = Post
16    template_name = 'blog/post_list.html'

Low-Level Caching

For more control, use Django's cache API directly:

pythonCopy
1from django.core.cache import cache
2
3
4def get_popular_posts():
5    """
6    Fetches popular posts from cache if available,
7    otherwise queries the database and stores result.
8    """
9    cache_key = 'popular_posts'
10    posts = cache.get(cache_key)
11    
12    if posts is None:
13        # Cache miss: fetch from database
14        posts = Post.objects.order_by('-view_count')[:10]
15        
16        # Store in cache for 30 minutes (60 * 30 seconds)
17        cache.set(cache_key, posts, 60 * 30)
18    
19    return posts
20
21
22def invalidate_post_cache(post_id):
23    """
24    Call this whenever a post is updated to ensure
25    stale data is removed from cache.
26    """
27    cache.delete(f'post_{post_id}')
28    cache.delete('popular_posts')

Cache Backends

Django supports multiple cache backends:

🔴 6. Setting Up Redis Caching

Redis is the industry-standard caching backend for Django production applications.

Installation

bashCopy
1pip install django-redis

Configuration

pythonCopy
1# myproject/settings.py
2
3CACHES = {
4    'default': {
5        'BACKEND': 'django_redis.cache.RedisCache',
6        'LOCATION': 'redis://127.0.0.1:6379/1',
7        'OPTIONS': {
8            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
9        }
10    }
11}
12
13# Optional: Use Redis for session storage too
14SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
15SESSION_CACHE_ALIAS = 'default'

Using Redis Cache

Once configured, the cache API works exactly the same — Django handles the backend transparently:

pythonCopy
1from django.core.cache import cache
2
3# This now stores data in Redis instead of local memory
4cache.set('my_key', 'my_value', 60 * 15)
5value = cache.get('my_key')

Why Redis?

• Persistent: Survives server restarts (optional)
• Distributed: Multiple Django servers can share the same cache
• Fast: In-memory storage with sub-millisecond response times
• Scalable: Handles millions of operations per second

🧪 7. Practice Task — Implement Redis Caching for API Responses

Task Requirements

Create a caching system for your blog API that:

1. Caches the post list API response for 5 minutes
2. Invalidates the cache when a post is created, updated, or deleted
3. Uses a signal to automatically clear cache on post save

Solution

pythonCopy
1# blog/signals.py
2
3from django.db.models.signals import post_save, post_delete
4from django.dispatch import receiver
5from django.core.cache import cache
6from .models import Post
7
8
9@receiver(post_save, sender=Post)
10@receiver(post_delete, sender=Post)
11def invalidate_post_cache(sender, instance, **kwargs):
12    """
13    Clear post-related caches whenever a post is saved or deleted.
14    """
15    cache.delete('api_post_list')
16    cache.delete(f'api_post_detail_{instance.slug}')
17
18
19# Don't forget to import in apps.py!

pythonCopy
1# api/views.py
2
3from django.core.cache import cache
4from rest_framework.response import Response
5from rest_framework import generics
6
7
8class PostListCreateView(generics.ListCreateAPIView):
9    # ... existing configuration ...
10    
11    def list(self, request, *args, **kwargs):
12        """
13        Override list to add caching.
14        """
15        cache_key = 'api_post_list'
16        cached_data = cache.get(cache_key)
17        
18        if cached_data:
19            return Response(cached_data)
20        
21        # If not cached, proceed with normal list logic
22        response = super().list(request, *args, **kwargs)
23        
24        # Cache the response data for 5 minutes
25        cache.set(cache_key, response.data, 60 * 5)
26        
27        return response

❌ Common Mistakes & How to Fix Them

✅ Module 13 Summary

🚪 What's Next?

In Module 14, we will master Testing in Django. You'll learn to write unit tests for models, views, and API endpoints using Django's TestCase and DRF's APITestCase. We'll cover test databases, factory data, mocking email sending, and achieving high test coverage for your blog application.

Before proceeding, make sure:

1. You created at least one custom middleware class
2. Your middleware is registered in settings.py and working
3. You created signals that trigger on user/post save
4. You understand the difference between select_related and prefetch_related
5. You can identify and fix an N+1 query problem
6. You implemented Redis caching configuration in settings.py
7. You understand when to use cache_page vs low-level cache.set

Your Django application is now optimized for speed and efficiency. Ready to write bulletproof tests? 🧪🚀