Shell Scripting Process Management: Control & Monitor Linux

Shell Scripting Process Management: Control & Monitor Linux

Every command you run in a shell creates a process. Understanding how to start, monitor, control, and terminate these processes is essential for building robust automation scripts. Whether you're running background tasks, handling timeouts, cleaning up temporary files, or managing service lifecycles, process management is what separates fragile scripts from production-grade tools.

In this guide, we'll cover running commands in the background, capturing output, controlling processes with signals, using trap for cleanup, and managing jobs interactively.

Running Commands

Not every command needs to block your terminal. Shell scripting provides several ways to execute commands — synchronously, asynchronously, with timeouts, and with output capture.

Running Commands in the Background

Append an ampersand & to run a command in the background, freeing your terminal immediately:

bashCopy
1#!/bin/bash
2
3echo "Starting long-running task..."
4long_running_task &
5pid=$!
6
7echo "Task started with PID: $pid"
8echo "You can continue using the terminal..."
9
10# Wait for the background job to finish
11wait $pid
12echo "Task completed with exit code: $?"

What happens:

• & forks the command into a background process
• $! captures the Process ID (PID) of the last background command
• wait $pid pauses script execution until that process finishes
• $? contains the exit status after wait completes

Practical Example: Parallel Downloads

bashCopy
1#!/bin/bash
2
3urls=(
4    "https://example.com/file1.zip"
5    "https://example.com/file2.zip"
6    "https://example.com/file3.zip"
7)
8
9pids=()
10
11for url in "${urls[@]}"; do
12    filename=$(basename "$url")
13    echo "Starting download: $filename"
14    curl -s -o "$filename" "$url" &
15    pids+=($!)
16done
17
18echo "Waiting for all downloads to complete..."
19for pid in "${pids[@]}"; do
20    wait $pid
21done
22
23echo "All downloads finished!"

Timeout: Prevent Hanging Scripts

The timeout command kills a process if it runs longer than a specified duration:

bashCopy
1#!/bin/bash
2
3# Kill the script if it runs longer than 30 seconds
4timeout 30 ./script.sh
5
6# Check if timeout occurred
7if [ $? -eq 124 ]; then
8    echo "Error: Script timed out after 30 seconds"
9fi

Timeout exit codes:

• 0 — Command completed successfully
• 124 — Timeout was reached
• 125 — The timeout command itself failed
• 126 — Command was found but could not be executed
• 127 — Command was not found

Capturing Command Output

Store both stdout and stderr in a variable for later processing:

bashCopy
1#!/bin/bash
2
3# Capture both stdout and stderr
4output=$(command 2>&1)
5
6# Check if command succeeded
7if command; then
8    echo "Success!"
9else
10    echo "Failed with exit code: $?"
11fi

Conditional Execution Based on Exit Status

bashCopy
1#!/bin/bash
2
3if ping -c 1 google.com &>/dev/null; then
4    echo "Internet connection is active."
5else
6    echo "No internet connection detected."
7fi

Process Control

Once processes are running, you need tools to inspect, signal, and manage them.

List Running Processes

bashCopy
1#!/bin/bash
2
3# Show all processes
4ps aux
5
6# Show processes for current user
7ps ux
8
9# Show process tree
10pstree
11
12# Show real-time process monitor
13top

Find Process IDs with pgrep

Instead of parsing ps output with grep, use pgrep:

bashCopy
1#!/bin/bash
2
3# Find PID by process name
4pgrep nginx
5
6# Find PID with full command line match
7pgrep -f "python app.py"
8
9# Show process name and PID
10pgrep -l nginx

Kill Processes

bashCopy
1#!/bin/bash
2
3# Graceful termination (allows cleanup)
4kill PID
5
6# Force kill (immediate, no cleanup)
7kill -9 PID
8
9# Kill by name
10pkill process_name
11
12# Kill all matching processes
13killall nginx

Signal Reference:

Wait for Processes

The wait command pauses script execution until background jobs complete:

bashCopy
1#!/bin/bash
2
3./backup_db.sh &
4./backup_files.sh &
5
6echo "Waiting for all backups to finish..."
7wait
8
9echo "All backups completed at $(date)"

You can also wait for specific PIDs:

bashCopy
1#!/bin/bash
2
3./task1.sh &
4pid1=$!
5
6./task2.sh &
7pid2=$!
8
9wait $pid1
10echo "Task 1 done"
11
12wait $pid2
13echo "Task 2 done"

Trapping Signals with trap

The trap command catches signals and runs cleanup code before your script exits. This is critical for removing temporary files, releasing locks, or closing database connections.

bashCopy
1#!/bin/bash
2
3function cleanup() {
4    echo "Cleaning up temporary files..."
5    rm -f temp.*
6    rm -f /tmp/script_*.lock
7}
8
9# Run cleanup on script exit, interrupt, or termination
10trap cleanup EXIT INT TERM
11
12echo "Creating temporary files..."
13touch temp.data
14touch temp.log
15
16# Simulate work
17sleep 10
18
19echo "Work completed normally."

Common Trap Signals:

Practical Example: Safe Temporary File Handler

bashCopy
1#!/bin/bash
2
3set -euo pipefail
4
5TEMP_DIR=$(mktemp -d)
6echo "Using temp directory: $TEMP_DIR"
7
8cleanup() {
9    local exit_code=$?
10    echo "Cleaning up: $TEMP_DIR"
11    rm -rf "$TEMP_DIR"
12    exit $exit_code
13}
14
15trap cleanup EXIT INT TERM
16
17# Your script logic here
18echo "Processing data..." > "$TEMP_DIR/processing.log"
19sleep 2
20
21echo "Done! Temp files will be cleaned up automatically."

Best Practice: Always pair temporary file creation with a trap cleanup EXIT. This guarantees cleanup even if the script fails or is interrupted.

Job Control

Job control lets you manage multiple processes within a single shell session — moving them between foreground and background, checking their status, and terminating them.

Running Jobs in the Background

bashCopy
1#!/bin/bash
2
3# Start a job in the background
4./script.sh &
5
6# Check background job status
7jobs
8
9# Output:
10# [1]+  Running                 ./script.sh &

Bring Background Job to Foreground

bashCopy
1#!/bin/bash
2
3# Bring job [1] to foreground
4fg %1

Now the job runs interactively in your terminal. Press Ctrl+Z to suspend it.

Resume Suspended Job in Background

bashCopy
1#!/bin/bash
2
3# Resume job [1] in the background
4bg %1

Kill a Background Job

bashCopy
1#!/bin/bash
2
3# Kill job [1] by job number
4kill %1

Job Control Reference

Real-World Example: Robust Task Runner

Here's a production-style script that combines background processes, timeouts, traps, and error handling:

bashCopy
1#!/bin/bash
2
3set -euo pipefail
4
5LOG_DIR="/var/log/myapp"
6TIMEOUT_SECONDS=300
7FAILED_TASKS=()
8
9# ─── Setup ───
10mkdir -p "$LOG_DIR"
11TEMP_DIR=$(mktemp -d)
12trap 'rm -rf "$TEMP_DIR"; echo "Cleanup complete."' EXIT INT TERM
13
14# ─── Task Functions ───
15
16run_task() {
17    local name=$1
18    shift
19    local logfile="$LOG_DIR/${name}_$(date +%s).log"
20    
21    echo "Starting: $name"
22    
23    if timeout "$TIMEOUT_SECONDS" "$@" > "$logfile" 2>&1; then
24        echo "  ✓ $name completed"
25        return 0
26    else
27        local exit_code=$?
28        echo "  ✗ $name failed (exit: $exit_code)"
29        FAILED_TASKS+=("$name")
30        return 1
31    fi
32}
33
34# ─── Main ───
35
36echo "=== Task Runner Started ==="
37echo "Timeout per task: ${TIMEOUT_SECONDS}s"
38echo ""
39
40# Run tasks in parallel
41run_task "database_backup" ./scripts/backup_db.sh &
42pid_db=$!
43
44run_task "file_sync" ./scripts/sync_files.sh &
45pid_sync=$!
46
47run_task "report_generation" ./scripts/generate_report.sh &
48pid_report=$!
49
50# Wait for all tasks
51echo ""
52echo "Waiting for all tasks to complete..."
53wait $pid_db
54wait $pid_sync
55wait $pid_report
56
57# ─── Summary ───
58echo ""
59echo "=== Summary ==="
60
61if [ ${#FAILED_TASKS[@]} -eq 0 ]; then
62    echo "All tasks completed successfully."
63    exit 0
64else
65    echo "Failed tasks: ${FAILED_TASKS[*]}"
66    exit 1
67fi

Best Practices for Process Management

1. Always capture $! immediately after backgrounding a command — other commands will overwrite it
2. Use timeout for any network or I/O operation that could hang
3. Trap EXIT for cleanup — it's the most reliable way to ensure resources are freed
4. Prefer kill -15 (SIGTERM) over kill -9 (SIGKILL) — give processes a chance to clean up
5. Use pgrep and pkill instead of ps | grep | awk — they're designed for this purpose
6. Check wait exit status — background processes can fail too
7. Quote PIDs and job numbers when passing them around
8. Use mktemp -d for temporary directories instead of hardcoded /tmp paths
9. Never ignore SIGKILL — it's uncatchable by design, use it only as last resort
10. Log process IDs in long-running scripts for easier debugging and monitoring

Quick Reference

bashCopy
1# Background execution
2command &                         # Run in background
3pid=$!                            # Get background PID
4wait $pid                         # Wait for specific process
5wait                              # Wait for all background jobs
6
7# Timeout
8timeout 30 command                # Kill after 30 seconds
9timeout -k 5 30 command           # Kill with SIGKILL after 5s grace
10
11# Process info
12ps aux                            # All processes
13pgrep -f pattern                  # Find PID by pattern
14pkill process_name                # Kill by name
15
16# Signals
17kill PID                          # SIGTERM (graceful)
18kill -9 PID                       # SIGKILL (force)
19kill -HUP PID                     # SIGHUP (reload)
20
21# Traps
22trap 'cleanup' EXIT               # Run on script exit
23trap 'cleanup' INT TERM         # Run on Ctrl+C or kill
24trap - INT                        # Reset INT handler
25
26# Job control
27jobs                              # List background jobs
28fg %1                             # Foreground job 1
29bg %1                             # Background job 1
30kill %1                           # Kill job 1

Process management is what makes shell scripts suitable for real-world automation. By understanding how to run tasks in parallel, handle timeouts, clean up resources, and respond to signals, you can build scripts that are resilient, efficient, and safe to run in production environments.

Next up: Error handling and debugging — learn how to write scripts that fail gracefully and are easy to troubleshoot.