Shell Scripting String Manipulation: Bash Text Processing Guide

Text is the native language of the command line. Whether you're parsing filenames, extracting data from logs, sanitizing user input, or generating reports, string manipulation is the skill that turns raw text into actionable information.

Bash provides powerful parameter expansion syntax for manipulating strings without needing external tools like sed, awk, or cut. These built-in operations are faster, more portable, and essential for writing efficient scripts.

In this guide, we'll cover every major string operation in Bash — from basic length checks to advanced pattern matching.

String Length

The simplest string operation is getting the length, using the ${#var} syntax.

bash
1#!/bin/bash
2
3str="Hello, World!"
4echo "Length: ${#str}"

Output:

Length: 13

Practical Use: Input Validation

bash
1#!/bin/bash
2
3password="Secure123"
4
5if [ ${#password} -lt 8 ]; then
6    echo "Error: Password must be at least 8 characters."
7    exit 1
8fi
9
10echo "Password length is acceptable: ${#password} characters"

Substring Extraction

Bash can extract portions of a string using ${var:start:length}.

bash
1#!/bin/bash
2
3str="Shell Scripting"
4
5# Extract from index 0, length 5
6echo "${str:0:5}"      # Shell
7
8# Extract from index 6, length 9
9echo "${str:6:9}"      # Scripting
10
11# Extract from index 6 to end
12echo "${str:6}"        # Scripting
13
14# Negative index (counts from end): last 9 characters
15echo "${str: -9}"      # Scripting
16
17# Negative index with length
18echo "${str: -9:4}"    # Scri

Important: When using negative indices, there must be a space between the colon and the minus sign: ${str: -5}. Without the space, Bash interprets it as a default value operator.

Practical Example: Extract Date Components

bash
1#!/bin/bash
2
3timestamp="20260816"
4year="${timestamp:0:4}"
5month="${timestamp:4:2}"
6day="${timestamp:6:2}"
7
8echo "$year-$month-$day"    # 2026-08-16

Substring Removal (Prefix/Suffix Stripping)

One of the most powerful features of Bash parameter expansion is the ability to remove patterns from the beginning or end of strings.

Remove Shortest Match from Beginning (`#`)

bash
1#!/bin/bash
2
3path="/home/user/documents/file.txt"
4echo "${path#*/}"        # home/user/documents/file.txt

Remove Longest Match from Beginning (`##`)

bash
1#!/bin/bash
2
3path="/home/user/documents/file.txt"
4echo "${path##*/}"       # file.txt

Remove Shortest Match from End (`%`)

bash
1#!/bin/bash
2
3path="/home/user/documents/file.txt"
4echo "${path%/*}"        # /home/user/documents

Remove Longest Match from End (`%%`)

bash
1#!/bin/bash
2
3path="/home/user/documents/file.txt"
4echo "${path%%/*}"       # (empty — removes everything up to last /)

Visual Reference

Syntax	Operation	Example
`${var#pattern}`	Remove shortest match from start	`"${path#*/}"`
`${var##pattern}`	Remove longest match from start	`"${path##*/}"`
`${var%pattern}`	Remove shortest match from end	`"${path%/*}"`
`${var%%pattern}`	Remove longest match from end	`"${path%%.*}"`

Practical Example: Filename Parser

bash
1#!/bin/bash
2
3filepath="/var/www/html/index.html"
4
5filename="${filepath##*/}"           # index.html
6directory="${filepath%/*}"           # /var/www/html
7extension="${filename##*.}"          # html
8basename="${filename%.*}"            # index
9
10echo "Path: $filepath"
11echo "Directory: $directory"
12echo "Filename: $filename"
13echo "Basename: $basename"
14echo "Extension: $extension"

Output:

Path: /var/www/html/index.html
Directory: /var/www/html
Filename: index.html
Basename: index
Extension: html

Search and Replace

Bash can replace substrings using parameter expansion — no need to pipe through sed for simple replacements.

Replace First Occurrence

bash
1#!/bin/bash
2
3str="foo bar foo baz"
4echo "${str/foo/FOO}"        # FOO bar foo baz

Replace All Occurrences

bash
1#!/bin/bash
2
3str="foo bar foo baz"
4echo "${str//foo/FOO}"       # FOO bar FOO baz

Replace from Beginning

bash
1#!/bin/bash
2
3str="foo bar foo baz"
4echo "${str/#foo/START}"     # START bar foo baz

Replace from End

bash
1#!/bin/bash
2
3str="foo bar foo baz"
4echo "${str/%baz/END}"       # foo bar foo END

Delete Substrings (Replace with Nothing)

bash
1#!/bin/bash
2
3str="foo-123-bar-456-baz"
4echo "${str//[0-9]/}"        # foo--bar--baz
5echo "${str//-/}"            # foo123bar456baz

Practical Example: Sanitize Input

bash
1#!/bin/bash
2
3filename="My Document v1.0 (Final).pdf"
4
5# Replace spaces with underscores
6safe_name="${filename// /_}"
7
8# Remove parentheses
9safe_name="${safe_name//[()]/}"
10
11echo "Original: $filename"
12echo "Safe: $safe_name"

Case Conversion

Bash 4.0+ provides built-in operators for changing case. For older versions, use tr as a fallback.

Uppercase

bash
1#!/bin/bash
2
3str="hello world"
4echo "${str^^}"              # HELLO WORLD
5
6# Uppercase first letter only
7echo "${str^}"               # Hello world

Lowercase

bash
1#!/bin/bash
2
3str="HELLO WORLD"
4echo "${str,,}"              # hello world
5
6# Lowercase first letter only
7echo "${str,}"               # hELLO WORLD

Toggle Case

bash
1#!/bin/bash
2
3str="Hello World"
4echo "${str~~}"              # hELLO wORLD

Convert Specific Characters

bash
1#!/bin/bash
2
3str="hello world"
4echo "${str^^[aeiou]}"       # hEllO wOrld (only vowels uppercase)

Fallback for Bash 3.x (macOS)

bash
1#!/bin/bash
2
3str="hello world"
4echo "$str" | tr '[:lower:]' '[:upper:]'    # HELLO WORLD

String Concatenation

Concatenating strings in Bash is straightforward — just place them adjacent to each other.

bash
1#!/bin/bash
2
3greeting="Hello"
4name="World"
5
6# Method 1: Adjacent variables
7message="$greeting, $name!"
8echo "$message"
9
10# Method 2: Literal concatenation
11prefix="file"
12suffix=".txt"
13filename="${prefix}${suffix}"
14echo "$filename"
15
16# Method 3: Append to existing variable
17log="Error occurred"
18log="$log at $(date)"
19echo "$log"

Building Paths

bash
1#!/bin/bash
2
3base="/home/user"
4subdir="documents"
5filename="report.txt"
6
7fullpath="${base}/${subdir}/${filename}"
8echo "$fullpath"              # /home/user/documents/report.txt

String Comparison

While not strictly "manipulation," comparing strings is fundamental to text processing.

bash
1#!/bin/bash
2
3str1="hello"
4str2="world"
5
6# Equality
7if [ "$str1" = "$str2" ]; then
8    echo "Strings are equal"
9else
10    echo "Strings are different"
11fi
12
13# Using [[ ]] for pattern matching
14if [[ "$str1" == h* ]]; then
15    echo "Starts with 'h'"
16fi
17
18# Check if empty
19if [ -z "$str1" ]; then
20    echo "String is empty"
21fi
22
23# Check if non-empty
24if [ -n "$str1" ]; then
25    echo "String is not empty"
26fi

Pattern Matching with `[[ ]]`

The double-bracket [[ ]] test supports wildcard and regex pattern matching on strings.

Wildcard Matching

bash
1#!/bin/bash
2
3filename="report.pdf"
4
5if [[ "$filename" == *.pdf ]]; then
6    echo "This is a PDF file."
7fi
8
9if [[ "$filename" == report.* ]]; then
10    echo "This is a report file."
11fi

Extracting String Components

bash
1#!/bin/bash
2
3name="document.tar.gz"
4
5# Extract extension
6ext="${name##*.}"
7echo "Extension: $ext"        # gz
8
9# Extract everything before last dot
10base="${name%.*}"
11echo "Base: $base"            # document.tar
12
13# Extract everything before first dot
14first="${name%%.*}"
15echo "First: $first"          # document

Parameter Expansion Reference Table

Expression	Description	Example
`${#var}`	String length	`"${#str}"` → `13`
`${var:offset}`	Substring from offset	`"${str:3}"`
`${var:offset:length}`	Substring with length	`"${str:0:5}"`
`${var#pattern}`	Remove shortest prefix	`"${path#*/}"`
`${var##pattern}`	Remove longest prefix	`"${path##*/}"`
`${var%pattern}`	Remove shortest suffix	`"${path%/*}"`
`${var%%pattern}`	Remove longest suffix	`"${path%%/*}"`
`${var/pattern/repl}`	Replace first match	`"${str/foo/bar}"`
`${var//pattern/repl}`	Replace all matches

Real-World Examples

Log Filename Generator

bash
1#!/bin/bash
2
3app_name="myapp"
4timestamp=$(date +%Y%m%d_%H%M%S)
5log_file="${app_name}_${timestamp}.log"
6
7echo "Logging to: $log_file"

URL Parser

bash
1#!/bin/bash
2
3url="https://user:pass@api.example.com:8080/v1/users?id=123#section"
4
5# Remove protocol
6no_protocol="${url#*://}"
7echo "Without protocol: $no_protocol"
8
9# Remove path and beyond
10domain_port="${no_protocol%%/*}"
11echo "Domain:Port: $domain_port"
12
13# Remove credentials and port
14domain="${domain_port#*@}"
15domain="${domain%:*}"
16echo "Domain: $domain"

Bulk Rename Script

bash
1#!/bin/bash
2
3for file in *.JPEG *.jpeg; do
4    [ -f "$file" ] || continue
5    
6    # Convert to lowercase extension
7    newname="${file%.[Jj][Pp][Ee][Gg]}.jpg"
8    
9    if [ "$file" != "$newname" ]; then
10        mv "$file" "$newname"
11        echo "Renamed: $file → $newname"
12    fi
13done

Environment Variable Sanitizer

bash
1#!/bin/bash
2
3# Ensure APP_NAME contains only lowercase letters and hyphens
4raw_name="My App 2.0 (Beta)"
5
6# Remove everything except alphanumeric and spaces
7clean="${raw_name//[^a-zA-Z0-9 ]/}"
8
9# Replace spaces with hyphens
10clean="${clean// /-}"
11
12# Lowercase
13clean="${clean,,}"
14
15echo "Original: $raw_name"
16echo "Sanitized: $clean"

Best Practices for String Manipulation

Always quote variables — especially when they contain spaces or special characters
Use ${var} syntax — curly braces prevent ambiguity: "${var}suffix" vs "$varsuffix"
Prefer parameter expansion over external tools — ${var//pattern/} is faster than echo "$var" | sed
Use [[ ]] for pattern matching — it's safer than [ ] and supports wildcards
Check string length before processing — empty strings can cause unexpected behavior
Be careful with greedy vs non-greedy matching — # vs ##, % vs %%
Test with edge cases — empty strings, strings with spaces, and special characters

Quick Reference

bash
1# Length
2${#var}
3
4# Extraction
5${var:3}          # From index 3 to end
6${var:3:5}        # From index 3, length 5
7${var: -5}        # Last 5 characters
8
9# Removal
10${var#*/}         # Remove shortest prefix
11${var##*/}        # Remove longest prefix
12${var%/*}         # Remove shortest suffix
13${var%%/*}        # Remove longest suffix
14
15# Replace
16${var/old/new}    # First occurrence
17${var//old/new}   # All occurrences
18${var/#old/new}   # Start only
19${var/%old/new}   # End only
20
21# Case
22${var^}           # First char uppercase
23${var^^}          # All uppercase
24${var,}           # First char lowercase
25${var,,}          # All lowercase
26
27# Defaults
28${var:-default}   # Default if unset/empty
29${var:=default}   # Set and return default
30${var:?error}     # Error if unset/empty
31${var:+alt}       # Alternate if set

String manipulation is where Bash truly shines as a text-processing tool. By mastering parameter expansion, you can handle most string operations without spawning external processes, making your scripts faster and more portable. Combined with arrays and loops from previous chapters, you now have the toolkit to process and transform data at scale.

Next up: File handling and I/O operations — reading, writing, and managing files like a pro.