Bash: Utility Commands Reference

Table of Contents

• File Permissions
  • Permission Values
  • Permission Structure: XYZ
  • Common Examples
• Time Command
• Compression and Decompression
  • tar (Standard Archive Tool)
  • pigz (Parallel gzip)
  • zstd (Zstandard - Fast Compression)
• Symbolic Links
• Mass File Deletion
• Find and Replace Across Files
• Process Management
• Postgres Commands
• Remote Backups with rsync
• Iperf3 usage
• RPM Package Management
• lsof (List Open Files)
• Git Remote Branch Cleanup
• XML Formatting

File Permissions

Unix file permissions use a three-digit octal notation (e.g., 755, 644) to control access.

Permission Values

Number	Permissions	Meaning
7	rwx	Read (4) + Write (2) + Execute (1)
6	rw-	Read (4) + Write (2)
5	r-x	Read (4) + Execute (1)
4	r—	Read only
3	-wx	Write (2) + Execute (1)
2	-w-	Write only
1	—x	Execute only
0	---	No permissions

Permission Structure: XYZ

Each permission number has three digits representing different user categories:

• X = File owner (user)
• Y = Group owner
• Z = Everyone else (other)

Common Examples

read/write/execute

# 755 = rwxr-xr-x
# Group: read/execute
# Others: read/execute
chmod 755 script.sh

# 644 = rw-r--r--
# Owner: read/write
# Group: read only
# Others: read only
chmod 644 file.txt

# 775 = rwxrwxr-x
# Owner: read/write/execute
# Group: read/write/execute
# Others: read/execute
chmod 775 shared_script.sh

Time Command

The time command measures how long a command takes to execute.

time python script.py

Output breakdown:

• real: Wall clock time (total elapsed time)
• user: CPU time spent executing user-level code
• sys: CPU time spent in kernel mode (system calls)

Compression and Decompression

Different compression tools offer tradeoffs between speed, compression ratio, and compatibility.

tar (Standard Archive Tool)

Create archives:

# Create uncompressed tar archive (verbose output)
tar --create --file /tmp/archive.tar --verbose /tmp/archive/
tar -cvf archive.tar directory/

# Create with specific compression
tar -czvf archive.tar.gz directory/     # gzip
tar -cjvf archive.tar.bz2 directory/    # bzip2
tar -cJvf archive.tar.xz directory/     # xz

# Use multi-threaded xz compression
tar -cvf archive.tar.xz -I 'xz -T 0' ./files/*

List archive contents:

tar -tzf archive.tar.gz     # gzip
tar -tjf archive.tar.bz2    # bzip2
tar -tJf archive.tar.xz     # xz

Extract archives:

# Auto-detects compression type
tar -xf archive.tar.gz

# Explicitly specify compression
tar -xzf archive.tar.gz     # gzip
tar -xjf archive.tar.bz2    # bzip2
tar -xJf archive.tar.xz     # xz

# Extract to specific directory
tar -xzf archive.tar.gz -C /destination/path/

pigz (Parallel gzip)

Much faster than standard gzip on multi-core systems.

Compress:

# Use pigz with tar for parallel compression
tar --use-compress-program=pigz -cf archive.tar.gz bin/*

# Or pipe to pigz directly
tar -cf - large_directory/ | pigz > archive.tar.gz

Decompress:

# Decompress and extract (benchmark: ~3 minutes)
pigz -dc archive.tar.gz | tar -xf - -C /destination/

# Alternative syntax
pigz -dc archive.tar.gz | tar --directory=/tmp/cpp -x -f -

zstd (Zstandard - Fast Compression)

Excellent compression ratio with very fast decompression. Recommended for large archives.

Compress:

# Fast compression with multi-threading (benchmark: ~2m 5s)
tar -cf - cpp/ | zstd --long -T0 -3 > archive.tar.zst

# Default compression (benchmark: ~4m 30s, better ratio)
tar -cf - cpp/ | zstd > archive.tar.zst

# Compression levels: -1 (fast) to -19 (slow, best compression)
tar -cf - data/ | zstd -10 -T0 > archive.tar.zst

Decompress:

# Very fast decompression (benchmark: ~1m 20s, 3x faster than compression)
zstd -d -c archive.tar.zst | tar -xf - -C /destination/

# Single-threaded decompression (multi-threading doesn't help much)
zstd -d archive.tar.zst

Compression Comparison:

Tool	Speed	Ratio	Use Case
gzip	Medium	Medium	Universal compatibility
pigz	Fast	Medium	Multi-core systems, fast compress
bzip2	Slow	Good	Better compression, slower
xz	Very Slow	Best	Maximum compression
zstd	Fast	Good	Modern balance of speed and ratio

Symbolic Links

Create symlinks to reference files or directories from another location.

# Read where a symlink points
readlink /tmp/mysymlink

# Create absolute symlink (recommended - less error prone)
ln -s /tmp/target.a /tmp/symlink_to_target.a

# Create relative symlink (can break if moved)
ln -s ../relative/path/file.txt symlink.txt

# Update existing symlink (force overwrite)
ln -sf /new/target.a /tmp/symlink_to_target.a

Best practice: Use absolute paths for symlinks to avoid issues when the symlink or target is moved.

Mass File Deletion

Use find with -delete to avoid “Argument list too long” errors with wildcards.

# Avoid this - can fail with too many files
# rm -rf ./*.dat

# Better approach - preview files first
find . -name "*.rst" -print0 | xargs -0 echo

# Delete files matching pattern
find . -name "*.rst" -delete

# Delete files older than N days
find . -name "*.log" -mtime +30 -delete

# Delete with confirmation
find . -name "*.tmp" -exec rm -i {} +

Find and Replace Across Files

Use sed to replace text patterns across multiple files.

# Find and replace in all matching files
find /path/dir -type f -name "*.txt" -exec sed -i 's/oldpattern/newpattern/g' {} +

# macOS: Use gsed (GNU sed) for consistent behavior
# Install with: brew install gnu-sed
find /path/dir -type f -name "*.txt" -exec gsed -i 's/oldpattern/newpattern/g' {} +

# Create backups before modifying
find /path/dir -type f -name "*.txt" -exec sed -i.bak 's/old/new/g' {} +

Process Management

List processes:

# Find processes by name
ps aux | grep python

# Show PIDs only
ps aux | grep python | awk '{ print $2 }'

# Show detailed process tree
pstree -p

Kill processes:

# Kill specific PID
kill 12345

# Force kill if not responding
kill -9 12345

# Kill all matching processes by name
pkill -f python

# Alternative using killall
killall python

# Kill with pattern matching (regex)
killall -r 'python.*'

# Kill all processes of a user
pkill -u username

# Interactive process killing
htop  # Press F9 to kill selected process

Postgres Commands

View and reload HBA File

SHOW hba_file;
SHOW data_directory;
SELECT pg_reload_conf();

pg_dump and pg_restore examples

pg_dump -Z 6 -F c db_name > /tmp/db_dump.dump
pg_restore -F c ./db_dump.dump -d db_name -v

Remote Backups with rsync

Automated backup script:

# Define hosts to backup
hosts=("server1" "server2" "db-server")

for host in "${hosts[@]}"; do
    echo "Backing up $host..."
    ssh "$host" "rsync -avP ~/ archivehost:/backup/$host/home/"
done

Parallel backups (faster but uses more resources):

# Run backups in background (parallel execution)
HOST=server1; ssh "$HOST" "rsync -avP ~/ archivehost:/backup/$HOST/home/" &
HOST=server2; ssh "$HOST" "rsync -avP ~/ archivehost:/backup/$HOST/home/" &

# Wait for all background jobs to complete
wait
echo "All backups complete"

rsync flags explained:

• -a: Archive mode (preserves permissions, timestamps, symlinks)
• -v: Verbose output
• -P: Show progress and keep partially transferred files

Iperf3 usage

iperf3 -s -p 5201 # On server
iperf3 -c <server-ip> -p 5201 # On Client
iperf3 -c <server-ip> -p 5201 -u # udp
for port in {5201..5210}; do iperf3 -c <server-ip> -p $port; done # multiple ports

RPM Package Management

Query installed RPM packages on RedHat/CentOS/Fedora systems.

# List all installed packages
rpm -qa

# Search for specific package
rpm -qa | grep package_name

# Show files installed by a package
rpm -ql package_name

# Show which package owns a file
rpm -qf /path/to/file

# Show package information
rpm -qi package_name

lsof (List Open Files)

Identify which processes are using specific files, ports, or resources.

# See what's using a specific file or directory
lsof /path/to/file

# Show all files opened by a user
lsof -u username

# Show processes using a specific port
lsof -i :8080

# Kill process using a port
kill -9 $(lsof -t -i :5000)

# List all open files by a specific PID
lsof -p 12345

# Show all network connections
lsof -i

# Show TCP connections only
lsof -i TCP

# Show listening ports
lsof -i -sTCP:LISTEN

Git Remote Branch Cleanup

Identify and clean up stale remote branches that have been merged.

# Fetch latest remote state
git fetch --prune

# Find merged remote branches older than a specific date
for branch in $(git branch -r --merged | grep -v HEAD | grep -v develop | grep -v master | sed /\*/d); do
    # Check if branch has commits since specified date
    if [ -z "$(git log -1 --since='Jun 15, 2020' -s ${branch})" ]; then
        # Show branch info: last commit date and author
        echo -e "$(git show --format="%ci %cr %an" ${branch} | head -n 1) \t$branch"
        remote_branch=$(echo ${branch} | sed 's#origin/##')
        # Uncomment to actually delete:
        # git push origin --delete ${remote_branch}
    fi
done

Safer alternative with confirmation:

# List stale branches
git branch -r --merged | grep -v master | grep -v develop | grep -v HEAD

# Delete specific remote branch
git push origin --delete branch-name

XML Formatting

Format XML files for better readability.

# Format XML file (output to new file)
xmllint --format input.xml > output.xml

# Format and recover from errors
xmllint --format --recover malformed.xml > fixed.xml

# Format in place (overwrite original)
xmllint --format input.xml --output input.xml

# Validate XML against schema
xmllint --schema schema.xsd input.xml