1# Shell Script Examples
2
3This directory contains practical examples for advanced bash scripting topics.
4
5## Directory Structure
6
7```
8Shell_Script/
9├── 02_parameter_expansion/ # Parameter expansion and declare
10├── 03_arrays/ # Array manipulation
11├── 05_function_library/ # Reusable function libraries
12├── 06_io_redirection/ # Advanced I/O redirection
13├── 08_regex/ # Regex validation and extraction
14├── 09_process_management/ # Parallel execution and cleanup
15├── 10_error_handling/ # Error framework and safe operations
16├── 11_argument_parsing/ # getopts, getopt, CLI tools
17├── 13_disaster_recovery/ # Disaster recovery backup/restore
18├── 13_testing/ # Bats tests and test runner
19├── 14_perf_diagnosis/ # Performance bottleneck diagnosis
20├── 14_task_runner/ # Makefile-like task runner project
21├── 15_deployment/ # Deployment automation scripts
22├── 16_monitoring/ # System monitoring dashboard
23└── README.md # This file
24```
25
26## Examples Overview
27
28### 02_parameter_expansion/
29
30Advanced string manipulation and variable attributes.
31
32#### string_ops.sh
33Demonstrates parameter expansion for string operations:
34- Path component extraction (filename, directory, extension)
35- Batch file renaming patterns
36- Config file parsing (key=value format)
37- URL parsing (protocol, host, port, path)
38- Default values and substitutions
39- String length and substrings
40- Case conversion
41
42**Run:**
43```bash
44cd 02_parameter_expansion
45./string_ops.sh
46```
47
48#### declare_demo.sh
49Shows various uses of the `declare` builtin:
50- Integer arithmetic with `declare -i`
51- Readonly variables with `declare -r`
52- Case conversion with `declare -l/-u`
53- Name references with `declare -n`
54- Array declarations (indexed and associative)
55- Export variables with `declare -x`
56- Function-local variables
57
58**Run:**
59```bash
60cd 02_parameter_expansion
61./declare_demo.sh
62```
63
64### 03_arrays/
65
66Practical applications of bash arrays.
67
68#### assoc_arrays.sh
69Demonstrates associative arrays (hash maps):
70- Word frequency counter
71- Config file loader (key=value into associative array)
72- Simple phonebook (add, lookup, delete, list operations)
73- Nested data structures using key prefixes
74
75**Run:**
76```bash
77cd 03_arrays
78./assoc_arrays.sh
79```
80
81**Features:**
82- Reads from stdin or uses sample data
83- Interactive-style demonstrations
84- Practical real-world use cases
85
86#### csv_parser.sh
87CSV file parser with advanced features:
88- Parse CSV files handling quoted fields
89- Query rows by column value
90- Pretty-print as formatted table
91- Extract column values
92- Handle commas inside quoted fields
93
94**Run:**
95```bash
96cd 03_arrays
97./csv_parser.sh
98```
99
100**Usage as library:**
101```bash
102source csv_parser.sh
103parse_csv "data.csv"
104print_table
105query_by_column "Name" "Alice"
106get_column "Email"
107```
108
109### 05_function_library/
110
111Reusable function libraries for common tasks.
112
113#### lib/logging.sh
114Comprehensive logging library:
115- Log levels: DEBUG, INFO, WARN, ERROR
116- Colored console output with timestamps
117- Optional file logging
118- Configurable log level filtering
119- Section headers and separators
120
121**Features:**
122- `log_debug()`, `log_info()`, `log_warn()`, `log_error()`
123- `set_log_level(level_name)`
124- `enable_file_logging(filename)`
125- `disable_file_logging()`
126- `log_separator()`, `log_section(title)`
127
128#### lib/validation.sh
129Input validation functions:
130- Email validation
131- IP address validation (IPv4)
132- Port number validation (1-65535)
133- URL validation
134- Hostname validation
135- MAC address validation
136- Hex color validation (#RGB, #RRGGBB)
137- Semantic version validation (semver)
138- Date validation (YYYY-MM-DD)
139- Integer checks (is_integer, is_positive, is_non_negative, is_in_range)
140- Path validation (file/directory exists)
141- Command existence check
142- String blank/not-blank checks
143
144#### demo.sh
145Demonstrates both libraries with comprehensive examples.
146
147**Run:**
148```bash
149cd 05_function_library
150./demo.sh
151```
152
153**Usage in your scripts:**
154```bash
155#!/usr/bin/env bash
156source "lib/logging.sh"
157source "lib/validation.sh"
158
159set_log_level DEBUG
160enable_file_logging "/var/log/myapp.log"
161
162log_info "Starting application"
163
164if ! validate_email "$user_email"; then
165 log_error "Invalid email address"
166 exit 1
167fi
168
169log_info "Email validated successfully"
170```
171
172### 06_io_redirection/
173
174Advanced I/O redirection techniques.
175
176#### fd_demo.sh
177File descriptor demonstrations:
178- Custom file descriptors (exec 3>file, 4>file)
179- Redirecting stdout and stderr separately
180- Swapping stdout and stderr
181- Logging to both console and file simultaneously
182- Read/write file descriptors
183- Saving and restoring stdout
184- Here-documents with file descriptors
185- Noclobber mode and forced overwrites
186
187**Run:**
188```bash
189cd 06_io_redirection
190./fd_demo.sh
191```
192
193#### process_sub.sh
194Process substitution examples:
195- Comparing output of two commands with `diff <(cmd1) <(cmd2)`
196- Feeding multiple inputs to commands
197- Avoiding subshell variable scope issues
198- Reading multiple streams simultaneously
199- Write process substitution `>(cmd)`
200- Complex data processing pipelines
201- Practical log analysis example
202
203**Run:**
204```bash
205cd 06_io_redirection
206./process_sub.sh
207```
208
209**Key patterns:**
210```bash
211# Compare outputs
212diff <(cmd1) <(cmd2)
213
214# Multiple inputs
215paste <(gen1) <(gen2) <(gen3)
216
217# Avoid subshell variable loss
218while read line; do
219 ((count++))
220done < <(command)
221
222# Split output to multiple destinations
223command | tee >(proc1) >(proc2) > output.txt
224```
225
226#### fifo_demo.sh
227Named pipe (FIFO) demonstrations:
228- Basic FIFO usage
229- Producer-consumer pattern
230- Bidirectional communication (two FIFOs)
231- Load balancing with multiple workers
232- Pipeline stages using FIFOs
233- Timeout handling with FIFOs
234- Automatic cleanup with trap
235
236**Run:**
237```bash
238cd 06_io_redirection
239./fifo_demo.sh
240```
241
242**Features:**
243- Creates temporary FIFOs in /tmp
244- Background processes for producers/consumers
245- Clean signal handling and cleanup
246- Practical patterns for IPC
247
248### 08_regex/
249
250Regular expression patterns for validation and data extraction.
251
252#### validate.sh
253Input validation using bash regex (`=~` and `BASH_REMATCH`):
254- Email, IPv4, date, semver, and URL validation
255- Comprehensive pass/fail demonstrations
256
257#### extract.sh
258Data extraction using regex:
259- URL, email, and phone number extraction from text
260- Log line parsing with structured output
261- CSV parsing with quoted field handling
262
263**Run:**
264```bash
265cd 08_regex
266./validate.sh
267./extract.sh
268```
269
270### 09_process_management/
271
272Parallel execution and signal handling patterns.
273
274#### parallel.sh
275Parallel execution with concurrency control:
276- `run_parallel()` with configurable concurrency limit
277- PID tracking and exit code collection
278- Sequential vs parallel performance comparison
279
280#### cleanup.sh
281Signal handling and cleanup patterns:
282- Trap handlers for EXIT, INT, TERM, HUP
283- Automatic temp file/directory cleanup
284- Lock file management
285- Graceful shutdown for long-running scripts
286
287**Run:**
288```bash
289cd 09_process_management
290./parallel.sh
291./cleanup.sh
292```
293
294### 10_error_handling/
295
296Reusable error handling frameworks.
297
298#### error_framework.sh
299Comprehensive error handling:
300- Predefined error codes (E_SUCCESS, E_INVALID_ARG, etc.)
301- Stack trace with BASH_LINENO and FUNCNAME
302- Try/catch simulation using subshells
303
304#### safe_ops.sh
305Safe file and command operations:
306- `safe_cd()`, `safe_rm()`, `safe_write()`, `safe_cp()`
307- `retry()` with exponential backoff
308- `require_cmd()` for dependency checking
309
310**Run:**
311```bash
312cd 10_error_handling
313./error_framework.sh
314./safe_ops.sh
315```
316
317### 11_argument_parsing/
318
319CLI argument parsing patterns.
320
321#### getopts_demo.sh
322POSIX getopts demonstration:
323- Short options: `-v`, `-o FILE`, `-n NUM`, `-h`
324- Proper usage function and error handling
325- Positional argument processing
326
327#### cli_tool.sh
328Professional CLI tool with:
329- Short and long options (`--verbose`, `--help`, `--version`)
330- Progress bar and spinner implementations
331- Colored output with verbose/quiet modes
332
333**Run:**
334```bash
335cd 11_argument_parsing
336./getopts_demo.sh -v -o output.txt file1 file2
337./cli_tool.sh --help
338```
339
340### 13_disaster_recovery/
341
342Disaster recovery backup and restore scripts.
343
344#### dr_backup.sh
345Automated disaster recovery backup script (~120 lines):
346- Full system backup (filesystem, databases, config)
347- PostgreSQL database dumps with pg_dump
348- Compression (gzip) and encryption (gpg)
349- Remote backup transfer via rsync
350- Backup rotation policy (daily/weekly/monthly)
351- Email notifications on success/failure
352- Comprehensive logging with timestamps
353- Incremental backup support
354
355**Features:**
356- Configurable backup sources
357- Database backup with custom format
358- Secure GPG encryption
359- Remote transfer to backup server
360- Automatic cleanup of old backups
361- Error handling and validation
362
363**Run:**
364```bash
365cd 13_disaster_recovery
366./dr_backup.sh # Full backup
367./dr_backup.sh -n # Local only (no remote)
368./dr_backup.sh -i # Incremental backup
369./dr_backup.sh --help # Show usage
370```
371
372#### dr_restore.sh
373Restore from disaster recovery backups (~100 lines):
374- List available backups by date
375- Selective restoration (all/filesystem/database/config)
376- Backup integrity verification with checksums
377- Decryption and decompression
378- Database restoration with pg_restore
379- Dry-run mode for safety
380- Comprehensive logging
381
382**Features:**
383- Interactive backup selection
384- SHA256 checksum verification
385- Selective component restore
386- Safe dry-run testing
387- Automatic decryption/decompression
388
389**Run:**
390```bash
391cd 13_disaster_recovery
392./dr_restore.sh --list # List backups
393./dr_restore.sh -d 20240215_120000 # Restore specific backup
394./dr_restore.sh -t database # Restore databases only
395./dr_restore.sh -n -d 20240215_120000 # Dry-run mode
396./dr_restore.sh -v -d 20240215_120000 # Verify integrity
397```
398
399### 13_testing/
400
401Shell script testing with Bats framework.
402
403#### math_lib.sh
404A math library to be tested: `add()`, `subtract()`, `multiply()`, `divide()`, `factorial()`, `is_prime()`, `gcd()`, `lcm()`.
405
406#### test_math_lib.bats
407Complete Bats test suite with 50+ test cases covering normal, edge, and error cases.
408
409#### run_tests.sh
410Automated test runner: runs Bats tests, ShellCheck validation, and generates summary reports.
411
412**Run:**
413```bash
414cd 13_testing
415./run_tests.sh # Run all tests
416bats test_math_lib.bats # Run Bats tests directly
417```
418
419### 14_perf_diagnosis/
420
421Performance bottleneck diagnosis tools.
422
423#### bottleneck_finder.sh
424System performance bottleneck analyzer (~150 lines):
425- CPU usage analysis with mpstat/top
426- Load average monitoring (1/5/15 min)
427- Memory and swap usage analysis
428- Disk I/O statistics with iostat
429- Network interface statistics
430- Process resource consumption
431- Automatic bottleneck detection
432- Color-coded warnings (green/yellow/red)
433
434**Features:**
435- Comprehensive system metrics collection
436- Configurable thresholds (warning/critical)
437- Top CPU and memory consumers
438- Zombie process detection
439- Network connection analysis
440- Summary report with recommendations
441- Export to file capability
442
443**Run:**
444```bash
445cd 14_perf_diagnosis
446./bottleneck_finder.sh # Full analysis
447./bottleneck_finder.sh -a cpu # CPU analysis only
448./bottleneck_finder.sh -a memory # Memory analysis only
449./bottleneck_finder.sh -o report.txt # Export to file
450./bottleneck_finder.sh -v # Verbose mode
451```
452
453**Analysis Types:**
454- `cpu`: CPU usage, load average, top consumers
455- `memory`: RAM/swap usage, memory consumers
456- `disk`: Disk space, I/O statistics
457- `network`: Interface stats, connections
458- `process`: Process counts, top CPU/memory users
459- `all`: Complete system analysis (default)
460
461### 14_task_runner/
462
463Complete task runner project (from Lesson 14).
464
465#### task.sh
466Makefile-like task runner:
467- Task discovery via `task::*` naming convention
468- Dependency resolution with `depends_on`
469- Help generation from `##` comments
470- Colored output with timestamps
471
472**Run:**
473```bash
474cd 14_task_runner
475./task.sh --help
476./task.sh build
477./task.sh deploy
478```
479
480### 15_deployment/
481
482Deployment automation scripts (from Lesson 15).
483
484#### deploy.sh
485Production-grade deployment automation:
486- SSH-based remote execution and file syncing
487- Rolling deployment with health checks
488- Rollback support with release history
489- Multi-environment support (staging/production)
490
491#### entrypoint.sh
492Docker container entrypoint script:
493- Environment variable validation
494- Template processing with envsubst
495- Wait-for-it functionality for dependencies
496- Signal handling for graceful shutdown
497
498**Run:**
499```bash
500cd 15_deployment
501./deploy.sh --help
502./deploy.sh status --env staging
503```
504
505### 16_monitoring/
506
507System monitoring tools (from Lesson 16).
508
509#### monitor.sh
510Interactive terminal dashboard:
511- Real-time CPU, memory, disk, load metrics
512- Terminal UI with tput (colors, boxes, cursor positioning)
513- Color-coded alerts (green/yellow/red)
514- Cross-platform support (Linux and macOS)
515
516#### health_check.sh
517Cron-safe health check automation:
518- System resource monitoring with thresholds
519- Process and HTTP endpoint checks
520- Webhook alerting (Slack-compatible)
521- Idempotent design for repeated cron execution
522
523**Run:**
524```bash
525cd 16_monitoring
526./monitor.sh # Interactive dashboard
527./health_check.sh # One-shot health check (for cron)
528```
529
530## General Usage Notes
531
532### Making Scripts Executable
533
534All scripts are already set up with the shebang `#!/usr/bin/env bash`, but you may need to make them executable:
535
536```bash
537chmod +x script_name.sh
538```
539
540### Script Safety
541
542All scripts use:
543```bash
544set -euo pipefail
545```
546
547This means:
548- `-e`: Exit on error
549- `-u`: Error on undefined variables
550- `-o pipefail`: Pipelines fail if any command fails
551
552### Sourcing vs Executing
553
554Some scripts can be both:
555- **Executed**: `./script.sh` - Runs the demo
556- **Sourced**: `source script.sh` - Loads functions for use in your shell
557
558Libraries in `05_function_library/lib/` should be sourced, not executed.
559
560### Temporary Files
561
562Scripts that create temporary files use `/tmp` and clean up after themselves. The FIFO demo uses `trap` to ensure cleanup even if interrupted.
563
564### Platform Notes
565
566These scripts are tested on:
567- Linux (bash 4.0+)
568- macOS (bash 3.2+ with some limitations on features like `declare -l/-u`)
569
570Some features require bash 4.0+:
571- Associative arrays
572- `declare -l/-u` (case conversion)
573- `${var,,}` and `${var^^}` syntax
574
575## Best Practices Demonstrated
576
5771. **Error Handling**: Using `set -euo pipefail` and checking return codes
5782. **Input Validation**: Comprehensive validation library
5793. **Logging**: Structured logging with levels and colors
5804. **Code Reuse**: Separating reusable functions into libraries
5815. **Documentation**: Comments explaining complex operations
5826. **Cleanup**: Proper resource cleanup with trap handlers
5837. **Portability**: Using `#!/usr/bin/env bash` for compatibility
584
585## Learning Path
586
587Suggested order to study these examples:
588
5891. **02_parameter_expansion/** - Master string manipulation and declare
5902. **03_arrays/** - Understand associative arrays and CSV parsing
5913. **05_function_library/** - Build reusable code
5924. **06_io_redirection/** - Advanced I/O, process substitution, FIFOs
5935. **08_regex/** - Regex validation and data extraction
5946. **09_process_management/** - Parallel execution and signal handling
5957. **10_error_handling/** - Error frameworks and safe operations
5968. **11_argument_parsing/** - CLI tool development
5979. **13_testing/** - Bats testing framework
59810. **13_disaster_recovery/** - Backup and restore automation
59911. **14_task_runner/** - Complete task runner project
60012. **14_perf_diagnosis/** - Performance analysis tools
60113. **15_deployment/** - Deployment automation
60214. **16_monitoring/** - System monitoring dashboard
603
604## Additional Resources
605
606- Bash Reference Manual: https://www.gnu.org/software/bash/manual/
607- Advanced Bash-Scripting Guide: https://tldp.org/LDP/abs/html/
608- ShellCheck (linting): https://www.shellcheck.net/
609
610## License
611
612These examples are provided as educational material under the MIT License.