Intro to bash scripting - Terminal Tooling

posts

Overview   cli posix

Bash is a (perhaps the ubiquitous) shell. As a member of the POSIX interface, bash provides a consistent access point to applications on your operating system.

Because it is all but guaranteed to be installed on any server and environment, learning this one interface will ensure you can be comfortable interacting with any computer.

Before we begin, it will be helpful to clear up a few ambiguous terms related to command line tools that often come up together. A terminal is a desktop application you start which presents you with a command line interface, running your chosen shell.

The terminal is the user interface to interact with your computers files, devices, and hardware.

The command line is a method of interacting with your computer where you input a command, the computer processes the command, computes the result, and prints it out.

The shell is the software that is responsible for the actual processing of your requests and returning the results to the user.

So in this case, we are interested in a shell, bash. Bash was written as a open source replacement to the bourne shell. First released in 1989, it's been a staple for decades, only recently being overtaken by zsh as the default shell on Mac OS among other systems.

Bash can be used to simply execute programs installed on your computer:

ls ~/Code/k8s-app
backend
docker-compose.yml
files
frontend
kubernetes
testing.yml

But it can do much more. Bash introduced control flow mechanisms allowing you to create reusable scripts along the lines of something you would expect from a python script.

Control flow

if, else, etc.

Like any scriptable, programable environment, bash supports branching features depending on provided conditions.

if true
then
    echo truthy
fi
truthy

If checks (perhaps confusingly if you're coming from a software development backgroud) if the condition evaluates to 0. If it does, the then branch is executed. Also note an if block is terminated with fi (if, backwards). Non zero results in if following the else branch if one is given.

if false
then
    echo truthy
else
    echo falesy
fi
falesy

If checks the exit status of it's condition in order to determine which branch to take. the true command simply sets its exit status to 0 (truthy). false does the inverse. You can check the exit status of the previous command with the variable $?.

true
echo $?

false
echo $?
0
1

Every command ran in the shell sets an exit command, so you can use this to branch based on the success or failure of any command.

if grep "foo" ~/Code/k8s-app/backend/main.go
then
    echo "found foo"
else
    echo "couldn't find foo"
fi
couldn't find foo

In the above example, we can tell if the grep command failed or not, but if we look at another example where grep does succeed, the results aren't probably what you're expecting.

if grep "http" ~/Code/k8s-app/backend/main.go
then
    echo "Found it!"
else
    echo "Couldn't find it."
fi
	"net/http"
	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
func indexHandler(app *App) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
func recipeHandler(app *App) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
	handler func(app *App) http.HandlerFunc
func initServer(app *App) *http.ServeMux {
	mux := http.NewServeMux()
		otelHandler := otelhttp.NewHandler(http.HandlerFunc(route.handler(app)), route.route)
	log.Fatal(http.ListenAndServe(":8080", mux))
Found it!

In this example, grep succeeded and because grep prints the lines that match its search term, those results showed up in the output as well. Sometimes this could be what you want, but often it's not.

What we need to do here is to test the exit status only, but not print anything that the command itself would otherwise output. This can be accomplished by redirecting grep's output to a black hole, /dev/null. This will be covered later on when we cover output redirection

while, for

Bash also supports while and for loops. Notice that for and while use the keyword do before the loop body, and done to close it.

for x in 1 2 3 4
do
    echo $x
done
1
2
3
4
for file in ~/Code/k8s-app/backend/*.go
do
    echo $file
done
/Users/jharder/Code/k8s-app/backend/app.go
/Users/jharder/Code/k8s-app/backend/db.go
/Users/jharder/Code/k8s-app/backend/main.go
/Users/jharder/Code/k8s-app/backend/tracing.go
/Users/jharder/Code/k8s-app/backend/tracing_test.go

Some of this wont make perfect sense yet until we cover varbiables and shell arithmetic, but hopefully you can get a sense of what it's doing:

x=5
while [ $x -gt 0 ]
do
    echo $x
    x=$(( $x - 1 ))
done
5
4
3
2
1

Case statements

Bash also supports case (also referred to as switch statements. The syntax may look a little foreign to our 21st century eyes, but it is handles the key concepts of most switch statements.

x=5
case $x in
1)
  echo "$x = 1"
  ;;
2 | 3 | 4 | 5)
  echo "$x is between 2 and 5"
  ;;
*)
  echo "$x is something else"
  ;;
esac

5 is between 2 and 5

Each arm of the case statement can be a raw value, or a pattern. Cases patterns end with ), and multiple patterns can be provided separated with |. Case blocks end with ;;. Lastly like if, case statements end with their keyword reversed, esac.

Output redirection

The output of a command can be redirected (from standard output) to another location. This could be a different stream (like standard out), a file, or a virtual device. /dev/null gobbles up all data sent to it so in this case the results grep prints are never shown.

if grep "http" ~/Code/k8s-app/backend/main.go > /dev/null
then
    echo "found it"
else
    echo "couldn't find it"
fi
found it

Sometimes there is no command which exists to check the condition you want. In these scenarios, you can use the command test.

test

The man page for test is very informative. It supports a number of flags which execute a number of different conditional checks. For integers it supports -eq, -ne, -gt, -ge, -lt, and -le, or ==, !=, >, >=, <, and <= expressed in a more familiar syntax.

if test 4 -gt 3
then
    echo "Four is greater than three"
else
    echo "Four is NOT greater than three"
fi
Four is greater than three

To make things a little nicer to look at, bash provides an alias of test called [.1

if [ 4 -gt 3 ]
then
    echo "Four is greater than three"
else
    echo "Four is NOT greater than three"
fi
Four is greater than three

NOTE: because [ is a bash command just like test or true, you must have a space after it. It's tempting to say if [condition] or even if[condition], but since [ is just an alias of test which is a regular command, this is illegal as it would be equivelant to if testcondition or iftestcondition respectively.

You can test for different file system conditions using test as well.

if [ -e ~/Code/k8s-app/backend/main.go ]
then
    echo "file exists!"
else
    echo "File not found"
fi
file exists!
if [ -d ~/Code/k8s-app/backend/ ]
then
    echo "file exists, and it's a directory"
else
    echo "either the file doesn't exist, or it does, but it's not a directory"
fi
file exists, and it's a directory
if [ -r ~/Code/k8s-app/backend/main.go ]
then
    echo "file exists and it has read permissions"
else
    echo "file may or may not exists, but if it does, it can't be read"
fi
file exists and it has read permissions

Variables

Simple variables

Bash also supports variable, which are set using var=value syntax. To set a variable you do not provide the $ symbol in front of the variable, but to retrieve it, you must reference the variable with the $ symbol.

my_var=5
echo $my_var
5

Variables can be set to the result of commands if you use a sub-shell to compute the result.

my_var=$(ls -l ~/Code/k8s-app/backend | tail -n1)
echo $my_var
-rw-r--r-- 1 jharder staff 65 Dec 15 13:03 tracing_test.go

You can also embed variables inside of a string to perform string interpolation.

my_name="Jon"
echo "Hello, $my_name"
Hello, Jon

If you want to put content directly after your variable, you'll need to use ${...} instead of $.

animal=cat
echo "I like ${animal}s"
I like cats

If we want to store arithmetic computations in a variable we will need to reach for some additional constructs because sadly, bash will not perform arithmetic operations by itself.

a=4+8
echo $a
4+8

To do arithmetic in the shell, you'll need to use….shell arithmetic.

Arrays

Bash also supports arrays.

numbers=(5 2 6 7)
echo $numbers
5

Is that what you expected? It sure wasn't what I expected. Referencing the variable just gives you the first element in the array, similar to C.

You can access elements of the array using ${var[n]} syntax. You can access all the elements by using @ as the array index.

numbers=(5 2 6 7)
echo ${numbers[2]}
echo ${numbers[@]}
6
5 2 6 7

The latter can be used in a for loop to handle every element.

my_array=(thing1 thing2 "thing three")
for x in "${my_array[@]}"; do
    echo "x = $x"
done
x = thing1
x = thing2
x = thing three

The double quoting around ${my_array} is to avoid splitting on the space in "thing three". Try removing the quotes and see what happens.

Shell arithmetic

a=$((4+8))
b=$((4<3))
c=$((4>3))

echo $a
echo $b
echo $c
12
0
1

You can reference variables inside (( )) without dollar signs if you want.

x=6

echo $((x*8))
echo $((4>3))
48
1

You can combine this with if for more concise conditions…

if $((4>3))
then
    echo "four is greater than three"
else
    echo "four is not greater than three"
fi
four is not greater than three

however, because shell arithmetic returns the more familiar 1 for true and 0 for false, you'll need to use test (or [) to mediate between math and your shell.

if [ $((4>3)) ]
then
     echo "four is greater than three"
else
     echo "four is not greater than three"
fi
four is greater than three

Functions

Definition

Bash also has basic support for defining and using functions, with some oddities.

function my_fun() {
  echo "hi"
}

my_fun
hi

Defining arguments

You would think that adding arguments inside the () would allow you to define parameters to your function. And you'd be wrong.

# THIS DOES NOT COMPILE!!!
function my_fun(a, b) {
  echo "$a $b"
}

Instead, you need to refer to the arguments by their positional index: $1, $2, …, $n.

function my_fun() {
  echo "arg 1 = $1"
  echo "arg 2 = $2"
}

my_fun 1 2
arg 1 = 1
arg 2 = 2

Local variables

You can make local variables to the function without polluting the global namespace using the keyword local.

x=3
echo "before my_fun, x=$x"

function my_fun() {
  local x=5
  echo "in my_fun, x=$x"
}

my_fun

echo "after my_fun, x=$x"
before my_fun, x=3
in my_fun, x=5
after my_fun, x=3

Returning values

Because functions operate just like any other command you may run in the shell, you can only set an exit status, not any arbitrary value. To return anything else, you can use standard output and capture that value in a subshell.

function my_fun() {
    echo "$1 times $2 is $(($1 * $2))"
}

result=$(my_fun 3 6)
echo $result
3 times 6 is 18

Conclusion

Bash as a bit of a quirky programming language, but being a programming language that's built right into your shell can be very handy. However, because it lacks any higher level language constructs like classes, higher order functions, or clear argument or type definitions, bash scripts have a maximum carrying capacity in the form of maintainability. When they get too large, it's probably best to look for a dedicated programming language.

Footnotes:

1

Bash also adds [[ as an extension on top of the posix [. It functions in much the same way but adds some niceties on top of it. Space does not allow me to go into detail, but starting here provides a succinct overview of the differences.