Bash Loops
So far our backup script functions as expected and its usability has been substantially increased in comparison with the initial code introduced at the beginning of this scripting tutorial. We can now easily backup any user directory by pointing the script to user’s home directory using positional parameters during the script’s execution.
The trouble only arises when we need to backup multiple user directories on a daily basis. Hence this task will very quickly become tedious and time-consuming. At this stage, it would be great to have the means to backup any number of selected user home directories with a single backup.sh script execution.
Fortunately, bash has us covered, as this task can be accomplished by use of loops. Loops are looping constructs used to iterate through any given number of tasks until all items in a specified list were completed or predefined conditions were met. There are three basic loop types available to our disposal.
For Loop
For loop is used to iterate through any given code for any number of supplied items in the list. Let’s start with a simple for loop example:
linuxconfig.org:~$ for i in 1 2 3; do echo $i; done 1 2 3 linuxconfig.org:~$
The above for loop has used the echo command to print all items 1, 2 and 3 in the list. Using a semicolon allows us to execute for loop on a single command line. If we were to transfer the above for loop into a bash script, the code would look like follows:
#!/bin/bash
for i in 1 2 3; do
echo $i
doneThe for loop consists of four Shell Reserved Words: for, in, do, done. The above code can therefore also be read as: FOReach item INlist 1, 2 and 3 assign each item temporarily into a variable i after which DOecho $i in order to print the item as STDOUT and keep printing until all items INthe list are DONE.
Printing numbers is undoubtedly fun but let’s try something more meaningful instead. Using command substitution as explained earlier in this tutorial we can create any kind of list to be a part of for loop construct. The following slightly more sophisticated for loop example will count characters of each line for any given file:
linuxconfig.org:~$ vi items.txt linuxconfig.org:~$ cat items.txt bash scripting tutorial linuxconfig.org:~$ for i in $( cat items.txt ); do echo -n $i | wc -c; done 4 9 8 linuxconfig.org:~$
Yes, when mastered, the power of GNU Bash knows no limits! Take your time to experiment before moving forward.
Exercise:Rewrite the above character count for loop to print names of all files and directories inside your current working directory along with the number of characters each file and directory name consists from. The for loop output should look similar to:
0_xvz has 5 backup.sh has 9 compare.sh has 10 date.sh has 7 file1.txt has 9 foobar has 6 function.sh has 11 hello-world.sh has 14 if_else.sh has 10 items.txt has 9
While Loop
The next loop construct on our list is while loop. This particular loop acts on a given condition. Meaning, it will keep executing code enclosed withing DOand DONEwhile the specified condition is true. Once the specified condition becomes false, the execution will stop. Consider the following example:
#!/bin/bash
counter=0
while [ $counter -lt 3 ]; do
let counter+=1
echo $counter
done
This particular while loop will keep executing the enclosed code only while the counter variable is less than 3. This condition is set on Line 4. During each loop iteration, on Lines 5the variable counter is incremented by one. Once the variable counter is equal 3, the condition defined on Lines 4 becomes false and while loop execution is terminated.
linuxconfig.org:~$ cat while.sh #!/bin/bash counter=0 while [ $counter -lt 3 ]; do let counter+=1 echo $counter done linuxconfig.org:~$ ./while.sh 1 2 3 linuxconfig.org:~$ vi while.sh linuxconfig.org:~$ ./while.sh 3 linuxconfig.org:~$
Subscribe to Linux Career NEWSLETTER and receive latest Linux news, jobs, career advice and tutorials.
Until Loop
The last loop we are going to cover in this scripting tutorial is until loop. The until loop does the exact opposite of the while loop. Until loop also acts on a preset condition. However, the code enclosed between DOand DONEis repeatedly executed only until this condition changes from false to true. The execution of until loop is illustrated using the below example:
#!/bin/bash
counter=6
until [ $counter -lt 3 ]; do
let counter-=1
echo $counter
done
If you understood the above while loop script, the until loop will be somewhat self-explanatory. The script starts with the variable counter set to 6. The condition defined on Line 4of this particular until loop is to keep executing the enclosed code until the condition becomes true.
linuxconfig.org:~$ cat until.sh #!/bin/bash counter=6 until [ $counter -lt 3 ]; do let counter-=1 echo $counter done linuxconfig.org:~$ ./until.sh 5 4 3 2 linuxconfig.org:~$ vi until.sh linuxconfig.org:~$ ./until.sh 3 2 linuxconfig.org:~$
At this stage, we can convert our understanding of loops into something tangible. Our current backup script is currently capable to backup a single directory per execution. It would be nice to have the ability to backup all directories supplied to the script on a command line upon its execution. Review the updated script below which implements such a new feature:
#!/bin/bash
# This bash script is used to backup a user's home directory to /tmp/.
function backup {
if [ -z $1 ]; then
user=$(whoami)
else
if [ ! -d "/home/$1" ]; then
echo "Requested $1 user home directory doesn't exist."
exit 1
fi
user=$1
fi
input=/home/$user
output=/tmp/${user}_home_$(date +%Y-%m-%d_%H%M%S).tar.gz
function total_files {
find $1 -type f | wc -l
}
function total_directories {
find $1 -type d | wc -l
}
function total_archived_directories {
tar -tzf $1 | grep /$ | wc -l
}
function total_archived_files {
tar -tzf $1 | grep -v /$ | wc -l
}
tar -czf $output $input 2> /dev/null
src_files=$( total_files $input )
src_directories=$( total_directories $input )
arch_files=$( total_archived_files $output )
arch_directories=$( total_archived_directories $output )
echo "########## $user ##########"
echo "Files to be included: $src_files"
echo "Directories to be included: $src_directories"
echo "Files archived: $arch_files"
echo "Directories archived: $arch_directories"
if [ $src_files -eq $arch_files ]; then
echo "Backup of $input completed!"
echo "Details about the output backup file:"
ls -l $output
else
echo "Backup of $input failed!"
fi
}
for directory in $*; do
backup $directory
done;
After reviewing the above script, you may have noticed that new function called backup on Lines 5 – 57was created. This function includes all of our previously written code. The function definition ends on Line 57after which we have implemented a new for loop on Lines 59 – 51to execute the newly defined backup function for each user directory supplied as an argument. If you recall, the $* variable contains all arguments supplied on a command line upon the script execution. Furthermore, a cosmetic change to the code on Line 44ensures a better readability of the script’s output by separating each directory backup info output block with a hash line. Let’s see how it works:
$ ./backup.sh linuxconfig damian ########## linuxconfig ########## Files to be included: 27 Directories to be included: 4 Files archived: 27 Directories archived: 4 Backup of /home/linuxconfig completed! Details about the output backup file: -rw-r--r-- 1 linuxconfig linuxconfig 236173 Oct 23 10:22 /tmp/linuxconfig_home_2017-10-23_102229.tar.gz ########## damian ########## Files to be included: 3 Directories to be included: 1 Files archived: 3 Directories archived: 1 Backup of /home/damian completed! Details about the output backup file: -rw-r--r-- 1 linuxconfig linuxconfig 2140 Oct 23 10:22 /tmp/damian_home_2017-10-23_102230.tar.gz
Exercise:The current script does not check for the existence of user directories prior to the backup function execution. This can lead to unforeseen consequences. Do you think that you would be able to create your own improved copy of the backup script by defining a separate loop to check the existence of all user directories before the backup for loop is reached? You for loop will exit the script’s execution if any of the user directories on the supplied list does not exist.
Bash Arithmetics
In the last section of this bash scripting tutorial, we will discuss some basics of bash arithmetics. Arithmetics in bash scripting will add another level of sophistication and flexibility to our scripts as it allows us to calculate numbers even with numeric precision. There are multiple ways on how to accomplish arithmetic operations within your bash scripts. Let’s go through some of them using few simple examples.
Arithmetic Expansion
The arithmetic expansion is probably to the most simple method on how to achieve basic calculations. We just enclose any mathematical expression inside double parentheses. Let’s perform some simple addition, subtraction, multiplication and division calculations with integers:
linuxconfig.org:~$ a=$(( 12 + 5 )) linuxconfig.org:~$ echo $a 17 linuxconfig.org:~$ echo $(( 12 + 5 )) 17 linuxconfig.org:~$ echo $(( 100 - 1 )) 99 linuxconfig.org:~$ echo $(( 3 * 11 )) 33 linuxconfig.org:~$ division=$(( 100 / 10 )) linuxconfig.org:~$ echo $division 10 linuxconfig.org:~$ x=10;y=33 linuxconfig.org:~$ z=$(( $x * $y )) linuxconfig.org:~$ echo $z 330 linuxconfig.org:~$
Exercise:Can you use the arithmetic expansion to perform a modulus operation? For example what is the result of modulus operation 99 % 10?
expr command
Another alternative to arithmetic expansion is the expr command. Using the expr command allows us to perform an arithmetic operation even without enclosing our mathematical expression within brackets or quotes. However, do not forget to escape asterisk multiplication sign to avoid expr: syntax error
:
linuxconfig.org:~$ expr 2 + 2 4 linuxconfig.org:~$ expr 6 * 6 expr: syntax error linuxconfig.org:~$ expr 6 \* 6 36 linuxconfig.org:~$ expr 6 / 3 2 linuxconfig.org:~$ expr 1000 - 999 1 linuxconfig.org:~$
let command
Similarly, as with expr command, we can perform bash arithmetic operations with let command. let command evaluates a mathematical expression and stores its result into a variable. We have alredy encountered the let command in one of our previous examples where we have used it to perform integer increment. The following example shows some basic operations using let command as well as integer increment and exponent operations like x3:
linuxconfig.org:~$ let a=2+2 linuxconfig.org:~$ echo $a 4 linuxconfig.org:~$ let b=4*($a-1) linuxconfig.org:~$ echo $b 12 linuxconfig.org:~$ let c=($b**3)/2 linuxconfig.org:~$ echo $c 864 linuxconfig.org:~$ let c++ linuxconfig.org:~$ echo $c 865 linuxconfig.org:~$ let c-- linuxconfig.org:~$ echo $c 864 linuxconfig.org:~$
bc command
After few minutes of experimentation with the above bash arithmetic methods, you may have noticed that they work perfectly with integer numbers however when it comes to the decimal numbers there is something amiss. To take our bash arithmetic to an entirely different level, we will need to use bc command. bc command with a proper syntax allows for more than simple integer calculations.
Operational manual of the bc command is quite extensive as it spans over more than 500 lines. However, it does not hurt to show some basic operations. The following example will perform a division operation with 2 and 30 decimal numbers and the square root of 50 with 50 decimal numbers. By default, the bc command will produce all results as an integer number. Use scale=x to instruct the bc command to show real numbers:
linuxconfig.org:~$ echo '8.5 / 2.3' | bc 3 linuxconfig.org:~$ echo 'scale=2;8.5 / 2.3' | bc 3.69 linuxconfig.org:~$ echo 'scale=30;8.5 / 2.3' | bc 3.695652173913043478260869565217 linuxconfig.org:~$ squareroot=$( echo 'scale=50;sqrt(50)' | bc ) linuxconfig.org:~$ echo $squareroot 7.07106781186547524400844362104849039284835937688474 linuxconfig.org:~$
Let’s put our new bash arithmetic knowledge to work and once again change our backup.sh script to implement a counter of all archived files and directories for all users:
#!/bin/bash
# This bash script is used to backup a user's home directory to /tmp/.
function backup {
if [ -z $1 ]; then
user=$(whoami)
else
if [ ! -d "/home/$1" ]; then
echo "Requested $1 user home directory doesn't exist."
exit 1
fi
user=$1
fi
input=/home/$user
output=/tmp/${user}_home_$(date +%Y-%m-%d_%H%M%S).tar.gz
function total_files {
find $1 -type f | wc -l
}
function total_directories {
find $1 -type d | wc -l
}
function total_archived_directories {
tar -tzf $1 | grep /$ | wc -l
}
function total_archived_files {
tar -tzf $1 | grep -v /$ | wc -l
}
tar -czf $output $input 2> /dev/null
src_files=$( total_files $input )
src_directories=$( total_directories $input )
arch_files=$( total_archived_files $output )
arch_directories=$( total_archived_directories $output )
echo "########## $user ##########"
echo "Files to be included: $src_files"
echo "Directories to be included: $src_directories"
echo "Files archived: $arch_files"
echo "Directories archived: $arch_directories"
if [ $src_files -eq $arch_files ]; then
echo "Backup of $input completed!"
echo "Details about the output backup file:"
ls -l $output
else
echo "Backup of $input failed!"
fi
}
for directory in $*; do
backup $directory
let all=$all+$arch_files+$arch_directories
done;
echo "TOTAL FILES AND DIRECTORIES: $all"
On Line 60 we have used addition to add all archived files using let command to a resulting variable all. Each for loop iteration adds new count for every additional user. The result is then printed using echo command on Line 62.
Example script execution:
$ ./backup.sh linuxconfig damian ########## linuxconfig ########## Files to be included: 27 Directories to be included: 6 Files archived: 27 Directories archived: 6 Backup of /home/linuxconfig completed! Details about the output backup file: -rw-r--r-- 1 linuxconfig linuxconfig 237004 Dec 27 11:23 /tmp/linuxconfig_home_2017-12-27_112359.tar.gz ########## damian ########## Files to be included: 3 Directories to be included: 1 Files archived: 3 Directories archived: 1 Backup of /home/damian completed! Details about the output backup file: -rw-r--r-- 1 linuxconfig linuxconfig 2139 Dec 27 11:23 /tmp/damian_home_2017-12-27_112359.tar.gz TOTAL FILES AND DIRECTORIES: 37