As subtle as a flying brick.

Posts tagged “File Management

How to check if a directory exists in a shell script

To check if a directory exists and is a directory use the following syntax:

[ -d "/path/to/dir" ] && echo "Directory /path/to/dir exits." || echo "Error: Directory /path/to/dir does not exits."

The following version also check for symbolic link:

[ -d "/path/to/dir" && ! -L "/path/to/dir" ] && echo "Directory /path/to/dir exits." || echo "Error: Directory /path/to/dir exits but point to $(readlink -f /path/to/dir)."

OR

[ -d "/path/to/dir" && ! -h "/path/to/dir" ] && echo "Directory /path/to/dir exits." || echo "Error: Directory /path/to/dir exits but point to $(readlink -f /path/to/dir)."

Finally, you can use the traditional if..else..fi:

if [ -d "/path/to/dir" ]
then
    echo "Directory /path/to/dir exits."
else
    echo "Error: Directory /path/to/dir does not exits."
fi

Shell script examples to see if a ${directory} exists or not

 
#!/bin/bash
dir="$1"

[ $# -eq 0 ] && { echo "Usage: $0 dir-name"; exit 1; }

if [ -d "$dir" -a ! -h "$dir" ]
then
   echo "$dir found and setting up new Apache/Lighttpd/Nginx jail, please wait..."
   # __WWWJailSetup "cyberciti.biz" "setup"
else
   echo "Error: $dir not found or is symlink to $(readlink -f ${dir})."
fi

In this example, create directories if does not exits:

# Purpose: Setup jail and copy files
# Category : Core
# Override : No
# Parameter(s) : d => domain name
#                action => setup or update
__WWWJailSetup(){
        local d="$1"
        local action="${2:setup}"       # setup or update???
        local index="$d

$d

" # default index.html
        local J="$(_getJailRoot $d)/$d" # our sweet home 
        local _i=""

        [ "$action" == "setup" ] && echo "* Init jail config at $J..." || echo "* Updating jail init config at $J..."
        __init_domain_config "$d"

        [ "$action" == "setup" ] && echo "* Setting up jail at $J..." || echo "* Updating jail at $J..."
        [ ! -d "$J" ] &&  $_mkdir -p "$J"

        for _i in $J/{etc,tmp,usr,var,home,dev,bin,lib64}
        do
                [ ! -d "$_i" ] &&  $_mkdir -p "$_i"
        done
        for _i in $_lighttpd_webalizer_base/$d/stats/{dump,out}
        do
                [ ! -d "$_i" ] &&  $_mkdir -p "$_i"
        done
        for _i in $_lighttpd_webalizer_prepost_base/$d/{pre.d,post.d}
        do
                [ ! -d "$_i" ] &&  $_mkdir -p "$_i"
        done
## truncated 
}

Summary

Use the following to check file/directory types and compare values:

  1. -L "FILE" : FILE exists and is a symbolic link (same as -h)
  2. -h "FILE" : FILE exists and is a symbolic link (same as -L)
  3. -d "FILE" : FILE exists and is a directory
  4. -w "FILE" : FILE exists and write permission is granted

 


Get only the latest version of a file from across mutiple directories

bash$ find . -name custlist\* | perl -ne '$path = $_; s?.*/??; $name = $_; $map{$name} = $path; ++$c; END { print $map{(sort(keys(%map)))[$c-1]} }'

Explanation

The purpose of this is to find the the “latest” version of the custlist_*.xls file from among multiple versions in directories and sub-directories, for example:

./c/custlist_v1.003.xls
./c/custlist_v2.001.xls
./d/b/custlist_v1.001.xls
./d/custlist_v1.002.xls

Let’s decompose the one-liner to the big steps:

  • find . -name custlist\* — find the files matching the target pattern
  • ... | perl -ne '...' — run perl, with the input wrapped around in a while loop so that each line in the input is set in the variable $_
  • $path = $_; s?.*/??; $name = $_; — save the full path in $path, and cut off the subdirectory part to get to the base name of the file and save it in $name
  • $map{$name} = $path; — build a mapping of $name to $path
  • ++$c; — we count the elements, to use it later
  • (sort(keys(%map)))[$c-1] — sort the keys of the map, and get the last element, which is custlist_v2.001.xls in this example
  • END { print $map{$last} }' — at the end of all input data, print the path of the latest version of the file

Limitations

Even if the latest version of the file appears multiple times in the directories, the one-liner will print only one of the paths. This could be fixed though if needed.


Bash For Loop Examples

How do I use bash for loop to repeat certain task under Linux / UNIX operating system? How do I set infinite loops using for statement? How do I use three-parameter for loop control expression?

A ‘for loop’ is a bash programming language statement which allows code to be repeatedly executed. A for loop is classified as an iteration statement i.e. it is the repetition of a process within a bash script.

For example, you can run UNIX command or task 5 times or read and process list of files using a for loop. A for loop can be used at a shell prompt or within a shell script itself.

for loop syntax

Numeric ranges for syntax is as follows:

for VARIABLE in 1 2 3 4 5 .. N
do
	command1
	command2
	commandN
done

This type of for loop is characterized by counting. The range is specified by a beginning (#1) and ending number (#5). The for loop executes a sequence of commands for each member in a list of items. A representative example in BASH is as follows to display welcome message 5 times with for loop:

#!/bin/bash
for i in 1 2 3 4 5
do
   echo "Welcome $i times"
done

Sometimes you may need to set a step value (allowing one to count by two’s or to count backwards for instance). Latest bash version 3.0+ has inbuilt support for setting up ranges:

#!/bin/bash
for i in {1..5}
do
   echo "Welcome $i times"
done

Bash v4.0+ has inbuilt support for setting up a step value using {START..END..INCREMENT} syntax:

#!/bin/bash
echo "Bash version ${BASH_VERSION}..."
for i in {0..10..2}
  do
     echo "Welcome $i times"
 done

Sample outputs:

Bash version 4.0.33(0)-release...
Welcome 0 times
Welcome 2 times
Welcome 4 times
Welcome 6 times
Welcome 8 times
Welcome 10 times

The seq command (outdated)

WARNING! The seq command print a sequence of numbers and it is here due to historical reasons. The following examples is only recommend for older bash version. All users (bash v3.x+) are recommended to use the above syntax.

The seq command can be used as follows. A representative example in seq is as follows:

#!/bin/bash
for i in $(seq 1 2 20)
do
   echo "Welcome $i times"
done

There is no good reason to use an external command such as seq to count and increment numbers in the for loop, hence it is recommend that you avoid using seq. The builtin command are fast.

Three-expression bash for loops syntax

This type of for loop share a common heritage with the C programming language. It is characterized by a three-parameter loop control expression; consisting of an initializer (EXP1), a loop-test or condition (EXP2), and a counting expression (EXP3).

for (( EXP1; EXP2; EXP3 ))
do
	command1
	command2
	command3
done

A representative three-expression example in bash as follows:

#!/bin/bash
for (( c=1; c<=5; c++ ))
do
	echo "Welcome $c times..."
done

Sample output:

Welcome 1 times
Welcome 2 times
Welcome 3 times
Welcome 4 times
Welcome 5 times

How do I use for as infinite loops?

Infinite for loop can be created with empty expressions, such as:

#!/bin/bash
for (( ; ; ))
do
   echo "infinite loops [ hit CTRL+C to stop]"
done

Conditional exit with break

You can do early exit with break statement inside the for loop. You can exit from within a FOR, WHILE or UNTIL loop using break. General break statement inside the for loop:

for I in 1 2 3 4 5
do
  statements1      #Executed for all values of ''I'', up to a disaster-condition if any.
  statements2
  if (disaster-condition)
  then
	break       	   #Abandon the loop.
  fi
  statements3          #While good and, no disaster-condition.
done

Following shell script will go though all files stored in /etc directory. The for loop will be abandon when /etc/resolv.conf file found.

#!/bin/bash
for file in /etc/*
do
	if [ "${file}" == "/etc/resolv.conf" ]
	then
		countNameservers=$(grep -c nameserver /etc/resolv.conf)
		echo "Total  ${countNameservers} nameservers defined in ${file}"
		break
	fi
done

Early continuation with continue statement

To resume the next iteration of the enclosing FOR, WHILE or UNTIL loop use continue statement.

for I in 1 2 3 4 5
do
  statements1      #Executed for all values of ''I'', up to a disaster-condition if any.
  statements2
  if (condition)
  then
	continue   #Go to next iteration of I in the loop and skip statements3
  fi
  statements3
done

This script make backup of all file names specified on command line. If .bak file exists, it will skip the cp command.

#!/bin/bash
FILES="$@"
for f in $FILES
do
        # if .bak backup file exists, read next file
	if [ -f ${f}.bak ]
	then
		echo "Skiping $f file..."
		continue  # read next file and skip cp command
	fi
        # we are hear means no backup file exists, just use cp command to copy file
	/bin/cp $f $f.bak
done

Recommended readings:

  • man bash
  • help for
  • help {
  • help break
  • help continue

Updated for accuracy!