As subtle as a flying brick.

Posts tagged “UNIX

Understanding /etc/shadow file

Q. Can you explain /etc/shadow file used under Linux or UNIX?

A. /etc/shadow file stores actual password in encrypted format for user’s account with additional properties related to user password i.e. it stores secure user account information.

All fields are separated by a colon (:) symbol. It has one entry per line for each user listed in /etc/passwd file Generally, shadow file entry looks as follows:

shadow-file

(Fig.01: /etc/shadow file fields)

  1. User name : It is your login name
  2. Password: It your encrypted password. The password should be minimum 6-8 characters long including special characters/digits
  3. Last password change (lastchanged): Days since Jan 1, 1970 that password was last changed
  4. Minimum: The minimum number of days required between password changes i.e. the number of days left before the user is allowed to change his/her password
  5. Maximum: The maximum number of days the password is valid (after that user is forced to change his/her password)
  6. Warn : The number of days before password is to expire that user is warned that his/her password must be changed
  7. Inactive : The number of days after password expires that account is disabled
  8. Expire : days since Jan 1, 1970 that account is disabled i.e. an absolute date specifying when the login may no longer be used

The last 6 fields provides password aging and account lockout features (you need to use chage command to setup password aging). According to man page of shadow – the password field must be filled. The encrypted password consists of 13 to 24 characters from the 64 character alphabet a through z, A through Z, 0 through 9, \. and /. Optionally it can start with a “$” character. This means the encrypted password was generated using another (not DES) algorithm. For example if it starts with “$1$” it means the MD5-based algorithm was used.


Understanding /etc/passwd

Q. Can you explain /etc/passwd file format for Linux and UNIX operating systems?

A. /etc/passwd file stores essential information, which is required during login i.e. user account information. /etc/passwd is a text file, that contains a list of the system’s accounts, giving for each account some useful information like user ID, group ID, home directory, shell, etc. It should have general read permission as many utilities, like ls use it to map user IDs to user names, but write access only for the superuser (root).

Understanding fields in /etc/passwd

The /etc/passwd contains one entry per line for each user (or user account) of the system. All fields are separated by a colon (:) symbol. Total seven fields as follows.

Generally, passwd file entry looks as follows (click to enlarge image):

passwd-file

Fig.01: /etc/passwd file format

  1. Username: It is used when user logs in. It should be between 1 and 32 characters in length.
  2. Password: An x character indicates that encrypted password is stored in /etc/shadow file.
  3. User ID (UID): Each user must be assigned a user ID (UID). UID 0 (zero) is reserved for root and UIDs 1-99 are reserved for other predefined accounts. Further UID 100-999 are reserved by system for administrative and system accounts/groups.
  4. Group ID (GID): The primary group ID (stored in /etc/group file)
  5. User ID Info: The comment field. It allow you to add extra information about the users such as user’s full name, phone number etc. This field use by finger command.
  6. Home directory: The absolute path to the directory the user will be in when they log in. If this directory does not exists then users directory becomes /
  7. Command/shell: The absolute path of a command or shell (/bin/bash). Typically, this is a shell. Please note that it does not have to be a shell.

Task: See User List

/etc/passwd is only used for local users only. To see list of all users, enter:
$ cat /etc/passwd
To search for a username called tom, enter:
$ grep tom /etc/passwd

/etc/passwd file permission

The permission on the /etc/passwd file should be read only to users (-rw-r–r–) and the owner must be root:
$ ls -l /etc/passwd
Output:

-rw-r--r-- 1 root root 2659 Sep 17 01:46 /etc/passwd

Reading /etc/passwd file

You can read /etc/passwd file using the while loop and IFS separator as follows:

#!/bin/bash
# seven fields from /etc/passwd stored in $f1,f2...,$f7
# 
while IFS=: read -r f1 f2 f3 f4 f5 f6 f7
do
 echo "User $f1 use $f7 shell and stores files in $f6 directory."
done < /etc/passwd

Your password is stored in /etc/shadow file

Your encrypted password is not stored in /etc/passwd file. It is stored in /etc/shadow file. In the good old days there was no great problem with this general read permission. Everybody could read the encrypted passwords, but the hardware was too slow to crack a well-chosen password, and moreover, the basic assumption used to be that of a friendly user-community.

Almost, all modern Linux / UNIX line operating systems use some sort of the shadow password suite, where /etc/passwd has asterisks (*) instead of encrypted passwords, and the encrypted passwords are in /etc/shadow which is readable by the superuser only.


How to check if a directory exists in a shell script

To check if a directory exists and is a directory use the following syntax:

[ -d "/path/to/dir" ] && echo "Directory /path/to/dir exits." || echo "Error: Directory /path/to/dir does not exits."

The following version also check for symbolic link:

[ -d "/path/to/dir" && ! -L "/path/to/dir" ] && echo "Directory /path/to/dir exits." || echo "Error: Directory /path/to/dir exits but point to $(readlink -f /path/to/dir)."

OR

[ -d "/path/to/dir" && ! -h "/path/to/dir" ] && echo "Directory /path/to/dir exits." || echo "Error: Directory /path/to/dir exits but point to $(readlink -f /path/to/dir)."

Finally, you can use the traditional if..else..fi:

if [ -d "/path/to/dir" ]
then
    echo "Directory /path/to/dir exits."
else
    echo "Error: Directory /path/to/dir does not exits."
fi

Shell script examples to see if a ${directory} exists or not

 
#!/bin/bash
dir="$1"

[ $# -eq 0 ] && { echo "Usage: $0 dir-name"; exit 1; }

if [ -d "$dir" -a ! -h "$dir" ]
then
   echo "$dir found and setting up new Apache/Lighttpd/Nginx jail, please wait..."
   # __WWWJailSetup "cyberciti.biz" "setup"
else
   echo "Error: $dir not found or is symlink to $(readlink -f ${dir})."
fi

In this example, create directories if does not exits:

# Purpose: Setup jail and copy files
# Category : Core
# Override : No
# Parameter(s) : d => domain name
#                action => setup or update
__WWWJailSetup(){
        local d="$1"
        local action="${2:setup}"       # setup or update???
        local index="$d

$d

" # default index.html
        local J="$(_getJailRoot $d)/$d" # our sweet home 
        local _i=""

        [ "$action" == "setup" ] && echo "* Init jail config at $J..." || echo "* Updating jail init config at $J..."
        __init_domain_config "$d"

        [ "$action" == "setup" ] && echo "* Setting up jail at $J..." || echo "* Updating jail at $J..."
        [ ! -d "$J" ] &&  $_mkdir -p "$J"

        for _i in $J/{etc,tmp,usr,var,home,dev,bin,lib64}
        do
                [ ! -d "$_i" ] &&  $_mkdir -p "$_i"
        done
        for _i in $_lighttpd_webalizer_base/$d/stats/{dump,out}
        do
                [ ! -d "$_i" ] &&  $_mkdir -p "$_i"
        done
        for _i in $_lighttpd_webalizer_prepost_base/$d/{pre.d,post.d}
        do
                [ ! -d "$_i" ] &&  $_mkdir -p "$_i"
        done
## truncated 
}

Summary

Use the following to check file/directory types and compare values:

  1. -L "FILE" : FILE exists and is a symbolic link (same as -h)
  2. -h "FILE" : FILE exists and is a symbolic link (same as -L)
  3. -d "FILE" : FILE exists and is a directory
  4. -w "FILE" : FILE exists and write permission is granted

 


Bash For Loop Examples

How do I use bash for loop to repeat certain task under Linux / UNIX operating system? How do I set infinite loops using for statement? How do I use three-parameter for loop control expression?

A ‘for loop’ is a bash programming language statement which allows code to be repeatedly executed. A for loop is classified as an iteration statement i.e. it is the repetition of a process within a bash script.

For example, you can run UNIX command or task 5 times or read and process list of files using a for loop. A for loop can be used at a shell prompt or within a shell script itself.

for loop syntax

Numeric ranges for syntax is as follows:

for VARIABLE in 1 2 3 4 5 .. N
do
	command1
	command2
	commandN
done

This type of for loop is characterized by counting. The range is specified by a beginning (#1) and ending number (#5). The for loop executes a sequence of commands for each member in a list of items. A representative example in BASH is as follows to display welcome message 5 times with for loop:

#!/bin/bash
for i in 1 2 3 4 5
do
   echo "Welcome $i times"
done

Sometimes you may need to set a step value (allowing one to count by two’s or to count backwards for instance). Latest bash version 3.0+ has inbuilt support for setting up ranges:

#!/bin/bash
for i in {1..5}
do
   echo "Welcome $i times"
done

Bash v4.0+ has inbuilt support for setting up a step value using {START..END..INCREMENT} syntax:

#!/bin/bash
echo "Bash version ${BASH_VERSION}..."
for i in {0..10..2}
  do
     echo "Welcome $i times"
 done

Sample outputs:

Bash version 4.0.33(0)-release...
Welcome 0 times
Welcome 2 times
Welcome 4 times
Welcome 6 times
Welcome 8 times
Welcome 10 times

The seq command (outdated)

WARNING! The seq command print a sequence of numbers and it is here due to historical reasons. The following examples is only recommend for older bash version. All users (bash v3.x+) are recommended to use the above syntax.

The seq command can be used as follows. A representative example in seq is as follows:

#!/bin/bash
for i in $(seq 1 2 20)
do
   echo "Welcome $i times"
done

There is no good reason to use an external command such as seq to count and increment numbers in the for loop, hence it is recommend that you avoid using seq. The builtin command are fast.

Three-expression bash for loops syntax

This type of for loop share a common heritage with the C programming language. It is characterized by a three-parameter loop control expression; consisting of an initializer (EXP1), a loop-test or condition (EXP2), and a counting expression (EXP3).

for (( EXP1; EXP2; EXP3 ))
do
	command1
	command2
	command3
done

A representative three-expression example in bash as follows:

#!/bin/bash
for (( c=1; c<=5; c++ ))
do
	echo "Welcome $c times..."
done

Sample output:

Welcome 1 times
Welcome 2 times
Welcome 3 times
Welcome 4 times
Welcome 5 times

How do I use for as infinite loops?

Infinite for loop can be created with empty expressions, such as:

#!/bin/bash
for (( ; ; ))
do
   echo "infinite loops [ hit CTRL+C to stop]"
done

Conditional exit with break

You can do early exit with break statement inside the for loop. You can exit from within a FOR, WHILE or UNTIL loop using break. General break statement inside the for loop:

for I in 1 2 3 4 5
do
  statements1      #Executed for all values of ''I'', up to a disaster-condition if any.
  statements2
  if (disaster-condition)
  then
	break       	   #Abandon the loop.
  fi
  statements3          #While good and, no disaster-condition.
done

Following shell script will go though all files stored in /etc directory. The for loop will be abandon when /etc/resolv.conf file found.

#!/bin/bash
for file in /etc/*
do
	if [ "${file}" == "/etc/resolv.conf" ]
	then
		countNameservers=$(grep -c nameserver /etc/resolv.conf)
		echo "Total  ${countNameservers} nameservers defined in ${file}"
		break
	fi
done

Early continuation with continue statement

To resume the next iteration of the enclosing FOR, WHILE or UNTIL loop use continue statement.

for I in 1 2 3 4 5
do
  statements1      #Executed for all values of ''I'', up to a disaster-condition if any.
  statements2
  if (condition)
  then
	continue   #Go to next iteration of I in the loop and skip statements3
  fi
  statements3
done

This script make backup of all file names specified on command line. If .bak file exists, it will skip the cp command.

#!/bin/bash
FILES="$@"
for f in $FILES
do
        # if .bak backup file exists, read next file
	if [ -f ${f}.bak ]
	then
		echo "Skiping $f file..."
		continue  # read next file and skip cp command
	fi
        # we are hear means no backup file exists, just use cp command to copy file
	/bin/cp $f $f.bak
done

Recommended readings:

  • man bash
  • help for
  • help {
  • help break
  • help continue

Updated for accuracy!