About bash scripting : operators, expansions, keywords & special variables

bash shell

Introduction

This page lists some (most) of the bash operators (unary, comparison) as well as bash special variables and most of the parameters expansions (for array please refer to the array page).

Feel free to add you comments or tips!

 

1 Bash special Variables

Here are some of the most useful bash specials variables.

Variable Value
$0 Filename of script (this is NOT a positional parameter)
$FUNCNAME the function name (attention: inside a function, $0 is still the $0 of the shell, not the function name)
$1 – $9 Positional parameter #1 (exists from 1 up to 9)
${10} Positional parameter beyond 9 (#10 here) (note the brackets usage “{}”)
$# Number of arguments passed to the script
$* or $@ All the positional parameters (with no protection) (eg: $1 $2 $3) NOT RECOMMENDED
"$*" All the positional parameters except $0 (as a single word) (eg: “$1 $2 $3”)
"$@" All positional parameters except $0 (as separate strings) (eg: “$1” “$2” “$3”) THIS IS THE RECOMMENDED FORM for most usage
$# Number of positional parameters
$? Return value for LAST command / function / script
$$ Process ID (PID) of script
$- Flags passed to script (using set)
$_ Last argument of previous command
$! Process ID (PID) of last job run in background

 

2 Bash unary operators

Unary means that those operators performs operation on one operand (file / dir etc…), on the opposite we have binary operators often called comparison operators.

Note : You should prefer the “[[ ]]” instead of “[ ]” to run test using operators, the former is much more robust. There is an exception when portability is required, in this very case prefer the latter (“[ ]”). Check this page out for more info.

  • Example : All of those operators may be used with the following form (using the “-f” test here)
    if [[ -f "${file}" ]]
  • A complete form may looks like
    if [[ -f "${file}" ]] ; then
      echo " "${file}" is a file"
    else
      echo " "${file}" is NOT a file"
    fi
Operator(s)
true if
-a <file> <file> exists, deprecated same as -e
-b <file> <file> exists, and is a block mode device
-c <file> <file> exists and is a character mode device
-d <file> <file> exists and is a directory
-e <file> <file> exists, same as -a
-f <file> <file> exists and is a regular file
-g <file> <file> exists and has its SGID bit active
-G <file> <file> exists and is owned by the real group ID (same GID as yours)
-h <file> <file> exists and is a SYMLINK, same as -l
-k <file> <file> exists and has its Sticky bit active
-L <file> <file> exists and is a SYMLINK, same as -h
-n <string> <string> is not null
-N <file> <file> has been modified since its last access
-O <file> <file> exists and is owned by the real user ID (you are the actual owner)
-p <file> <file> exists and is a pipe , named or not ( FIFO file)
-r <file> <file> exists and can be read
-s <file> <file> exists and is not empty
-S <file> <file> exists and is a socket
-t N The file descriptor N linking to a terminal
-u <file> <file> exists and has its SUID bit active
-w <file> <file> exists and is can be modified
-x <file> <file> exists and can be executed, OR <file> is a directory that can be listed
-z <string> <file> has a lenght equal to zero (is empty)

 

3 Bash comparison operators

  • Example : All of those operators may be used with the following form (using the = test here)
    if [[ "${var_1}" = "${var_2}" ]]
  • A complete form may looks like
    if [[ "${var_1}" = "${var_2}" ]] ; then
      echo " "${var_1}" is the same as "${var_2}" "
    else
      echo " "${var_1}" and "${var_2}" are different "
    fi
Operator(s) true if example
<fileA> -nt <fileB> the last mod time of <fileA> is newer than <fileB>
<fileA> -ot <fileB> the last mod time of <fileA> is older than <fileB>
<fileA> -ef <fileB> <file> exists and is a character mode device
<stringA> = <stringB> <stringA>is equal to <stringB> (POSIX version)
<stringA> == <stringB> <stringA> is equal to <stringB>
<stringA> ! <stringB> <stringA>is different than <stringB>
<stringA> =~ regexp <stringA> matches regexp (use only within a [[…]] structure !)
<stringA> > <stringB> <stringA> is before <stringB> in lexicographic order
<stringA> < <stringB> <stringA> is after <stringB> in lexicographic order
<exprA> -eq <exprB> arithmethic <exprA> is equal to arithmethic <exprB>
<exprA> -ne <exprB> arithmethic <exprA> is not equal to arithmethic <exprB>
<exprA> -lt <exprB> arithmethic <exprA> is lesser than arithmethic <exprB>
<exprA> -gt <exprB> arithmethic <exprA> is greater than arithmethic <exprB>
<exprA> -le <exprB> arithmethic <exprA> is lesser than or equal to arithmethic <exprB>
<exprA> -ge <exprB> arithmethic <exprA> is greater than or equal to arithmethic <exprB>
<exprA> -a <exprB> <exprA> AND <exprB> is true
<exprA> -o <exprB> <exprA> OR <exprB> is true

 

4 Bash redirections

Here are some bash redirections.

Redirections syntaxes can be tricky sometimes, especially when the coder does not have a “writing clear code”  habits. And sometimes even with a “guy who cares” it can be difficult to read and/or predict. I post here some basic examples to try to help us to understand this cryptic bash feature.

  • Default file descriptors
    Bash opened by default (for any new shell) 3 files (each associated to a file descriptor) as :

    • The standard input (stdin) : FD=0 (usually your keyboard)
    • The standard output (stdout) : FD=1 (usually your monitor)
    • The standard error (stderr) : FD=2 (usually also your monitor)

     

    Those file descriptor can be use to redirect part (or complete) information given by the shell when executing commands to different location, such as a file or another FD.

  • Basic redirections
    • Redirect stdout to a file
      echo "redirection testing ..." > file.txt  # This will put the echo stdout (redirection testing ...) in file.txt, overwriting previous file.txt content
    • Redirect (append) stdout to a file
      echo "redirection testing ..." >> file.txt  # This will <em>append</em> the echo stdout (redirection testing ...) in file.txt
    • Redirect stderr to a file (e.g : create an error log). Note : Appending can be done the same way it was for the stdout
      cat non-existent_file.txt 2> error.log  # error.log would contains the cat command error message (cat: file.t: No such file or directory)
    • Taking command input from stdin (as opposite to “taking input from a file“) can be done by using the “-” sign which actually stands for stdin
      cat -  # this would give you an empty prompt and echo anything you'd enter at this empty prompt

      It is very common to use a pipe “|” to send stdout from a former command to the stdin of a latter command, as :

      echo "testing string" | cat -  # this would print "testing string" from the cat command (not from the echo command)

      Note : Some commands does NOT take their input from stdin ! (see the echo command for instance)

    • Redirect stdout AND stderr to a file
      cat existent-file.txt non-existent-file.txt &> cat.log  # both syntax are equivalent (>& and &>)
      cat existent-file.txt non-existent-file.txt >& cat.log

      Note : I spoke about cryptic form above, here is an example that does the same as the previous redirection :

      cat existent-file.txt non-existent-file.txt >cat.log 2>&1

      The bad news is : the latter syntax is RECOMMANDED
      The good news is : there is no good news. USE THE RECOMMENDED FORM !

See the wiki-bash-hackers redirection page for the most complete reference out there.

Operator(s) true if example
<cmd1> | <cmd2> pipe : <cmd1> stdout is sent to <cmd2> stdin
> <file> stdout is sent to <file>
< <file> stdin is read from <file>
&> <file> stdout & stderr are sent to <file>
2>&1 stderr is redirected to stdout (the “&” define the following argument, “1” in this case, is not a file but a file descriptor (“0” stands for stdin, “1” stands for stdout, “2” for stderr …)
2>&- stderr to a closing file descriptor (may replace the good ol’ /dev/null sometimes)
>> <file> stdout is sent to <file>, if <file> already exists it (stdout) is appended
>| <file> stdout is sent to <file>, even if noclobber is set
n >| <file> stdout of file descriptor n is sent to <file> even if noclobber is set
<> <file> <file> is used as stdout and stdin
n<> <file> <file> is used as stdout and stdin for file descriptor n
<< <flag> here document. Note the one space between the 2 “chevrons” and the <flag>.
<<<“string” here string. Note the lack of space between the 3 “chevrons” and the <string>.
n> <file> file descriptor n is sent to <file>

 

5 Bash parameters expansions

(from wiki bash hackers)

 

6 Differences between [ ] and [[ ]]

The [ ] construct is exactly the same as the test command. [ ] is known as the old or classic test command, in opposition to the new test command [[ ]].

The [[ ]] construct is known as the “new” test command or the “conditional expression“.

They both have specific and common operators, see the table below (which i stole from here, i just changed the examples because i found them not simple enough).

Feature New test Old test Example
string comparison > \> (*)
< \< (*)
= (or ==) =
!= !=
integer comparison -gt -gt
-lt -lt
-ge -ge
-le -le
-eq -eq
-ne -ne
conditional evaluation && -a (**)
|| -o (**)
expression grouping (...) \( ... \) (**)
Pattern matching = (or ==) (not available)
RegularExpression matching =~ (not available)

Note See the GreyCat wiki for complete references.

 

7 Differences between [[ ]] and (( ))

TODO !

 

8 Handy tips

This section gives out some tips for everyday scripting situations.

8.1 Test if a variable is “set/not-set” or “set but null”

In this section we will  try to define if a variable is “set or not” or “set but empty”

  • Let us first define the “not set” expression :
    The “not set” term describe a variable that has not been initialized : it has no defined value

    unset test_variable    # This would set a variable to a "not set" status


  • Now let’s define the “null” expression :
    The “null” term describe a variable that has been set but without any value

    Test_variable=      # These 2 forms would set Test_variable to an empty value : null.
    Test_variable=""

 

Note : You might see a lot of different ways to test whether a variable is set or not, or null or not on the web, but the forms i present here are the only ones that are sure, convenient and “portable”, i would recommend to use these.

 

8.1.1 Test if a variable is set or not

We here want to know if a variable called testVar is set or not (we do not care to check for its value), if not we want to print a message.

unset testVar        # unset the variable
echo ${testVar?}     # test the variable
bash: testVar: parameter null or not set     # Get an error message for a "not set" variable

We can see in this example that the ${var_name?} form return an error if the tested variable s not set, but if the tested var is set but empty (set to a “null” value) this form will not return any error, instead it returns the variable (empty) value, as :

declare testVar=""     # set the variable to an "empty or null value "
echo ${testVar?}       # test the variable  
# empty line here      # Get NO error message for a "set to a null value" variable

 

8.1.2 Test if a variable is null or not set

This second form will return an error whether  the variable is not-set or set to a null value.

unset testVar           # unset the variable
echo ${testVar:?}       # test the variable
bash: testVar: parameter null or not set     # Get an error message for a "not set" variable
declare testVar=""      # set the variable to an "empty or null value "
echo ${testVar:?}       # test the variable  
bash: testVar: parameter null or not set     # Get an error message for a "not set" variable

 

I am sure you noticed the slight difference between the two forms : the semicolon (“:”), this is how we define if we want to test only for the “set/not-set” (without the semicolon) or for both “set/not-set” AND “null/not-null” status (with the semicolon).

 

8.2 Test if a variable set AND not-null

For this purpose we use the test buit-in command in its two different forms : test and [ ], as well as the new test command : [[ ]].
Once again the best way to avoid error when using the test command (or the [[ ]] form) is to use double-quoted and bracketed variable name, this way you will be protected against “missing parameter” error or the other “[: too many arguments"

  1. The classic test command
    [User@Test ~]$ declare TESTVAR      # Let's first declare the TESTVAR as an empty variable
    [User@Test ~]$ test "${TESTVAR}"    # Now we try the test command
    [User@Test ~]$ echo $?              # We can see that it returns 1 because the variable exists but is EMPTY
    1
    [User@Test ~]$ declare TESTVAR=1      # Let's now declare the TESTVAR as NOT empty variable
    [User@Test ~]$ test "${TESTVAR}"      # Now we use the test command again
    [User@Test ~]$ echo $?                # We can see that it returns 0 because the variable exists AND is NOT EMPTY
    0
  2. The not less classic [ ]
    The exactly same as above, unless we change the test command to the [ ] one.

    [User@Test ~]$ declare TESTVAR     # Let's first declare the TESTVAR as an empty variable
    [User@Test ~]$ [ "${TESTVAR}" ]    # Now we try the test command
    [User@Test ~]$ echo $?             # We can see that it returns 1 because the variable exists but is EMPTY
    1
    [User@Test ~]$ declare TESTVAR=1     # Let's now declare the TESTVAR as NOT empty variable
    [User@Test ~]$ [ "${TESTVAR}" ]      # Now we use the test command again
    [User@Test ~]$ echo $?               # We can see that it returns 0 because the variable exists AND is NOT EMPTY
    0
  3. We could also use the newer operator [[ ]]
    [User@Test ~]$ declare TESTVAR     # Let's first declare the TESTVAR as an empty variable
    [User@Test ~]$ [[ "${TESTVAR}" ]]    # Now we try the test command
    [User@Test ~]$ echo $?             # We can see that it returns 1 because the variable exists but is EMPTY
    1
    [User@Test ~]$ declare TESTVAR=1     # Let's now declare the TESTVAR as NOT empty variable
    [User@Test ~]$ [[ "${TESTVAR}" ]]      # Now we use the test command again
    [User@Test ~]$ echo $?               # We can see that it returns 0 because the variable exists AND is NOT EMPTY
    0
    bash-3.2$ unset test
    bash-3.2$ if [[ "$test" ]] ; then echo "SET AND NOT-NULL" ; else echo "NOT-SET OR NULL" ; fi
    NOT-SET OR NULL
    bash-3.2$ declare test
    bash-3.2$ if [[ "$test" ]] ; then echo "SET AND NOT-NULL" ; else echo "NOT-SET OR NULL" ; fi
    NOT-SET OR NULL
    bash-3.2$ declare test=1
    bash-3.2$ if [[ "$test" ]] ; then echo "SET AND NOT-NULL" ; else echo "NOT-SET OR NULL" ; fi
    SET AND NOT-NULL

 

 

Resources

Leave a Reply

Your email address will not be published. Required fields are marked *

This site supports SyntaxHighlighter via WP SyntaxHighlighter. It can highlight your code.
How to highlight your code: Paste your code in the comment form, select it and then click the language link button below. This will wrap your code in a <pre> tag and format it when submitted.