Variable Assignment Bash Shell Script Variables

Variables!

Temporary stores of information

Introduction

For those of you that have dabbled in programming before, you'll be quite familiar with variables. For those of you that haven't, think of a variable as a temporary store for a simple piece of information. These variables can be very useful for allowing us to manage and control the actions of our Bash Script. We'll go through a variety of different ways that variables have their data set and ways we can then use them.

Variables are one of those things that are actually quite easy to use but are also quite easy to get yourself into trouble with if you don't properly understand how they work. As such there is a bit of reading in this section but if you take the time to go through and understand it you will be thankful you did later on when we start dabbling in more complex scripts.

How do they Work?

A variable is a temporary store for a piece of information. There are two actions we may perform for variables:

  • Setting a value for a variable.
  • Reading the value for a variable.

Variables may have their value set in a few different ways. The most common are to set the value directly and for its value to be set as the result of processing by a command or program. You will see examples of both below.

To read the variable we then place its name (preceded by a $ sign) anywhere in the script we would like. Before Bash interprets (or runs) every line of our script it first checks to see if any variable names are present. For every variable it has identified, it replaces the variable name with its value. Then it runs that line of code and begins the process again on the next line.

Here are a few quick points on syntax. They will be elaborated on and demonstrated as we go into more detail below.

  • When referring to or reading a variable we place a $ sign before the variable name.
  • When setting a variable we leave out the $ sign.
  • Some people like to always write variable names in uppercase so they stand out. It's your preference however. They can be all uppercase, all lowercase, or a mixture.
  • A variable may be placed anywhere in a script (or on the command line for that matter) and, when run, Bash will replace it with the value of the variable. This is made possible as the substitution is done before the command is run.

Command line arguments

Command line arguments are commonly used and easy to work with so they are a good place to start.

When we run a program on the command line you would be familiar with supplying arguments after it to control its behaviour. For instance we could run the command ls -l /etc. -l and /etc are both command line arguments to the command ls. We can do similar with our bash scripts. To do this we use the variables $1 to represent the first command line argument, $2 to represent the second command line argument and so on. These are automatically set by the system when we run our script so all we need to do is refer to them.

Let's look at an example.

mycopy.sh

  1. #!/bin/bash
  2. cp $1 $2
  3. echo Details for $2
  4. ls -lh $2

Let's break it down:

  • Line 4 - run the command cp with the first command line argument as the source and the second command line argument as the destination.
  • Line 8 - run the command echo to print a message.
  • Line 9 - After the copy has completed, run the command ls for the destination just to verify it worked. We have included the options l to show us extra information and h to make the size human readable so we may verify it copied correctly.
  1. ./mycopy.sh /projects/file1.data ./results.data
  2. Details for ./results.data
  3. -rw-r--r-- 18 ryan users 3.4M Feb 14 07:18 results.data

We'll discuss their usage a little more in the next section ( 3. Input ).

Other Special Variables

There are a few other variables that the system sets for you to use as well.

  • $0 - The name of the Bash script.
  • $1 - $9 - The first 9 arguments to the Bash script. (As mentioned above.)
  • $# - How many arguments were passed to the Bash script.
  • $@ - All the arguments supplied to the Bash script.
  • $? - The exit status of the most recently run process.
  • $$ - The process ID of the current script.
  • $USER - The username of the user running the script.
  • $HOSTNAME - The hostname of the machine the script is running on.
  • $SECONDS - The number of seconds since the script was started.
  • $RANDOM - Returns a different random number each time is it referred to.
  • $LINENO - Returns the current line number in the Bash script.

If you type the command env on the command line you will see a listing of other variables which you may also refer to.

Some of these variables may seem useful to you now. Others may not. As we progress to more complex scripts in later sections you will see examples of how they can be useful.

Setting Our Own Variables

As well as variables that are preset by the system, we may also set our own variables. This can be useful for keeping track of results of commands and being able to refer to and process them later.

There are a few ways in which variables may be set (such as part of the execution of a command) but the basic form follows this pattern:

variable=value

This is one of those areas where formatting is important. Note there is no space on either side of the equals ( = ) sign. We also leave off the $ sign from the beginning of the variable name when setting it.

Variable names may be uppercase or lowercase or a mixture of both but Bash is a case sensitive environment so whenever you refer to a variable you must be consistent in your use of uppercase and lowercase letters. You should always make sure variable names are descriptive. This makes their purpose easier for you to remember.

Here is a simple example to illustrate their usage.

simplevariables.sh

  1. #!/bin/bash
  2. myvariable=Hello
  3. anothervar=Fred
  4. echo $myvariable $anothervar
  5. echo
  6. sampledir=/etc
  7. ls $sampledir

Let's break it down:

  • Lines 4 and 6 - set the value of the two variables myvariable and anothervar.
  • Line 8 - run the command echo to check the variables have been set as intended.
  • Line 9 - run the command echo this time with no arguments. This is a good way to get a blank line on the screen to help space things out.
  • Line 11 - set another variable, this time as the path to a particular directory.
  • Line 13 - run the command ls substituting the value of the variable sampledir as its first command line argument.
  1. ./simplevariables.sh
  2. Hello Fred
  3. a2ps.cfg aliases alsa.d ...

It is important to note that in the example above we used the command echo simply because it is a convenient way to demonstrate that the variables have actually been set. echo is not needed to make use of variables and is only used when you wish to print a specific message to the screen. (Pretty much all commands print output to the screen as default so you don't need to put echo in front of them.)

Variables can be useful for making our scripts easier to manage. Maybe our script is going to run several commands, several of which will refer to a particular directory. Rather than type that directory out each time we can set it once in a variable then refer to that variable. Then if the required directory changes in the future we only need to update one variable rather than every instance within the script.

Quotes

In the example above we kept things nice and simple. The variables only had to store a single word. When we want variables to store more complex values however, we need to make use of quotes. This is because under normal circumstances Bash uses a space to determine separate items.

  1. myvar=Hello World
  2. -bash: World: command not found
  • Remember, commands work exactly the same on the command line as they do within a script.

Because commands work exactly the same on the command line as in a script it can sometimes be easier to experiment on the command line.

When we enclose our content in quotes we are indicating to Bash that the contents should be considered as a single item. You may use single quotes ( ' ) or double quotes ( " ).

  • Single quotes will treat every character literally.
  • Double quotes will allow you to do substitution (that is include variables within the setting of the value).
  1. myvar='Hello World'
  2. echo $myvar
  3. Hello World
  4. newvar="More $myvar"
  5. echo $newvar
  6. More Hello World
  7. newvar='More $myvar'
  8. echo $newvar
  9. More $myvar

Command Substitution

Command substitution allows us to take the output of a command or program (what would normally be printed to the screen) and save it as the value of a variable. To do this we place it within brackets, preceded by a $ sign.

  1. myvar=$( ls /etc | wc -l )
  2. echo There are $myvar entries in the directory /etc

Command substitution is nice and simple if the output of the command is a single word or line. If the output goes over several lines then the newlines are simply removed and all the output ends up on a single line.

  1. ls
  2. bin Documents Desktop ...
  3. Downloads public_html ...
  4. myvar=$( ls )
  5. echo $myvar
  6. bin Documents Desktop Downloads public_html ...

Let's break it down:

  • Line 1 - We run the command ls. Normally its output would be over several lines. I have shortened it a bit in the example above just to save space.
  • Line 4 - When we save the command to the variable myvar all the newlines are stripped out and the output is now all on a single line.

When playing about with command substitution it's a good idea to test your output rather than just assuming it will behave in a certain way. The easiest way to do that is simply to echo the variable and see what has happened. (You can then remove the echo command once you are happy.)

Exporting Variables

Remember how in the previous section we talked about scripts being run in their own process? This introduces a phenomenon known as scope which affects variables amongst other things. The idea is that variables are limited to the process they were created in. Normaly this isn't an issue but sometimes, for instance, a script may run another script as one of its commands. If we want the variable to be available to the second script then we need to export the variable.

script1.sh

  1. #!/bin/bash
  2. var1=blah
  3. var2=foo
  4. echo $0 :: var1 : $var1, var2 : $var2
  5. export var1
  6. ./script2.sh
  7. echo $0 :: var1 : $var1, var2 : $var2

script2.sh

  1. #!/bin/bash
  2. echo $0 :: var1 : $var1, var2 : $var2
  3. var1=flop
  4. var2=bleh

Now lets run it and see what happens.

  1. ./script1.sh
  2. script1.sh :: var1 : blah, var2 : foo
  3. script2.sh :: var1 : blah, var2 :
  4. script1.sh :: var1 : blah, var2 : foo

The output above may seem unexpected. What actually happens when we export a variable is that we are telling Bash that every time a new process is created (to run another script or such) then make a copy of the variable and hand it over to the new process. So although the variables will have the same name they exist in separate processes and so are unrelated to each other.

Exporting variables is a one way process. The original process may pass variables over to the new process but anything that process does with the copy of the variables has no impact on the original variables.

Exporting variables is something you probably won't need to worry about for most Bash scripts you'll create. Sometimes you may wish to break a particular task down into several separate scripts however to make it easier to manage or to allow for reusability (which is always good). For instance you could create a script which will make a dated (ie todays date prepended to the filename) copy of all filenames exported on a certain variable. Then you could easily call that script from within other scripts you create whenever you would like to take a snapshot of a set of files.

Summary

$1, $2, ...
The first, second, etc command line arguments to the script.
variable=value
To set a value for a variable. Remember, no spaces on either side of =
Quotes " '
Double will do variable substitution, single will not.
variable=$( command )
Save the output of a command into a variable
export var1
Make the variable var1 available to child processes.
Formatting
The presence or absence of spaces is important.
Manageability
If a particular value is used several times within a script (eg a file or directory name) then using a variable can make it easier to manage.

Activities

Let's explore variables.

  • A good place to start is to create a simple script which will accept some command line arguments and echo out some details about them (eg, how many are there, what is the secone one etc).
  • Create a script which will print a random word. There is a file containing a list of words on your system (usually /usr/share/dict/words or /usr/dict/words). Hint: Piping will be useful here.
  • Expand the previous activity so that if a number is supplied as the first command line argument then it will select from only words with that many characters. Hint: Grep may be useful here.
  • Take a copy of the two files script1.sh and script2.sh above then experiment by tweaking them and running them and observing the output. This will help you get a feel for how exporting variables works.
  • Now let's create a script which will take a filename as its first argument and create a dated copy of the file. eg. if our file was named file1.txt it would create a copy such as 2018-03-13_file1.txt. (To achieve this you will probably want to play with command substitution and the command date)
  • Challenge: To make it a bit harder, see if you can get it so that the date is after the name of the file (eg. file1_2018-03-13.txt (The command basename can be useful here.)
  • Challenge: Now see if you can expand the previous question to accept a list of files on the command line and it will create a named copy of all of them. (The command xargs may be useful here.)

Bash variables and command substitution

An essential feature of programming is the ability to use a name or a label to refer to some other quantity: such as a value, or a command. This is commonly referred to as variables.

Variables can be used, at the very least, to make code more readable for humans:

However, variables really come into use in more advanced programming, when we're in a situation in which the actual values aren't known before executing a program. A variable acts as a placeholder that gets resolved upon actual execution time.

For example, imagine that contains a list of website addresses. The following routine reads each line (via , which isn't best practice…but will do for now) into a loop, which then downloads each URL:

Variables

Basic variable usage and syntax

Setting a variable

The following command assigns to the variable named , and to

Unlike most modern languages, Bash is pretty picky about the syntax for setting variables. In particular, no whitespace is allowed between the variable name, the equals sign, and the value.

All of these examples would cause Bash to throw an error:

Referencing the value of a variable

Whenever Bash encounters a dollar-sign, immediately followed by a word, within a command or in a double-quoted string, it will attempt to replace that token with the value of the named variable. This is sometimes referred to as expanding the variable, or parameter substitution:

Failure to dereference

When a dollar-sign doesn't precede a variable name, or a variable reference is within single-quotes, Bash will interpret the string literally:

Concatenating strings

Variables can be very useful for text-patterns that will be repeatedly used:

If your variable name butts up against a literal alphanumeric character, you can use this more verbose form, involving curly braces, to reference a variable's value:

Valid variable names

Variable names can contain a sequence of alphanumeric characters and underscores. For variables created by you, the user, they should start with either an alphabetical letter or an underscore (i.e. not a number):

Valid variable names:

    When we write functions and shell scripts, in which arguments are passed in to be processed, the arguments will be passed int numerically-named variables, e.g. , ,

    For example:

    Inside , commands will use to refer to , to , and for

    The variable reference, , will expand to the current script's name, e.g.


    Command substitution

    The standard output of a command can be encapsulated, much like a value can be stored in a value, and then expanded by the shell.

    This is known as command substitution. From the Bash documentation:

    Command substitution allows the output of a command to replace the command itself. Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting.

    As an example, consider the seq command, which will print a sequence of numbers starting from the first argument to the second argument:

    With command substitution, we can encapsulate the result of into a variable by enclosing the command with and , and pass it as an argument to another command:

    As a GIF:

    Variables and command expansion

    When a command is replaced by its standard output, that output, presumably just text, can be assigned to a variable like any other value:

    The loss of newlines in command substitution

    Earlier, I quoted from the Bash documentation on command expansion. Here's an emphasized version of the excerpt:

    Command substitution allows the output of a command to replace the command itself. Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting.

    What does that mean? Consider being called normally, and then, via command substitution, and note the change in formatting:

    Why do the newlines get removed during the command expansion? It's something we'll encounter later (and there's a section on it at the end of this tutorial) and deals with how Bash interprets space and newline characters during expansion. Anyway, it's worth noting the behavior for now, as it may be new to you if you're coming from another programming language.

    Arithmetic expansion

    To do basic calculations, you can enclose an expression inside :

    Check the Bash documentation for the full set of arithmetic operators. Math at the command-line can be a bit clunky so we won't be focusing too much on it.

    The bc utility

    An aside: if you want to do more advanced math from the command line, use bc, which reads in from stdout and evaluates the expression:

    Word-splitting in the wild

    This section covers more technical details of how Bash handles space characters when it does an exapansion. It's not necessary to memorize for the specific assignments in this class. However, as many of you are wont to copy and paste code directly from things you've seen on the Internet, it might be worth knowing all the different ways you could accidentally harm yourself, due to the way Bash handles spaces and newline characters.

    Here's the Bash documentation for the concept known as "word-splitting"

    The internal field separator

    The global variable is what Bash uses to split a string of expanded into separate words…think of it as how Excel knows to split a CSV (comma-separated-values) text file into a spreadsheet: it assumes the commas separate the columns.

    Let's pretend that has been set to something arbitrary, like a capital . When Bash expands a variable that happens to contain a , the value of that variable will be split into separate words (and the literal will disappear):

    By default, the variable is set to three characters: newline, space, and the tab. If you , you won't see anything because those characters…well, how do you see a space character if there aren't any visible characters?

    The upshot is that you may see code snippets online in which the variable is changed to something like (which stands for the newline character).

    Imagine a textfile that contains a bunch of lines of text that, for example, may refer to filenames:

    When Bash reads each line of the file, the default value of , which includes a space character, will cause Bash to treat the file named as two files, and , because the space character is used to split words.

    With set to just the newline character, is treated as a single filename.

    This concept will make sense when it comes to reading text files and operating on each line. I don't expect you to fully understand this, but only to be aware of it, just in case you are haphazardly copy-pasting code from the Internet.

    The dangers of unquoted variables

    In an ideal world, everyone would keep their string values short and without space/newline, or any other special characters. In that ideal world, the following unquoted variable reference would work just fine:

    But when people start adding special characters to filenames, such as spaces, expanding variables, without the use of double quotes, can be dangerous.

    In the following example, the programmer intends the file named to be deleted:

    Unexpected word-splitting

    However, when referenced without double-quotes, Bash sees as containing two separate values, and . The subsequent command will attempt to delete those two files, and not :

    Unexpected special characters in filenames

    Ah, no harm done, you say, because those files didn't exist in the first place. OK, but what happens when someone puts a star (i.e. asterisk) into a filename? You're aware of what happens when you do and – the star acts as a wildcard, grabbing every file.

    So you'll see the previous errors, since and don't exist. But in between those attempted deletions, will run on …so say bye-bye to every file in that directory.

    Here's the animated GIF version:

    Notice how affects only the file that is named, .

    So the main takeaway here is: double-quote your variable references whenever possible.

    To reiterate

    Expanding a variable can lead to unexpected and sometimes catastrophic results if the variable contains special characters:

    Expanding a variable within double-quotes can prevent such problems:

    Who would do such a thing?

    You might think, Who the hell puts star characters in their filenames? Well, besides people who really enjoy star-shaped symbols, malicious hackers and pranksters. And variables usually aren't just manually assigned by the result of human typing. As you've read above, sometimes the result of commands are stored in a variable. And if such commands are processing raw data, it's not unimaginable that the raw data, quite innocently, contains special characters that are destructive to certain Bash programs.

    For the purposes of the CompCiv course, the assignments will try to stay far from untrusted sources of data. But keep in mind the dangers of just pasting in seemingly safe-looking code. Bash's syntax and behavior in handling strings is hard to fully comprehend, which is why developers use other languages for more complex applications.

    You can read more about quoting variables. There's a lot of minutiae, but the main takeaway, besides general safety, is to have a general understanding how Bash, and any other programming environment, uses certain conventions and syntax rules to deal with the myriad ways that users want to pass around values in their programs.

    0 Thoughts to “Variable Assignment Bash Shell Script Variables

    Leave a comment

    L'indirizzo email non verrà pubblicato. I campi obbligatori sono contrassegnati *