I believe mastering the for loop in Bash on Linux is one of the fundamentals for Linux sysadmins (and even developers!) that takes your automation skills to the next level. In this post I explain how they work and offer some useful examples.

Update 07/06/2016: lots of critique on Reddit (granted: well deserved), so I updated most of the examples on this page for safer/saner defaults.

Let me first start by saying something embarrassing. For the first 4 or 5 years of my Linux career – which is nearing 10 years of professional experience – I never used loops in Bash scripts. Or at the command line.

The thing is, I was a very fast mouse-clicker. And a very fast copy/paster. And a good search & replacer in vim and other text editors. Quite often, that got me to a working solution faster than working out the quirky syntax, testing, bugfixing, … of loops in Bash.

And, to be completely honest, if you’re managing just a couple of servers, I think you can get away with not using loops in Bash. But, once you master it, you’ll wonder why you haven’t learned Bash for-loops sooner.

The Bash For Loop: Example

First, let me show you the most basic – and the one you’ll see most often – form of a Bash loop.

#!/bin/bash
for i in 1 2 3 4 5; do
  echo "counter: $i"
done

If you execute such a script, it outputs like this.

$ ./script.sh
counter: 1
counter: 2
counter: 3
counter: 4
counter: 5

Pretty basic, right? Here’s what it breaks down to.

The first part, #!/bin/bash, is the shebang, somethings called a hashbang. It indicates which interpreter is going to be used to parse the rest of the script. In short, it’s what makes this a Bash script.

The rest is where the Bash for loop actually comes in.

1. for: indicates that this is a loop, and that you’d like to iterate (or “go over”) multiple items.
2. i: a placeholder for a variable, which can later be referenced as $i. i is often used by developers to loop or iterate over an array or a hash, but this can be anything ^(*). For clarity, it could also have been named counter, the variable to reference it later on would then be $counter, with a dollar sign.
3. in: a keyword, indicating the separator between the variable i and the collection of items to run over.
4. 1 2 3 4 5: whichever comes between the in keyword and the ; delimiter is the collection of items you want to run through. In this example, the collection “1 2 3 4 5” is considered a set of 5 individual items.
5. do: this keyword defines that from this point on, the loop starts. The code that follows will be executed n times, where n is the amount of items that’s in the collection, in this case a set of 5 digits.
6. echo "counter: $i": this is the code inside the loop, that will be repeated – in this case – 5 times. The $i variable is the individual value of each item.
7. done: this keyword indicates that the code that should be repeated in this loop, has finished.

^{(*) Technically, the variable can’t be anything as there are limitations of what characters can be used in a variable, but that’s beyond the scope here. Keep it to alphanumerics without spaces or special chars, and you’re probably safe.}

Lots of text, isn’t it?

Well, look at the screenshot again and just remember the different parts of the for loop. And remember also that this isn’t limited to actual “scripts”, it can be concatenated to a single line for use at the command line, too.

$ for i in 1 2 3 4 5; do echo "counter: $i"; done

The same breakdown occurs there.

There is one, important difference though. Right before the last done keyword, there is a semicolon ; to indicate that the command ends there. In the Bash script, that isn’t needed, because the done keyword is placed on a new line, also ending the command above.

Actually, that very first example I showed you? It can be rewritten without a single ;, by just new-lining each line.

#!/bin/bash
for i in 1 2 3 4 5
do
  echo "counter: $i"
done

You can pick which ever style you prefer or find most readable/maintainable.

The result is exactly the same: a set of items is looped and for each occurrence, an action is taken.

Values to loop in Bash

Looping variables isn’t very exciting in and of itself, but it gets very useful once you start to experiment with the data to loop through.

For instance:

$ for file in *; do echo "$file"; done
file1.txt
file2.txt
file3.txt

Granted, you can just do ls and get the same value, but you can use * inside your for statement, which is essentially the same as an $(ls), but with safe output. It’s execute before the actual for-loop and the output is being used as the collection of items to iterate.

This opens up a lot of opportunities, especially if you take the seq command in mind. With seq you can generate sequences at the CLI.

For instance:

$ seq 25 30
25
26
27
28
29
30

That generates the numbers 25 through 30. So if you’d like to loop items 1 until 255, you can do this:

$ for counter in $(seq 1 255); do echo "$counter"; done

When would you use that? Maybe to ping a couple of IPs or connect to multiple remote hosts and fire of a few commands.

$ for counter in $(seq 1 255); do ping -c 1 "10.0.0.$counter"; done
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
...

Now we’re talking.

Ranges in Bash

You can also use some of the built-in Bash primitives to generate a range, without using seq. The code below does exactly the same as the ping example above.

$ for counter in {1..255}; do ping -c 1 10.0.0.$counter; done

More recent Bash version (Bash 4.x at least) can also modify this command increase the step by which each integer increments. By default it’s always +1, but you can make that +5 if you like.

$ for counter in {1..255..5}; do echo "ping -c 1 10.0.0.$counter"; done
ping -c 1 10.0.0.1
ping -c 1 10.0.0.6
ping -c 1 10.0.0.11
ping -c 1 10.0.0.16

Looping items like this can allow you to quickly automate an otherwise mundane task.

Chaining multiple commands in the Bash for loop

You obviously aren’t limited to a single command in a for loop, you can chain multiple ones inside the for-loop.

#!/bin/bash
for i in 1 2 3 4 5; do
  echo "Hold on, connecting to 10.0.1.$i"
  ssh root@"10.0.1.$i" uptime
  echo "All done, on to the next host!"
done

Or, at the command line as a one-liner:

$ for i in 1 2 3 4 5; do echo "Hold on, connecting to 10.0.1.$i"; ssh root@"10.0.1.$i" uptime; echo "All done, on to the next host"; done

You can chain multiple commands with the ; semicolon, the last command will be the done keyword to indicate you’re, well, done.

Bash for-loop examples

Here are a couple of “bash for loop” examples. They aren’t necessarily the most useful ones, but show some of the possibilities.

For each user on the system, write their password hash to a file named after them

One-liner:

$ for username in $(awk -F: '{print $1}' /etc/passwd); do grep $username /etc/shadow | awk -F: '{print $2}' > $username.txt; done

Script:

#!/bin/bash
for username in $(awk -F: '{print $1}' /etc/passwd)
do
  grep $username /etc/shadow | awk -F: '{print $2}' > $username.txt
done

Rename all *.txt files to remove the file extension

One-liner:

$ for filename in *.txt; do mv "$filename" "${filename%.txt}"; done

Script:

!#/bin/bash
for filename in *.txt
do
  mv "$filename" "${filename%.txt}"
done

Use each line in a file as an IP to connect to

One-liner:

$ for ip in $(cat ips.txt); do ssh root@"$ip" yum -y update; done

Script:

#!/bin/bash
for ip in $(cat ips.txt)
do
  ssh root@"$ip" yum -y update
done

Debugging for loops in Bash

Here’s a one way I really like to debug for-loops: just echo everything. This is also a great way to “generate” a static Bash script, by catching the output.

For instance, in the ping example, you can do this:

$ for counter in {1..255..5}; do echo "ping -c 1 10.0.0.$counter"; done

That will echo each ping statement. Now you can also catch that output, write it to another Bash file and keep it for later (or modify manually if you’re struggling with the Bash loop – been there, done that).

$ for counter in {1..255..5}; do echo "ping -c 1 10.0.0.$counter"; done > ping-all-the-things.sh
$ more ping-all-the-things.sh
ping -c 1 10.0.0.1
ping -c 1 10.0.0.6
ping -c 1 10.0.0.11
...

It may be primitive, but this gets you a very long way!