Read a file line by line assigning the value to a variable

Read a File Line by Line and Assign to a Bash Variable

Learn various robust methods to read text files line by line in Bash, assigning each line's content to a variable for processing.

Reading a file line by line is a fundamental operation in shell scripting. Whether you're processing configuration files, log data, or lists of items, knowing how to iterate through each line and assign its content to a variable is crucial. This article explores several common and robust methods to achieve this in Bash, highlighting their advantages and potential pitfalls.

Method 1: Using a while read Loop

The while read loop is the most common and generally recommended way to read a file line by line in Bash. It's robust against filenames with spaces and handles most line-ending scenarios gracefully. The read command reads a single line from standard input, and the while loop continues as long as read successfully reads a line.

#!/bin/bash

FILE="my_file.txt"

while IFS= read -r line; do
  echo "Processing line: $line"
  # You can now use the '$line' variable for further processing
  # For example, store it in an array:
  # my_array+=("$line")
  # Or perform operations:
  # echo "Length of line: ${#line}"

done < "$FILE"

# After the loop, the '$line' variable will contain the last line read.
# If you need to access all lines, consider storing them in an array as shown above.

Method 2: Using for Loop with cat (Less Recommended)

While seemingly simpler, using a for loop with cat is generally discouraged for line-by-line processing, especially if lines can contain spaces or special characters. The for loop iterates over words, not lines, by default, and cat can introduce issues with large files or binary content. However, for very simple files with single-word lines, it might appear to work.

#!/bin/bash

FILE="my_file.txt"

# This is generally NOT recommended for line-by-line processing
# unless you are absolutely sure about the file content (no spaces, special chars).
for line in $(cat "$FILE"); do
  echo "Processing word (not necessarily line): $line"
done

# To make it work for lines with spaces, you'd need to change IFS,
# but even then, it's less robust than 'while read'.
# IFS=$'\n' # Set Internal Field Separator to newline only
# for line in $(cat "$FILE"); do
#   echo "Processing line: $line"
# done

Method 3: Storing All Lines in an Array First

For scenarios where you need to access all lines multiple times or process them in a non-sequential order, it can be beneficial to read the entire file into a Bash array first. This consumes more memory for large files but offers flexibility in processing.

#!/bin/bash

FILE="my_file.txt"

# Read all lines into an array
# The mapfile (or readarray) command is efficient for this.
mapfile -t lines < "$FILE"

# Now iterate through the array
for i in "${!lines[@]}"; do
  line="${lines[$i]}"
  echo "Line $i: $line"
  # You can access lines by index, e.g., ${lines[0]} for the first line
done

# Example: Access a specific line
if [ ${#lines[@]} -gt 2 ]; then
  echo "Third line: ${lines[2]}"
fi

Flowchart of the while read loop for line-by-line file processing.