How can I recursively find all files in current and subfolders based on wildcard matching?

Recursively Find Files with Wildcard Matching in Linux

Learn how to effectively locate files across directories using wildcards and powerful command-line tools like find and globstar in Bash.

Finding specific files within a complex directory structure is a common task for any Linux user or system administrator. While simple ls commands work for the current directory, locating files recursively based on patterns requires more advanced tools. This article will guide you through using find and Bash's globstar option to efficiently search for files using wildcard matching in current and subdirectories.

Understanding Wildcards and Globbing

Before diving into commands, it's crucial to understand how wildcards (also known as globbing patterns) work in the shell. These special characters allow you to match multiple files or directories with a single pattern. The most common wildcards are:

• *: Matches any sequence of zero or more characters.
• ?: Matches any single character.
• []: Matches any one of the characters enclosed in the brackets (e.g., [abc] matches 'a', 'b', or 'c').
• [!...]: Matches any character NOT enclosed in the brackets.

These patterns are expanded by the shell before the command is executed. This process is called globbing.

flowchart TD
    A[User enters command with wildcard] --> B{Shell performs globbing}
    B --> C{Wildcard pattern expanded to matching files/dirs}
    C --> D[Expanded list passed to command]
    D --> E[Command executes on list of files/dirs]

How shell globbing processes wildcard patterns

Using the find Command for Recursive Search

The find command is the most powerful and flexible tool for searching files and directories in Linux. It can traverse directory trees, apply various criteria (like name, type, size, modification time), and execute actions on the found items. When combined with wildcards, it becomes incredibly effective for recursive file searching.

find . -name "*.log"
# This command searches for all files ending with '.log' in the current directory and its subdirectories.

Basic find command to locate files by name pattern

Let's break down the command:

• find: The command itself.
• .: Specifies the starting directory for the search. . means the current directory. You could replace this with /home/user/documents or any other path.
• -name "*.log": This is the primary criterion. -name tells find to match files by their name. The "*.log" is the wildcard pattern. It's crucial to enclose the pattern in double quotes ("") to prevent the shell from expanding the wildcard before find gets it. If you don't quote it, the shell might try to expand *.log in the current directory, which is not what we want for a recursive search.

find . -type f -name "config_*.txt"
# Finds all regular files (not directories) whose names start with 'config_' and end with '.txt'.

find /var/log -type f -name "access_log-????-??-??.gz"
# Finds gzipped access logs with a specific date pattern in /var/log.

More advanced find examples with type and specific patterns

Leveraging Bash's globstar Option

For users who prefer a more shell-centric approach without relying on find, Bash (version 4.0 and later) offers the globstar option. When enabled, ** will match all files and zero or more directories and subdirectories. This allows for recursive globbing directly within your shell commands.

shopt -s globstar
# Enable the globstar option

ls -l **/*.txt
# List all .txt files in the current directory and all subdirectories.

cat **/*.conf
# Concatenate the contents of all .conf files found recursively.

# To disable globstar:
shopt -u globstar

Using globstar for recursive wildcard matching with ls and cat

Choosing Between find and globstar

Both find and globstar offer powerful ways to recursively find files with wildcards, but they have different strengths:

• find: More powerful and flexible for complex searches (e.g., by size, modification time, permissions), executing commands on found files (-exec), and handling very large numbers of files without hitting argument limits. It's a separate utility.
• globstar: More convenient for simple recursive pattern matching directly within shell commands (like ls, grep, cat). It's a shell feature, so it integrates seamlessly with other shell operations. However, it can hit argument limits with too many matching files.

1. Identify your search criteria

Determine what kind of files you are looking for (e.g., all .jpg files, files starting with report_, files modified in the last day).

2. Choose your tool

For simple name-based recursive searches, globstar might be quicker to type. For complex criteria or when dealing with potentially thousands of files, find is the safer and more robust choice.

3. Construct your wildcard pattern

Use *, ?, [] to define your pattern. Remember to quote patterns for find -name.

4. Execute and verify

Run your command. If using globstar for a destructive operation, always test with ls first.