aws cli: How can I query list values?

Mastering AWS CLI: Efficiently Querying List Values

Learn how to effectively query and filter list values from AWS CLI output using JMESPath, enhancing your automation and scripting capabilities.

The AWS Command Line Interface (CLI) is a powerful tool for managing your Amazon Web Services resources. While it's excellent for retrieving data, extracting specific information, especially from list values within complex JSON outputs, can sometimes be challenging. This article will guide you through using JMESPath, a query language for JSON, to precisely filter and extract list values from your AWS CLI commands.

Understanding AWS CLI Output and JMESPath

AWS CLI commands typically return JSON output. This output often contains arrays (lists) of objects or strings. To effectively work with these lists, you need a way to navigate and filter the JSON structure. This is where JMESPath comes in. JMESPath allows you to specify how to extract elements from a JSON document, similar to how XPath works for XML. The AWS CLI integrates JMESPath directly through the --query parameter.

flowchart TD
    A[AWS CLI Command] --> B{JSON Output}
    B --> C["JMESPath Query (--query)"]
    C --> D["Filtered/Extracted Data"]
    D --> E["Further Processing (e.g., scripting)"]
    subgraph JSON Structure
        B -- "Contains Lists/Arrays" --> B1[List of Objects]
        B -- "Contains Lists/Arrays" --> B2[List of Strings]
    end
    style C fill:#f9f,stroke:#333,stroke-width:2px
    style D fill:#bbf,stroke:#333,stroke-width:2px

AWS CLI Querying Process with JMESPath

Basic List Extraction

Let's start with a simple example: listing S3 bucket names. The aws s3api list-buckets command returns a JSON object containing a Buckets array. To get just the names, we can use a JMESPath query that selects the Buckets array and then projects the Name attribute from each item in that array.

aws s3api list-buckets --query 'Buckets[].Name'

Querying S3 bucket names

In this query, Buckets selects the array, and [] is the projection operator, which applies the subsequent expression (.Name) to each element in the array. The result will be a flat list of bucket names.

Filtering List Values with Conditions

Often, you don't want all items in a list, but only those that meet certain criteria. JMESPath allows you to filter arrays using [?expression]. For instance, if you want to find EC2 instances that are currently running, you would query the Reservations array, then the Instances array within each reservation, and finally filter those instances where State.Name is 'running'.

aws ec2 describe-instances \
    --query 'Reservations[].Instances[?State.Name==`running`].InstanceId'

Filtering running EC2 instance IDs

Here, [?State.Name==running] filters the Instances array, keeping only those instances where the State.Name property equals 'running'. Note the backticks ` around running to denote a literal string in JMESPath.

Extracting Nested List Values and Complex Objects

Sometimes, the list you're interested in is deeply nested, or you need to extract multiple attributes from each item in a filtered list. JMESPath handles this with ease. Consider extracting the IpPermissions from a security group, specifically looking for rules that allow SSH access (port 22).

aws ec2 describe-security-groups \
    --group-ids sg-0abcdef1234567890 \
    --query 'SecurityGroups[].IpPermissions[?FromPort==`22` && ToPort==`22`].IpRanges[].CidrIp'

Extracting CIDR IPs for SSH access from a security group

This query first selects the security group, then filters its IpPermissions for rules where both FromPort and ToPort are 22. Finally, it projects the CidrIp from the IpRanges list within those filtered permissions. This demonstrates chaining filters and projections to pinpoint specific data.