What do the brackets mean in NASM syntax for x86 asm?

Understanding Brackets in NASM x86 Assembly Syntax

Demystify the use of square brackets [] in NASM assembly for x86 architecture, covering memory addressing modes and operand types.

When writing assembly code for x86 processors using the NASM (Netwide Assembler) syntax, you'll frequently encounter square brackets []. These brackets are not merely decorative; they play a crucial role in defining how the assembler interprets an operand, specifically indicating a memory access rather than a direct value or register. Understanding their precise meaning is fundamental to correctly manipulating data in memory.

The Core Concept: Memory Dereferencing

In NASM, square brackets [] are used for memory dereferencing. This means that the value inside the brackets is treated as a memory address, and the instruction will operate on the data stored at that address, not on the address itself. Without brackets, an operand might be interpreted as an immediate value (a literal number) or the value of a register.

flowchart TD
    A[Operand] --> B{Has Brackets?}
    B -- Yes --> C[Treat as Memory Address]
    C --> D[Access Data at Address]
    B -- No --> E[Treat as Immediate Value or Register Content]
    E --> F[Operate on Value/Content]

Decision flow for interpreting NASM operands with and without brackets.

Common Scenarios and Examples

Let's look at practical examples to illustrate the difference. The MOV instruction is perfect for this, as it moves data from a source to a destination.

; --- Example 1: Immediate Value vs. Memory Content ---

section .data
    my_var dw 1234h ; Define a word (2 bytes) variable initialized to 1234h

section .text
    global _start

_start:
    ; MOV AX, my_var
    ; This would attempt to move the *address* of my_var into AX.
    ; NASM will likely give a 'type mismatch' warning or error here
    ; because my_var is a memory location, not a direct value.
    ; If it were a label for an immediate value, it would work.

    MOV AX, WORD my_var ; Moves the *address* of my_var into AX (e.g., 0x0000000000402000)
                        ; This is often not what you want when dealing with variables.

    MOV AX, [my_var]    ; Moves the *content* of my_var (1234h) into AX.
                        ; The brackets dereference the memory address 'my_var'.

    ; MOV AX, 1234h     ; Moves the immediate value 1234h into AX.

    ; --- Example 2: Register as Value vs. Register as Address ---

    MOV EBX, 0x1000     ; EBX now holds the value 0x1000
    MOV ECX, EBX        ; ECX now holds the value 0x1000 (content of EBX)

    ; MOV EAX, [EBX]    ; Attempts to load data from memory address 0x1000 into EAX.
                        ; This is memory dereferencing using a register as the address.
                        ; If 0x1000 is not a valid readable address, this will cause a segmentation fault.

    ; --- Example 3: Indexed Addressing ---

    MOV ESI, my_array   ; ESI points to the start of my_array
    MOV AL, [ESI+4]     ; Loads the byte at (address of my_array + 4 bytes) into AL.
                        ; This is useful for accessing elements in an array.

    ; --- Example 4: Base-Indexed Addressing ---

    MOV EBP, my_stack_frame ; EBP points to a stack frame base
    MOV EAX, [EBP+EDI*4] ; Accesses an element at an offset from EBP, scaled by EDI*4.
                         ; Common for accessing local variables or array elements.

    ; Exit program
    MOV EAX, 1          ; sys_exit
    XOR EBX, EBX        ; exit code 0
    INT 0x80

Illustrative NASM code snippets demonstrating bracket usage.

Addressing Modes and Brackets

The brackets are integral to various x86 addressing modes. They allow you to construct complex memory addresses based on registers, displacements, and scale factors. Here's a breakdown of common patterns:

Visual representation of common x86 addressing modes using brackets.

1. Direct Addressing: [variable_name] - The simplest form, where variable_name is a label representing a fixed memory address. The instruction operates on the data at that address.
2. Register Indirect Addressing: [register] - The value in register (e.g., EBX, ESI, EDI) is treated as the memory address. This is highly flexible for iterating through data or accessing dynamically determined locations.
3. Base-Relative Addressing: [register + displacement] - An offset (displacement) is added to the value in register to form the effective address. Useful for accessing fields within a structure or local variables on the stack (e.g., [EBP-4]).
4. Scaled-Index Addressing: [base_register + index_register * scale] - base_register holds a base address, index_register holds an index, and scale (1, 2, 4, or 8) multiplies the index. This is perfect for accessing elements in arrays where each element has a fixed size (e.g., [EAX + EBX*4] for an array of 4-byte integers).
5. Full Addressing Mode: [base_register + index_register * scale + displacement] - Combines all elements for the most complex address calculation. (e.g., [EBP + ESI*4 + 8]).

Operand Size Directives

Sometimes, NASM needs help determining the size of the memory operand you're referring to. This is where operand size directives come in, often used in conjunction with brackets. These include BYTE, WORD, DWORD, QWORD, TWORD, and OWORD.

section .data
    my_byte db 0FFh
    my_word dw 1234h
    my_dword dd 5678ABCDh

section .text
    global _start

_start:
    MOV AL, [my_byte]       ; NASM infers BYTE size from AL
    MOV BX, [my_word]       ; NASM infers WORD size from BX
    MOV ECX, [my_dword]     ; NASM infers DWORD size from ECX

    ; What if the destination register doesn't imply size, or you want to be explicit?
    MOV AL, BYTE [my_word]  ; Loads the first byte (34h) of my_word into AL
    MOV AX, WORD [my_byte]  ; Loads a word starting at my_byte's address into AX
                            ; (0x1234 if my_byte is followed by my_word's first byte)

    ; This is crucial when using generic registers or memory-to-memory moves
    MOV EAX, 0x1000
    MOV BYTE [EAX], 0x55    ; Writes byte 0x55 to memory address 0x1000
    MOV WORD [EAX+1], 0xAABB ; Writes word 0xAABB to memory address 0x1001

    ; Exit program
    MOV EAX, 1
    XOR EBX, EBX
    INT 0x80

Using operand size directives with brackets for explicit memory access.

These directives explicitly tell the assembler how many bytes to read from or write to the memory location specified by the brackets. This is particularly important when the destination register doesn't implicitly define the size (e.g., when moving data between two memory locations or using a generic register like EAX for different sizes).