Why does my base64 encoded SHA-1 hash contain 56 chars?

Understanding SHA-1 and Base64 Encoding: Why 56 Characters?

Explore the journey of a SHA-1 hash from its raw binary form to a 56-character Base64 encoded string, demystifying the encoding process and common misconceptions.

When working with cryptographic hashes like SHA-1, you might expect a fixed-length output. SHA-1, for instance, produces a 160-bit hash. However, if you then Base64 encode this hash, you'll often find the resulting string is 56 characters long, not 40 (which is what a hexadecimal representation would yield). This article will break down the process, explaining why Base64 encoding transforms a 160-bit SHA-1 hash into a 56-character string.

The Nature of SHA-1 Hashes

SHA-1 (Secure Hash Algorithm 1) is a cryptographic hash function that takes an input (or 'message') and returns a fixed-size, 160-bit (20-byte) hash value. This hash is typically represented in hexadecimal format, where each byte is represented by two hexadecimal characters. Since 20 bytes * 2 characters/byte = 40 characters, a raw SHA-1 hash is commonly seen as a 40-character hexadecimal string.

import hashlib

message = "Hello, World!"
sha1_hash_object = hashlib.sha1(message.encode('utf-8'))

# Raw bytes output (20 bytes)
raw_bytes = sha1_hash_object.digest()
print(f"Raw bytes length: {len(raw_bytes)} bytes")

# Hexadecimal representation (40 characters)
hex_digest = sha1_hash_object.hexdigest()
print(f"Hexadecimal digest: {hex_digest} (length: {len(hex_digest)} characters)")

Generating a SHA-1 hash and its hexadecimal representation in Python.

Understanding Base64 Encoding

Base64 is an encoding scheme that converts binary data into an ASCII string format. It's commonly used to transmit binary data over mediums that are designed to handle text, such as email or URLs. The core principle of Base64 is to take 3 bytes (24 bits) of binary data and represent them as 4 Base64 characters. Each Base64 character represents 6 bits of data (2^6 = 64 possible characters, hence 'Base64').

flowchart LR
    subgraph Input
        A[Binary Data (3 bytes)]
    end
    subgraph Process
        A --> B{Divide into 6-bit chunks}
        B --> C[4 x 6-bit chunks]
        C --> D{Map to Base64 character set}
    end
    subgraph Output
        D --> E[4 Base64 Characters]
    end
    style A fill:#f9f,stroke:#333,stroke-width:2px
    style E fill:#bbf,stroke:#333,stroke-width:2px

The fundamental 3-byte to 4-character conversion in Base64 encoding.

The Calculation: 160 bits to 56 Characters

Now, let's apply the Base64 encoding logic to a 160-bit SHA-1 hash.

1. SHA-1 Output: 160 bits
2. Base64 Character Representation: Each Base64 character represents 6 bits.
3. Total Base64 Characters (initial calculation): 160 bits / 6 bits/character = 26.666...

Since you can't have a fraction of a character, Base64 encoding always rounds up to the next multiple of 4 characters and uses padding characters (=) to fill the gaps.

To find the exact number of characters, we need to consider the 3-byte to 4-character block conversion:

• A SHA-1 hash is 20 bytes.
• We divide 20 bytes by 3 bytes/block: 20 / 3 = 6 with a remainder of 2 bytes.
• This means we have 6 full 3-byte blocks, which convert to 6 * 4 = 24 Base64 characters.
• The remaining 2 bytes (16 bits) need to be encoded. These 2 bytes will form a partial block. According to Base64 rules, 2 bytes will be encoded into 3 Base64 characters, and then one padding character (=) will be added to make it a full 4-character block.
• So, 24 characters (from full blocks) + 3 characters (from partial block) + 1 padding character = 28 characters.

Wait, this is still not 56 characters! What's missing? The key is often how the raw binary output of the SHA-1 hash is handled before Base64 encoding. If the SHA-1 hash is first converted to its hexadecimal string representation (40 characters) and then that hexadecimal string is Base64 encoded, the calculation changes significantly.

Let's re-evaluate based on the common scenario where the hexadecimal string of the SHA-1 hash is Base64 encoded:

1. SHA-1 Hex String Length: 40 characters.
2. Bytes of Hex String: Each character in the hex string is 1 byte (in ASCII/UTF-8). So, 40 characters = 40 bytes.
3. Base64 Encoding of 40 Bytes:
   • Number of 3-byte blocks: 40 / 3 = 13 with a remainder of 1 byte.
   • Full blocks: 13 * 4 = 52 Base64 characters.
   • Remaining 1 byte: This will be encoded into 2 Base64 characters, and then two padding characters (==) will be added to complete the 4-character block.
   • Total: 52 characters (from full blocks) + 2 characters (from partial block) + 2 padding characters = 56 characters.

This is the most common reason for a 56-character Base64 encoded SHA-1 hash: the Base64 encoding is applied to the hexadecimal string representation of the SHA-1 hash, not its raw binary form.

import hashlib
import base64

message = "Hello, World!"
sha1_hash_object = hashlib.sha1(message.encode('utf-8'))

# Scenario 1: Base64 encode the raw binary digest
raw_bytes = sha1_hash_object.digest() # 20 bytes
base64_raw_digest = base64.b64encode(raw_bytes).decode('utf-8')
print(f"Base64 of raw bytes: {base64_raw_digest} (length: {len(base64_raw_digest)} characters)")
# Expected length: ceil(20 * 8 / 6) = ceil(160 / 6) = ceil(26.66) = 27. Then padded to multiple of 4, so 28.

# Scenario 2: Base64 encode the hexadecimal string digest
hex_digest = sha1_hash_object.hexdigest() # 40 characters (40 bytes)
base64_hex_digest = base64.b64encode(hex_digest.encode('utf-8')).decode('utf-8')
print(f"Base64 of hex string: {base64_hex_digest} (length: {len(base64_hex_digest)} characters)")
# Expected length: ceil(40 * 8 / 6) = ceil(320 / 6) = ceil(53.33) = 54. Then padded to multiple of 4, so 56.

Demonstrating the two common Base64 encoding scenarios for SHA-1 hashes.

Why the Confusion?

The confusion often arises because developers might implicitly convert the raw binary hash to a hexadecimal string for display or logging purposes, and then inadvertently pass this string to a Base64 encoder. Many programming languages' hash libraries provide a hexdigest() method that returns the hexadecimal string, and it's easy to use this directly without realizing the intermediate conversion. If you intend to Base64 encode the actual binary hash, ensure you are using the method that returns the raw bytes (e.g., digest() in Python, or similar in other languages).