WEP: Base64 Encoding API

Status update (AsByteSlice generalization)

The encode / decode API below has since been generalized over AsByteSlice, which supersedes the concrete signatures documented in this WEP:

• encode / encode_url / encode_with now take &S for any S: AsByteSlice (ByteSlice, ByteList, ByteArray, List<u8>, or a String's UTF-8 bytes), not &List<u8>.
• decode now takes the same &S and returns Option<ByteList>, absorbing the former decode_bytes(&List<u8>) — which has been removed.

The lenient-decode and alphabet/padding behaviour described below is unchanged. See core:base64 for the current signatures.

Context

Base64 is a fundamental encoding used across web, security, and data interchange contexts — JWT tokens, data URIs, HTTP authentication, S3 object metadata, PEM certificates, and more. A standard library without Base64 support forces every project to implement its own, leading to subtle bugs (padding handling, alphabet confusion) and unnecessary dependency overhead.

Design Goals

1. Simple things simple: The common cases (standard Base64 encode/decode) should be one function call with no configuration
2. Complex things possible: Edge cases (URL-safe alphabet, padding control) should be expressible without separate function names for every combination
3. Lenient decode: Decoding should accept all reasonable inputs — padding or no padding, either alphabet or both mixed
4. Wasm-friendly: Pure computation, no I/O, no effects — suitable for any world

Prior Art

Observations:

• The 2×2 matrix (standard/URL-safe × padded/unpadded) is the fundamental design space. Go covers it with 4 globals, Rust with 4 engine constants — both lead to combinatorial explosion of named objects
• JavaScript's modern TC39 API (Uint8Array.toBase64(), 2025) uses an options bag — closest to Wado's flags approach, but still requires the caller to specify the alphabet for decoding
• Python keeps it simple with module functions but requires calling the correct decode function for the alphabet
• Rust (base64 v0.22+) uses an Engine trait with GeneralPurpose config — powerful but verbose for simple cases (BASE64_STANDARD.encode(b"hello"))
• No mainstream language auto-detects the alphabet on decode, even though the character set is unambiguous: +/ never appears in URL-safe and -_ never appears in standard

Key Insight: Asymmetric API

Encoding requires the caller to specify the output format — you must choose between +/ and -_, padding or no padding. But decoding needs no configuration: the decode table maps both + and - to value 62, and both / and _ to value 63. The two alphabets share the remaining 62 characters. This means a single decode table handles all valid Base64 input — standard, URL-safe, or even mixed — with zero detection overhead.

Padding is equally simple — just strip trailing = and compute the output length from remaining characters.

Decision

Module: core:base64

use { encode, decode, Encoding } from "core:base64";

Encoding Flags

Use Wado's flags type to control encoding variants:

pub flags Encoding {
    UrlSafe,    // use -_ alphabet instead of +/
    NoPadding,  // omit trailing = padding
}

Encode Functions

Encoding requires explicit format specification — the caller must decide the output format:

/// Standard Base64 (RFC 4648 §4, padded).
pub fn encode(data: &List<u8>) -> String

/// URL-safe Base64 (RFC 4648 §5, no padding).
/// Equivalent to `encode_with(data, Encoding::UrlSafe | Encoding::NoPadding)`.
pub fn encode_url(data: &List<u8>) -> String

/// Encode with custom flags.
pub fn encode_with(data: &List<u8>, encoding: Encoding) -> String

Decode Functions

Decoding is always lenient — auto-detects alphabet and accepts padding regardless:

/// Decode Base64 from a string.
/// Accepts both standard (+/) and URL-safe (-_) alphabets, with or without padding.
/// Returns null on invalid input.
pub fn decode(encoded: String) -> Option<List<u8>>

/// Decode Base64 from raw bytes (e.g., HTTP body, file content).
/// Same lenient behavior as `decode`.
pub fn decode_bytes(encoded: &List<u8>) -> Option<List<u8>>

No Encoding parameter on decode — there is no configuration to pass.

Complete API Summary

5 public functions. 1 flags type.

Usage Examples

use { encode, decode, encode_url, encode_with, decode_bytes, Encoding } from "core:base64";

export fn run() {
    let data: List<u8> = [72, 101, 108, 108, 111] as List<u8>;

    // Standard encode/decode
    let b64 = encode(&data);
    assert b64 == "SGVsbG8=";

    if let Some(decoded) = decode(b64) {
        assert decoded.len() == 5;
        assert decoded[0] == 72 as u8;
    }

    // Padding-less input works too
    if let Some(decoded) = decode("SGVsbG8") {
        assert decoded.len() == 5;
    }

    // URL-safe encode
    let url = encode_url(&data);
    // decode auto-detects — no separate decode_url needed
    if let Some(decoded) = decode(url) {
        assert decoded.len() == 5;
    }

    // Decode from raw bytes (e.g., HTTP response body)
    let raw: List<u8> = [83, 71, 86, 115, 98, 71, 56, 61] as List<u8>;
    if let Some(decoded) = decode_bytes(&raw) {
        assert decoded.len() == 5;
    }

    // Custom: standard alphabet, no padding
    let no_pad = encode_with(&data, Encoding::NoPadding);
    assert no_pad == "SGVsbG8";

    // Invalid input returns null
    assert decode("!!!") matches { None };
    // Mixed alphabets accepted (lenient)
    assert decode("abc+def_ghi") matches { Some(_) };
}

Error Cases

decode / decode_bytes return None for:

• Non-Base64 characters (control chars, spaces, symbols outside the alphabet)
• Truncated input (length mod 4 == 1 after stripping padding, which is structurally invalid)

Whitespace (newlines, spaces) is not stripped automatically. Callers handling MIME-formatted Base64 (76-character lines) should strip whitespace before calling decode. This keeps the core implementation simple and predictable.

Implementation Notes

The implementation will be pure Wado, following the same pattern as core:zlib and core:collections. No new compiler features are required.

Internally, the encode/decode loop operates on List<u8> with the standard 3-byte-to-4-character mapping. Lookup tables are global arrays initialized at module load.

Consequences

Positive

• 5 functions cover all use cases — compared to Go's 4 encoding objects or Rust's engine configuration
• Lenient decode eliminates a class of bugs — callers never need to worry about padding, alphabet choice, or alphabet mixing
• flags type is idiomatic — reuses Wado's existing flags feature, bitwise combinable, zero overhead
• decode_bytes serves real-world needs — HTTP bodies, file contents, and network buffers arrive as bytes, not strings
• No effects — pure functions, usable in any context

Negative

• No alphabet validation on decode — mixed standard and URL-safe characters are silently accepted. In practice this is harmless since both map to the same values
• No streaming API — the entire input must be in memory. Streaming Base64 (e.g., for multi-gigabyte files) is out of scope; it can be added later as core:base64/stream if needed
• Whitespace not stripped — MIME Base64 users must pre-strip newlines. This is intentional (explicit > implicit) but may surprise users coming from lenient implementations like Python's

Comparison

Related WEPs

• Tuple and List Literal Syntax — List<u8> usage
• Global Variables — lookup tables as module globals

References

• RFC 4648 — The Base16, Base32, and Base64 Data Encodings
• RFC 7515 — JSON Web Signature (JWS) — uses Base64url without padding