WEP: core:icu — Internationalization as One Facade

Context

Internationalization is the standard library's largest capability and the one least shaped by code. Unicode and CLDR are data, and the data separates along two axes that behave nothing alike.

By capability: in the wado-bundled-icu/ spike an all-features ICU4X component is ~3.7 MB, of which segmentation is ~2.35 MB and collation ~1.12 MB, against ~44 KB for the character property tries.

By locale, measured with bdp-spike/datagen against icu4x 2.2 and CLDR 48.2.0:

The two rows invert. Collation is a root table with tailorings hung off it, so slicing by locale saves at most 475 KB. Formatting is almost nothing but locale data, at a 206× ratio — and that is why this design exists. No amount of dead-code elimination substitutes: a locale is reached through the same constructor whatever the program declares.

Two questions follow: what a program sees, and how it ends up with only the data it uses. This WEP answers both, and the second answer is a lookup in a bundled archive rather than a slicing run — most of what compile-time data providers offers goes unused here.

Feasibility is settled. The spike established that full ICU4X compiles to wasm cleanly, that it tree-shakes by reachability, and that a no_std build yields a zero-import self-contained component. The bdp-spike/ follow-up established the other half this design rests on: a capability can take its data from a postcard blob at run time instead of baking it, with the blob supplied by a component composed in.

Decision

core:icu is the entire user-facing surface

One module. A program says use icu from "core:icu" and reaches every ICU capability through it; nothing else is nameable and no dependency is added.

The facade is ordinary Wado source, organized into submodules and re-exported through one entry. That matters more than namespacing convenience: since Provider Metadata made the CM ABI stop being the library contract, a core:* API is no longer obliged to be CM-representable. The facade can therefore be Wado-native — generic, trait-using, value-semantic — while the WIT beneath it is an internal ABI shaped for the boundary rather than for users. Concretely, the facade is where ICU capabilities acquire Wado shapes: a collator that satisfies Ord, segmentation exposed through the iterator traits, Display and Inspect on locale identifiers.

Three API shapes, chosen by what the thing is

A free function covers normalization, case folding, is_nfc, character properties, and one-shot locale-sensitive casing.

What separates the two handle shapes is who ends the object's life. It is not the line the data slicing draws — that one is capability and locale, and it runs across both shapes.

Statefulness forces the affine shape for a second, independent reason: copying a token aliases its referent, which two copies of a lazy iterator would show by advancing each other. So the segmenter is a token and the iteration is not — a full pass returns a List<u32> of boundaries, while lazy iteration is affine, an eager list being pathological when the caller wants the first few.

Where a capability serves both a one-shot and a loop, the facade offers both: a convenience function that constructs and discards, and the handle for when the construction should be hoisted. The facade makes the cost visible where it is paid.

Non-owning token

A collator over a declared locale, plural rules, a formatter over a fixed skeleton, a segmenter: immutable, and configured only from what with { locales: [...] } and the program's types already bound. The implementation asset interns these, so the set is finite by construction and never needs freeing, and the handle keeps the value semantics the rest of core:* has — it sits in a global, and a closure captures it by copy, so list.sort_by(|a, b| c.compare(a, b)) is written the obvious way.

Affine resource

A collator tailored from user-supplied rules, a formatter compiled from a runtime skeleton or message pattern, anything keyed by an Accept-Language string, a lazy segmentation iterator. The far side allocates these per call from input the program does not bound, so the object has an end and something must reach it: the handle carries a dtor and is move-only.

One implementation asset, collected per program

Underneath the facade sits a single prebuilt ICU asset, not a partition. It is shipped relocatable, so the linking and reloc.CODE sections survive and the collection wasm-ld runs for --gc-sections can run again against the exports one program reaches. wado-wasm-embed performs it, over data as well as code: an active segment is pruned by the byte, from a map of which data ranges each function reads, and every surviving run becomes a segment of its own. Measured, that reproduces what rebuilding the asset with a narrower WIT surface would produce — 89 KB for locale + casemap against a ~92 KB rebuild, the slices runtime-checked, Turkish casing included — and reaches granularity a rebuild cannot: a grapheme pass is 23 KB where segmentation's four operations are 2.31 MB. Slicing needs no rebuild, so one asset serves every program.

So the capability axis is reachability's, end to end: code, and the data that does not vary by locale, which is baked into the asset and collected away when unreached. What no collection separates is locales, which a constructor reaches through the same symbol whatever the program declares. That axis alone is the package's.

Reachability granularity

The facade carries the coarse split in its types. Where two operations differ in data by orders of magnitude — a grapheme pass needs kilobytes where word segmentation pulls a multi-megabyte dictionary — they belong to different types, so naming a type is already a data decision and the cheap user never mentions the expensive one.

That split stays load-bearing, because the collection's root set is not free of Wado's own liveness: the roots are the asset's exports that surviving Wado code calls, and the liveness pass seeds every method of a live type as a live root by design. A single Segmenter type carrying all four operations would keep all four exports rooted and collect nothing, whatever the program called. Method granularity is the refinement that would lift that.

Locales are a compile-time declaration

use icu from "core:icu" with { locales: ["ja", "en-US"] };

• Omitted means root only — collation's root UCA, no tailorings. Minimal by default.
• The declared set is the union across all use sites, expanded with likely-subtags and the fallback chain.
• A runtime langid outside the set falls back per ICU, to the nearest available and ultimately to root. A literal langid outside the set is a compile-time diagnostic.
• The declaration is transitive and additive across the dependency graph: a library declaring ja forces ja into any program linking it. This is correct — the library genuinely needs it — but an application inherits locales it never declared. A later extension may add an application-level cap; out of scope here.

A single facade means one declaration site per file rather than one per capability, so the union has one obvious place to look.

The ICU package

core:icu is a provider-backed package in the sense the data-provider WEP defines, with one difference: it ships with the toolchain rather than being fetched, so use icu from "core:icu" works with nothing to add. That changes where the package comes from and nothing about how it works.

Its three parts:

• The facade — the Wado source above.
• The implementation asset — the spike's ICU component, promoted to a first-party artifact and shipped relocatable. Its locale-independent data is baked in; its locale-bearing capabilities take a buffer provider.
• Data assets — the image below, keyed by (capability, locale).

The asset ships in the text format, as wado-bundled-libm does. A binary checked into the toolchain is a thing no reader can inspect, and the text form says what the component is. The compiler tells a component from a core module after parsing, so an asset is one or the other however it is written.

Assembling a program's data is a lookup: the reached locale-bearing capabilities name their marker sets, the declared locales expand through likely-subtags and the fallback chain, and the entries at those keys are handed over as blobs. ICU4X's fork provider lets the asset read several blobs, so nothing merges or re-encodes them.

Marker granularity finer than the capability is deliberately not pursued. For a program declaring ja and formatting dates, the locale declaration is worth 4.3 MB where choosing within datetime's 57 markers is worth tens of KB inside the 47 KB that remains. Fixing the marker set per capability is what removes the ICU-version-specific symbol-to-marker map, and with it the recording test that would have had to catch the map drifting on every ICU upgrade.

The version of Unicode is the version of Wado

The toolchain carries one ICU build, so the Unicode and CLDR versions it answers for are the release's, not a dependency a program pins. ICU is a sharp tool: it brings behaviour a program did not ask for and bytes a program did not expect, so the version it speaks belongs to the language rather than to a package resolution.

Character properties are the first capability

They are resource-free, need no locale, and their data does not vary by one, so they need neither the handle kinds above nor the data image below.

The surface names a property two ways. A name written into the source is a WIT enum case: it resolves at the call site, carries no string, and cannot fail, which is what a per-character predicate like is_upper needs. A name a program reads at run time stays a string, matched loosely per UAX #44 and answered with none where it does not resolve. That is the \p{...} in a grammar. Both reach the same baked data, so neither approximates the other.

The enum is not a size knob. ICU4X bakes every property as a static and the dispatch reaches every case, so naming a property as an enum case does not narrow what the call roots. What a program carries is decided by the collection instead.

Three UCD properties are deliberately not answered:

• Block, which ICU4X ships no data for.
• Decomposition_Type, likewise.
• The contributory Other_* properties, which UAX #44 defines as inputs to derived properties rather than as properties to query, and which ICU4X therefore omits.

An unknown name is refused rather than approximated. A caller complementing an approximate set admits real members of the property instead of missing some, which is the failure that matters for a \P{...} in a grammar. Nothing in antlr/grammars-v4's 1,150 grammars uses any of the three in a live rule.

The data image

One compressed, indexed archive of the locale-bearing data, which keeps wado a self-contained binary while letting a build extract only the entries it needs.

• An entry is exactly what one capability loads for one locale, so a build concatenates entries instead of slicing inside them. The index stays small because capabilities are tens, where a per-(marker, locale) key would run to tens of thousands.
• The index maps entry name to offset and length. Random access is required for selective extraction, so a streamed archive format does not qualify.
• A locale's entry is stored deduplicated against its fallback parent, which the runtime fallbacker undoes. Measured, that is 5.16 MB down to 4.34 MB on formatting and 23 bytes on collation — CLDR patterns inherit heavily where collation tailorings are genuinely distinct.
• Compression is per-entry zlib, reversed host-side, so nothing the program carries links a decompressor. zlib is chosen over zstd deliberately: it is already a compiler dependency and pure Rust, where zstd would add a C dependency to an otherwise runtime-free compiler. Measured on the ICU blobs, zstd-19's best case is ~24% on mid-size locale data and less elsewhere — not enough to buy that. Compression happens once at toolchain-build so its speed is irrelevant, and decompression is already negligible against a build that pulls a handful of entries.

Collation's runtime dependency on the normalizer's NFD markers — ~37 KB, the one real cross-capability data dependency measured — needs nothing here: NFD does not vary by locale, so it is baked into the asset and collation reads it there.

Alternatives considered

A resource-free surface keyed by (locale, options)

Every ICU object could be a pure function of a key, with the asset memoizing internally: compare(locale, options, a, b) instead of a constructed collator. Attractive because it keeps the surface value-semantic and needs no resource support in the CM import path. Rejected on four grounds.

The far-side memo is a cache with no eviction policy and no visibility. Compile time bounds the locale data, not the keys — a service reading Accept-Language sees unbounded key strings, and the program cannot inspect or bound what accumulates.

It hides cost. A constructor says where the expensive thing happens; a call that is expensive only the first time reads as free at every call site, and a profile of the hot comparison cannot show the construction inside it.

The key is not small. Collator options alone are combinatorial, and a tailored collator is compiled from user rules — not a key at all.

Most decisively, it does not scale to the full surface. Formatters compiled from skeletons and message patterns, and segmentation iterators, are functions of no compile-time-bounded key. A design that holds for case mapping and breaks at date formatting is not a design for core:icu.

The counterargument — that move-only resources clash with Wado's value semantics — is real, and the three-shape split above is the answer to it rather than a concession: the objects that want to be copyable values are copyable values. What stays move-only is what genuinely has an end.

One handle shape for everything

Making every handle affine adds no kind, but move-only on an immutable interned object is a discipline with no safety payoff — nothing can be double-freed because nothing is freed — and it costs what the hot paths need: a collator in a global, and a collator captured by a comparison closure. It would also put the entire surface behind CM resource import rather than the part owning a dtor.

Making every handle a bare index is the mirror error. It reintroduces what sank the memoizing surface above — an object built per request from Accept-Language accumulates far-side with no eviction and no visibility, a leak rather than a cache — and it silently shares the state of every object that has some. Representation was never the question: a CM handle is already an index into a per-instance table.

Three user-visible modules

An earlier plan split the surface into core:text, core:collation, and core:segmentation so that unused capabilities linked nothing. Rejected: it made a naming decision out of what reachability already decides, so the split bought nothing — while costing users a memorized map from capability to module, spreading the locale declaration across three use sites to be unioned, and freezing an internal boundary into the public API where moving it becomes a breaking change.

Baking all of the data

ICU4X's compiled-data mode throughout, which the original spike used. Simpler, and needs no data assets at all, but it cannot slice by locale — a program declaring two locales still carries every locale CLDR ships. Survivable for casing and normalization; for formatting it is 4.34 MB where the declared locales need 47 KB. So baking stays, for the data that does not vary by locale.

Splitting the asset into components

Several prebuilt components, one per capability, so an unused one is never composed. Rejected: it coarsens what the collection already does. A partition is guessed when the artifacts are built and then freezes each capability's granularity at that guess, where the collection's root set is whatever the program reached; it also costs a ~10 KB floor per line drawn.

It would additionally need a shared data component to recover what one asset shares for free — measured, that sharing is ~37 KB between collator and normalizer and essentially nothing between casemap, properties and segmenter, dedup following genuine runtime data dependencies rather than taxonomy.

Consequences

• Users learn one module, and there is no partition to learn or to maintain: one asset serves every program.
• Per-program data is bounded by what the program reached and the locales it declared; an unreached capability contributes nothing, code and data alike.
• The API tells the truth about cost. A constructed object is visibly constructed, and the one-shot helpers beside it are visibly one-shot.
• Toolchain size grows by the bundled ICU package, in exchange for a zero-dependency core: experience.
• Only locale-bearing data is runtime-loaded, and it pays for that in zero-copy-from-static and a deserialization cost per capability. Baking is otherwise preferred, being what the collection can reach.
• No ICU code runs at build time, so the build has nothing to sandbox, cache by content, or keep deterministic beyond reading its own archive.
• The CM import blocker splits. The compile-time-bounded surface needs only a dtor-less imported handle; the tailored, runtime-configured, and stateful surface waits on full resource import, a gap shared with every resource-bearing third-party component.

Known gaps

• The collection reaches an interface's lifts, not a world's. A world-level lift is a root whether or not the program calls it, because the import plan lists every world function the dependency declares and used_wasi_functions keys effect operations alone (#2068). core:icu exports interfaces only, so it costs nothing here.

• The collection stops at the export. A program reaching one property carries the code and data of every property that export's dispatch can reach. has matches over all 63 binary properties, so naming one case does not narrow what the function roots, and a program wanting Uppercase alone pays for all of them.

  Splitting that needs a root finer than an export, which neither the collection nor a re-link can express today. The export-level split already separates the two paths: the string-named ranges and contains pull the enumerated properties, General_Category and the name tables, which the enum-named has and set do not.

• A dropped lift leaves a stub, not a hole. A lift the program does not import keeps its name and signature in the core module, because the component's canon lift still aliases it. Dropping it outright would mean renumbering the component's own index spaces. What remains is one short function per unused lift — the data and code it reached are already gone, so the cost is bytes rather than kilobytes.

Open questions

• [ ] What bounds a token's interned set. Tokens forbid eviction — an entry freed under a live token dangles — so the set is a permanent high-water mark and must be small, not merely finite. The bound wanted is on the key type's cardinality, which is a facade-authoring decision rather than anything the compiler checks; whether it should instead be a distinct type per bounded configuration is undecided. Unrelated to data slicing.
• [ ] Holding an affine handle in a global, and capturing one in a closure (resource ownership). The token form needs neither, so this bounds only the tailored, runtime-configured, and stateful surface — where a service caching a collator per Accept-Language lives.
• [ ] Does ICU4X expose collation sort keys? A bulk key-extraction call would let a sort cross the boundary once instead of per comparison, which dominates any handle-versus-lookup difference.
• [ ] Where the facade's date/time formatting meets core:temporal, which takes its zone IANA canonicalization from here — WASI's timezone answers only for the host's own zone, so it cannot serve one, and ICU4X's own zone data stops at identifiers: TimezonePeriodsV1 is 6.8 KB for every zone and holds a standard/daylight pair, not the transitions that say which is in effect. Open: whether the tzdb the offsets do need rides this facade's data component as one more marker set, or arrives beside it.
• [ ] The full capability inventory, and which of its data is locale-bearing. The first cut covers what the spikes measured — locale identity, normalization, case mapping, character properties, segmentation, collation — with formatting, plural rules, and transliteration following.

Implementation

• [ ] Non-owning token support in CM component import (Wasm CM Component Import): a dtor-less imported handle decoded as a copyable newtype. It is what normalization, case mapping, collation over the declared locales, and grapheme boundaries by eager pass need beyond what character properties already use.
• [ ] Affine resource support in the same path — methods, static constructors, borrow<T> parameters, resource.drop — gating the tailored, runtime-configured, and stateful surface, not the package.
• [ ] The data-provider mechanism itself, reduced here to an archive lookup: core:icu runs no provider component and needs no compile-time execution.
• [ ] Ship the asset relocatable, and move the locale-bearing capabilities onto a buffer provider with the rest left baked. The asset is first-party and bundled already; what it is not yet is either of those.
• [x] Collect over a component asset. Each core module it holds is embedded and every other section copied byte for byte, so nothing is renumbered and each alias core export still resolves. The root set is the lifted functions the program imports; the rest keep their name and signature as a stub. A program reaching one property carries 104 KB of the 303 KB asset.
• [x] Carry reloc.DATA — a pointer stored in the data itself, reaching another data range or a function. A live data range can root a function, so the data and code edges close over one worklist rather than the code seeding the data once. libm has none; ICU has 146.
• [x] Bake ICU's data-reference map at asset-build time, as libm's is. That is what keeps the .wat: the map's offsets are segment-relative, so a print/reparse preserves them where a reloc entry's offset into the encoded code section would not. Resolving the map at compile time instead would need the asset to ship relocatable, and ship in binary with it.
• [ ] Grow the facade past character properties: re-exports, the Wado-native shapes (Ord, iterators, Display), and the one-shot helpers.
• [ ] Build the data image — per-(capability, locale) zlib entries plus index, deduplicated against the fallback parent — and bundle it with the toolchain.
• [ ] Feed several blobs to the asset through ICU4X's fork provider.

References

• Compile-Time Data Providers — the mechanism this consumes, here reduced to an archive lookup.
• research: splitting large libraries — the capability-axis measurements. The locale-axis table in Context comes from bdp-spike/datagen's coll-loc and fmt-loc sets, which reproduce it.
• Wasm CM Component Import — how the implementation asset is consumed and composed.
• Provider Metadata — why a core:* API need no longer be CM-representable.
• Resource Ownership — the token and affine handle kinds the facade draws on.