Wado

WEP: Marl — Markdown Renderer and Formatter

Historical note: this WEP absorbs the former "Marl Format" WEP (wep-2026-07-10-marl-format.md), which added the Markdown-to-Markdown formatter; the two documents are merged here.

Context

The Wado project needs a blog at wado-lang.github.io. Rather than reach for an existing generator, we build one in Wado and compile it with Wado's own toolchain — a dogfooding vehicle for the language, the package system, and the WASI filesystem bindings. The blog generator itself is a thin static transform; the substantive missing piece in the ecosystem is a Markdown-to-HTML processor, plus an HTML escaping helper. That processor is Marl, and it gets its own package.

The design goal is stated up front: start small, grow big. The minimal processor must be genuinely small yet leave clean seams for later growth (wider Markdown coverage, more backends).

The formatter grew out of the same dogfooding impulse. wado-dev-tools format-md (invoked by mise run format) formats the repo's Markdown by embedding the dprint-plugin-markdown Rust crate; Marl adds a Markdown-to-Markdown formatter (format) written in Wado, shipped together with the renderer as a wasi:cli/command program — alongside Kiln, another instance of Wado dogfooding its own toolchain.

The formatter's bar is "sufficiently reasonable Markdown output," not byte-for-byte parity with dprint. dprint stays the design reference: it is mature, and this repo's Markdown is already in its canonical form, which makes it a convenient corpus for round-trip / idempotency testing.

Decision

Ship Marl, a Markdown toolkit — a Markdown-to-HTML renderer (render) and a Markdown-to-Markdown formatter (format) — as a package-* workspace member (not stdlib), mirroring package-gale.

Name

The ecosystem's tool names are short, evocative natural nouns — Gale (Grammar Adaptive LL Engine), Tide (the WebIDL binding generator), Kiln (Keyed IDL Lowering Notation). Read as elements, they cover air, water, and fire.

Marl — Markdown Adaptive Rendering Layer. Marl is a calcareous earth (a mudstone), completing the elemental set with earth, and its spelling echoes "Markdown".

One parser, two backends

render and format share a single CommonMark + GFM parser (fmt_parse_block.wado / fmt_parse_inline.wado) producing one AST (fmt_ast.wado). The AST retains enough of the source — marker characters, delimiter choices, table alignment — for the formatter to normalize rather than merely echo it. Two backends walk the same tree:

lib.wado re-exports render, RenderResult / HeadingInfo, format, and the escape_text / escape_attr helpers for HTML-templating consumers.

Public API: WIT-representable, no references

The public surface takes and returns values only — no &T — so it maps directly onto a WIT interface (string, list, records). render and format are export: a --lib build lowers them into a CM interface marl (with RenderResult / HeadingInfo riding along render's signature), so any Component Model consumer can pull Marl from a registry. escape_text / escape_attr stay pub (Wado-native, no CM boundary). Reference-taking workhorses stay internal behind the value-based facade, so the hot path avoids copies.

Rendering: the HTML backend

The renderer has no I/O — a single pure entry point over strings, which keeps it trivially testable and reusable by any consumer. It returns the HTML plus the heading outline collected during rendering, so a consumer (e.g. a CMS) can opt into a table of contents by walking headings, or ignore it for plain HTML.

pub struct RenderResult {
    pub html: String,
    pub headings: List<HeadingInfo>,
}

pub struct HeadingInfo {
    pub level: u8,     // 1..=6
    pub id: String,    // GFM slug, unique in the document; also the `id` attr
    pub text: String,  // plain-text content: slug source and TOC label
}

export fn render(source: String) -> RenderResult { ... }

Heading ids (GFM slugs)

Every heading is emitted as <hN id="slug">…</hN> and recorded in RenderResult.headings in document order (headings nested in blockquotes and list items included). The slug follows github-slugger:

Fidelity is exact for ASCII and for caseless scripts (CJK). The one accepted divergence from GitHub: non-ASCII letters are not case-folded (an uppercase Ü stays Ü), since the guest has ASCII-only case tables and full Unicode case folding is out of scope for this iteration.

Supported blocks (v0.1):

Block Notation
ATX heading # … ######
Paragraph
Fenced code ``` / ~~~, with language label
List - / 1., nestable
Blockquote >
Thematic break --- / ***
Table (GFM) \| a \| b \|

Supported inline (v0.1): emphasis (*em*) and strong (**strong**), inline code (`code`), links ([text](url), with optional title), images (![alt](url)), strikethrough (~~del~~), autolinks (bare URLs), and backslash hard line breaks.

Fence rules follow CommonMark: an opening fence is three or more of the same character (` or ~); the closing fence uses the same character and is at least as long as the opening one. This lets a code block that itself contains a triple-backtick fence be wrapped in a longer fence — useful for a compiler blog that shows Markdown and shell snippets.

HTML output is safe by construction: render escapes <, >, &, and " in text; renders raw HTML — block and inline — as escaped literal text, never passthrough; and scheme-filters link / image / autolink destinations (sanitize_url: only http, https, mailto, tel, and scheme-less (relative) URLs survive; javascript:, data:, and any other scheme are dropped to an empty destination, so untrusted Markdown cannot inject an executable link). Reference-style links resolve against the document's definition table; link reference definitions and front matter produce no output. This keeps the renderer simple and the output safe by construction.

Marl scope: CommonMark non-goals and deferrals

Reference links, raw HTML, and indented code all parse; raw HTML still renders as escaped literal text, never passthrough. A few CommonMark features stay out of scope and render as ordinary text:

Simplifications: loose/tight lists follow a simplified model; tabs count as a single indentation column (no tab-stop expansion, though tabs in content are preserved); blockquote lazy continuation, URL normalization, GFM footnotes, www./email autolinks, CJK-aware emphasis flanking, and nested-paren or multi-line link titles are not implemented.

Accepted ambiguity: a --- line is a thematic break, or a table delimiter row when it follows a pipe header. Both constructs matter, and setext (which would also claim ---) is excluded, so no further disambiguation is needed.

A standalone CLI

package-marl/wado.toml declares [world]."wasi:cli/command" = "src/main.wado" alongside its lib, so the package stays importable as a library and is also directly runnable. The CLI mirrors format_md.rs's surface (positional paths, --check) with WASI file I/O: walk the preopened tree for *.md (skipping vendor / target / .git / node_modules / .vscode-test), then read / format / compare / write. No Rust, no wado-dev-tools dependency, no mise or CI wiring.

Unicode width

Table-column alignment is the one place display width determines the exact bytes emitted. unicode_width.wado ships the full East Asian Width Wide+Fullwidth table (Unicode 15.0) plus a small zero-width set (combining marks, variation selectors, ZWJ/ZWNJ); no grapheme clustering. The brief allowed an approximation, but the exact table is small enough to ship whole.

Testing

Non-goals (this iteration)

Consequences

Positive

Negative

Trade-offs

A hand-written parser over Gale: Marl does not use the Gale parser generator, because Markdown is not context-free. The CommonMark fence rule (a closing fence at least as long as the opening one) is context-sensitive and cannot be expressed in a context-free grammar; block structure is defined as a line-based stack machine with lazy continuation, and emphasis resolution uses a delimiter stack with backtracking. These defeat an ANTLR-style LL grammar, which is why every serious Markdown implementation is hand-written. Gale remains the right tool for context-free DSLs, not for Markdown.

Progress

Phase 0: Scaffolding

Phase 1: Marl v0.1 (renderer)

Within the implemented subset, rendering follows CommonMark: emphasis uses the delimiter-stack procedure (flanking, rule of three, nested ***), ATX headings strip closing sequences, lists split on marker change and emit start, and links carry titles. Remaining divergences are all in features left out of the subset (setext headings, blockquote lazy continuation, raw HTML and entity passthrough) and are tracked as growth lines. List tightness still follows a simplified model.

Building the blog site at -O2 surfaced an optimizer miscompile (a value-copy of a &mut referent dropped when the referent is reassigned), filed as wado-lang/wado#1522. It is now handled at copy-insertion time (lower::plan::value_copy): a value sourced through a reference deref is not owned, so the fold inserts the copy and no pass removes it. The minimal reproduction lives as the e2e regression fixture opt_move_out_string_reset_1522.wado, and Marl's flush_text uses the natural move-out again with no workaround.

Phase 2: Marl Format (formatter)

Known defects (deferred)

A recall-oriented review surfaced these CommonMark/GFM conformance defects. None is triggered by this repo's corpus (the acceptance checks stay green), so they are deferred — fix them if Marl is ever pointed at arbitrary Markdown.

Growth lines (post-MVP)