WEP: Loam — Lowered Operators, Ahead-of-time Modules
Context
Wado has no tensor story. The closest thing in the tree is
example/microgpt.wado, a scalar autograd graph where every arithmetic
operation allocates a Value node. It is a faithful port and a useful GC
benchmark. It is not a foundation for machine learning work.
What Loam is for is inference on the user's own device, on WebGPU: a gaming PC's GPU, and a phone. A browser runs the same module, but a model worth running outgrows what a page should download, so there it serves a demonstration. The two targets share two constraints that a datacenter never has. The module ships inside an application, so it must carry only what the model uses. And the process starts cold, so there is no warmup to amortize a JIT against. An engine that ships an interpreter and the graph as data pays both; a module compiled from the graph ahead of time pays neither, and that is the shape Loam takes.
Compiling ahead of time buys more than size, and this is the design's center. An engine that runs a graph finds a mismatch when it runs. Loam turns the graph into typed Wado before anything runs, so most of what could go wrong stops the build instead:
- Every extent is settled at build time. The generator evaluates the shape computation the graph carries, and an extent condition it cannot discharge stops the build. So does a node in a form no kernel computes: a build error, never a wrong number.
- Axes are types. Each axis the graph mentions becomes a marker type, and each kernel's signature carries the axes it reads, so the ordinary Wado compiler checks every generated call. A caller whose tensor names the wrong axes does not compile.
- A checkpoint is checked against the graph twice: its header at build time, and the bytes the loader is given at load time. A disagreement names the tensor and the axis.
What is left for run time is only what a value decides: the checkpoint's bytes
and the extents of the tensors forward receives. Both are checked where they
arrive, before any kernel reads them, and a disagreement names the tensor and
the axis. load returns it as a LoadError. forward stops with it, since an
input that disagrees is a bug in the caller. A runtime cannot offer this,
however good its kernels are. Its graph is data, and data is checked when it is
read. That is why Loam builds again what inference runtimes already provide.
Four findings from the 2026 literature drive this proposal.
docs/research-ml-inference-webgpu.md records the survey they come from.
Compile-time shape checking has become implementable. Dependent and refinement types both demand proofs to decide type equality, which is why neither was adopted in practice. Staging replaces the proofs with assertions evaluated as compile-time computations, and the ECOOP 2026 work validated it on ported ocaml-torch programs.
Axis notation has converged on einx, whose bracket marks the axes an operation actually sees. Static checking of it exists only as external stubs, because Python cannot say in a type what an operation does to a shape. That is a constraint of the host language rather than of the idea, and it is the reason this design has no notation layer at all.
Quantization formats are a moving target parameterized by block size, scale representation, and rotation, not a fixed list of names.
Fusion's objective function is not what it was. On WebGPU at batch size 1, dispatch count dominates and kernel quality does not. That objective belongs to the GPU work, and naming it here is what bounds the claims a CPU MVP can make.
Gale is the architectural precedent: a static compiled fast path with escalation to a runtime tier. This WEP adopts four properties of it:
- The escalation boundary is a decidability result. Gale escalates because the lookahead language of a recursive ambiguous prefix is non-regular, so subset construction provably does not converge.
- The escalation sites are enumerable and named. Each site Gale's
simulator decides is pinned by a fixture and visible through
gale dump --atn. - Cost routes as much as correctness does. Sites that belong on the simulator for correctness stay static where measurement showed the simulator too slow, and the divergence is pinned rather than hidden.
- An unused mechanism costs nothing. A grammar needing no simulator carries none and is byte-for-byte unaffected.
Gale also inlines its runtime into the generated file. Loam does not, for the reason in "The Runtime Is a Library".
Kiln makes the staging available without a language change. Wado has neither const generics nor general compile-time evaluation, so an extent cannot be carried as an integer in type position. A Kiln generator, however, runs arbitrary Wado at compile time and emits Wado source, which is staging. Gale already works this way. Loam is therefore a package, not a language feature.
Decision
Name
Loam — Lowered Operators, Ahead-of-time Modules.
A model that submits to fixed shapes at build time gets fused kernels, and gets its shape errors at build time instead of wrong numbers at run time. The constraint is the mechanism, not a limit on it.
Loam is the earth a foundry mould is made of, and a mould holds its shape before any metal is poured. Kiln fires such a mould, and Loam is a Kiln generator. The name is a short natural noun, as Gale, Kiln, Jade, and Marl are.
A Kiln Generator, Not a Library
Loam is invoked as a Kiln generator over a model graph and its checkpoint:
use { Weights, forward } from "./model.onnxtext" with {
type: "onnxtext",
generator: {
module: "wado-lang:loam@0.1",
inputs: [{ path: "./model.safetensors", type: "safetensors" }],
},
};
The use site's type says how Loam reads the file, never its extension:
"onnx" is a graph as the protobuf an exporter writes, "onnxtext" a graph in
ONNX's text form, and "hf-tokenizer" a tokenizer in Hugging Face's
tokenizer.json format
(the tokenizer). A file in inputs is
type: "safetensors", a checkpoint. Every file states its type, since no
kind is the obvious default, and one with any other type, or none, stops the
build.
A library would place shape checking at run time, where every other framework already has it. Only a generator can place it early enough for a shape mismatch to stop the build.
The generator reads the graph and the header of the checkpoint named in
inputs, and checks one against the other.
Two properties follow from the choice, and each answers a standing problem in the field:
- A Kiln generator runs in a full Wasm sandbox. A build-time code generator that
reads untrusted model files is an obvious attack surface, and both
build.rsandsetup.pyexecute with the developer's privileges. Loam cannot. - The generated module is ordinary Wado. It is readable, debuggable, and optimized by the same pipeline as any other Wado source.
Loam reads its graph with a Gale-built parser, so its own source imports a
module Kiln produces. The reader names ../grammar/Onnx.g4 at its use site, and
Kiln builds the parser when it builds the generator. Kiln's WEP covers that
under §"A generator may use a generator".
The Runtime Is a Library
The generated module imports its runtime from Loam's library rather than
carrying it. The library holds Tensor, the kernels, Backend with its
implementations, and the checkpoint reader. A consumer lists the package under
[dependencies] as well as [build-dependencies], and names the generator at
the use site by the spec it gave the package. A use site naming it by a path is
refused, since a path names no package to import from. This is
Kiln's runtime library arrangement.
use { Cpu, Tensor } from "lib:loam";
use { Batch, In, Weights, forward } from "./model.onnxtext" with {
type: "onnxtext",
generator: { module: "lib:loam" },
};
The reason is the types the modules share. Every generated module, and the
program calling it, reads one Tensor and one Backend. So a backend is
written once, and a tensor one module returns is a value another reads. When
each module carried its own runtime, each declared its own Tensor, and
neither held.
The axis markers stay each module's own, since an axis is named by its graph. A
tensor passing from one module to another is retyped to the axes the
receiving one names, and its forward checks the extents at its boundary as it
checks any input.
The cost is the version drift inlining ruled out. The two tables may resolve to different versions of the package, and the generated code may then name a kernel the runtime lacks. That is a compile error against the generated module, never a wrong number, since the module is checked against the runtime it imports.
The Graph Is ONNX
A forward pass is read rather than written. Loam takes a model graph in a language the field already has, and the first one is ONNX, with StableHLO to follow as a second front end.
The operators mean what ONNX says they mean: broadcasting, each operator's definition, and the opset version that fixes it. Loam declares the opset range it reads and reinvents none of it. This is where most of the robustness comes from. A graph language of Loam's own would have had semantics of Loam's own, and every rule of them would have been Loam's to specify, to test, and to get wrong. ONNX's rules come with a reference implementation and a conformance suite, and that suite is the oracle the kernels are tested against.
A graph arrives in either of ONNX's two forms, and the generator picks by
extension. .onnx is the protobuf form, which is what an exporter writes and
what the backend test data ships; package-loam/src/onnx_proto.wado reads it
through the types Grog generates from ONNX's own onnx.proto. .onnxtext is
the textual format onnx.parser reads and onnx.printer writes, named as
onnx-mlir names it, and a Gale grammar parses it.
A node is read as its opset defines it, and the readers restate it in the
current definition. That covers Concat before opset 4, whose axis defaulted
to 1, and every operator that only moved an attribute to an operand. Two older
meanings have no current spelling, and each lowers to a call of its own.
Softmax before opset 13 normalized every axis from axis on as one lane,
which softmax_flattened computes. Arithmetic before opset 7 with broadcast
aligned B at A's axis rather than at A's end. Stage 0 reads B followed by A's
remaining axes at extent 1, and read_at reads it so. onnxruntime runs no opset
that old, so its oracle case runs a graph that unsqueezes B instead.
Both produce the Graph in package-loam/src/graph.wado, so nothing after the
reader knows which form it came from. A fixture written here takes the text
form, since a textual graph is a few kilobytes and shows a diff, as
package-loam/example/mlp.onnxtext does. The .onnx files under
package-loam/tests/onnx/ are committed too, and they are copies of ONNX's own
test data rather than anything edited here. The two are not equivalent
encodings. Protobuf is ONNX's canonical form, so where they disagree it is the
one that wins.
An inline value in the text form is read as onnx.parser reads it, into the
protobuf field it would fill. A float16 or bfloat16 element is therefore an
integer, the bit pattern int32_data holds, and 15360 is 1.0. A float may
end at its point (1.) but not start there (.5). It may also be a word:
nan, inf or infinity in any case, with an optional minus. The word stays
an identifier in the grammar, so a tensor named inf still parses.
The graph names its weights, and either the graph or a checkpoint supplies
them. A .onnx may carry its initializers' data itself, and that is enough to
run. Otherwise an initializer's name is the key into the checkpoint, so the
graph is the from literal and the checkpoint an inputs entry, and a sharded
checkpoint is several entries.
An ONNX shape is positional, with a dim_param where a dimension is symbolic.
That is the extent half of an axis and none of the identity half. The identity
half comes from the operators, as Axis Names Are Inferred From the Graph says,
and the shape algebra carries the axes and their extents through every operator
by its ONNX signature. A dim_param is a symbolic extent; a dim_value is a
static one.
A dim_param may spell arithmetic, as an exporter writes
past_sequence_length + 1 for the attention mask of a decoder with a past. One
whose words are names and integers, separated by +, - and * with spaces
around them, reads as that polynomial, with * binding first. Any other
dim_param is one opaque name, so seq-len names one symbol. forward binds
a name from an input axis that is the name alone. A name no input carries
alone is read back from the first input axis whose length is linear in it,
once the other names that length reads are bound. A spelling over a name no
input axis yields this way is read as one opaque name after all, so
batch * seq on an input alone names one symbol. The check that every input
axis is as long as its spelling then catches a length that no whole name
spells, and a name read back as negative stops the run.
A graph also declares the shapes of its own outputs, and those are merged
against what the nodes produced on ONNX's terms (mergeInDimensionInfo): two
static extents that differ are the conflict, and a ? names nothing. A
dim_param is declared as one on an input is, as Axis Names Are Inferred From
the Graph says. A rank that differs is a conflict.
The invocation may fix what the graph leaves open, before stage 0 reads it.
dims gives a dim_param a static extent, as onnxruntime's free dimension
overrides do. outputs names the graph outputs forward returns, and the graph
is cut down to the nodes, inputs and initializers they read, so a decoder's
present.* caches leave with the nodes only they need. A name in either option
that the graph does not declare stops the build. dims fixes a name inside a
spelled dim_param too. Fixing some of its names and not the others stops the
build, and so does fixing them to a negative extent.
last maps a graph output to an axis, and forward returns only the last
position along it, as a language model's next token reads only the last
position's logits. The cut moves back from the output through every node that
computes each position along the axis from the same position of its operands:
an elementwise operator, a reduction that keeps the axis, and a product along
its rows. Those nodes then compute the one position kept, and the operands they
read from elsewhere are cut where they are read. A node that reads across the
positions, such as attention, computes them all. The output keeps the axis at
extent 1, under its name, and an empty axis has no last position, so the
inputs owe one. An output that the graph reads itself, or an axis it lacks,
stops the build.
Without outputs, forward returns every output the graph declares. A single
output is returned as its tensor. Several are returned as an Outputs struct,
one field per output in the order the graph declares them, so a decoder hands
back its present.* keys and values beside its logits.
forward is pass over inputs on the host, with its outputs read back to the
host. pass takes the inputs and returns the outputs as the backend holds
them, a Passed struct where there are several, so a caller reads back only
the part it needs: a language model's next token reads the last position's
logits, not the whole sequence's. Its caller begins the pass
(backend.begin()), which releases what the backend holds for the pass before,
then uploads the inputs. forward does both. A tensor the caller retains
(backend.retain(&t)) lives through the next begin, so a decoder feeds each
pass's present.* back as the next one's past without reading it back.
A graph computes much of its own shape. An exporter writes x.view(x.size(0), -1) as Shape, Gather, Unsqueeze and Concat feeding a Reshape, and
attention's scale as a Cast and a Pow of an extent. Stage 0 evaluates every
node whose operands the build knows, by the operator's ONNX definition. The
build knows an int64, int32 or bool initializer, a Constant, the extents
Shape reads, a symbolic one as its polynomial, and whatever stage 0 has
already evaluated.
Such a node emits no code. Stage 0 evaluates nothing over a float initializer:
a weight belongs to the checkpoint, and the build may not have it.
A folded tensor that a kernel reads or the graph returns becomes a Tensor
literal, as a transformer's position ids do once Range has folded. A
one-element float is the exception where an elementwise operator can take it: a
binary elementwise operand whose partner is a tensor, or a Gemm bias. There it
folds into the operator's Expr as a constant and no tensor is allocated.
Stage 0 also tracks each tensor's element type. An int64 tensor, such as a
token id or an attention mask, stays i64 through the kernels that move
elements without computing on them: the regroupings, Transpose and
Identity. Gather reads it as indices, and Cast converts it. Every other
kernel computes on a float, f32, f16 or bf16, and stops the build on any
other operand.
This settles who Loam is for. A graph reaching it has already run once, in the framework that exported it, so its shapes are consistent by construction. What the build catches is the seam that framework never saw: a checkpoint whose extents disagree with the graph, and a caller whose tensors disagree with the boundary.
The Graph Is the Schema, the Checkpoint Is Data
The graph is the schema, and the generated code is a function of it. A checkpoint carries data, behind a header of its own: a safetensors header is a length-prefixed JSON blob at the front of the file carrying tensor names, positional shapes, dtypes, and offsets. The build checks that header against the graph and reads nothing further down the file.
Kiln's probe is what makes that pay. A generator reports how many leading bytes determine its output, and the build keys on that range; the mechanism and the two invariants a probe owes are in the Kiln WEP. Loam's probe reads a safetensors header, which satisfies both. The header length is in the first eight bytes, so the extent is determined by bytes inside it.
Without it the build would hash whole checkpoints on every incremental build, proving that bytes the output never depended on did not change.
The graph file is probed too, and its extent is the whole file, since every byte
of a textual graph is schema. An extent is a prefix, so a format that
interleaves its schema with its data has none, and this is what a protobuf
.onnx does: graph and initializers in one message, interleaved, so no prefix
excludes the weights. A model whose weights are large must therefore keep them
in external data files, which is
onnx.save_model(..., save_as_external_data=True) and is what every model over
2 GB already does, since protobuf caps a message there. A stated input
requirement is what keeps the extent a length instead of a range set; the
alternative was generalizing the extent so one input could name several disjoint
ranges, which complicates the key, the record, and every generator that never
needed it.
Loam is the reason Kiln's probe receives the invocation's options:
inline_weights makes the output depend on the whole file rather than on the
header, so a probe blind to it would either report the whole file every time,
losing what the extent was for, or report the header and let a payload-only
change hit the cache and emit a module carrying the old weights.
Per-input probing is what a real checkpoint needs. Large models ship sharded —
model-00001-of-00003.safetensors beside an index — so several huge inputs
arrive at one invocation, each with a header of its own.
The generated module contains the typed graph, the kernels, and a loader that checks the incoming payloads against the shapes the header declared. A checkpoint that does not match its header reports which axis disagreed, at load time, rather than producing wrong numbers.
The loader is
Prepared::load(backend: &mut Bk, checkpoint: &ByteList) -> Result<Prepared<Bk>, LoadError>.
It takes the checkpoint's bytes rather than a path, so the generated module
declares no effect of its own: a command reads the file through core:fs, and
a page fetches it. The build checks the header of the checkpoint the invocation
names, and the loader checks the one it is given, since the two need not be the
same file. An initializer the graph carries data for is embedded, and the
checkpoint supplies the rest. See Weights Are Prepared Once for why the loader
takes the backend.
A checkpoint is keyed by the graph's initializer names, and a published one
rarely is. GPT-2's drops the transformer. prefix its exported graph carries,
and ties the output projection to the embedding where the graph holds the
transpose as a tensor of its own. Loam does not map one onto the other, since a
mapping is knowledge of one model. package-loam/tools/onnx_split.wado splits
an .onnx into the two files Loam reads: a .onnxtext whose float initializers
carry no values, and a .safetensors holding them under the graph's names. The
build then reads a textual graph, which a diff can show.
A small model may set inline_weights to have the payload emitted as a byte
string literal read by List::from_le_bytes, which makes the component
self-contained. The compiler folds that call into a data segment, so the Wasm
grows by the payload alone and loading it runs no decode loop. This is for test
fixtures and small models, not the normal path. A .onnx carrying its own
weights takes the same path, and costs the cache nothing, since its extent is
the whole file already.
The literal was chosen over the alternatives by measurement. A float literal
per element costs every compiler phase a node, about 11 s and 460 MB per MiB.
A decode at startup, from protobuf, base64 in __DATA__ or zlib, costs 8 to
40 ms per MiB and grows the Wasm or the startup. The folded byte literal
compiles at under 1 s per MiB and loads in one array.new_data.
Axis Identity Is a Type, Extents Are Values
An axis has a name rather than a position. Transposition is then a change of description rather than a movement of data, and an operation states which axes it consumes.
An axis is a marker type, and a tensor carries its axis list as one type
parameter: Tensor<f32, [Batch, Seq]>. Naming the same axis variable twice in
a signature makes the two positions agree, so a contraction checks itself:
fn matmul<M, K, N>(a: &Tensor<f32, [M, K]>, b: &Tensor<f32, [K, N]>) -> Tensor<f32, [M, N]>
example/tensor_axes.wado is the working form, with a forward pass built on it.
Rank-generic operations take the axis list as a plain type parameter, and an
operation that changes the rank destructures it with a pack, so a reduction is
<..Rest, Last>(t: &Tensor<f32, [..Rest, Last]>) -> Tensor<f32, [..Rest]>.
This needs no language change. It was the open question when this WEP was written, because Wado has no const generics, and it is closed: an extent cannot be carried in a type, but an axis identity can, and that is the half that catches the mistakes people actually make.
Two axes an operation meets are one axis. Lengths that agree do not make two
axes one; only a meeting does. An axis of 1 is the exception, since it holds one
element and broadcasting stretches it to any axis, so it keeps an identity of
its own. The rule holds wherever an operator meets axes: a contraction, a
broadcast, the axes a Concat does not join, and a Conv's channels. A grouped
Conv meets two axes by length alone, because a group's share of the input
channels is an axis of its own. See Axis Names Are Inferred From the Graph.
Loam first took two names whose lengths agreed as one axis, under the first
name. A mistake in a layout then built a module that computed, with generated
types stating an axis identity the graph never had. No graph Loam runs relied
on it.
Extents therefore stay runtime values, and stage 0 is what checks them. A static
extent is a dim_value the graph states, and a checkpoint's header is checked
against it. A dynamic extent, such as sequence length or batch size, is a named
symbol that stage 0 carries through the shape algebra without knowing its value.
Stage 0 counts two tensors carrying one symbol as the same length. The caller
owes that for the inputs, and forward checks it before any kernel runs.
An extent in general is a polynomial over those symbols with integer
coefficients. A merge multiplies its parts, and an exporter's shape arithmetic
adds and subtracts them: a causal mask is sliced at the key length less the
query length, and a decoder with a past extends the sequence by it. So the
integers stage 0 folds hold polynomials too. Shape of a tensor with a
symbolic extent yields one, Gather, Concat and the arithmetic operators
carry it, and a Reshape target built from it regroups by proving the element
counts equal for every value of the symbols. Where a value depends on what a
symbol is, as a Range over the sequence does, the node becomes a kernel call,
and forward binds each symbol from the first input axis carrying it.
Some shape arithmetic leaves the polynomials. ONNX clamps a Slice bound to
its axis and reads a negative one from the end. It counts the steps of a Slice
or a Range rounding up, and never below none. A Reshape's -1 rounds a
quotient down, and an integer Div truncates one toward zero. An int32 keeps
the low 32 bits of its value, and a half rounds to the nearest it holds. Where the build can decide such an operation for every input, it
takes the answer. A bound past any length a list holds is one such case, and it
is how an exporter writes "to the end". Elsewhere the result is a derived
symbol, such as min(N, 8) or ceil(N / 2): a value forward computes from
the input symbols before any kernel runs. Stage 0 carries a derived symbol as it
carries an input symbol, within the range its operation leaves. The same
operation on equal operands derives the same symbol.
Errors split across the two layers. The generated boundary carries these types, so a caller passing the wrong tensor gets a type error from the ordinary compiler. A disagreeing extent inside the graph, or between the graph and its checkpoint, is discharged as compile-time computation in the generator, which is the ECOOP 2026 formulation.
An extent condition that stage 0 cannot discharge is a compile error, with one
exception: an inequality over the input symbols. The caller owes it, as it owes
the inputs' agreement on a symbol. forward checks it before any kernel runs,
and refuses the inputs naming the node that relies on it. Stage 0 first tries
to prove the inequality from what every symbol is. An input symbol is a length
between 0 and i32::MAX, and a derived one lies within what its operation
leaves. One it shows to hold for every input is not owed, and one it shows false
for every input stays a compile error. Anything else stage 0 cannot discharge has no
runtime tier to defer to, because the MVP has no tier below the static one.
The rule first had no exception. Every condition stage 0 could not discharge was a compile error, "the MVP's boundary, stated rather than worked around". That could not hold once the sequence length became a symbol: GPT-2's causal mask slice needs the length within the context, and only the run knows the length. A condition on the inputs is one the inputs can be checked against, so it became the one exception.
Two lengths of one axis are owed the same way. Where an operation meets two
axes of one name whose lengths are both symbolic, and stage 0 can show them
neither equal nor unequal, the inputs owe the equality as two inequalities. A
merged decoder reaches this. Its mask carries a dim_param of its own, and the
keys it masks carry the past's length plus the sequence's. A number meeting a
symbol stays a compile error, since dims can bind the symbol at build time.
A derived symbol is the common source of one. GPT-2 slices its causal mask to
sequence_length from a buffer as long as its context, so the mask's extent is
min(sequence_length, 1024). The scores it masks carry sequence_length
itself, and the two must be one length. Where a proof needs two extents equal,
or one extent to be 1, and they are not, stage 0 reads each derived symbol in
their difference as the operand it takes unchanged. A clamp reads as its bound,
a count clamped at none as its span, an index counted from the end as counted
from the start, and a wrap as the value it wraps. The reading holds only where the operation does
take that operand, so the inputs owe it. GPT-2's mask owes
sequence_length <= 1024. A rounded quotient takes no operand unchanged, so
only its operand is read.
A broadcast reads the same way. Where a 1 meets a derived symbol that reads as 1, the result keeps the 1, and the inputs owe the reading. A decoder with a past reaches this: one position's causal mask is sliced to a derived length, which reads as that 1.
Derived symbols replaced a stricter rule. A clamp leaves no polynomial, so Loam
first read every bound as unclamped, and the inputs owed that it lay within the
axis. A wrap was read the same way, and the inputs owed that the value fit
int32. That refused inputs ONNX accepts, such as a Slice to N over an axis
of 8 at N = 10, and it had no answer where the step was not 1.
package-loam/conformance/semantics_test.wado holds Loam to onnxruntime's
answers at two lengths.
More conditions are owed where only the run knows a length:
- A
Reshapetarget extent owes that it is not 0, which ONNX reads as the input's extent at that position, unless it is that extent or reads as it. - A
Reshapetarget's-1owes that the other extents do not multiply to 0, since ONNX then finds no extent for it. Where the build cannot divide them out of the count, the-1is the quotient rounded down, and it owes that they divide the count. - A step a
Rangecounts by owes that it is not 0. - A window owes that the padded input holds the kernel's reach.
- A
Splitowes that each part lies within the axis, and that the parts sum to it. - An output the graph declares a length for, where the nodes produced a polynomial, owes the equality as two inequalities. The declaration constrains the inputs, and only the inputs the run is given can say whether they meet it.
An int64 that some input would carry past i64 is refused rather than owed.
forward computes every owed slack in i64 as well, so the check itself would
wrap.
The generated module writes each axis as a marker type and the tensors in terms
of them, so a caller says Tensor<f32, [Batch, In]> where it means the input.
A marker whose axis every input, output and weight fixes to one length carries
it as EXTENT, a weight's length being its checkpoint header's. A caller writes
Seq::EXTENT rather than the number, so it cannot disagree with the graph, and
a graph that stops fixing the axis no longer compiles the caller.
It emits no per-tensor alias. Wado's type is a newtype, so an alias would need
an as at every construction to restate what the axis list already says, and it
would make two tensors of one shape distinct for no reason.
Axis Names Are Inferred From the Graph
Neither the graph nor the checkpoint names an axis. An ONNX shape and a safetensors header both give a tensor's rank and its lengths and nothing about what they mean. The graph's operators do say which axes are one, since two axes an operation meets are one axis. Loam takes each meeting as an equation and solves them all over the graph, as a type checker unifies type variables. Each class of axes the equations make one is one axis, and one marker type.
An axis takes its name from a dim_param on a graph input or output, and where
none reaches the class, from a name the generator mints. The exporter wrote the
dim_param, and the graph may join two of them. A merged decoder joins
sequence_length with total_sequence_length in the branch without a past,
where the queries are the keys. The class then takes the name the graph declares
first, among its inputs and then its outputs, and a branch's outputs after the
graph's.
A dim_param reaching two classes joins them. ONNX reads one dim_param as one
length, and the exporter means one axis by it: a decoder's past inputs share
past_sequence_length, and their lengths meet nowhere in the graph, since each
is joined to the new positions apart.
A dim_param names an axis only where it is one name and the axis is no merge,
since a merge takes its parts' names. The type is that name in PascalCase, as
core:serde spells a name_policy, so batch_size is BatchSize and
num_KV_heads is NumKvHeads. A minted name is the first
tensor the class reaches, in PascalCase, with the axis's position after it:
past_key_values.0.key's second axis is PastKeyValues0Key1. The tensors are
read in the order a caller meets them: the graph's inputs, its outputs, its
weights, then each node's outputs. A ? names nothing, and its length is a
symbol of the tensor and the position it was declared at. Both kinds of name are
minted from the module's set, as Every Name Is Minted From One Set says.
Three readings follow from what a meeting is. A broadcast stretches an extent of
1 only over another length, so two extents of 1 meet, while a 1 stretched over
another stays an axis of its own. The branches of an If meet axes in the
classes the graph around them solves, so a weight or a constant a branch reads
has one name everywhere. A dim_param that the dims option binds to a length
still declares its axes, so they stay one class and take its name. Nothing joins
one layer's heads with another's, so each layer's cache has a type of its own,
which A Decoder's Cache Is One Value gathers into one struct.
This is safe because of who Loam is for: a graph reaching it has already run in the framework that exported it, so its meetings are consistent by construction (see The Graph Is ONNX). What the build catches is the seam that framework never saw, and inference leaves the seam as it was. A checkpoint's header is checked against the extents the graph states. The boundary's types carry the axes a caller must name. Two axes made one must still agree in length, or stage 0 owes the equality or stops the build, as it does for one name.
A caller that wants a word of its own for an axis renames it where it imports
it, as any import renames, so use { X1 as In } from "./mlp.onnxtext" reads the
input's features as In.
Loam's settings are options rather than a second file, because Kiln already
rejected a schema a reader has to go elsewhere to find (§"A manifest-side
invocation registry"). Options are typed by the generator's own Options
struct, so the use site gets validation and completion that a sidecar document
could not have, and they are canonically encoded into the cache key, so
reformatting one does not regenerate. Kiln's probe receives them too, which
inline_weights needs.
Loam first required a layout option to name every input, every initializer
and every axis an operator creates, and refused a minted name as a positional
fallback. A minted name is not positional: the equations give its axis an
identity that the graph's dataflow proves, and no position plays a part.
Requiring the names put knowledge of one model at each use site, a rule per kind
of weight and per regrouping, and the built-in tables an architecture option
was to select would have put the same knowledge in Loam. Both are withdrawn, so
a graph from an exporter builds with no option naming its model.
layout then stayed as an override that gave an axis a word, and it is
withdrawn too. A word it gave could only agree with the classes the graph
solves or contradict them: an import rename gives the word without the second
case, and two words reaching one class, or one reaching two, no longer need a
rule.
A Regrouped Axis Is Named by Its Parts
Flatten, Reshape and Unsqueeze produce axes that no input carries. Most of
them are regroupings, and a regrouping names its result from its parts.
Merging adjacent axes produces one axis whose identity is the ordered list of
its parts, Merged<[C, H, W]>, and whose extent is their product. Splitting
it again yields the parts back. A Reshape in general takes its input apart
into those parts and gives each output axis the consecutive run of them whose
extents multiply to its own, so a merge followed by a different split is still
named.
A weight that reads a flattened activation, as the first dense layer after a convolution does, contracts the merge, and the contraction checks as any other does.
An extent-1 axis holds no elements to regroup, so a merge leaves it out of its
parts, and one that no output axis takes is dropped. [N, C, 1, 1] flattens to
[N, C], not to a merge of C with two unit axes.
The axis a merge takes one into still remembers it, though no type shows it, so
a split that puts an extent-1 axis back beside that axis puts back the same one.
An exporter's linear layer over a batch of one flattens [Batch, Seq, Embed]
to [Seq, Embed] and restores the product to [Batch, Seq, Hidden], and the
product computes over the leading axes only because the restored Batch is the
input's.
An axis that no regrouping determines is new: the split of an axis that was
never merged, the extent-1 axis Unsqueeze inserts, the axis Concat joins
along, the axes of a folded
tensor, a one-element one included, and every spatial axis a window produces, since
Conv and MaxPool place their output on a grid of their own. A new axis joins
whatever the operators reading it meet it with, and takes its name as the
previous section says. Gemma's attention splits the hidden axis into heads, and
the per-head axis meets the head norm's weight, so the two are one axis. A
window whose output keeps its input's extent is still a new axis.
Unsqueeze and Squeeze carry every other axis by its position, so an inserted
axis never takes the name of an extent-1 axis beside it. Concat sums its
operands' extents on the axis it joins, and they must agree on every other axis,
by name and by extent. A join of more than two operands is a chain of pairwise
concat calls from the left, and each partial result names the joined axis as
the output does.
Two things follow. An extent is a product of symbols as well as a symbol, since
the parts of a merge may be dynamic. And Reshape reads its target shape at
build time, so the generator reads an integer tensor, which the graph carries as
a Constant or an initializer.
A Branch Is a Block of forward
An exporter can write a decoder as one graph that branches on whether it is
given a past. ONNX's If reads a bool of one element and runs one of two
subgraphs, each reading the tensors of the graph around it. Loam emits it as one
if over that element, with a block of forward for each branch. Both branches
compile, and the run takes one.
This is what lets one module hold one set of weights and still read a prompt in
one pass. The prompt runs through the branch without a past, all its positions
at once, and each step after it through the branch with one. Feeding the prompt
one position at a time through the decoder with a past, starting from an empty
past, also needs one graph and matches onnxruntime. It was rejected because a
prompt of n tokens then costs n dispatches where one would do, and a design
that loses performance on WebGPU is not taken.
Stage 0 checks each branch in a copy of the scope around it. What a branch owes
is checked at the top of its block rather than of forward. A condition one
branch relies on would otherwise refuse inputs the other branch accepts: the
branch without a past owes that the mask is as long as the sequence, and the
branch with one that it is as long as the past and the sequence together.
The axis equations are not copied: the branches and the graph around them solve one set of classes, as Axis Names Are Inferred From the Graph says, so axes one branch joins are one in the other too. The node's outputs take the axes both branches give them. An axis the two give apart is a new axis of the output, named as any other, and each block relabels its value to it. A length the two branches give differently becomes a symbol of its own, which only the run knows. The two branches must agree on each output's rank and element type, and the build stops where they do not.
A constant of the graph around the branches takes the axes that graph gives it
and is spelled once, ahead of the if, so a mask as large as the context is not
written into the module twice.
Without a past, a merged decoder's keys are its sequence, so that branch joins its key axis with its query axis. With a past the keys are the past and the sequence together, two lengths of that one axis, so its types do not tell a key from a query.
A Decoder's Cache Is One Value
A decoder hands back the state it computed, one output per layer and kind, and its caller feeds each output back as the matching input on the next pass. An attention layer's state is its keys and values. A recurrent layer's is what its recurrence and its causal convolution carry. Spelled input by input, that is one argument and one field per layer at every call, and the caller's code then names how many layers the model has.
Hugging Face's exporters name the two sides by one convention: an input whose
name starts with past pairs with the output whose name starts with present
and goes on alike. past_conv.<i> pairs with present_conv.<i>, and
past_recurrent.<i> with present_recurrent.<i>. Keys and values are the one
irregular spelling, past_key_values.<i>.key beside present.<i>.key, so
past_key_values. reads as present.. Loam reads that convention, as it reads
Hugging Face's tokenizer.json.
Where every such input pairs with an output, the generated module declares
Cache<Bk>, with one field per pair. pass and forward take one cache in
place of the past inputs and return the next one in place of the presents, as
the backend holds it: forward reads back the other outputs alone.
cache.retain(&mut backend) keeps it through the next begin.
Cache::empty(&mut backend) is the state before the first pass. A field with an
axis the past's length gives holds no positions on it. Every other field is a
state of fixed extents and holds zeros, which is what onnxruntime's
LinearAttention and CausalConvWithState read where no past is given.
A struct rather than a list, because the pairs need not share their axes. Gemma 4's global layers keep heads twice as long as its local layers do, and its later layers read the caches of earlier ones and keep none of their own. Qwen 3.5 interleaves attention layers, which keep keys and values, with recurrent layers, which keep a convolution's window and a recurrence's matrix.
A graph that names only one side, as a decoder without a past names only the
presents, declares no Cache and keeps those tensors as they are. So does a
graph whose outputs option cuts the presents away. Where both sides are
named, a past that no present pairs with, or the reverse, stops the build naming
it, since a partial cache is a mistake in the export rather than a design.
Wado Stores a Half, a Device Computes In It
A checkpoint's elements are mostly half precision, and most LLM checkpoints on
the Hub are bf16. Some exports keep the activations and the attention cache in
half precision too, and gpt-oss's is published in no other form. So f16 and
bf16 are primitives that carry bits and no arithmetic, specified in
WEP: Half-Precision Primitives.
A tensor keeps the element type the graph gives it, in every scope: an initializer, an input, and every value between two nodes. Each kernel stores its output in that type. How it computes the value is the backend's:
Cpuwidens each half operand to f32, which is exact, computes in f32, and rounds once to the output's type. Where an operator is one of+,-,*,/and the square root, that is the result IEEE half arithmetic gives: f32's 24-bit significand is at least twice a half's 11 plus 2, so the second rounding loses nothing a single one would keep.WebGpuBackendcomputes in f16 where the device offersshader-f16, which packs twice the lanes into a register and, on most phone GPUs, doubles the arithmetic rate. On a device without it, it computes in f32, throughunpack2x16floatandpack2x16float, which WGSL always has.- A bf16 tensor takes the f32 path on every backend, since WGSL has no bf16 type.
- A sum, as a product, a reduction and a softmax take one, accumulates in f32 on every path. A long sum in f16 loses its small terms, and a product's cost is reading its operands, not adding them.
- On
Cpu, a value past a half's range is an infinity, as the graph says it is. Keeping a model in range is the model's work, as Gemma 4's per-block scales do.
ONNX leaves open how precisely an operator computes a half result, so a backend computing in f16 is within the graph's meaning, held to the tolerance every path is (see Results Are Held to a Tolerance, Not to the Bit).
Half arithmetic stays out of Wado. Exact half arithmetic in Wasm is a widening
and a rounding around each operation, which is what Cpu does already, so
operators on f16 would add a spelling and no speed. A device is where
computing in half pays.
Each kernel is generic over the element type it stores, so its signature states
the dtype beside the axes, and a tensor of one precision handed where another
is meant is a type error from the ordinary compiler. That is the check a
missing Cast would otherwise slip past.
A dtype becomes a Wado primitive only where a tensor holds its elements one by
one, as an activation or an output logit does. A weight-only format never does.
int4 is two elements to a byte with a scale per block, and f8 is never shipped
without its scales either; WGSL has no type for either and unpacks both from
u32 with shifts. They stay bytes in the
checkpoint and a tag inside the generator, and never reach Wado's type system.
The Signature Is the Notation
einx exists because Python cannot say in a type what an operation does to a shape, so it says it in a string. Wado can say it in a type. The runtime declares each kernel, and the generator the model's boundary, as a generic signature that carries every part of the notation:
fn matmul<..Bs, M, K, N>(
a: &Tensor<f32, [..Bs, M, K]>,
b: &Tensor<f32, [..Bs, K, N]>,
) -> Tensor<f32, [..Bs, M, N]>
A repeated variable is einx's bracket: K appears in both operands and not in
the result, which is what contraction means. The unrepeated scalars are the
vectorized axes, the pack is einx's leading ellipsis, and the return type is its
arrow. One signature covers rank 2, 3 and 4, because the pack also matches
empty.
So there is no notation layer. A tagged template could not have been one
anyway: ReflectTemplate exposes the hole types and the literal text as a
value, with no associated type derived from the literal, and einx's entire
content is in that literal. A tag could only return a tensor whose axes had
been erased, discarding what the type layer just won.
A parameter list may declare several packs, each settled by the argument that
carries it. So an operation whose input and output axis lists are independent
names both, which is what a rank-changing operator such as Reshape needs.
A contraction names its axis where the axis sits. Two packs in one tuple settle
neither on their own, so [..Pre, K, ..Post] needs its ends settled by something
else, and a generated call spells them in a turbofish. The generator writes both
sides of that call, so spelling them costs it nothing.
Whether a middle-axis contraction is then transposed into the trailing position is the kernel's decision, taken for locality rather than forced by the signature. A rank-2 weight broadcasts across batch axes by putting a pack on one side only.
The Backend Is a Trait
A Backend trait, monomorphized at each use, selects where kernels run. Cpu
is the MVP's implementation, and a first WebGPU one follows it.
The backend owns the tensor's representation, as a generic associated type.
Each kernel is a method, and its signature is the free function's with
Tensor read as Self::Tensor, so the notation survives the seam:
trait Backend with _ {
type Tensor<E: Element, Axes>;
fn matmul<..Bs, M, K, N>(
&mut self,
a: &Self::Tensor<f32, [..Bs, M, K]>,
b: &Self::Tensor<f32, [..Bs, K, N]>,
) -> Self::Tensor<f32, [..Bs, M, N]>;
}
impl Backend for Cpu {
type Tensor<E: Element, Axes> = Tensor<E, Axes>;
...
}
The parameter is generic because half the kernels are: gather, concat and
the regroupings move elements of any type without computing on them. One
associated type per element type would split each of them into one method per
type. Element bounds it to the types a kernel stores, f32, f16, bf16,
i64 and bool, and converts one to another as ONNX's Cast does, so cast
is one kernel generic over both ends.
The rest of the seam follows from what a GPU backend needs:
- The backend is a value.
forward<Bk: Backend>takesbackend: &mut Bkahead of its inputs, since a GPU backend holds a device and a queue. - The boundary stays
Tensor, data on the host.forwarduploads each input and downloads each output but theCache, which is nothing on the CPU. Each crosses in the element type the graph gives it, so nothing converts there. The weights are the backend's already: see Weights Are Prepared Once. - An elementwise operator is passed as data, never as a closure. An
Expris a list of steps over the kernel's operands (operand(0),operand(1), and on), the constants folded into it (constant(0.5 as f32)), and the unary and binary operators ONNX names, each reading the steps before it. A closure cannot become a shader, and fusion composes expressions. One kernel,fused, runs everyExpr, and each operand reaches it through a read that checks it against the output's axes (see Fusion Is Decided by Operator Kind). - The CPU backend evaluates an
Exprone step at a time over a run of 1024 output elements, each step a loop with its operator fixed before it starts: read step by step per element, anExprran four to five times slower than the closure it replaced. The run keeps what one step hands the next in cache rather than in a tensor. An operand holding the output's trailing axes is read in place, repeated; any other broadcast operand is stretched to the output first. - The trait declares
with _. A GPU backend callswasi:webgpu, and the CPU backend instantiatesforwardwith no effect. - The shape checks stay on the host and outside the trait, ahead of the
uploads, reading the inputs as given. An
Ifdownloads its condition, which on a GPU is a synchronization. BackendandCpuare the runtime library's, asTensoris. A test backend,package-loam/conformance/opaque_backend.wado, holds each tensor behind a handle with no field aTensorhas. The operator and semantics cases run through it, so aforwardreaching past the seam does not compile there.
The WebGPU Backend
WebGpuBackend (package-loam/src/runtime/webgpu.wado) is the seam's second
implementation, and forward runs on it unchanged. Its design is in
Loam on WebGPU.
- A tensor is
Placed: a hostTensor, or a view of a device buffer holding the tensor's element type, a half as two to a word and aboolas 1.0 and 0.0. WGSL stores noi64, so an index stays on the host, and a gather writes its indices to the device as it runs. keepholds every weight on the device for as long as the backend, and a pass reads nothing back but what it returns.- A regrouping is a new view of the same buffer wherever strides can say it.
- An elementwise kernel and a row kernel run a shader generated from their
Expr, compiled once. A product runs one tiled shader, its epilogue in the same dispatch. - The convolutions and
max_poolrun onCpu, downloading their operands. with _on an impl method is an effect parameter nothing resolves. So each method spells thewasi:webgpuresources it touches.
package-loam/conformance/webgpu.wado runs every shader path against Cpu,
and package-loam/conformance/webgpu_gpt2.wado runs the tiny GPT-2 on the
device. wado-run-webgpu's tests run both, since wado test reaches no
device. package-loam/example/webgpu/mlp.wado times a three-layer MLP at batch
256 under wado run-webgpu, both backends on one host. The figure is from the
first cut, which uploaded the weights on every call:
| Device | Speed over Cpu, second call |
Largest deviation |
|---|---|---|
| GeForce RTX 4060 Ti | 19x | 1.9e-6 |
An effect was considered and set aside. Wado's effects lower through stack switching, whose per-operation cost is unmeasured here, and a tensor dispatch is a hot path. A trait's cost is known to be zero.
The CPU backend is for correctness and for the seam, not for speed. Wasm SIMD
loads from linear memory and a Wado List<T> is a WasmGC array with no address,
so core:simd offers no load: a vector is built by splat, by tuple coercion,
or by reinterpreting another vector. The pattern that does pay is core:prng's,
where vectors stay in the dataflow and never come from a list, which suits a
tile held across a long computation and little else. The performance story
belongs to the GPU backend, and a place with no GPU, such as an edge worker, is
not a target until the CPU path has a vector load.
Results Are Held to a Tolerance, Not to the Bit
Loam promises no bit equality: not between two backends, not between two devices, and not between a fused module and its unfused one. A backend may reorder a sum, contract a product and an add into one fused multiply-add, use Wasm's relaxed SIMD, whose results the specification lets differ between hosts, or compute in f16 on a device. Each of these is where speed comes from, and holding every path to one bit pattern would rule each of them out.
What Loam promises is that each output lies within a tolerance of what onnxruntime computes from the same graph, as the conformance suites already hold it, and that a decoder picks the tokens onnxruntime picks. Two of Loam's own paths, a fused module and its unfused one say, are held to each other by the same tolerance.
Weights Are Prepared Once
forward reads the weights as the backend holds them, never as the host form
does. Moving a weight onto the device or into a product's layout on every call
would make incremental decoding, one forward per token, move the whole model
per token.
The generated module carries two structs over the same initializers:
Weightsis the host form, typed as the graph holds each tensor. A caller that builds weights by hand builds this one, andweights.prepare(&mut backend)turns it into the other.Prepared<Bk>is whatforwardtakes. Each field is aBk::Tensorin the element type the graph gives it, or aBk::Rhs(below).
A weight of rank 2 that every read takes as a matrix product's right operand,
and that no output names, is prepared for the product alone, as a
Bk::Rhs<K, N> only matmul_rhs reads. The backend chooses its layout: Cpu
packs the rows into f32x4 vectors, and WebGpuBackend keeps the matrix
column by column where one binding holds it, in bf16 where that loses nothing. The weight is a product's
input rather than a tensor in another layout, so no other kernel can be handed
it, and none has to read two layouts.
A Gemm that transposes its right operand has it prepared transposed by
keep_rhs_transposed, so no pass transposes it. A weight two products read in
the two orientations, or one beside a rank-1 left operand, stays a tensor.
Prepared::load(&mut backend, checkpoint) and Prepared::embedded(&mut backend)
build Prepared directly. Each reads one tensor, prepares it, and moves on, so
beside the prepared weights the host holds one host tensor at a time rather
than a whole second copy of the model. Wado has no move, so a host form that
outlives its preparation is the caller's to drop.
Backend::prepare is where a tensor becomes the backend's. It keeps the element
type the graph gives the weight (see Wado Stores a Half, a Device Computes In
It). Loam first widened every half weight to f32 here, and computed the whole
pass in f32. That doubled the bytes a pass
reads from every half tensor, and it left a half graph's intermediate values
unrounded, which is not what the graph computes.
The MVP Is Tier 1 Only
Gale's two tiers become three here. This WEP builds the first:
| Tier | Decided at | Covers |
|---|---|---|
| Static (tier 1) | build time | Static shapes, weights, fused blocks |
| Bucketed (tier 2) | first use, cached | Residual sequence and batch extents |
| Dynamic (tier 3) | per token | Speculation, early exit |
Tiers 2 and 3 are deliberately absent, not undiscovered.
A mixture of experts is tier 1. Its routing picks which weights a token reads,
not which shapes the pass computes: the output of an MoE layer has the shape of
its input whatever experts are chosen. onnxruntime's QMoE, which gpt-oss's
export uses, holds the routing inside one operator, so the device chooses the
experts and the host neither waits nor branches. Loam first listed MoE routing
under tier 3, by its data dependence alone.
Tier 2 is a shape-class cache rather than a simulator. Generating a kernel is expensive where Gale's ATN step is cheap, and the generation must finish before the dispatch it serves can run. Tier 3 is the genuine analogue of Gale's ATN: data-dependent control flow that no bucketing covers.
The Generator Reports Every Fusion Decision
gale dump is what makes Gale's escalation auditable, and its absence is why
torch.compile's graph breaks arrive as a mystery. Loam's generator reports,
per operation, what it fused, what it did not, and why.
The MVP has one tier and reports anyway. A declined fusion carries the reason it was declined, and those reasons are what will route the later tiers.
loam dump is the report. It says how forward lowers each node (a kernel
call, folded by stage 0, an alias of its operand, a branch, or a step of a fused
kernel), where each axis took its name from, each tensor's axes, and what each
scope leaves forward to check.
loam dump --format json says the same for a test to read. loam gen runs the
generator outside Kiln, so its cost can be profiled.
onnxruntime is the report's yardstick, and mise run loam-ort-inspect is how
it is read. Its own report names each graph transformer that changed a model and
how many times, and nothing about where or why one did not apply. Its fusions
of a normalization and an activation are named node sequences
(LayerNormFusionL2, GeluFusionL2). It also runs the shape arithmetic an
export carries: the tiny GPT-2 keeps 38 Shape nodes after every optimization,
where stage 0 folds them all.
Fusion Is Decided by Operator Kind
One pass, plan, runs between stage 0 and emission and groups the nodes into
kernels. Emission and loam dump read its one answer, as both read the
lowering. The rules are over the kind of mapping each operator makes from
its inputs to its output, as TVM (Chen et al., OSDI 2018) and DNNFusion (Niu et
al., PLDI 2021) classify them, rather than over named sequences:
- An elementwise operator reads each operand at the output's position, or stretched to it.
- A contraction,
MatMul,GemmandConv, reads whole rows of each operand. Its output takes elementwise followers, and its input takes none. - A reduction,
ReduceMean,ReduceSumandSoftmax, reads an axis whole. - A regrouping,
Transpose,Reshapeand the rest, moves elements without computing on them.
The CPU objective is the bytes an intermediate tensor writes and reads back, which fusion removes when two operators cover one iteration space (Ivanov et al., "Data Movement Is All You Need", MLSys 2021). On WebGPU it is the dispatches a pass makes, and every rule below lowers both, so one plan serves every backend:
- A chain of elementwise operators becomes one
Expr. - A contraction absorbs the elementwise chain after it, its bias among them.
MNIST's
Conv,AddandRelubecome one call, as do itsMatMulandAdd. - A reduction or a softmax along the last axis joins a row kernel with the elementwise operators around it: a layer normalization is one kernel.
- A
Whereover floats is a step, so a mask joins the kernel reading it.
A value fuses where every node reading it is in one kernel, and it is not
returned by its scope. The plan reads the nodes last to first, so the readers'
kernels are known when a value is decided, and the Expr computes a value two
readers share once. A value two kernels read would be computed twice, and that
trade is not taken. A read inside a branch counts, so nothing fuses across an
If. A folded reader reads only extents and an alias is its operand under
another name, so neither counts. One kernel follows at most one contraction,
and a kernel folding rows takes none. The exception is two matrix products of
one left operand over one iteration space, as a gated layer's gate and value
are: they pair in one kernel, which reads the left operand once
(Backend::matmul_rhs_paired). Where both right operands are weights read by
their products alone, the backend holds them as one matrix whose columns
interleave theirs (Paired<N>). Products of one left operand that do not pair,
each a kernel alone, by a weight as stored that its product alone reads, are
one product by their weights' columns concatenated (Concat<A, B>), each part a
view of its columns, as attention's query, key and value are. A part with an
operator after it stays a kernel of its own. A kernel that takes no
contraction in, read only as the left operand of one product kernel by
weights, and that writes whole rows where it folds them, is that kernel's
prologue: the product computes it as it reads its left operand
(Backend::matmul_rhs_from, Backend::matmul_rhs_paired_from), so a
normalization before a layer is written by no kernel of its own. The plan finds
prologues once every kernel is formed, since only a whole kernel shows whether
it takes a contraction in. The WebGPU backend's narrow product computes one
block of rows at a time into workgroup memory, each workgroup folding the
block's rows itself. The other backends write it with the kernel it would
have had. A linear layer an exporter writes as a
Gemm between a flattening and a restoring Reshape is lowered as one product
over the leading axes, so what follows it joins it.
Every kernel that writes its output names why, one case of Reason, so a test
asserts on the case rather than on its wording: fusion is off, the operator
computes no kernel's input (a window, a regrouping), the scope returns it, two
kernels read it, a reader is a kind that takes no operator before it, its
readers' kernel covers other elements, that kernel already follows a
contraction it does not pair with, or a contraction and a fold would share a
kernel. Choosing among
overlapping groups optimally, as Korch (Hu et al., ASPLOS 2024) does, is later
work that these reasons route.
A kernel that takes others in reports the bytes their outputs would hold, the
writes it never makes and the reads that never follow. An extent stage 0 left
symbolic stays a symbol: a product of [N, 3] taken into its bias add keeps
12 * N bytes out of memory, for whatever batch the run brings.
Fusion keeps the graph's order: each element goes through the operators as the
graph lists them, and a contraction's epilogue applies after its sum is
complete. A fused step keeps its value at the precision the kernel computes in
rather than rounding it to its node's half type, since nothing reads it in
between. So a fused module may differ from the unfused one by those roundings,
within the tolerance (see Results Are Held to a Tolerance, Not to the Bit). The
option fuse: false generates the unfused module, which is what the tests
compare against and what loam gen writes for a comparison by hand. One
program cannot hold both, since Kiln runs a graph under one set of options per
program, so package-loam/conformance/fusion_test.wado builds each as the generator
would and runs it under core:eval.
A contraction takes the operators after it as then, an
Option<Epilogue<Bk::Operand<Out>>> whose Expr reads the contraction's
element as operand(0), and writes in place. The CPU's product applies it to
each batch element's matrix as soon as that matrix is complete, while it is
still in cache. Its convolution applies it once the whole output is complete,
since completing one output channel's plane before the next measured slower:
the tap-by-tap order reads each input element once for every channel. The
WebGPU backend's product applies it in the same dispatch.
On the CPU, MNIST-12's forward pass measures as below, by
wado run package-loam/example/mnist_bench.wado with fuse set each way. Its
elementwise work is a small share beside its two convolutions, so fusion
saves little time there, and the number is a floor, as What the CPU MVP Cannot
Validate says.
| Module | Time, relative to unfused |
|---|---|
fuse: false |
1.00 |
| fused | 0.997 |
| fused, the convolution finishing plane by plane | 1.03 |
An Expr reads any number of operands. Each operand has its own axes, and Wado
has no type for a list of tensors over different axis lists, since a pack does
not map through a type constructor. So each operand is read through a typed call
first, which relates its axes to the output's and yields the backend's
Operand<Axes> under the output's axes alone:
readtakes a tensor of the output's axes.read_suffix::<Pre, Suf>takes one of the trailing axesSufand repeats it acrossPre.read_at::<Pre, Mid, Post>takes one of the axesMidmet beforePost, as arithmetic before opset 7 aligned.
fused::<Axes> then takes one list of operands of one type, and the Expr.
Every check a two-operand kernel made of its operands is made by the read.
Every Name Is Minted From One Set
Every identifier the generated module declares is minted from one set. The
module binds the runtime names it imports, forward, Weights and the
weights parameter before it names anything the graph brings. It imports one
fixed list whatever the graph reads, so which names are taken does not depend on
the graph.
An ONNX tensor name is any string and comes out of a model file, so weights,
map and Batch are all names a graph may carry, and one that is taken gets a
trailing _. An axis name, from a dim_param or one Loam mints, is minted from
the set the same way.
The language's own share of that set is package-wadopoet's is_reserved: every
keyword the parser does not take as a name, and every name the prelude puts in
scope. A name keyword such as type or from stays free. The predicate is
generated from what wado syntax --format json reports, so a keyword added to
the compiler is reserved here without anything else being edited. Gale reserves
the same set for a grammar's rule names.
What the CPU MVP Cannot Validate
The MVP validates the shape algebra, the generator architecture, and the backend seam. It does not validate the fusion objective.
On CPU the objective is memory traffic. On WebGPU at batch size 1 it is dispatch count. They are different objectives, and a fusion policy tuned against the first is not evidence about the second. The WebGPU backend will have to re-derive its policy from measurement rather than inherit this one.
The CPU arm cannot even reach its own objective's ceiling, for the reason in "The Backend Is a Trait": no vector load from a WasmGC array. A number this MVP produces is a correctness result and a floor, never a claim about the design.
The Tensor Kernels Stay in Loam
The kernels in package-loam/src/runtime/tensor.wado stay in Loam until Loam's
own use cases settle them. An ndarray API has to serve several backends
(scalar, SIMD, WebGPU), and Rust keeps that layer out of std, split across
crates. Moving any of it into the standard library comes after.
Roadmap
- [x] The kernels. Each operator lowers to a call into the runtime
(
package-loam/src/runtime/tensor.wado), and the generated module carries aWeightsstruct for the graph's initializers and aforwardthat runs the nodes in order. Each kernel carries the axis notation in its own signature.matmul<..Bs, M, K, N>contractsK, andread_suffix<..Pre, ..Suf>broadcasts a trailing axis list, so a generated call is checked by the ordinary compiler rather than by Loam.package-loam/conformance/checks them against ONNX's own outputs: graphs from the backend test data compile through Loam and reproduce the tensors shipped beside them. The models are committed underpackage-loam/tests/onnx/. - [x] The half-precision element types.
float16andbfloat16map to the primitives WEP: Half-Precision Primitives adds.Weightsandforwardstate the graph's own element type, andwiden/narrowconvert at the boundary over theHeldtrait the two halves implement.package-loam/example/half.wadocompiles such a graph through Kiln and prints its tensor. - [x] A model's own weights. Where a graph carries every initializer's data,
the generated module adds
Prepared::embedded, which returns them. Each payload is a byte literal read byList::from_le_bytes, which the compiler folds into a data segment, so an ONNX file alone is enough to run inference. The conformance graphs with initializers run fromPrepared::embedded. - [x] Minimal inference on ONNX alone. MNIST-12 from the ONNX Model Zoo runs
from its
.onnxfile and reproduces the logits shipped beside it, which tookConv,MaxPool,Reshapeand a stretched bias. The model lives inpackage-loam/tests/models/with its provenance, apart from ONNX's own test data.package-loam/example/mnist.wadoclassifies digits drawn in its source. - [x] A small transformer over static shapes. A GPT-2 decoder as an exporter
writes it, with its batch and sequence length fixed at build time. Most
of such a graph computes shapes rather than values, so this is where
stage 0 evaluates that computation. A tiny GPT-2 with random weights runs
from its
.onnxand reproduces the logits onnxruntime computes for it. The model lives inpackage-loam/tests/models/tiny-gpt2/with the scripts that wrote its oracle data.package-loam/example/gpt2.wadotakes text in and gives text out. Its tokenizer, GPT-2's byte-level BPE, gives the ids Hugging Face's tokenizer gives, and greedy decoding picks the tokens onnxruntime picks. - [x] A symbolic sequence length. The same model with its sequence length left to the run. Finished when one generated module reproduces onnxruntime's logits at two sequence lengths. The tiny GPT-2 does at four tokens and at seven, and both GPT-2 examples feed the tokens so far rather than a padded context.
- [x] The checkpoint reader. safetensors in the generator, its header checked
against the graph. safetensors comes first because its header is already
JSON, with no quantization to model. The generated loader checks each
payload against the shape its header declared. Finished when a real
checkpoint's shapes check against its graph and its payloads load, and a
deliberately mismatched header or payload names the axis. The real
checkpoint is GPT-2's 124M-parameter export, split by
onnx_split.package-loam/example/gpt2-124m/downloads it and continues a prompt with the tokens onnxruntime picks. It builds from the graph and the checkpoint's header, both committed, sowado testcompiles it without the download. - [x] Incremental decoding.
forwardreturns every output the graph names, so a decoder hands back the attention keys and values it computed. GPT-2'sdecoder_with_past_model.onnxtakes them back aspast_key_values, so a step after the prompt computes one position against a past that grows, which needs step 6. Finished when GPT-2 continues a prompt with the tokens onnxruntime picks, computing one position per step after the prompt. The tiny GPT-2's decoder with a past lives inpackage-loam/tests/models/tiny-gpt2/with-past/, andpackage-loam/conformance/tiny_gpt2_past_test.wadoreproduces onnxruntime's logits for the prompt and for two steps after it.package-loam/example/gpt2.wadoreads the prompt once and then one token per step. - [x] One graph for both. An exporter merges the two decoders into one graph
that branches with an
Ifon whether a past is given, so one generated module holds one set of weights and still reads a prompt in one pass. Finished when GPT-2 continues a prompt with the tokens onnxruntime picks through one module. The tiny GPT-2's merged decoder lives inpackage-loam/tests/models/tiny-gpt2/merged/, andpackage-loam/conformance/tiny_gpt2_merged_test.wadoreproduces onnxruntime's logits for the prompt and two steps after it.package-loam/example/gpt2.wadoruns on it. - [x] The tokenizer. Text reaches a model through a tokenizer Loam ships,
rather than through code beside each example. Its design is in
the Loam tokenizer. Both GPT-2 examples
encode and decode through a
Tokenizergenerated from theirtokenizer.json, with the ids Hugging Face's tokenizer gives. - [x] The backend seam. The
Backendtrait with its CPU implementation. Finished when the MLP runs through the trait and matches the reference within tolerance, and a test backend whose tensor is an opaque handle runs it too, which no generated read of.datasurvives. The conformance graphs run throughCpu, and the MLP, anIfand every operator and semantics case run through the handle backend inpackage-loam/conformance/opaque_backend.wado, the MLP and theIfto the CPU's output exactly. - [x] Prepared weights.
Preparedholds each weight as the backend computes it, converted once whereloadorembeddedbuilds it, rather than widened on every call toforward. The form belongs to the backend, so this follows step 11. It is whatf16andbf16entered the language for: a half-precision model is downloaded and stored at half its size.package-loam/conformance/backend_test.wadoruns a half-precision model'sforwardin a loop through the handle backend, which counts each tensor it converts: the weights convert once, each call converts its input alone, and the output matches what converting the weights per call gives. See Weights Are Prepared Once. - [x] Fusion and its report. The generator fuses operators into one kernel
where the CPU objective favours it, and reports each decision, following
what
gale dumpestablished. See Fusion Is Decided by Operator Kind. Finished when the step-4 model runs with its elementwise chains and its contractions' epilogues fused and matches the unfused output bit for bit, and every operation reports a decision and a reason. The timeforwardtakes on the CPU before and after is recorded, as a floor. MNIST-12'sConv,AddandRelurun as one call twice over, and itsMatMulandAddas one, as onnxruntime fuses them;package-loam/conformance/fusion_test.wadoholds the two modules to one result. - [x] The WebGPU backend. A second
Backendoverwasi:webgpu, with the fusion objective re-derived for dispatch count. GPT-2 (124M) runs through it underwado run-webgpuand picks the tokens onnxruntime picks, one read back a token. The design and the measurements are in Loam on WebGPU. Abenchmark/row against onnxruntime waits for a GPU to measure on, since llvmpipe's number says nothing of one.
Status
This WEP is complete. Further development of Loam, its roadmap and its known
gaps included, is tracked in package-loam/ (TODO.md, AGENTS.md) rather
than here.
