WEP: Variant Return Scalarization at NIR

Context

A function returning a variant — overwhelmingly Result<T, E> and Option<T> — is compiled by wir_build as a single GC struct result. Every return Ok(v) allocates. wir_optimize::sroa_variant_return then rediscovers the pattern from WIR shapes and unpicks that boxing into a flat [i32 discriminant, payload…] multi-value result.

Everything about that is late. The scalarization happens after every NIR pass has run, so no NIR pass ever sees through a Result-returning call: to const_fold, dce, sroa, store_load_forward, and inline alike, the result is one opaque boxed value. The point of moving the work to NIR is not to relocate it — it is to let it compose with the rest of the optimizer, which is where the compounding wins are.

That goal decides the shape of the change, and it rules out the obvious design.

Decision

A ReturnAbi marker cannot deliver the goal

The natural reading of issue #1742 is: add ReturnAbi::Variant beside ReturnAbi::MultiValue, classify variant returns in optimize::multi_value_return, and let wir_build emit the flat signature.

That is not enough, and the issue's own caveat says why. ReturnAbi::MultiValue does not change a function's NIR return_type — the type stays the aggregate, and only the WIR-level ABI shifts. The classifier also runs after the fixed-point loop, among the backend-required rewrites. So a marker is by construction invisible to every NIR pass: nothing in the loop reads return_abi, and nothing could usefully read it, because the NIR the passes see is unchanged.

A marker buys real but narrow things — wir_build emitting the intended ABI instead of operating on boxing it introduced itself, and WIR peephole/DCE seeing clean code. It buys no pass interaction at all.

The design: scalarize the return in NIR, reuse the tuple ABI

Rewrite the function, not a flag. A variant-returning function becomes a tuple-returning function:

fn parse(s: String) -> Result<i32, String>
  ⇒
fn parse(s: String) -> (i32, i32, Option<String>)
//                      tag  Ok    Err

• return Ok(v) becomes return (0, v, None).
• return Err(e) becomes return (1, 0, Some(e)).
• match parse(s) { Ok(v) => A, Err(e) => B } becomes let t = parse(s); match t.0 { 0 => { let v = t.1; A } _ => { let e = t.2!; B } }.

From there every existing NIR pass applies unchanged, because there is no variant left in the return position — only a tuple, an integer tag, and locals. And the Wasm-level ABI needs no new machinery: optimize::multi_value_return already turns a tuple return whose call sites destructure into a flat multi-value signature. The variant stops being a special case.

This is the return-position dual of optimize::sroa_param, which already rewrites parameter signatures interprocedurally inside the loop, deliberately placed before nir/inline "so the inliner sees post-SROA signatures and can propagate the scalar through call chains" (sroa_param.rs). nir/dae and nir/drve likewise change signatures mid-loop. Signature rewriting in the loop is an established pattern here, not a new risk.

What it unlocks

known_case folds a match whose scrutinee is a known VariantConstruct within one function. The rewrite extends that to the cross-call case for free: the case is an i32 the constant machinery already handles, and the payload is an ordinary opaque local.

None of these are available to a marker-only design at any effort.

What it costs: a pass keyed on the variant shape stops firing

The same reach that unlocks the table above is what a pass recognising the variant shape loses. labeled_block_fusion deletes the Option an inlined helper leaves at a discriminating consumer; after this pass the intermediate is a tuple, its breaks carry a tuple literal and the consumer reads temp.0, so the recogniser declined and the tuple stayed heap-allocated — worse than the variant it replaced.

The rule that follows: a pass that matches on VariantConstruct / VariantTest / VariantPayload at a call boundary has to learn the scalarized shape as well, and the two belong in one recogniser rather than two that can drift. Finding them is empirical — a struct.new "tuple// in a hot loop is the signature — so read the WIR of a workload after enabling this pass, not only the benchmark totals.

Design

The slot typing rule

The one thing NIR cannot express directly is the padding. WIR pads an unused result slot with ref.null and widens the slot with WirType::as_nullable; NIR types carry no nullability modifier, and a String-typed slot holding null would be a lie.

The rule that resolves it uses only types NIR already has. It keys on the payload's Wasm representation, not its NIR shape, because that is what decides whether the wrapper costs anything:

The Option<T> wrapper in row 3 is free. wir_optimize::nullable_ref lowers a 2-case {Unit, Payload(ref T)} variant to a bare nullable ref, so Option<T> for a non-nullable ref T is ref null T at Wasm level, Some(v) is v, and None is ref.null. The resulting signature is byte-identical to what the WIR pass produces today — parse_leaf in tests/fixtures/opt_sroa_variant_tail_call.wado expects -> [i32, i32, ref null "String"], which is exactly (i32, i32, Option<String>) after lowering.

Row 2 is what stops the wrapper from ever costing a box. Option<String> is already ref null String, so wrapping it again would produce Option<Option<String>> — whose payload is a nullable ref, which is_nullable_ref_eligible rejects, so it would allocate. Using the payload type as its own slot type instead makes the pad and a genuine None payload indistinguishable, which is harmless: the tag discriminates. Option<i32> is not in this row — it is an ordinary boxed variant struct, so row 3 applies and Option<Option<i32>> lowers to ref null Option<i32>, again free.

A pad is never read — the reader tests the tag first. If a future pass gets that wrong, an Option slot unwraps through RefAsNonNull and traps, rather than silently producing a mistyped value. A zero-padded resource slot has no such guard, which matches what the WIR pass emitted (default_value_for_type → I32Const(0)).

v128, i128 and u128 are declined outright. The pad is what decides it: every row above needs a zero the pass can mint as a promoted value, and ValueKind has no v128 literal — while i128 / u128 occupy two Wasm results, which the slot count does not model. The WIR pass could pad all three, because default_value_for_type works on WirType after lowering; retiring it gives the shapes up. Nothing measured exercises them (tests/fixtures/opt_variant_return_v128_payload.wado pins the decline).

Layout: shared and per-case

The same two shapes wir_optimize's layout.rs computes today, decided now on NIR types:

• Shared, when every payload-bearing case has the same slot type: one payload slot. Option<char> → (i32, char).
• Per-case otherwise: one slot per payload-bearing case, in case order. Result<i32, String> → (i32, i32, Option<String>).

Cap: at most 8 tuple elements (MAX_PER_CASE_RESULT_FIELDS today), which bounds a per-case layout at seven payload-bearing cases.

A WIR variant case has at most one payload slot — wir_build/types.rs:380 and :910 map a case's single NIR payload TypeId to vec![] when it is Unit and to a one-element vector otherwise, and a multi-field case is one slot holding a tuple ref (Shape::Rectangle([f64, f64]) becomes payload_0: ref "tuple//[f64, f64]", tests/generated/fixtures/assert_inspect_matches.wir.wado:61). So MAX_SHARED_RESULT_FIELDS = 4, and the 1 + max_payload_count > 4 guard in layout.rs can never bind. Its payload_{j} loops are dead generality, and the NIR layout must not carry them forward.

The pass: nir/sroa_variant_return

Modeled on optimize::sroa_param: gate-aware, interprocedural, placed in the loop before nir/inline, and firing at most once per function (state carried like param_spec::ParamSpecState) so the fixed point terminates.

Phase 1 — candidates

A function qualifies when its return type is a variant (or a GenericInstance resolving to one), it is neither an ABI boundary nor address-taken, and its case payloads fit the slot rule and the cap.

The exclusion set mirrors what the WIR pass derives precisely from collect_pinned_func_ids (exports, element-table entries, RefFunc operands), plus the value-copy helpers, whose calls WIR build adds after NIR:

is_trait_method() is not on this list, and must not be — see Trait methods carry the benefit.

A variant that nullable_ref will erase (2 cases, one Unit, one ref-represented payload — i.e. Option<T> for ref T) is excluded: it already returns a bare nullable ref, which beats any tuple.

Phase 2 — validation, to a fix-point

Every Return must produce a fresh variant value:

• Return(VariantConstruct) of this variant;
• Return(Local(t)) where t is single-def / single-use and defined by a VariantConstruct — the shape field_scalarize leaves behind, and the whole reason wir_optimize's return_temp.rs exists;
• Return(Call(g)) where g is another candidate returning the same variant — the tail-call rule;
• Return(null), the Option::None literal;
• Block / If / Match / Switch whose every leaf is one of the above;
• a diverging (Never-typed) value.

Every call site must destructure the result immediately:

• Match { expr: Call(f), .. } — the call is the scrutinee. This is what ? desugars to (match f(x) { Ok(v) => v, Err(e) => { return Err(e) } }), and it is the dominant shape, not an edge case.
• let t = Call(f); … where every use of t is a Match / Switch scrutinee, a VariantTest, a VariantPayload, or a VariantTag.
• Return(Call(f)) inside another candidate.

Arm patterns must be one level deep over this variant (Ok(x) / Err(e) / _). A nested pattern (Ok(Some(x))) rejects the candidate rather than triggering a peel — the peel is where the WIR pass's complexity lives and it should not be recreated. nir/inline and nir/sroa reach most of those cases anyway once the outer level is scalarized.

Both rules couple caller and callee candidacy, so validation iterates to a fixed point (optimistic assume-then-refute, so mutually tail-recursive groups are accepted), exactly as sroa_param and widen.rs do.

Phase 3 — rewrite

Callee: return_type becomes the tuple type; each Return leaf's VariantConstruct becomes a TupleLiteral of tag plus slots, per the layout.

Call sites: the bound temp's type becomes the tuple; VariantTest becomes FieldAccess(t, "0") == k; VariantPayload becomes the slot read (unwrapped through VariantPayload(slot, Some) for an Option slot); a Match over variant patterns becomes a Match / Switch over the tag with the payload binding prepended to each arm body. match f(x) { … } is first normalized into { let t = f(x); match t { … } }, so every site is the one let-bound shape.

A candidate reached from a global initializer is dropped: an initializer body has no locals list to mint the binding in.

Phase 4 — variant locals

A local holds the same tuple a return does. let mut s: Option<i64> = null; … s = Some(v); … match s { … } is the derived deserializer's slot per scalar field, and each Some(v) was a heap object. The local qualifies when every write builds a case (null included) and every read is a tag test, a payload read or a one-level match. A local read whole, passed or returned, stays a variant. Each write becomes the case's tuple literal, the reads go through the Phase 3 call-site rewrite, and sroa splits the tuple into scalars: a whole-literal assign t = [tag, v] becomes one write per field. The writes come after every field is evaluated, each but a constant into a temp. Written as it went, the tag landed before a payload that reads t or leaves the statement: s = Some(if c { break done; } else { 1 }) read as Some(0) after the break.

Four shapes are declined, each for a reason of its own:

• A local two Lets bind. sroa leaves it whole, so the tuple would allocate at every write, null included, where the variant's None did not.
• A local a guard of a match over it writes. The Phase 3 rewrite reads the payload slot again at each guard and arm body. That is one read only for a call's temp, which nothing reassigns. After such a guard, a later arm would read the pad.
• A local a promoted operand reads. The rewrite rebuilds a read it has a node for, and a promoted read has none, so it would go on naming the variant.
• A layout whose slot is itself a variant with a layout. Once sroa splits the tuple, that slot is a variant local of its own, and for Option<Option<i32>> it is the very same type, so taking it never terminates.

On json-catalog this removes all 18 Option<i64> constructions. Against the commit before it, best of three alternating rounds on 2026-09-28:

Every copy of a type has to move together

A rewrite that changes what a function returns has to retype four things that each carry their own copy, and missing any one produces invalid Wasm rather than a missed optimization:

• the function's return_type;
• every Call node targeting it — its type_id is what wir_build reads;
• the let binding: the NirLocal, the StmtKind::Let's own type_id, and every ExprKind::Local node reading it;
• the merge points a return value passes through (Block / If / Match / Switch), whose type_id still names the variant.

The same applies to the let t = Ok(v); … return t shape: the initializer is rewritten in place, so the binding's declared type has to follow it.

return null is not a node

null is Wado's Option::None literal, and lowering promotes it to a pure value: the Return carries an Operand::Value, not an ExprId. A rewrite that only ever overwrites expression nodes cannot touch it, so rewrite_return_value hands back a replacement operand and each of its three write-back positions — the Return statement, a block-tail Expr statement, a Match arm body — stores it.

Which case the value denotes comes from the return type, exactly as wir_build resolves a promoted Null (translate.rs), so the two agree by construction.

This is not an edge case: it is how every hand-written Iterator::next signals exhaustion. Rejecting it cost both of syntax_highlight's functions, the only one sqlite_parse has, and two or three on each serde workload.

Constants must carry their type

ValuePool::intern is type-erased and records no type; ValuePool::alloc_unshared records one. Every tag and pad this pass mints is a promoted operand the WIR extractor will ask the type of, so all of them go through alloc_unshared — intern compiles fine and panics in wir_build.

Option's declaration has to outlive its uses

The pass mints Option<T> slots after the early DCE run, and wir_build::register_mono_variants registers an instance off the declaration. A program that used no Option had the declaration dropped, so the minted slot type had no WIR registration and codegen panicked. remove_unreachable_types now keeps the option compiler-item declaration unconditionally; that costs nothing, since WIR registers instances, not declarations.

Keeping a declaration means keeping its case payload types too: register_mono_variants substitutes the declaration's payloads against each instance's type args, so a surviving declaration whose payload TypeId was pruned panics just the same. One set therefore gates both the payload walk and the retain — two predicates that have to agree, and did not.

Required changes to optimize::multi_value_return

The tuple classifier becomes the sole owner of the Wasm-level decision, and two of its gates block the traffic this pass sends it.

Trait methods carry the benefit

Confirmed SROA candidates at -O2 (WADO_TRACE=sroa_variant_return):

Iterator::next, Deserializer::*, and FieldSchema::lookup are 78–82 % of every widened function on the parser workloads. multi_value_return.rs:171 rejects a trait method outright; keeping that gate caps the whole change at ~20 % of the available benefit. A trait method is an ordinary direct-call target after monomorphization, and the WIR pass widening 123 of them on cbor_twitter today is the existence proof that the ABI change is safe for them. Dropping the gate also lets the tuple/struct path reach trait methods returning tuples — a separate win, to measure on its own.

The arity cap

(2..=4).contains(&result_types.len()) must widen to the layout cap (8), or the classifier that is supposed to serve this pass rejects every per-case variant layout. This also affects source-level tuples; measure the size effect.

A tail-call group is decided together

return g(x) ties the two signatures together: neither can take the tuple ABI unless both do. Candidates come from the gate's dirty set, so a mutually tail-recursive pair can arrive one at a time, and each is then refused for the other's absence — in every round, forever. The candidate set therefore pulls in the tail-call closure of everything the gate offered, so the group is decided as the unit it is. A self-recursive function never showed this, its partner being itself; the WIR pass never showed it either, running once over the whole module with no gating.

The tail-call rule, again

The classifier required every return to build a fresh aggregate and every call site to bind one, so return g(x) disqualified both functions at once: g had an unbound call site, and its caller returned nothing it had built. That is the same rule this pass applies to variants, and without it a scalarized return g(x) chain lands a boxed tuple where widen.rs used to emit multi-value. Accepting it couples caller and callee candidacy, so classification became a refute-to-fix-point loop like this pass's own.

Both halves of the rule must read the assumed set, not the candidate set as it stands mid-round. Reading the latter let a caller spare its callee and then be invalidated later in the same round: the callee took the ABI, the caller did not, and the caller's return g(x) pushed N results through a one-ref signature — 239 fixtures of invalid Wasm, at -O0 as well as -O2.

They must also agree on where a tail call may sit. The return side accepts one in an if branch, a match arm, a switch arm and their nestings; a call-site walk that recognised only a bare return counted the rest as escapes, invalidated the callee, and the fix-point then withdrew the caller too, so both kept the heap-tuple ABI. A disagreement here costs optimization rather than correctness, which is why it survives a green suite — the shapes have to be enumerated once and shared, not re-derived per half. Break values are the one deliberate asymmetry, refused on both sides: a break names a label, and one naming a block nested inside the return value exits somewhere the rewrite below never reaches, so break L: g(x) would strand g's N results on the stack.

An arm tail is not a break value. A dense match lowers to a br_table, so a tail call in an arm becomes an exit carrying the callee's results rather than a branch tail carrying the aggregate — the rewrite returns the results directly there, exactly as it unpacks a StructNew at the same exit.

What a let may bind is the lowering's question

inline and let_block_flatten leave a call at a block tail with its receiver hoisted in front of it, and wir_build::try_emit_multi_value_let is what has to split N results into N locals there. Classification and lowering therefore ask the same question, and answering it twice put the narrower answer in the classifier: a wrapped call was refused, and its callee kept the heap tuple. One recogniser (block_tail_call) settles it for both, so the shape the classifier accepts is the shape the lowering can split, by construction rather than by an assertion policing the gap.

Consequences

What retired in WIR

widen.rs's entire job — rediscovering the pattern from WIR shapes, the two fix-points, the return rewriting, the call-site rewriting — is subsumed. return_temp.rs normalized temps field_scalarize left and wir_build materialized; those temps are gone at their source rather than recognised afterwards — see the return temp.

4,188 lines to 1,735. wrapper.rs shrank further than planned: ResultStep recorded the wrapper path only so the call-site rewriter could re-walk what validation had already checked, and the one surviving caller discards the path.

The line count is the least interesting number here — this design is justified by pass interaction, not by size.

Two things the WIR pass could do that this one cannot, both found by the suite after the deletion rather than by the benchmarks before it:

• A v128 / i128 / u128 payload. default_value_for_type pads from WirType, after lowering; NIR has no v128 literal to pad with, and the 128-bit types occupy two Wasm results the slot count does not model.
• Nothing else. The mutually tail-recursive pair that also broke is a defect in this pass, not a missing capability — see the tail-call group.

widen = 0 across every benchmark was not evidence of deadness. Benchmarks carry the common shapes; the fixture suite is where the unusual ones live, and half its coverage sits behind WADO_FULL_TEST. Stub the pass out and run the full suite before deleting anything of this size.

The candidate sets converged

The two analyses look at different programs: the WIR pass saw post-nullable_ref, post-propagate_trivial_copies shapes; the NIR pass sees NIR, and sees it before inlining rather than after everything. So they did not coincide at first. On cbor_twitter the NIR pass took 74 functions and left 87 to the WIR side — more than it took.

Closing those shapes one at a time took the leftovers to zero. Counted at -O2 with WADO_TRACE=sroa_variant_return, widen is what the WIR pass still confirmed behind the NIR one on the run that cleared widen.rs for deletion:

The one slot-flatten on gale_gen is by design — that pass stays. widen is dead on every workload, which is what made deleting it safe.

The NIR pass pays for padding and Some(_) / None nodes at return sites that the WIR pass did not, because it worked in WIR directly. No rebox fires on any workload.

Read a widened-function count only from a paired run — same build, flag on and off. An earlier draft recorded sqlite_parse dropping from 67 to 1 and blamed the multi_value_return gate lift; a merge-base build widens 1 too.

Termination and monotonicity

The pass changes signatures inside a loop that runs to a fixed point. It needs no "already rewritten" state to terminate: a rewritten function returns a tuple, and Phase 1 only admits a variant return, so it can never be a candidate again. The candidate set strictly shrinks and the pass cannot keep the loop alive.

Validation is a snapshot; inline runs after

Validation sees the call sites that exist when it runs. nir/inline runs later in the same iteration and can plant new ones — a let mut t = <block ending in a candidate call> whose local still carries the variant the callee no longer returns. The signature is already committed by then, so this is invalid Wasm, not a missed optimization (serde_json_synth_variant.wado).

Placing the pass after nir/inline does not fix it: a callee rewritten in one iteration cannot be un-rewritten when the next iteration plants a site, whatever the order within an iteration. Soundness has to come from the rewrite itself.

The mechanism is to rebox — wrap any call the fast path did not bind back into the variant it used to return, f(x) becoming { let t = f(x); match t.0 { 0 => Ok(t.1!), _ => Err(t.2!) } }. The surrounding context then sees exactly the type it was written against, and validation becomes a profitability filter rather than a correctness precondition. The rebox costs the allocation the scalarization was meant to avoid, so it is a backstop const_fold and sroa are expected to collapse once the tag is known — not a target.

Three rules make it work, and all three were learned by getting them wrong:

• The call-site rewrite must run over everything scalarized so far, not just the round's candidates. Reboxing a site inlining just planted gives up the optimization on exactly the sites inlining made hot; trying the fast path first and reboxing only the remainder does not. The rewrite then has to decline per binding, since a site inherited from an earlier round was never validated.
• The fast path, the repair, and the invariant check must agree on what counts as a binding fed by a call, and all three have to look through a block tail — by the time a later round sees a binding, let_block_flatten may have wrapped its call in a block. A narrower rule in any one of them makes the repair rebox the fast path's own work, which is what produced a tuple-typed binding holding a variant.
• Exiting a labeled block of the callee's tuple type is a handled position too, not just a let of that type. Inlining a scalarized callee into a rewritten call site turns its body into a labeled block typed by the tuple, and the callee's own tail calls then exit through break L: f(x) — or through the branches of one, since the tail-call positions the return side accepts arrive with it. Whether a call already delivers the tuple is the consumer's declared type talking, so every consumer reads its value through the same walk, not just the one that binds a local. Reboxing a call there hands a variant to a tuple-typed position — invalid Wasm from the repair itself. The call node's own type_id looks like a simpler test, but it is not one: the site rewrite retypes every call to a rewritten callee before deciding which consumers it can rewrite, so that type says nothing about the consumer.

With those in place the E2E suite runs clean and emits no invalid Wasm anywhere.

A payload binding that is a reference cell

An arm pattern binds one name and the rewrite re-mints it as let v = t.k. When v's address is taken (Some(v) => sink.put(&v)), lower::plan::boxing promoted its slot from T to Box<T>, so a plain let stores a bare slot read into a box-typed local. The re-mint boxes it instead, exactly as lowering did. Refusing the shape instead cost 39 bindings on gale_gen — enough to keep four functions boxed there — and recovering them is size-neutral.

The test has to be structural: the declared type is genuinely the Box<T> struct of what the pattern names. "The declared type differs, so it was boxed" is unsound, because local indices are pooled — a declaration matching nothing the pattern names belongs to another binding, and boxing against it emits a ref where an i32 is read. A binding the test does not recognise is refused, and validation consults the same predicate so the two decline together.

A local in stores_aliased_locals is refused outright either way: that alias is established outside the body, and a let does not re-establish it.

A rebox is a signal, not a cost to absorb

Reboxing is sound but it reinstates the very allocation the pass removes, so a rebox firing in steady state means the candidate should not have been taken. On cbor_twitter fifty of them fired, every one a return g(x) in a caller that still returned the variant: invalidate_bad_call_sites accepted a tail-position call without checking that the caller had a tuple to pass it through. Refusing the callee in that case — and letting the fix-point propagate the refusal, which is what widen.rs's cascade did — took the reboxes to zero.

The rule that follows: the repair exists for sites that appear after the fact, not for sites validation could have refused. A non-zero steady-state rebox count is a bug in the candidate set, and worth asserting on once the fixture churn is settled.

? re-padding is free

After the rewrite, a ? in a scalarized caller reads the callee's slots and re-pads into the caller's own layout. Each function's returns are rewritten to its own layout independently, so no cross-function agreement beyond "same variant type" is needed — which is what the tail-call rule already requires.

slot_flatten stays in WIR

It splits a ref W result slot whose W is itself a small variant, and it fires 1–2 times per benchmark — once across gale_gen, twice on cbor_twitter. A NIR analogue means a tree-shaped layout through compute_layout, build_result_tuple, the call-site reconstruction and the arity cap, for those one or two functions. This pass declines the shape instead, which restores the WIR splits it had been displacing.

A ?-unwrapped slot, and the arm's result type

DeserializeSeq::next_element returns Result<Option<T>, DeserializeError>. Single-level SROA gives it [i32, ref Option<T>, ref E], and for a scalar T that ref Option<T> is a real heap object — Option<f64> cannot erase to a nullable ref because f64 is not a GC ref. Left boxed it is a struct.new on every element of every sequence decode, in JSON and CBOR alike; flattening it is worth 12-21% on the deserialize rows.

Two things had to meet for slot_flatten to reach it.

then_is_pure_slot_copy matches the ?-unwrap then-arm as [single] or as [LocalSet t, LocalGet t]. The arm can also arrive as one Seq([LocalSet, LocalGet]) node, one level of nesting on from that shape, so it peels the Seq.

Widening the predicate alone emits invalid Wasm. The desugar nests the error test as an else if carrying the binding's type, so rewrite_unwrap_to_guard turning the outer if into a statement leaves the inner arm still declaring a result — "values remaining on stack at end of block". drop_tail_result clears the declaration along the retained arm's tail chain, touching only nodes that diverge. A diverging node produces no value to declare. This is the hazard all_returns_decompose documents from the other side: the validator's coverage and the rewriter's have to move together.

The slot is split whatever the caller holds

Splicing the slot buys one fewer heap object and costs layout.len() more values live across the call. That reads like the rule SROA width answers to: past the register file, a value live across a call is a spill slot reloaded at every call boundary.

The first measurements agreed. A cap declined the callee when any call site sat in a function declaring more than 128 locals, and cbor-twitter, whose User and Status decoders declared 307 and 186, lost 6.3% without it.

The cap is gone. A declared local is not a live one: by the time this pass runs, inlining has left Area's two-field decoder declaring 200 locals and reading 70 of them, so the cap declined next_field on json-catalog's hottest struct. And flattening everything no longer loses anywhere. Against the cap counted on the locals a caller reads, best of four alternating rounds on 2026-09-28:

A settled binding is not a mut binding

A let mut t = f(x) that nothing ever assigns is an immutable binding, and refusing it on the mut alone left 32 functions on cbor_twitter boxed. settled_locals accepts it, and this pass and multi_value_return share the predicate — a site only one of them accepts leaves the tuple boxed, which is worse than the variant it replaced.

The rule counts writes through a local, not just assignments to it: h.seen = h.inner.next() makes h's mut load-bearing, and treating h as settled splits a local the field write then has nowhere to land in, trapping at -O0.

A variant payload takes its own tag and slots

Result<Option<i32>, E> — a payload that is itself a variant — accounted for 30 more. The inner variant gets its own tag and slots in the outer tuple.

A nested layout that reconstructs the inner value at the call site was implemented first and discarded: it produced exactly the intended signatures and cost more than it recovered, because a site binding the whole inner value has to rebuild what the callee just took apart. The form that pays pushes the destructure into the call site, which is what slot_flatten does — so a payload the split would still reach is left to it.

The return temp belongs to field_scalarize

The temp is not this pass's to sink: field_scalarize hoists the field write-back after the returns are already tuple literals, one let per branch, so the local carries two defs and two reads and no single-def rule in this pass reaches it. Sinking the VariantConstruct wrapper at this pass's own rewrite site was tried and is inert — the shape does not exist yet at that point.

The constraint that creates the temp is "the write-back must run between the value's evaluation and the return jump". That applies to the value's components, not to the aggregate: an aggregate literal is a pure construction over its operands, so binding the operands and rebuilding the literal after the sync evaluates the same effects in the same order and leaves no aggregate local. field_scalarize::capture_for_sync states that once for the three sites that used to hand-roll it. Two details it has to get right:

• A literal constant is re-emitted after the sync without a binding. A promoted ValueKind::FieldAccess is not — it is re-emitted at its use, and the sync may have written the field it reads.
• So is a local the sync does not assign, which is decided by reading the sync statements: a re-read ($hfs_F = local.F) assigns its scalar local, a write-back (local.F = $hfs_F) only reads one. This is sound for a ref-typed local too — the binding copies the reference, not the pointee, so a field the sync writes is visible either way.
• Each operand holds its pooled temp until the whole literal is built, or alloc_temp's free list hands one index to two live slots.

Cursor::scan goes from a boxed ref tuple to -> [i32, i32, ref null Fail], and CborDeserializer::is_null — the last boxed tuple this pass left on cbor_twitter — flattens with it.

The aggregate rule alone was not enough to retire return_temp.rs. It removed the shape for a literal, and left it for a bare local — $hfs_call_N = hit; self.pos = $hfs_pos; return $hfs_call_N survived on sqlite_parse and syntax_highlight, which the WIR pass had been cleaning up unnoticed. The local rule above is what closed it. Removing a shape at its source is still cheaper than recognising it afterwards; the caution is that "removed" has to be checked, not assumed from the case that motivated the rule.

A LabeledBlock return value arriving via break L: v

Rejected outright. The return-value rewrite walks the block tail; a value reaching the return through break L: v is not in the tail, so the candidate is declined rather than mis-rewritten. Sound, but it gives up functions whose returns pass through a labeled block.

multi_value_return does take that shape, which inlining an early-returning helper produces, so wir_build has to turn every exit into a Return of the N fields before dropping the block's result type. Dropping it while one exit still yields a struct leaves the function returning nothing, which traps rather than failing validation — so the rewrite reports whether it reached them all and the result type is dropped on that answer.

"Every exit" reaches under value operands too: let x = f()? lowers to a LocalSet whose value is an If, and a break in there exits the enclosing block like any other. Only Block, Loop and If bodies open a Wasm label, so everything else is walked at the enclosing depth — but an operand's own fall-through tail feeds the operand, not the block, and rewriting it into a Return is how a StructNew bound to a local became the function's result.

Alternative considered: ReturnAbi::Variant

Add a Variant case to nir::ReturnAbi carrying nothing — a permission, since the physical layout is a WIR representation fact wir_build must be free to veto (nullable_ref erasure, payload-type eligibility, caps). The classifier sets it; wir_build computes the layout, emits the flat signature, lowers VariantConstruct at the single StmtKind::Return translation site (translate.rs:1770), and binds call results to split locals as try_emit_multi_value_let already does for tuples.

That design is sound, and cheaper: no signature rewriting inside the loop, no new pass, and the same WIR files retire. It is the right fallback if the NIR rewrite proves infeasible — for instance if arm-pattern rewriting turns out to carry the complexity the WIR peel does.

It is not the design here because it cannot serve the motivation. A flag only wir_build reads produces no interaction with any NIR pass, and no refinement of it will, because the NIR the passes see is byte-for-byte what it was before. The measurable difference between the two designs is everything in What it unlocks.

References

• Issue #1742.
• wado-compiler/src/optimize/sroa_param.rs — the in-loop interprocedural signature rewrite this mirrors.
• wado-compiler/src/optimize/multi_value_return.rs — the tuple ABI classifier this pass feeds.
• wado-compiler/src/wir_optimize/sroa_variant_return/ — the WIR pass this replaces.
• wado-compiler/src/wir_optimize/nullable_ref.rs — the lowering that makes the Option<T> slot free.
• Variant Wasm GC Representation
• NIR Optimizer Architecture