Git Dependency Resolution — Design

Design for making wado-cli resolve, fetch, and build git-repository dependencies. This is Phase 6 of the Dependency Management Implementation Plan; the user-facing surface is already fixed by the Package Manifest WEP and the CLI Subcommands WEP. This document settles the implementation model and the seams each layer touches.

Context

A git dependency is declared in wado.toml today and parses correctly:

"user:router" = { git = "https://github.com/user/router.git", version = "^1.0.0" }
"user:router" = { git = "https://github.com/user/router.git", ref = "main" }
"org:foo"     = { git = "https://github.com/org/monorepo.git", version = "^1.0.0", directory = "packages/foo" }

Everything downstream of parsing is stubbed:

• resolve.rs returns ResolveError::UnsupportedSource { kind: "git" }.
• FilesystemProvider's three git methods return "git dependency backend is not wired yet".
• wado_lsp::host::dependency_index_from skips DependencySource::Git, so a git dep never reaches the compiler.
• RawDependency does not deserialize directory, so that field is silently dropped even though the WEP specifies it.

The scaffolding that is in place: the DependencyProvider trait already has git methods with an in-memory implementation, LockFile/LockedPackage already model a git entry (git+… id + resolved-ref, no integrity), and the path-dependency source-compilation pipeline is fully wired.

Core Decision: a git dependency is a source dependency

The single most important modeling choice. A registry dependency is a published, standalone Component Model artifact consumed across the CM boundary (WEP 2026-06-26). A git dependency is the opposite: a source repository carrying Wado source plus its own wado.toml with transitive dependencies. There is no prebuilt component to pull.

Therefore a git dependency is materialized as a source working tree in the cache and compiled into the consumer exactly like a path dependency — via DependencyIndex.resolved and ModuleSource::Dependency, not via DependencyIndex.components. The only differences from a path dep are:

1. its source lives in the shared ~/wado/ cache at a pinned commit (a git worktree) rather than at a relative path, and
2. it must be acquired (clone/fetch/worktree) before it can be read.

This reuses the entire path-dependency machinery (loader, name resolution, transitive traversal, [package].lib entry discovery) and keeps the compiler agnostic — it still only sees ModuleSource values.

Git backend: shell out to the system git

The DependencyProvider git methods are implemented by invoking the system git binary as a subprocess, mirroring how wado publish shells out to wkg.

Rationale, and the trade-off against the plan's "consuming a dependency must not require an external binary" principle:

• That principle was written for the registry hot path, where every import may trigger a network pull; it justified a native oci-client over wkg. Git acquisition is comparatively cold: a repo is cloned once per (url, commit) and then served from a warm checkout.
• git is effectively universal on developer and CI machines, far more so than wkg. A pure-Rust git (gix/git2) adds a large dependency surface (transport, auth, protocol) for a cold-path feature.
• Cargo itself defaults to shelling out to git.

The provider trait is the seam: a later swap to gix changes only FilesystemProvider, never the resolver or the compiler wiring. A missing git binary is reported as a clear, actionable ProviderError, not a panic.

Submodules are populated by default (safe side): a dependency's submodules are part of its source, so materialization runs git submodule update --init --recursive in the worktree (a no-op without submodules). Wado has no build scripts, so a checkout is inert source with no code-execution risk.

git invocations

Cache layout: a canonical clone with nested per-version worktrees

The cache must satisfy two requirements that pull against each other: stay ghq-compatible — ~/wado/{host}/{owner}/{repo} is itself a real, browsable git working tree, so GHQ_ROOT=~/wado ghq list and editor/fuzzy-cd tooling see one clean entry per repo — and hold multiple versions of one repo at once, since different consumers pin different commits. The WEP's original layout ({owner}/{repo}/{version}-{ref}/) fails the first: {owner}/{repo} becomes a container of version dirs, not a checkout.

git worktree, with the version worktrees nested inside the canonical clone, resolves the tension:

~/wado/github.com/user/router/                        # canonical clone: default
                                                       # branch, ghq entry, object
                                                       # store + worktree admin
~/wado/github.com/user/router/.worktrees/1.0.2-abc1234d/  # linked worktree @ commit
~/wado/github.com/user/router/.worktrees/2.1.0-def56789/  # linked worktree @ commit

• The canonical clone at {owner}/{repo} is a normal (non-bare) working tree on the remote's default branch — what ghq and browsing tools see — and it hosts the shared object store and worktree metadata. A bare clone would save one working-tree's worth of disk but forfeit ghq compatibility (no working tree at {owner}/{repo}); decided in favor of compat, since Wasm packages are small (the WEP's stance).
• Each consumed commit is a linked worktree under {owner}/{repo}/.worktrees/{version}-{short-ref}. Nesting inside the repo is what keeps ghq clean: ghq list stops descending the moment it finds the repo's .git, so nested worktrees are never enumerated as their own entries.
• Worktrees share the object store, so a version costs only its checked-out working files, not a second copy of history. Even a redundant worktree of an already-present commit (a monorepo whose two subdirectory packages carry different [package].versions at one commit) costs only working-file bytes — which is why the dir name keeps the readable {version}-{short-ref} rather than collapsing to a bare commit id.

The worktrees live in .worktrees/, not build/: a git dependency is itself a Wado package whose own wado build writes build/<world>.wasm, so build/ is a path the dependency may legitimately track — nesting our checkouts there could collide with the upstream tree. .worktrees/ is dedicated and dot-hidden. On creating the canonical clone, the tool appends .worktrees/ to {repo}/.git/info/exclude so the canonical working tree never reports our checkouts as untracked and a stray git status/git clean stays quiet.

short-ref is the first 8 hex of the resolved SHA; it disambiguates two commits that resolve to the same version tag.

A worktree is disposable derived state: it is reproducible from the locked SHA via git worktree add, so it can be deleted and rebuilt at will (see wado clean). The sources of truth are the canonical clone's object store and the lock's resolved-ref — never the checked-out files.

The Wado root and its config

The cache tree is the Wado root — the same concept ghq calls its "root", a flat {host}/{owner}/{repo} store of cloned source. Naming and control:

• The term is root, not home: home (à la CARGO_HOME) connotes the tool's whole private state dir, whereas this is a source-checkout root meant to be interchangeable with ghq's. The existing WADO_ROOT env var and cache_root() resolver already use "root".

• An XDG config file $XDG_CONFIG_HOME/wado/config.toml (defaulting to ~/.config/wado/config.toml when XDG_CONFIG_HOME is unset) sets it:

  root = "~/ghq"   # point the Wado root at an existing ghq root
  

• Resolution precedence: WADO_ROOT (env) → root in $XDG_CONFIG_HOME/wado/config.toml → default ~/wado. ~ and $VARS expand. The resolver lives where both the CLI (which fetches) and the LSP (which reads offline) already share cache_root(), so both agree on one location.

Pointing the root at ~/ghq is a first-class use case, and it is precisely the nested-.worktrees/ layout that makes it safe: a git dependency's canonical clone lands at ~/ghq/{host}/{owner}/{repo} — exactly where ghq get would put it, so the two tools interoperate — while the per-version worktrees stay hidden inside it and never pollute ghq list. The @ref-sibling layout, by contrast, would have littered a real ghq root with repo@ver entries.

Registry components continue to live under the same root ({host}/{ns}/{name}/{version}/component.wasm); they carry no .git, so ghq ignores them.

Worktrees are global, not the project's build/

The per-version worktrees live under the Wado root, not in the consuming project's build/. build/ holds only this project's compiled outputs (build/<world>.wasm, build/kiln/…); dependency source is shared machine-wide, mirroring the existing rule for registry components (see wado-cli/src/cache.rs: the ~/wado/ cache exists so packages are shared "instead of re-downloading into each project's build/").

Materializing into build/ was considered and rejected:

• It would not remove the global clone or its locking — the object store and the ghq-compatible canonical checkout still have to live under the root — so it only relocates the leaf checkout while adding a coupling: rm -rf build/ (a routine wipe) orphans git worktree admin entries in the global clone until a prune.
• It loses cross-project sharing: N projects on one machine would each re-check-out the same commit.
• It diverges from registry deps, which are global — git deps behaving differently would be a needless inconsistency.

The global model keeps one shared, ghq-browsable copy per commit and confines per-build churn to wado clean.

Acquisition: resolve reads a blob, materialize adds a worktree

Two tiers, so resolution never pays for a working-tree checkout:

Resolution-time (no worktree) — used by the resolver to read a manifest and its transitive deps:

1. Ensure the canonical clone exists (git clone <url> <repo> if absent).
2. Ensure the commit's objects are present: git fetch origin <ref> (prefer a shallow --depth 1; fall back to an unshallowed fetch when the server rejects a by-SHA want, uploadpack.allowReachableSHA1InWant off).
3. Read the manifest without checking anything out: git -C <repo> show <sha>:<directory>/wado.toml.

Materialize-time (worktree) — used by wado fetch / build when source files must be on disk:

1. Compute the worktree dir {owner}/{repo}/.worktrees/{version}-{short-ref}.
2. Warm hit if it exists and git -C <worktree> rev-parse HEAD == <sha> — done (a commit is immutable, so no re-fetch).
3. Otherwise git -C <repo> worktree add --detach <worktree> <sha> (objects are already present from the fetch above — no second network trip).

Concurrency and crash safety

Multiple processes (parallel builds, the LSP alongside a CLI invocation) may clone/fetch/worktree add the same repo at once. Mutations of one repo are serialized by a per-repo advisory file lock (flock on <repo>/.git via the already-present libc on unix; LockFileEx, or a small cross-platform lock crate such as fs4, on Windows). Reads of an already-materialized worktree take no lock — a completed worktree is immutable.

Crash safety: worktree add is not atomic, so a crash can leave a partial worktree. The lock holder verifies HEAD == <sha> after adding; a worktree that fails the check is git worktree remove --forced and rebuilt. A worktree dir deleted out from under the clone (by wado clean or by hand) is reconciled by git worktree prune. Because every mutation runs under the per-repo lock, a second process never races a partial tree — it waits, then sees either a good worktree or repairs the leftover.

Implementation

Where each piece lives (see the code for detail):

wado clean evicts derived state: it removes every {owner}/{repo}/.worktrees/ directory and runs git worktree prune, leaving the canonical clones and registry components unless --all is given. Safe because a worktree is reproducible from its locked SHA.

Reproducibility, offline, integrity

• The resolved-ref SHA pins the exact tree; a warm checkout under --offline needs no network. Optionally assert git rev-parse HEAD == resolved_ref on a warm checkout to detect tampering.
• --locked forbids re-resolution: a git dep whose manifest requirement no longer matches its lock entry is an error, not a silent re-resolve.
• These land with the Phase 3 --locked/--offline/--frozen flags; git deps need no special-casing beyond honoring them.

Tests

Unit tests cover the cache-path parser and the resolver (against the in-memory provider); wado-cli/tests/git_dependency.rs drives the real git backend end to end against local file:// repos (basic, inline, submodule, monorepo).

Open questions

• Fetch cost: the PubGrub prefetch fetches every in-range version's manifest, and materialization does a full fetch of the ref. Both are conservative; optimize (bounded prefetch, by-SHA shallow fetch) once measured.
• Cross-platform locking: flock (libc) covers unix; Windows needs LockFileEx or a lock crate (fs4).
• Nested-group vs local URLs: a remote URL keeps its full path (so GitLab subgroups work); file:///absolute paths key on the trailing three segments. A deeper local layout could still collide — revisit if it matters.