Dependency caching belongs at the registry layer, not the package store
Caching the package manager store treats a symptom. A pull-through proxy registry is the layer where cross-runner dependency caching actually compounds.
For months I treated dependency caching as a store problem. Cache /pnpm-store. Cache .m2. Cache .uv_cache. Bridge the tarball to GitHub Actions cache or S3, key it on the lockfile, restore it before install. The work was real, the speedups were real, and I was solving the wrong layer.
What I actually wanted was narrower than I admitted: the second CI run, on a fresh runner, should not re-download react. That is a registry concern. The store is just a local mirror of the registry, and ferrying a mirror between runners is a workaround for not having a shared registry to mirror from.
Four layers, not one
Dependency and build caching happens at four distinct layers, and conflating them is what produced the confusion.
| Layer | What it caches | Examples |
|---|---|---|
| Package manager store | downloaded tarballs, content-addressed | ~/.pnpm-store, ~/.m2/repository, pip cache |
| Build-tool layer cache | intermediate build steps | Dagger withExec layer cache, Dockerfile RUN results |
| Registry / pull-through proxy | upstream packages, fetched once and served | Nexus proxy repo, Verdaccio, a private pnpm registry |
| Build cache (task outputs) | compiled outputs keyed by input hash | Gradle build cache, Bazel action cache, ccache |
Store-caching, the tarball-to-S3 dance, lives entirely in the first row. It asks: how do I move this mirror between runners? The third row asks a better question: do I need a per-runner mirror at all?
The reframe
A pull-through proxy registry is the shared mirror. When pnpm, Maven, or pip points at a proxy repo, the first request for a package fetches it from npmjs, Maven Central, or PyPI and caches it; every later request, from any runner or CI provider or laptop, is served warm. You stop ferrying tarballs. The local store still exists, but it is always warm because the registry behind it is always warm.
I had written the store approach up as if it were the architecture. It is a fallback for when you cannot, or will not, run a registry.
Where the mistake came from
I hit the same wall twice and misread it both times.
On frangonf.com, I moved the Dagger container to self-contained mise with two cache volumes mounted at /app/.mise and /pnpm-store, and I wrote that the setup delivered a 9× speedup because “subsequent runs reuse the cached contents.” That speedup was real on my laptop and within a single CI run, where the Dagger engine stays alive. It was not true across CI runs on a free Dagger Cloud plan. Cache volumes are scoped to the engine instance; on an ephemeral GitHub-hosted runner with a local engine, the volume is recreated empty every run. I saw reused 0, downloaded 731 on every run and kept treating it as a tuning problem instead of a layer problem.
The cache volumes were not wrong. They were solving the store layer, and the store layer was not where the cross-run win lived.
Which tool covers which layer
Once the layers are separate, the tooling map gets small.
- Dependencies (npm, Maven, PyPI, Docker base images): a Nexus Repository proxy repo. One warm source for every CI, runner, and laptop. This replaces the per-CI store cache for dependencies entirely.
- Images you publish: a Nexus or Harbor hosted repo.
- Docker build layers:
--cache-to type=registryinto a hosted repo, as abuildcachetag. The registry is the build-cache backend; no sidecar required. I already do this in a polyglot monorepo with buildx registry cache. - Compiled task outputs (Gradle build cache, Bazel actions, ccache): a build-cache daemon. Nexus does not touch these; it is a repository manager, not a build cache.
The honest correction: I first sketched this platform with Nexus and a build-cache sidecar as co-equal layers. They are not. Nexus covers everything you download. A build-cache daemon covers only what you compile. In most polyglot stacks the dependency and image layers dominate, and the build-cache daemon is an optional second-order win, not a core layer.
The open-source menu, by layer
The selection above is what fits a polyglot monorepo. The broader menu of open-source, self-hostable options, grouped by the layer each tool actually serves:
| Tool | Layer | Formats / scope |
|---|---|---|
| Nexus Repository OSS | Registry proxy + hosted | Universal: npm, Maven, PyPI, Docker, Helm, Go, NuGet, APT, Conan, RubyGems |
| Verdaccio | Registry proxy + hosted | npm/Node only; lightweight, single process |
| Harbor | Registry proxy + hosted | Docker/OCI; pull-through proxy, replication, scanning (CNCF graduated) |
Distribution (registry:2) | Registry proxy | OCI reference registry with a pull-through proxy mode |
| Quay | Registry proxy + hosted | Container registry with upstream mirroring and scanning |
| devpi | Registry proxy + hosted | Python/PyPI; the Verdaccio equivalent for pip |
| Pulp | Registry proxy | Python, RPM, container, deb, file (Red Hat) |
| GitLab Package Registry | Registry proxy + hosted | Built into self-hosted GitLab; npm, Maven, PyPI, Conan, Terraform |
| MinIO | Store blob backend | S3-compatible object storage; the universal backend for store tarballs |
| Garage / SeaweedFS | Store blob backend | Alternative OSS S3-compatible object stores |
| GitLab Runner distributed cache | Store ferry | S3-backed cache archives for .pnpm-store, .m2, .uv_cache |
| Dagger cache volumes | Store (engine-scoped) | withMountedCache; persists only while the engine lives |
| Dagger Engine | Build layer cache | Content-addressed withExec layers, built in |
| Docker BuildKit / buildx | Build layer cache | Cache mounts plus backends: local, registry, gha, s3 |
| omni-cache | Build cache | S3-backed; speaks gha, Bazel, Gradle, ccache, HTTP |
| bazel-remote | Build cache | Bazel REAPI; S3, GCS, or disk backend |
| BuildBuddy / BuildBarn | Build cache | Bazel REAPI backends, OSS editions |
| sccache | Build cache | Rust, C, C++; S3, Redis, GCS remote backends (Mozilla) |
| ccache | Build cache | Compiler cache with HTTP, Redis, or S3 remote storage |
| Gradle build cache | Build cache | Any HTTP server with PUT; Develocity is the commercial host |
Two patterns show up in the menu. Layer three, the registry, has the most options because it is the layer that compounds across runners, CIs, and teams; pick one universal (Nexus) or compose specialists (Verdaccio for npm, devpi for Python, Harbor for images). Layer four, the build cache, fragments by ecosystem because each compiler invented its own protocol; there is no universal build cache, only one per tool family.
One honest gap: there is no open-source, self-hosted universal Artifactory drop-in anymore. JFrog retired the old universal OSS edition, and its current free tiers are SaaS or Conan-only. Nexus is the practical open-source answer to “I want one self-hosted registry for everything.”
Scope, honestly
Store-caching is the right call for a single repo on a single CI where the install is cheap and you do not want to operate a service. frangonf.com is that case, and it will stay on a lightweight store cache.
The registry layer earns its place at the scale where the store approach starts to multiply: a polyglot monorepo across GitHub Actions and GitLab, a fleet of runners, a team pulling the same dependencies. There, every new runner and every CI provider rebuilding its own mirror from cold is the cost you are actually paying, and a proxy registry is the one change that compounds across all of them.
The line I keep returning to: store-caching optimizes how runners rebuild a mirror. Registry-caching removes the need to rebuild one. The second framing is the one I wish I had started with.
References
- Nexus Repository 3. Universal package manager; proxy, hosted, and group repositories across npm, Maven, PyPI, Docker, Helm, and more
- Docker buildx registry cache. Stores BuildKit build cache as a registry tag, served by any OCI registry host including Nexus and Harbor
- Dagger Cache Volumes. Scoped to the engine instance; the source of the cross-run confusion on ephemeral runners
- Harbor. CNCF graduated registry with scanning, signing, and replication for the image-governance layer
- Verdaccio. Lightweight open-source npm proxy and hosted registry
- devpi. Open-source PyPI proxy, host, and dev index
- Distribution /
registry:2. The OCI reference registry with a pull-through proxy mode - MinIO. Open-source, S3-compatible object storage; the universal store-cache backend
- sccache. Shared compiler cache with cloud storage backends (Mozilla)
- bazel-remote. Open-source Bazel remote cache with S3/GCS/disk backends
- pnpm content-addressed store. The local mirror a proxy registry keeps warm
- Alternatives to JFrog Artifactory (Buildkite). The distinction between a repository manager and a build cache
- Designing a GitLab CI cache strategy for a polyglot monorepo. Where I first wrote that scaling past one runner means standing up Nexus or Artifactory
- Self-Contained mise in Dagger. The post whose cross-run cache claim I am walking back here
This post was written with AI assistance.