Designing a GitLab CI cache strategy for a polyglot monorepo on a self-hosted runner

A polyglot monorepo (Java + Node + Python + React Native + Terraform) runs on a single self-hosted GitLab runner, and I layered a cache taxonomy on top to keep builds warm without an external cache service to maintain.

Runner and repository constraints

I composed the pipeline out of roughly eleven per-project CI files, assembled across five stages: build, test, sync, deploy, admin. Each subproject speaks a different toolchain. The Spring Boot backend pulls dependencies through Maven. The Node BFF and the Nuxt apps use pnpm. Python tooling is pinned with mise and resolved with uv. I build four service images with Docker buildx.

Default caching does not survive that shape. A default cache: block re-archives large package trees into GitLab cache tarballs on every job, and the runner is set to concurrent = 2, so two jobs sharing a buildx builder will delete or reconfigure each other’s builder mid-build. A polyglot stack means many distinct caches that each want their own invalidation rule, and one warm host to serve them all.

Cache design

MR-authoritative workflow

I dropped the automatic push pipeline for feature branches. The merge-request pipeline is the canonical review signal; the default branch push still auto-runs. Every job carries interruptible: true, so a new commit cancels the superseded run. web and api pipelines stay available for manual triggers.

The reasoning is direct: runner minutes spent on work-in-progress branches are wasted the moment I force-push. The MR is where review happens, so the MR is where CI happens. I get local parity for the same pipeline from gitlab-ci-local; I validate the assembled YAML before push with glab ci lint.

The five cache mechanisms

Five distinct mechanisms, and I chose each because the alternative loses on this runner.

1. GitLab cache archives for tool/package dirs

The committed mise wrapper keeps its cache under .mise/cache/, so I archive .mise/ to capture both installed tools and the wrapper-managed cache. The key is file-hash plus a versioned prefix, which makes the invalidation rule declarative: change mise.lock or the relevant uv.lock, the key changes, the cache misses cleanly.

# Toolchain cache template — uv + mise
.cache:python:
  cache:
    policy: pull-push
    key:
      files:
        - mise.lock
        - infrastructure/scripts/uv.lock
      prefix: uv-v1
    paths:
      - .mise/
      - .uv_cache/

I give every Python-touching job (the Spring backend, the two Nuxt apps, the CMS, the RN app) its own variant of this template, keyed on its own uv.lock. I still cache .mise/ alone for one deploy-unit validation job that never touches Python, under the mise-v1 prefix, because it runs the repo’s mise wrapper.

2. Host-mounted pnpm store: host mount instead of a cache archive

This is the one I want flagged. I do not use a GitLab cache archive for the pnpm store in the Node build/test jobs. The runner bind-mounts a host path into every container at /pnpm-store, and pnpm reads its content-addressed store directly from the host.

The alternative (archiving .pnpm-store/ into a GitLab cache tarball per job) is the standard approach. I tried it first; the tar round-trip on every job was slower than reusing the content-addressed store already on the host, so I switched to the host mount. On a single self-hosted runner the archive loses because the store is large and the host already has the bytes warm. Reusing the store in place is faster, and the store accumulates packages across jobs naturally. Clearing it is a single rm -rf on the host.

The tradeoff is real and covered later: see Tradeoffs.

3. Docker host-daemon layer cache

I run every Docker image build against the host daemon via the mounted /var/run/docker.sock. That gives three things without extra config: host daemon layer reuse across jobs, BuildKit cache mounts declared inside the Dockerfiles (pnpm and Maven cache mounts live here), and warm local pull/push state on the runner host. No extra configuration, no external backend.

4. Registry buildx cache for the four service images

The host daemon cache is per-host and per-prune. To survive both, I push and pull a registry-backed buildx cache for the four service images:

docker buildx build \
  --cache-from "type=registry,ref=${CI_REGISTRY_IMAGE}/backend:buildcache-v1" \
  --cache-to   "type=registry,ref=${CI_REGISTRY_IMAGE}/backend:buildcache-v1,mode=max" \
  --tag "${CI_REGISTRY_IMAGE}/backend:${CI_COMMIT_SHA}" \
  --push .

I give each of the four images (the Spring Boot backend, the main Nuxt frontend, the Payload CMS, the mobile API) its own buildcache-v1 tag, continuously refreshed per build. mode=max keeps intermediate layers in the cache, not just the final image. Reset is deleting the tag.

5. Maven host mount aligned with the pipeline

The runner exposes a persistent host bind mount at /root/.m2. I set the matching env in the pipeline so Maven writes where the mount lives:

.backend:test:
  variables:
    MAVEN_REPO_LOCAL: "/root/.m2/repository"
    MAVEN_OPTS: "-Dmaven.repo.local=/root/.m2/repository"

Without this, Maven defaults to a project-local .m2/repository inside the job container, re-downloading every backend dependency on every run. Aligning the variable with the mount means backend test jobs reuse the host-mounted dependency cache directly.

Per-job buildx builders to avoid concurrency collisions

The runner is concurrent = 2. Two image builds can land on the host at the same time. If they share a buildx builder, the second job’s docker buildx create or docker buildx rm mutates the builder the first job is still using; builds fail with “no such builder” or get silently reconfigured mid-run.

I wrote a small helper that every image-build job invokes. It derives a builder name from CI_JOB_ID (unique per job, per run), creates it before the build, and removes it in a finally block after:

# ci/docker_build.py: per-job builder lifecycle
import os
import subprocess

builder = f"ci-{os.environ['CI_JOB_ID']}"
subprocess.run(["docker", "buildx", "create", "--name", builder, "--use"], check=True)

try:
    subprocess.run(
        [
            "docker", "buildx", "build",
            "--builder", builder,
            "--cache-from", f"type=registry,ref={os.environ['CI_REGISTRY_IMAGE']}/backend:buildcache-v1",
            "--cache-to", f"type=registry,ref={os.environ['CI_REGISTRY_IMAGE']}/backend:buildcache-v1,mode=max",
            "--tag", f"{os.environ['CI_REGISTRY_IMAGE']}/backend:{os.environ['CI_COMMIT_SHA']}",
            "--push", ".",
        ],
        check=True,
    )
finally:
    subprocess.run(["docker", "buildx", "rm", builder], check=False)

Each concurrent job runs with its own builder; nobody steps on anyone else’s buildx inspect or cache state. The registry cache from mechanism 4 still keeps the builds warm across those isolated builders.

Local CI parity with safety guards

gitlab-ci-local runs the same pipeline on a laptop, but the runner config guards it so local runs cannot touch prod:

• id_tokens: {}: no OIDC tokens are minted locally.
• LOCAL_CI_VALIDATION=true: a flag short-circuits any deploy/sync stage.
• /local-ci-bin on PATH: a shim directory that stubs out aws, the registry credential helpers, and any ssm calls. Local parity for build and test, with no AWS, no registry push, no SSM read.

A ci:lint:dry-run task inlines the include: chains into one document and runs glab ci lint against the assembled pipeline. YAML errors, bad includes, and template typos get caught before push, which matters more here than usual because I composed the pipeline out of ~11 files across 5 stages.

Measured behavior

The two host mounts (pnpm store, Maven .m2) ground the “warm” row qualitatively: they skip the archive round-trip that the GitLab cache path pays per job, because the store and .m2 stay warm on disk between runs. I did not measure end-to-end job times before and after, so I am not claiming a specific speedup, only that the bytes are already on the host when the job starts.

runners.cache exists in the runner config but has no S3/MinIO/GCS backend; only MaxUploadedArchiveSize = 0. Cache archives stay local to the runner machine. For one host, I kept it local: low complexity, warm caches, nothing else to operate.

Tradeoffs

I tuned this cache design for one self-hosted runner. The host-mounted pnpm store and the Maven .m2 mount work because one warm host serves every job. The moment I scale the setup to multiple runners or a shared fleet, those host mounts stop being shared; each new runner starts cold and stays partially cold, because nothing propagates the store or .m2 between hosts.

At that point I would stop pretending the host mounts generalize and stand up a distributed cache backend: S3/MinIO/GCS for the runner cache, a pnpm store served over the network or a read-only fallback, and a Maven repository manager (Nexus, Artifactory) in front of .m2. The GitLab cache archives and the registry buildx cache already travel; the host mounts are the part that doesn’t. I would state that tradeoff plainly before scaling.

References

• GitLab CI cache keyword: keys, policy: pull-push, prefixes, paths.
• GitLab Runner concurrent configuration: the concurrent = 2 constraint that drives the per-job buildx builder fix, and the runners.cache section with no distributed backend.
• Docker buildx --cache-from / --cache-to registry backend: registry cache, mode=max, per-image buildcache tags.
• Docker BuildKit cache mounts: --mount=type=cache inside Dockerfiles for pnpm and Maven.
• gitlab-ci-local: local pipeline parity, guarded to build/test stages only.
• glab ci lint: CI YAML validation against the assembled (includes-inlined) pipeline.
• pnpm content-addressed store: why a direct host mount beats tar round-trips on a single runner.
• GitLab CI_JOB_ID variable: the per-job unique value used to derive isolated buildx builder names.