Adding Dagger CI to a Polyglot Monorepo

I ported the Dagger CI module from my Astro blog post to a monorepo with a Python service and a couple of pnpm frontend workspaces. The Node side copied cleanly. The Python side broke the caching pattern I had relied on.

Polyglot install constraints

The repo has two runtimes. The backend is a Python package managed with uv, living under backend/. The frontend is two pnpm workspaces under apps/. The existing CI workflow ran separate jobs on separate runners, each installing its own toolchain. No local parity.

I had a working module from the previous post: one Ci class, one private nodeBase(), a check() method chaining every step on one container. The question was whether the same shape worked when the repo had two runtimes.

Pipeline design and fixes

I kept the single Ci class and added a second private base. The module now has nodeBase() and pythonBase().

nodeBase() is the Astro post’s base with one change: this repo has no .npmrc, so I dropped that line. The same Bug-2 lesson from the previous post, applied ahead of time.

pythonBase() builds a python:3.12-slim container, installs ffmpeg and libsndfile1 via apt, installs uv via pip, and runs uv sync. I applied the previous post’s caching advice directly: copy pyproject.toml and uv.lock first, run uv sync, then copy the full source.

// ci/src/index.ts — first attempt
private pythonBase(source: Directory) {
  return dag.container()
    .from("python:3.12-slim")
    .withEnvVariable("DEBIAN_FRONTEND", "noninteractive")
    .withExec(["sh", "-c", "apt-get update && apt-get install -y --no-install-recommends ffmpeg libsndfile1 && rm -rf /var/lib/apt/lists/*"])
    .withExec(["pip", "install", "--no-cache-dir", "uv==0.7.13"])
    .withMountedCache("/root/.cache/uv", dag.cacheVolume("uv-cache"))
    .withFile("/backend/pyproject.toml", source.file("backend/pyproject.toml"))
    .withFile("/backend/uv.lock", source.file("backend/uv.lock"))
    .withWorkdir("/backend")
    .withExec(["uv", "sync", "--extra", "ci"])
    .withDirectory("/", source, { exclude: SOURCE_EXCLUDES });
}

The first dagger call check-backend --source=. failed.

Bug 1: README.md missing

OSError: Readme file does not exist: README.md

pyproject.toml declares readme = "README.md". hatchling, the build backend, validates that field when it builds the package. I had only copied pyproject.toml and uv.lock, so the README was absent at the install layer. I added .withFile("/backend/README.md", source.file("backend/README.md")) and ran it again.

Bug 2: no source to ship

ValueError: Unable to determine which files to ship inside the wheel

hatchling builds the project as an editable install. uv delegates to the build backend declared in [build-system]. Building requires the src/ tree so the backend can locate the package. I had not copied src/ because I was following the lockfile-first caching pattern.

This is where the two runtimes diverge. pnpm never builds the root project when it installs dependencies. It reads package.json, resolves the tree, and writes node_modules. The root source is irrelevant to the install, so copying package.json and pnpm-lock.yaml before pnpm install works: the install layer caches, and the source copy lands later.

uv syncs the project itself, not just its dependencies. It builds an editable wheel by invoking the build backend, and the backend reads the source. The lockfile-first trick assumes the install does not touch the source. For Python editable installs, it does.

If you are unfamiliar with Python’s build pipeline, the frontend/backend split, wheels, and editable installs, I wrote a companion post: Python’s Build System with uv.

The fix: two-phase sync

uv sync has --no-install-project: install all third-party dependencies, skip building the root project. That splits the work.

1. Copy pyproject.toml, uv.lock, and README.md. Run uv sync --no-install-project --extra ci. This layer caches on lockfile changes only.
2. Copy the full source.
3. Run uv sync --extra ci again. The editable build runs with src/ present. The dependencies are already in .venv (excluded from the source copy), so this step rebuilds only the project wheel.

// ci/src/index.ts — final
private pythonBase(source: Directory) {
  return dag.container()
    .from("python:3.12-slim")
    .withEnvVariable("DEBIAN_FRONTEND", "noninteractive")
    .withExec(["sh", "-c", "apt-get update && apt-get install -y --no-install-recommends ffmpeg libsndfile1 && rm -rf /var/lib/apt/lists/*"])
    .withExec(["pip", "install", "--no-cache-dir", "uv==0.7.13"])
    .withMountedCache("/root/.cache/uv", dag.cacheVolume("uv-cache"))
    .withFile("/backend/pyproject.toml", source.file("backend/pyproject.toml"))
    .withFile("/backend/uv.lock", source.file("backend/uv.lock"))
    .withFile("/backend/README.md", source.file("backend/README.md"))
    .withWorkdir("/backend")
    .withExec(["uv", "sync", "--no-install-project", "--extra", "ci"])
    .withDirectory("/", source, { exclude: SOURCE_EXCLUDES })
    .withExec(["uv", "sync", "--extra", "ci"])
    .withWorkdir("/backend");
}

I also mounted a cache volume at /root/.cache/uv. The Astro post never needed this, because Docker layer caching covered pnpm. With uv, if the lockfile changes the dependency layer re-runs, and the cache volume keeps the package downloads warm across runs regardless of layer invalidation.

Verified pipeline

The workflow makes two Dagger calls: check-backend and check-frontend. Both share one engine, so the second call hits the first call’s layers.

One mise run ci:dagger runs both. The two-phase uv sync means dependency installation caches at the lockfile layer, the same goal the previous post had. Only the editable rebuild runs per commit.

Follow-up

I copied the previous post’s caching advice to the Python base without checking whether the build backend had different requirements. The lockfile-first trick works when the package manager does not build the root project at install time. pnpm does not. uv does. That distinction is the whole post.

Generalized: before porting a caching pattern across runtimes, check what the install step actually reads. If it touches the source, the source has to land before the cached layer.

References

• Adding Dagger CI to an Astro Blog. The prior post this builds on
• Python’s Build System with uv. Companion post explaining the build pipeline
• uv: sync command reference. --no-install-project and project syncing
• hatchling. The build backend whose build_editable() hook caused both errors
• PEP 660: Editable installs. The build_editable() hook and editable wheel mechanism
• Dagger. Containerized CI engine
• Dagger caching. Cache volumes and layer caching
• pnpm. Does not build the root project on install
• mise. Local parity via mise run ci:dagger