Dagger in a GitLab CI pipeline: assessment

Dagger is a programmable pipeline engine that runs functions in containers. It provides local execution, cache volumes, and pipeline definitions in Go, TypeScript, or Python. I evaluated it against the monorepo pipeline described below.

Dagger’s execution model

Dagger supplies the pipeline engine while GitLab CI, GitHub Actions, or Jenkins continues to provide orchestration. The runner and .gitlab-ci.yml remain in place. Instead of shell scripts and raw docker buildx commands, a job calls dagger call functions written in Go, TypeScript, or Python.

# Today:
backend:build:image:
  image: docker:latest
  script:
    - docker buildx build --cache-from type=registry,ref=$CACHE_IMAGE --push -t $IMAGE .
    - docker push $IMAGE

# With Dagger:
backend:build:image:
  extends: [.dagger]
  script:
    - dagger call build-and-push --source=. --tag=$CI_COMMIT_SHA

The Dagger Engine uses BuildKit to manage container lifecycle, cache volumes, and layer reuse. The Dagger CLI sends GraphQL queries to the engine.

How it integrates with GitLab CI

Dagger needs a container runtime to run its engine. In GitLab CI, you have two common local-engine options:

Option A: Docker executor with socket. The runner starts a Docker container that runs dagger call. Dagger starts its engine through the Docker socket. This uses the existing config.toml, socket mount, and privileged: true configuration.

# infrastructure/gitlab-runner/config-dagger.toml snippet
.dagger:
  image: alpine:latest
  before_script:
    - apk add curl
    - curl -fsSL https://dl.dagger.io/dagger/install.sh | BIN_DIR=/usr/local/bin sh

The installer example is abbreviated. A production pipeline should pin the Dagger CLI version instead of installing the latest version on every job.

Option B: Docker-in-Docker. The runner starts a docker:dind service alongside the job container. Dagger’s engine runs inside that DinD daemon, so the job does not use the host Docker socket. The DinD configuration still requires the privileges documented by GitLab and Docker.

.dagger:
  image: docker:latest
  services:
    - docker:dind
  variables:
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: "/certs"

Dagger’s caching model vs. host-mounted volumes

The cache behavior differs between the current runner and Dagger. Our current runner uses host-mounted volumes for persistence:

# config.toml on our Hetzner box
volumes = [
  "/var/run/docker.sock:/var/run/docker.sock",
  "/srv/gitlab-runner/maven-repo:/root/.m2",
  "/srv/gitlab-runner/pnpm-store:/pnpm-store",
]

Pull policy is set to if-not-present. Docker layers are cached on the host filesystem. Maven dependencies live on the host NVMe. Each CI job starts with the existing host caches and performs the incremental build.

Dagger’s equivalent is cache volumes - content-addressed, persisted on the engine host:

func (m *Backend) Build(ctx context.Context, src *dagger.Directory) *dagger.Container {
    mavenCache := dag.CacheVolume("maven-repo")
    return dag.Container().
        From("eclipse-temurin:25-jre").
        WithMountedCache("/root/.m2", mavenCache).
        WithDirectory("/src", src).
        WithWorkdir("/src").
        WithExec([]string{"mvn", "package", "-DskipTests"})
}

When the Dagger Engine runs on the same persistent host as your current runner, cache volumes persist on that host. They survive engine restarts and use content-addressed storage. The current runner also retains its warm host caches across jobs.

When the engine is ephemeral, cache volumes end with the engine host. Dagger Cloud provides a managed distributed cache for supported plans and versions, adding a separate service and network I/O to cache operations.

GitLab Runner’s distributed cache supports S3, MinIO, GCS, and Azure Blob. A cache: block in .gitlab-ci.yml can point to shared storage so multiple runners reuse the cache. Dagger Cloud is a separate Dagger-managed persistence option, not a replacement for host-mounted package caches.

What Dagger changes

1. Running the same container workflow locally and in CI

# Run the same module locally and in CI:
dagger call test --source=.
dagger call build-and-push --source=. --tag=localhost:5000/test:latest

A local dagger call uses the same module, container images, and BuildKit-based execution model as the CI invocation. This removes the need to reproduce the step through a separate gitlab-ci-local environment.

2. Pipeline portability across runners

Dagger functions are code in Go, TypeScript, or Python. The same module can run on a GitLab runner, GitHub Actions, a local machine, or a Dagger-managed engine. Moving between engines may require changing DAGGER_HOST and the surrounding credentials and cache configuration.

For this monorepo, long-running builds could remain on the Hetzner box while quality checks run on GitLab SaaS runners. The module and cache configuration would still need to support both engines.

3. Moving engine execution away from the host Docker socket

Our pipeline uses privileged: true and mounts the host Docker socket. Dagger can run through DinD or a remote engine instead. This changes how runners connect to the engine, while the runner and engine still require their respective permissions and host resources.

4. Programmatic pipelines in general-purpose languages

Dagger functions are written in Go, TypeScript, or Python and can use the type checking, IDE support, unit tests, and review tools for those languages. This changes how the monorepo expresses its 15 CI includes and parent-child pipeline triggers compared with shell commands embedded in YAML.

What remains unchanged

1. The runner host remains required

The CI runner still provides the machine that runs the jobs. The Hetzner box would run dagger-engine alongside gitlab-runner instead of running Docker directly. The hardware dimensions remain local versus cloud, x86 versus ARM, and dedicated versus shared.

2. Cache behavior on ephemeral runners

If the Dagger Engine runs on an ephemeral EC2 instance that terminates after each job, cache volumes end with it. Dagger Cloud’s distributed cache is one alternative, subject to its plan and version support. GitLab’s built-in cache provides another option through cache: in .gitlab-ci.yml. Dagger cache volumes are content-addressed; GitLab’s cache is archive-based. Both can require network I/O when the cache is outside the runner host.

3. Mobile CI (Android emulator / iOS simulator)

Our pipeline includes a Maestro E2E suite that requires KVM for the Android emulator, Postgres via Docker Compose, and a release APK build. Dagger can build the APK and run the Postgres services. The Android emulator still needs /dev/kvm passed through to a privileged container and therefore requires a KVM-capable host.

4. The ARM64 / x86_64 divergence

If production runs on x86_64 and CI runs on ARM64, cross-platform Docker builds use QEMU emulation. The measured builds in this repository were 3-10× slower than native builds. Dagger’s BuildKit engine supports multi-platform builds, but the architecture difference remains. Native hardware avoids this emulation step.

Assessment for our monorepo

Our monorepo runs ~2,000 CI jobs/month across 15 CI includes. Builds take 45 seconds to 6 minutes depending on the job. Jobs use caches on the persistent host and the host Docker socket.

A full Dagger migration would require rewriting each job as a Dagger function. The current pipeline runs on a closet i5-7500T with a concurrency limit of two jobs. The current measurements identify hardware and concurrency as the limiting factors for this setup.

Decision for this repository:

1. I chose a Hetzner dedicated box with 3× the CPU and 64 GB RAM. This changes the runner hardware without changing the pipeline files.

2. Applying Dagger to one job: the Docker image build for the backend and API services. This moves that job’s pipeline logic into a Dagger module while the rest of the pipeline remains YAML. The Docker engine permissions remain part of the runner configuration.

3. Re-evaluating broader Dagger use if workloads move across GitLab SaaS runners and the Hetzner box, or if another team needs to run the same pipeline on different infrastructure.

Conditions for adopting Dagger

References

• Dagger quickstart for CI
• Dagger GitLab CI integration
• Dagger cache volumes
• Dagger Cloud: managed distributed cache and observability
• Dagger Engine configuration
• GitLab Runner distributed cache