Drastically Reduce and Improve the CI/CD Feedback Loop by Going Local

This first article will be followed by several others where I'll detail how I implemented this approach.

I spent 6 fantastic years at GitLab wearing various hats (TAM, CSM and simultaneously CSE). I remain convinced that GitLab is an incredible product for managing projects of all sizes. One of the features that made it successful is GitLab CI and its runners. From a DevOps perspective, nothing is more natural than creating CI/CD pipelines to automate tests, builds, deployments, vulnerability scans, reporting, etc.

Now, if I put myself strictly in a developer's shoes, I must admit that the feedback loop could be improved. Indeed, every time I push code, I have to wait for the pipeline to execute on runners hosted by GitLab (or on my own runners). Depending on the runner load and the size of my pipeline, this can take anywhere from a few seconds to several minutes... And it's painful.

Let's be clear, this is not a criticism of GitLab CI (I love GitLab CI). It's just the very nature of CI/CD pipelines that are designed to be executed in isolated and reproducible environments, often in the cloud. So I have the same issue with other CI/CD platforms like GitHub Actions, Jenkins, etc.

During the Devoxx France 2025 conference, I was able to attend an excellent presentation Pour une autre idée de la CI, sur la machine du développeur, avec Dagger by Yves Brissaud (ex-colleague, now at Dagger and Docker Captain) where Yves explains how Dagger allows running CI/CD pipelines locally on the developer's machine, using containers to ensure reproducibility and isolation. It really inspired me. Particularly the remark that quite often our laptops are more powerful than our CI servers and also the reference to David Heinemeier Hansson's blog post: We're moving continuous integration back to developer machines

The author explains that 37signals developers will now run continuous integration (CI) directly on their local machines instead of using remote servers. Thanks to the increased power of recent computers, tests and checks are now much faster locally.

And then 💡 the lightbulb moment... But I can do my CI locally with Docker Compose!🎉 (and I love YAML ❤️). What if I replaced (partially or entirely depending on the project size) my GitLab pipelines and GitHub actions with Docker Compose and a compose.yml file?

Usually, Docker Compose is used to orchestrate multi-container applications. But ultimately, nothing prevents us from using Docker Compose to orchestrate CI/CD tasks locally.

"Compose CI" with a First Small Project

Today, I'm going to use a simple Go project to illustrate this approach. The project is a small web service that exposes a REST API with 3 endpoints: /text which returns plain text, html which returns HTML and /json which returns a JSON object:

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"
)

type Human struct {
    Name string `json:"name"`
    Age  int    `json:"age"`
    City string `json:"city"`
}

func textHandler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/plain")

    human := Human{
        Name: "John Doe",
        Age:  30,
        City: "Paris",
    }

    fmt.Fprintf(w, "Name: %s\nAge: %d\nCity: %s", human.Name, human.Age, human.City)
}

func htmlHandler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/html")

    human := Human{
        Name: "John Doe",
        Age:  30,
        City: "Paris",
    }

    html := fmt.Sprintf(`
<!DOCTYPE html>
<html>
<head>
    <title>Human Info</title>
</head>
<body>
    <h1>Human Information</h1>
    <p><strong>Name:</strong> %s</p>
    <p><strong>Age:</strong> %d</p>
    <p><strong>City:</strong> %s</p>
</body>
</html>
`, human.Name, human.Age, human.City)

    fmt.Fprint(w, html)
}

func jsonHandler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")

    human := Human{
        Name: "John Doe",
        Age:  30,
        City: "Paris",
    }

    json.NewEncoder(w).Encode(human)
}

func rootHandler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/html")
    html := `
<!DOCTYPE html>
<html>
<head>
    <title>API Routes</title>
</head>
<body>
    <h1>Available Routes</h1>
    <ul>
        <li><a href="/text">/text</a> - Text response</li>
        <li><a href="/html">/html</a> - HTML response</li>
        <li><a href="/json">/json</a> - JSON response</li>
    </ul>
</body>
</html>
`
    fmt.Fprint(w, html)
}

func main() {
    http.HandleFunc("/", rootHandler)
    http.HandleFunc("/text", textHandler)
    http.HandleFunc("/html", htmlHandler)
    http.HandleFunc("/json", jsonHandler)

    log.Println("Server starting on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

The project structure is as follows:

.
├── go.mod
├── main.go
├── main_test.go
└── reports/

Let's see how I can set up a local CI with Docker Compose for this project.

What Do I Need for My CI?

For this first blog post, I'm going to implement a simple CI with the following steps:

Run unit tests and generate a test report in /reports.
If the tests pass, build the application and generate the artifact in /builds.
If the build succeeds, run the application and perform simple load tests and generate the results in /reports.

I obviously find my inspiration in GitLab CI. So a CI job runs in a Docker container, with sequential steps and dependencies between jobs. This ensures that each job runs in a clean and isolated environment.

Note: Each job can share artifacts via Docker volumes. In our case we'll use bind mounts to share artifacts between jobs and the host system.

Today, I'm going to focus on the first two jobs: unit tests and application build.

Initialize the CI Compose File with Unit Tests

Start by creating a compose.ci.yml file at the root of your project with the following content:

# Compose CI pipeline
x-documentation: >
  Project: hello-compose-ci
  Start the CI pipeline:
  docker compose -f compose.ci.yml up --build

# Define global variables for the CI pipeline
x-common-variables: &common-variables
  PROJECT_NAME: "hello-compose-ci"
  PROJECT_DESCRIPTION: "A simple web service application"

services:

  # Run unit tests
  unit-tests:
    image: golang:1.25.2-alpine
    environment:
      <<: *common-variables
    working_dir: /project
    command:
      - /bin/sh
      - -c
      - |
        echo "🧪 Running unit tests for $${PROJECT_NAME}"
        echo "📝 Description: $${PROJECT_DESCRIPTION}"

        go mod download
        if go test -v > ./reports/test-output.txt; then
          echo "✅ Tests completed"
          echo "📄 Test report generated at ./reports/test-output.txt"
          go test -cover > ./reports/test-coverage.txt
          exit 0
        else
          echo "❌ Tests failed"
          exit 1
        fi

    volumes:
      - ./:/project

Some Explanations About the `compose.ci.yml` File

1. Documentation (x-documentation)

x-documentation: >
  Project: hello-compose-ci
  Start the CI pipeline:
  docker compose -f compose.ci.yml up --build

Extension not interpreted by Docker Compose, serves only as reference

2. Global Variables (x-common-variables)

x-common-variables: &common-variables
  PROJECT_NAME: "hello-compose-ci"
  PROJECT_DESCRIPTION: "A simple web service application"

&common-variables: YAML anchor allowing reuse of these variables
Defines environment variables shared between services
Uses YAML anchor/alias syntax to avoid duplication

3. Service: unit-tests

Base Image:

image: golang:1.25.2-alpine

Uses official Go image version 1.25.2
Alpine variant for a lightweight container

Environment Variables:

environment:
  <<: *common-variables

<<: *common-variables: injects previously defined variables
Makes PROJECT_NAME and PROJECT_DESCRIPTION available in the container

Working Directory:

working_dir: /project

Defines /project as working directory in the container
All relative paths will start from this directory

Execution Command:

command:
  - /bin/sh
  - -c
  - |
    echo "🧪 Running unit tests for $${PROJECT_NAME}"
    echo "📝 Description: $${PROJECT_DESCRIPTION}"
    ...

/bin/sh -c: executes a shell script
$$: escaping to access environment variables (a single $ in Docker Compose represents Compose variables)
go mod download: downloads Go dependencies
go test -v > ./reports/test-output.txt:
- Runs tests in verbose mode
- Redirects output to a report file
Conditional structure:
- If success:
  - Displays confirmation message
  - Generates coverage report with go test -cover
  - Exits with exit 0 (success)
- If failure:
  - Displays error message
  - Exits with exit 1 (failure)

Mounted Volume:

volumes:
  - ./:/project

Mounts current directory (.) into /project in the container
Allows container to access source code
Generated reports are also accessible from the host

Generated Artifacts:

File	Description
`./reports/test-output.txt`	Detailed output of unit tests
`./reports/test-coverage.txt`	Code coverage report

It's now time to run our CI pipeline locally with Docker Compose!

Running the Pipeline

Use the following command to start the CI pipeline:

docker compose -f compose.ci.yml up

You'll get output similar to this:

Attaching to unit-tests-1
unit-tests-1  | 🧪 Running unit tests for hello-compose-ci
unit-tests-1  | 📝 Description: A simple web service application
unit-tests-1  | go: no module dependencies to download
unit-tests-1  | ✅ Tests completed
unit-tests-1  | 📄 Test report generated at ./reports/test-output.txt
unit-tests-1 exited with code 0

Verify Test Reports

You can check the test reports generated in the reports directory:

.
├── compose.ci.yml
├── go.mod
├── main.go
├── main_test.go
└── reports
    ├── test-coverage.txt
    └── test-output.txt

**test-output.txt**:

=== RUN   TestTextHandler
--- PASS: TestTextHandler (0.00s)
=== RUN   TestHtmlHandler
--- PASS: TestHtmlHandler (0.00s)
=== RUN   TestJsonHandler
--- PASS: TestJsonHandler (0.00s)
=== RUN   TestRootHandler
--- PASS: TestRootHandler (0.00s)
=== RUN   TestHumanStruct
--- PASS: TestHumanStruct (0.00s)
=== RUN   TestHumanJsonMarshalling
--- PASS: TestHumanJsonMarshalling (0.00s)
PASS
ok      hello    0.002s

**test-coverage.txt**:

PASS
coverage: 68.4% of statements
ok      hello    0.004s

Advantages of This Approach

Isolation: tests executed in a clean environment
Reproducibility: same Go version guaranteed
Portability: works on any machine with Docker
Traceability: automatic report generation
CI/CD Integration: easy to integrate in an automated pipeline

Add the Build Job

I'd like to add a build job that runs only if the unit tests succeed. Here's what needs to be added to the compose.ci.yml file:

  # Build application
  build-application:
    image: golang:1.25.2-alpine
    environment:
      <<: *common-variables
      TARGETOS: ${TARGETOS:-linux}
      TARGETARCH: ${TARGETARCH:-arm64}
    working_dir: /project
    command:
      - /bin/sh
      - -c
      - |
        echo "🏗️ Building application for $${PROJECT_NAME}"
        go mod download
        CGO_ENABLED=0 GOOS=$${TARGETOS} GOARCH=$${TARGETARCH} go build \
            -ldflags="-s -w" \
            -o ./builds/hello-$${TARGETOS}-$${TARGETARCH} main.go

        if [ $? -eq 0 ]; then
          echo "✅ Build succeeded"
          echo "📦 Artifact generated at ./builds"
          exit 0
        else
          echo "❌ Build failed"
          exit 1
        fi

    volumes:
      - ./:/project
    depends_on:
      unit-tests:
        condition: service_completed_successfully

Some Explanations About the New `build-application` Service

1. Environment Variables:

common-variables: Common variables inherited via YAML merge (<<:)
TARGETOS: Target operating system (default: linux)
TARGETARCH: Target architecture (default: arm64)

2. Build Command:

Download dependencies: go mod download
Cross-platform compilation:
- CGO_ENABLED=0: Disables CGO for a static binary
- GOOS and GOARCH: Define target platform
- -ldflags="-s -w": Reduces binary size (removes debug symbols)
- Output: ./builds/hello-{OS}-{ARCH}
Verification: Displays success ✅ or failure ❌

👋 And the important part: 3. Dependencies:

unit-tests: This service only runs if unit tests succeed (service_completed_successfully)

Let's run our CI pipeline again.

Running the Pipeline

We still use the same command:

docker compose -f compose.ci.yml up

You'll get output similar to this:

unit-tests-1  | 🧪 Running unit tests for hello-compose-ci
unit-tests-1  | 📝 Description: A simple web service application
unit-tests-1  | go: no module dependencies to download
unit-tests-1  | ✅ Tests completed
unit-tests-1  | 📄 Test report generated at ./reports/test-output.txt
unit-tests-1 exited with code 0
build-application-1  | 🏗️ Building application for hello-compose-ci
build-application-1  | go: no module dependencies to download
build-application-1  | ✅ Build succeeded
build-application-1  | 📦 Artifact generated at ./builds
build-application-1 exited with code 0

Docker Compose logs display outputs from each service in real-time. And you can note that each log line is prefixed with the service name, which makes reading easier.

Tip of the day: You can review logs with the command:

docker compose -f compose.ci.yml logs

And to get logs from a specific service:

docker compose -f compose.ci.yml logs unit-tests

You can verify the binary was built in the builds directory:

.
├── builds
│   └── hello-linux-arm64
├── compose.ci.yml
├── git.sh
├── go.mod
├── main.go
├── main_test.go
├── README.md
└── reports
    ├── test-coverage.txt
    └── test-output.txt

Note that the generated binary is for Linux ARM64. You can modify the TARGETOS and TARGETARCH environment variables to generate binaries for other platforms (for example darwin for macOS). But how to do this without modifying the CI script (our compose.ci.yml file)?

At GitLab, there are GitLab CI Variables (https://docs.gitlab.com/ci/variables/).

To simulate CI Variables you can define these variables in a .env file at the root of your project:

.env:

TARGETOS="darwin"
TARGETARCH="arm64"

On the next pipeline execution, the binary will be generated for macOS ARM64:

.
├── builds
│   └── hello-darwin-arm64
├── compose.ci.yml
├── git.sh
├── go.mod
├── main.go
├── main_test.go
├── README.md
└── reports
    ├── test-coverage.txt
    └── test-output.txt

🤚 It's of course recommended to add the .env file to your .gitignore to avoid committing sensitive information.

Note: To do multi-platform builds of our Go application, it would be entirely possible to modify the build service command or add new build services.

Example:

  multi-arch-build-application:
    image: golang:1.25.2-alpine
    environment:
      <<: *common-variables
    working_dir: /project
    command:
      - /bin/sh
      - -c
      - |
        echo "🏗️ Building application for $${PROJECT_NAME}"
        go mod download

        for OS_ARCH in "linux/amd64" "linux/arm64" "darwin/amd64" "darwin/arm64"; do
          OS=$${OS_ARCH%/*}
          ARCH=$${OS_ARCH#*/}
          echo "Building for $$OS/$$ARCH..."
          CGO_ENABLED=0 GOOS=$$OS GOARCH=$$ARCH go build \
              -ldflags="-s -w" \
              -o ./builds/hello-$$OS-$$ARCH main.go
          if [ $? -ne 0 ]; then
            echo "❌ Build failed for $$OS/$$ARCH"
            exit 1
          fi
        done

        echo "✅ All builds succeeded"
        echo "📦 Artifacts generated at ./builds"
    volumes:
      - ./:/project
    depends_on:
      unit-tests:
        condition: service_completed_successfully

Conclusion

That's it for this first article, you can see that it's quite simple to set up a local CI with Docker Compose. The advantage is that the feedback loop is considerably reduced since everything runs locally on your development machine.

You can iterate quickly, fix bugs and validate your changes before pushing code to a remote repository. And of course nothing prevents me from then running this in a GitLab CI or GitHub Actions pipeline (we'll come back to that later).

In the next article, I'll detail how to add load tests to our local CI pipeline with Docker Compose. 👋 Stay tuned!

Next, I plan to explore more advanced topics such as:

Building Docker images for the application.
Security scanning with Docker Scout.
Publishing images to Docker Hub.
Deploying to a local Kubernetes cluster.
Integrating AI Agents using Docker Model Runner and Agentic Compose.

Find the source code here: https://github.com/Triton-CI/hello-compose-ci/tree/v0.0.1

Drastically Reduce and Improve the CI/CD Feedback Loop by Going Local with Docker Compose

"Compose CI" with a First Small Project

What Do I Need for My CI?

Initialize the CI Compose File with Unit Tests

Some Explanations About the `compose.ci.yml` File

Running the Pipeline

Verify Test Reports

Advantages of This Approach

Add the Build Job

Some Explanations About the New `build-application` Service

Running the Pipeline

Conclusion

Comments

Local CI with Docker Compose

Significantly Reduce and Improve the CI/CD Feedback Loop by Going Local with Docker Compose

More from this blog

GoloScript will be the scripting language for AI

GoloScript and WebAssembly

GoloScript and WebAssembly Support with Wazero

A bit of Golo for the weekend

Playground & Getting Started with Golo 🤓

Command Palette

"Compose CI" with a First Small Project

What Do I Need for My CI?

Initialize the CI Compose File with Unit Tests

Some Explanations About the compose.ci.yml File

Running the Pipeline

Verify Test Reports

Advantages of This Approach

Add the Build Job

Some Explanations About the New build-application Service

Running the Pipeline

Conclusion

Comments

Local CI with Docker Compose

Significantly Reduce and Improve the CI/CD Feedback Loop by Going Local with Docker Compose

More from this blog

Some Explanations About the `compose.ci.yml` File

Some Explanations About the New `build-application` Service