Drastically Reduce and Improve the CI/CD Feedback Loop by Going Local with Docker Compose

For this 5th part, I'm going to cover adding simple generative AI features to our CI pipeline using the new Docker Agentic Compose and Docker Model Runner functionalities.

Prerequisites:

• Read Part 1: Tests & Build
• Read Part 2: Run and Stress Tests
• Read Part 3: Modularize CI Jobs
• Read Part 4: Image Analysis, Build and Publish

This year Docker introduced two exciting new features that make it easier to integrate generative AI into our development workflows:

• Docker Model Runner: a simple way to run generative AI models locally.
• Docker Agentic Compose: a tool to orchestrate generative AI applications and agents in Docker Compose environments.

In this part, we'll explore how to use these tools to enrich our CI pipeline with AI capabilities, even using "very small" local models.

My Need

My need is simple: I want to be able to provide an LLM with the content of a file to "ask" it to analyze that content and provide a report.

Constraints:

• I'd like to only use "very small" local models to demonstrate that we can do interesting things without needing large models hosted in the cloud.
• I've chosen models small enough that you can test them even without a GPU.

To meet this need, I created a very simple agent:

the "mini agent" source code is in the project repository, but for reuse in other projects, we could imagine packaging it in a dedicated Docker image in a separate project.

You'll find the agent code in the ai.analyzer folder of the repository:

ai.analyzer
├── Dockerfile
├── go.mod
├── go.sum
└── main.go

The agent code is very simple:

func main() {
    ctx := context.Background()

    engineURL := os.Getenv("MODEL_RUNNER_BASE_URL")
    chatModelId := "openai/" + os.Getenv("CHAT_MODEL_ID")

    fmt.Println("🌍 LLM URL:", engineURL)
    fmt.Println("🤖 Chat Model:", chatModelId)

    file, err := os.ReadFile(os.Getenv("FILE_PATH_TO_ANALYZE"))
    if err != nil {
        log.Fatalln("😡:", err)
    }
    fileContent := string(file)

    oaiPlugin := &openai.OpenAI{
        APIKey: "I💙DockerModelRunner",
        Opts: []option.RequestOption{
            option.WithBaseURL(engineURL),
        },
    }
    g := genkit.Init(ctx, genkit.WithPlugins(oaiPlugin))

    response, err := genkit.Generate(ctx, g,
        ai.WithModelName(chatModelId),
        ai.WithSystem(os.Getenv("SYSTEM_INSTRUCTIONS")),
        ai.WithMessages(
            ai.NewSystemTextMessage(fileContent),
        ),
        ai.WithPrompt(os.Getenv("USER_MESSAGE")),

        ai.WithStreaming(func(ctx context.Context, chunk *ai.ModelResponseChunk) error {
            fmt.Print(chunk.Text())
            return nil
        }),
    )
    if err != nil {
        fmt.Println("😡 Error during generation:", err)
    }

    // Save full response to a file
    outputFile := os.Getenv("REPORT_FILE_PATH")
    err = os.WriteFile(outputFile, []byte(response.Text()), 0644)
    if err != nil {
        fmt.Println("😡 Error writing output file:", err)
    } else {
        fmt.Println("✅ Full response saved to", outputFile)
    }
}

Code Explanation

This Go code (which uses Google's GenKit framework) is an analyzer that uses an LLM (Large Language Model) to analyze files.

Required inputs (environment variables):

• MODEL_RUNNER_BASE_URL: the LLM server URL
• CHAT_MODEL_ID: the model name to use (prefixed by "openai/", as we're using an OpenAI-compatible API)
• FILE_PATH_TO_ANALYZE: the path to the file to analyze
• SYSTEM_INSTRUCTIONS: system instructions for the LLM
• USER_MESSAGE: the user's message/question
• REPORT_FILE_PATH: the path where to save the generated report

How it works:

1. Reads the file specified in FILE_PATH_TO_ANALYZE
2. Connects to the LLM server via the OpenAI API
3. Sends the file content and instructions to the model
4. Displays the model's response in real-time (streaming)
5. Saves the complete response to the file specified by REPORT_FILE_PATH

Agent Dockerization

To run this agent in a Docker container, we need a Dockerfile which is extremely simple:

FROM golang:1.25.2-alpine
WORKDIR /app
COPY . .
RUN go mod tidy
RUN go build -o analyzer main.go
ENTRYPOINT ["./analyzer"]

CI Component with Docker Agentic Compose

I started by creating a compose.ci.ai-analysis.yml file dedicated to the analysis agent (or agents, because I'm actually going to call two of them).

Vulnerability Analysis Report

I'd like my agent to take the vulnerability report generated by Docker Scout and generate a summary of the vulnerabilities found and possibly some recommendations to fix them (but that will depend a lot on the size and capability of the model). So I created the security-agent service in the compose.ci.ai-analysis.yml file, and I configured the environment variables so it can analyze the vulnerability report:

models:
  lucy:
    model: hf.co/menlo/lucy-gguf:q4_k_m

services:

  security-agent:
    build:
      context: ./ai.analyzer
      dockerfile: Dockerfile
    volumes:
      - ./reports:/reports
    environment:
      SYSTEM_INSTRUCTIONS: |
        You are an AI security analyst specialized in container image vulnerability assessment.
        Your task is to analyze vulnerability reports generated by Docker Scout CLI.
        Provide a concise summary of the vulnerabilities found, their severity, and actionable recommendations to remediate them.
      USER_MESSAGE: |
        Please read and analyze the vulnerability report.
        Generate a summary of the vulnerabilities found in the image along with recommendations to fix them.
        Add a table from the results with the following columns: Vulnerability ID, Severity, Description, Recommendation.
      REPORT_FILE_PATH: /reports/cves.analyze.md
      FILE_PATH_TO_ANALYZE: /reports/cves.report.text

    models:
      lucy:
        endpoint_var: MODEL_RUNNER_BASE_URL
        model_var: CHAT_MODEL_ID

The models section defines the lucy model we're going to use for this analysis. Docker Compose will automatically download the hf.co/menlo/lucy-gguf:q4_k_m model (from Huggingface in this case, but it could be Docker Hub or your private registry) and inject the MODEL_RUNNER_BASE_URL and CHAT_MODEL_ID environment variables into the security-agent service so the agent can use the model.

If you launch this service with the command:

docker compose -f compose.ci.ai-analysis.yml up --build

The agent will read the /reports/cves.report.text file, analyze it with the lucy model, and generate a summary report in the /reports/cves.analyze.md file.

You should see a markdown document like this:

### Vulnerability Summary
The Docker Scout report identifies the following vulnerabilities in the image `hello-local-ci:demo-from-ci`:
- **High Severity**: `CVE-2025-58187` (golang/stdlib 1.25.2)
- **Medium Severity**: `CVE-2025-58050` (pcre2 10.43-r1)
- **Low Severity**: Two vulnerabilities (`CVE-2025-46394` and `CVE-2024-58251`) in busybox 1.37.0-r19

---

### Vulnerability Table

| Vulnerability ID | Severity | Description | Recommendation |
|------------------|----------|-------------|------------------|
| CVE-2025-58187   | HIGH     | High-severity vulnerability in golang/stdlib. Affected versions >=1.25.0 <1.25.3. | Update to 1.25.3. |
| CVE-2025-58050   | MEDIUM   | Medium-severity vulnerability in apk/alpine/pcre2. Affected versions <10.46-r0. | Update to 10.46-r0. |
| CVE-2025-46394   | LOW      | Low-severity vulnerability in apk/alpine/busybox. Affected versions <=1.37.0-r19. | Update to 1.37.0-r19. |
| CVE-2024-58251   | LOW      | Low-severity vulnerability in apk/alpine/busybox. Affected versions <=1.37.0-r19. | Update to 1.37.0-r19. |

Source Code Analysis Report

In this example I'm only analyzing one file (my agent can only handle one file at a time), but we could imagine improving it to handle multiple files or even a complete directory. And here again, of course, the quality of the report will depend a lot on the size and capability of the model used.

I added a second model in the models section, this time a model specialized in source code: hf.co/qwen/qwen2.5-coder-3b-instruct-gguf:q4_k_m:

models:
  lucy:
    model: hf.co/menlo/lucy-gguf:q4_k_m

  coder-model:
    model: hf.co/qwen/qwen2.5-coder-3b-instruct-gguf:q4_k_m

And I created a second quality-agent service that will analyze the agent's own main.go file to generate a report on code quality:

services:

  quality-agent:
    build:
      context: ./ai.analyzer
      dockerfile: Dockerfile
    volumes:
      - ./reports:/reports
      - ./:/project
    environment:
      SYSTEM_INSTRUCTIONS: |
        You are a world class Golang expert.
        Your job is to analyze the source code and summarize it.
        Provide recommendations and improvements to increase code quality.
        Generate the summary in markdown format
      USER_MESSAGE: |
        Please read and analyze the provided Go source code.
        Generate a summary of the code along with recommendations to improve its quality.
      REPORT_FILE_PATH: /reports/quality.analyze.md
      FILE_PATH_TO_ANALYZE: /project/main.go

    models:
      coder-model:
        endpoint_var: MODEL_RUNNER_BASE_URL
        model_var: CHAT_MODEL_ID

You'll notice that this time, we're injecting the coder-model model into the quality-agent service.

If you launch this service with the following command using the quality-agent service name to launch only this service:

docker compose -f compose.ci.ai-analysis.yml up quality-agent

You'll get a markdown report in the reports/quality.analyze.md file that will look like this:

Extract from the report generated by the agent:

### Summary of the Code

The provided Go source code defines a simple HTTP server that serves different types of responses based on the requested URL path. The server is designed to handle five different endpoints:

1. **Root Endpoint (`/`)**: Returns an HTML page listing available API routes.
2. **Text Endpoint (`/text`)**: Returns a plain text response with information about a human.
3. **HTML Endpoint (`/html`)**: Returns an HTML response with information about a human.
4. **JSON Endpoint (`/json`)**: Returns a JSON response with information about a human.
5. **Health Check Endpoint (`/health`)**: Returns a JSON response indicating the server's health status.

### Recommendations to Improve Code Quality

1. **Use Consistent Naming Conventions**: Ensure consistent naming conventions for functions, variables, and package names. This makes the code easier to read and maintain.

2. **Separate Logic into Functions**: Move the human data creation and JSON encoding logic into separate functions to improve readability and reusability.

3. **Use Named Returns in Function Signatures**: Use named returns in function signatures to improve clarity and make it easier to understand the return values.

4. **Avoid Magic Strings**: Replace hard-coded strings with constants or variables for better maintainability and readability.

5. **Use Error Handling**: Implement proper error handling to manage any issues that may occur during HTTP request processing or data encoding.

6. **Consider Using Libraries for HTTP Handling**: While this example uses native Go functions, consider using third-party libraries like `net/http/pprof` for additional features like profiling and debugging.

7. **Validate Input**: If the server receives input through the request, consider adding input validation to ensure the data is correct and secure.

8. **Document the Code**: Add comments and documentation to explain the purpose and functionality of each part of the code to help others (or yourself) understand it more easily.

Here is an improved version of the code incorporating some of these recommendations:

... improved Go code ...

This version of the code follows the recommended best practices and improves readability, maintainability, and functionality.

All that's left is to integrate our new CI jobs into the main CI pipeline

Integration into the CI Pipeline

First, we need to add the models section to the main compose.ci.yml file:

models:
  lucy:
    model: hf.co/menlo/lucy-gguf:q4_k_m

  coder-model:
    model: hf.co/qwen/qwen2.5-coder-3b-instruct-gguf:q4_k_m

Then, we need to add the two services to the services section of the compose.ci.yml file:

  quality-agent:
    extends:
      file: compose.ci.ai-analysis.yml
      service: quality-agent

As well as:

  security-agent:
    extends:
      file: compose.ci.ai-analysis.yml
      service: security-agent

    depends_on:
      image-analysis:
        condition: service_completed_successfully

And here's what the complete pipeline looks like with the two new AI analysis jobs integrated:

Conclusion

In this part, we saw how to integrate generative AI capabilities into our CI pipeline using Docker Model Runner and Docker Agentic Compose. I used Go and GenKit to create a simple agent, but you can very well use Langchain, LangchainJS, Langchain4J, or any other framework of your choice. I just wanted to demonstrate that with "very small" local models, we can already do interesting things and that it's very simple to integrate using Docker Model Runner and Docker Agentic Compose.

You can find the complete code for this project on GitHub, right here: https://github.com/Triton-CI/hello-compose-ci/tree/v0.0.5

Stay tuned for the next part (under construction).