Hello Nova!
For the past 2 years, I've been deeply involved in generative AI, both professionally and personally. I love developing in Go, but unfortunately I haven't found exactly what I needed in terms of AI agent frameworks for working with (local) LLMs: very often the tools are primarily designed to work with external APIs (OpenAI, Anthropic, Gemini...) and the specificities of their associated models, rather than for local models (and without taking their limitations into account), which can make it difficult to optimize the use of these same local LLMs. I mainly use Docker Model Runner (DMR) (dogfooding 😉) to serve LLMs on my machines (MacBook Pro M2, MacBook Air M4, and a few Raspberry Pi 5 - yes, even without GPU, you can do generative AI).
So as I was saying, I hadn't found my happiness in terms of AI agent frameworks in Go, so I decided to make my own. In 2 years, I actually made several, but at the end of 2025, I decided to consolidate my knowledge and experience to create NOVA (Neural Optimized Virtual Assistant): an AI agent framework in Go, open source, easy to use, modular, and designed to work first with local models.
The goal of NOVA is to provide ready-to-use AI agent structures that can be easily composed to create virtual assistants tailored to your needs.
The main agents provided by NOVA are:
The Chat Agent: a simple conversation agent.
The RAG Agent (in memory for now): an agent for similarity search.
The Tools Agent: for function calling and using external tools (MCP).
The Compressor Agent: to summarize and compress information (useful when conversation history becomes too long).
The Structured Agent: to extract structured information from free text.
The Orchestrator Agent: to detect the topic/intent from user messages and redirect to the appropriate agent.
Chat Agent Example:
agent, err := chat.NewAgent(
ctx,
agents.Config{
EngineURL: "http://localhost:12434/engines/llama.cpp/v1",
SystemInstructions: "You are Bob, a helpful AI assistant.",
KeepConversationHistory: true,
},
models.Config{
Name: "ai/qwen2.5:1.5B-F16",
Temperature: models.Float64(0.8),
},
)
if err != nil {
panic(err)
}
result, err := agent.GenerateStreamCompletion(
[]messages.Message{
{Role: roles.User, Content: "Who is Jean-Luc Picard?"},
},
func(chunk string, finishReason string) error {
// Simple callback that receives strings only
if chunk != "" {
fmt.Print(chunk)
}
if finishReason == "stop" {
fmt.Println()
}
return nil
},
)
if err != nil {
panic(err)
}
Structured Agent Example:
type Country struct {
Name string `json:"name"`
Capital string `json:"capital"`
Population int `json:"population"`
Languages []string `json:"languages"`
}
ctx := context.Background()
agent, err := structured.NewAgent[Country](
ctx,
agents.Config{
EngineURL: "http://localhost:12434/engines/llama.cpp/v1",
SystemInstructions: `
Your name is Bob.
You are an assistant that answers questions about countries around the world.
`,
},
models.Config{
Name: "hf.co/menlo/jan-nano-gguf:q4_k_m",
Temperature: models.Float64(0.0),
},
)
if err != nil {
panic(err)
}
response, finishReason, err := agent.GenerateStructuredData([]messages.Message{
{Role: roles.User, Content: "Tell me about Canada."},
})
if err != nil {
panic(err)
}
fmt.Println(response.Name)
fmt.Println(response.Capital)
fmt.Println(response.Population)
fmt.Println(response.Languages)
And the output will look like this:
Canada
Ottawa
38000000
[English French]
In the future, I'll write a more complete "getting started" for NOVA, but in the meantime, you can check out the GitHub repository: https://github.com/SnipWise/nova.
Since these "little ready-to-use agents" are easy to make work together, I started adding new types of agents to NOVA, or rather "agent teams" for more complex tasks. And in particular the Crew Agent.
I've developed other types of agents, such as an agent that exposes a REST API, but that will also be the subject of a future article.
The Crew Agent
Imagine you want to create a virtual assistant capable of handling several types of user requests, such as programming, philosophy, cooking, etc. Instead of creating a single agent that tries to do everything, you can create several specialized agents, for example 4 Chat agents each using a different LLM and prompt:
agentCrew := map[string]*chat.Agent{
"coder": coderAgent,
"thinker": thinkerAgent,
"cook": cookAgent,
"generic": genericAgent,
}
And then, we use a Crew Agent to orchestrate their use.
// ------------------------------------------------
// Define the function to match agent ID to topic
// ------------------------------------------------
matchAgentFunction := func(currentAgentId, topic string) string {
fmt.Println("Matching agent for topic:", topic)
var agentId string
switch strings.ToLower(topic) {
case "coding", "programming", "development", "code", "software", "debugging", "technology", "software sevelopment":
agentId = "coder"
case "philosophy", "thinking", "ideas", "thoughts", "psychology", "relationships", "math", "mathematics", "science":
agentId = "thinker"
case "cooking", "recipe", "food", "culinary", "baking", "grilling", "meal":
agentId = "cook"
default:
agentId = "generic"
}
fmt.Println("Matched agent ID:", agentId)
return agentId
}
orchestratorAgentSystemInstructions := `
You are good at identifying the topic of a conversation.
Given a user's input, identify the main topic of discussion in only one word.
The possible topics are: Technology, Health, Sports, Entertainment, Politics, Science, Mathematics,
Travel, Food, Education, Finance, Environment, Fashion, History, Literature, Art,
Music, Psychology, Relationships, Philosophy, Religion, Automotive, Gaming, Translation.
Respond in JSON format with the field 'topic_discussion'.
`
orchestratorAgent, err := orchestrator.NewAgent(
ctx,
agents.Config{
Name: "orchestrator-agent",
EngineURL: engineURL,
SystemInstructions: orchestratorAgentSystemInstructions,
},
models.Config{
Name: "hf.co/menlo/lucy-gguf:q4_k_m",
Temperature: models.Float64(0.0),
},
)
if err != nil {
panic(err)
}
// ------------------------------------------------
// Create the Crew agent
// ------------------------------------------------
crewAgent, err := crew.NewAgent(
ctx,
crew.WithAgentCrew(agentCrew, "generic"),
crew.WithMatchAgentIdToTopicFn(matchAgentFunction),
crew.WithExecuteFn(executeFunction),
crew.WithConfirmationPromptFn(confirmationPromptFunction),
crew.WithToolsAgent(toolsAgent),
crew.WithRagAgentAndSimilarityConfig(ragAgent, 0.4, 7),
crew.WithCompressorAgentAndContextSize(compressorAgent, 8500),
crew.WithOrchestratorAgent(orchestratorAgent),
)
if err != nil {
panic(err)
}
result, err := crewAgent.StreamCompletion(question, func(chunk string, finishReason string) error {
// Use markdown chunk parser for colorized streaming output
if chunk != "" {
fmt.Print(chunk)
}
if finishReason == "stop" {
fmt.Println()
}
return nil
})
So, this is just a code snippet, but basically, the Crew Agent will analyze the user's question, identify the main topic using the Orchestrator Agent, then select the appropriate specialized agent to answer the question. If necessary, it can also use a RAG Agent to search for relevant information in a knowledge base, a Tools Agent to execute external functions, and a Compressor Agent to manage the size of the conversation history.
Btw, there's also a REST API version of the Crew Agent in NOVA.
Crew Agent Architecture
Here's a diagram illustrating the architecture and relationships between the different Crew Agent components:
🍒 Cherry on top 🍰
As a base library, I use the official OpenAI Go SDK, which means NOVA also works with:
Ollama
Hugging Face API
Cerebras API
And certainly others, but I've only tested those for now.
Stay tuned, I'll soon publish more content on NOVA which I use every day on personal projects, but also for Docker Model Runner demos, testing the capabilities of new LLMs, etc.