Parakeet, an easy way to create GenAI applications with Ollama and Gol

What is "Parakeet"... and Why

Several frameworks exist for developing "Generative AI" applications that interact with LLMs. The most well-known include LangChain, LangChain JS, LangChain4J, and LangChain for Go.

These frameworks are useful as they provide abstractions to use various GenAI API systems (like OpenAI, LLamaFiles, and Ollama) and tools for document processing, connecting to vector databases, and more.

However, I find them complex, with a steep learning curve for simple needs like mine:

I mainly use Ollama.
I want to query one or more LLMs,
And I want to easily:
- Manage conversation memory,
- Give instructions to the model,
- Add context to my questions.

For this, Ollama's API is simple and sufficient. The Ollama project provides two SDKs (JavaScript and Python) that are easy to master.

However, since I am increasingly developing in Go, I decided to create a library in Go to easily use the Ollama API. My main requirement is that writing a GenAI application with Ollama and Go should be straightforward, and the code should remain easy to review later.

This led to the creation of Parakeet. Now, let's look at some code examples.

Hello World

Everything should start with "👋 Hello World 🌍". So this is an example of a simple completion (simple completion means different from a chat completion):

package main

import (
    "github.com/parakeet-nest/parakeet/completion"
    "github.com/parakeet-nest/parakeet/llm"
    "github.com/parakeet-nest/parakeet/enums/option"
    "fmt"
    "log"
)

func main() {
    ollamaUrl := "http://localhost:11434"
    model := "tinydolphin"


    options := llm.SetOptions(map[string]interface{}{
        option.Temperature: 0.5,
    })

    helloWorldQuery := llm.Query{
        Model: model,
        Prompt: `Generate a 'Hello, World!' program in Golang.`,
        Options: options,
    }

    _, err := completion.GenerateStream(ollamaUrl, helloWorldQuery,
        func(answer llm.Answer) error {
            fmt.Print(answer.Response)
            return nil
        })

    if err != nil {
        log.Fatal("😡:", err)
    }
}

You should get something like this:

Here is a simple Golang program that will print "Hello, World!" 
to the console:

```go
package main

import (
    "fmt"
)

func main() {
    fmt.Println("Hello, World!")
}
```
This program defines a function `main` which is the entry point of 
your application. 
Inside this function, we first print "Hello, World!" to the console 
using the `fmt.Println()` function.

Remark: I use TinyDolphin model (https://ollama.com/library/tinydolphin) because it's a small LLM (or SLM), and then Ollama can serve it very correctly on a Pi5 8GB RAM.

We saw that asking for a simple thing is pretty straightforward. But How can I create a chat with a "real" conversation using a conversational memory?

Chat completion

Chat with Memory

Chat completion is very easy to use; you only have to send a list of messages to the LLM, like some instructions and a question. You need to specify if the message is a system or user message.

Then, to keep a conversational memory, you only need to add the answer to the messages list for the next query (you could add the former question, too by the way):

package main

import (
    "fmt"
    "log"

    "github.com/parakeet-nest/parakeet/completion"
    "github.com/parakeet-nest/parakeet/llm"
    "github.com/parakeet-nest/parakeet/enums/option"
)

func main() {
    ollamaUrl := "http://localhost:11434"
    model := "tinydolphin"

    systemContent := `You are an expert with the star trek universe?`

    options := llm.SetOptions(map[string]interface{}{
        option.Temperature: 0.5,
    })

    query := llm.Query{
        Model: model,
        Messages: []llm.Message{
            {Role: "system", Content: systemContent},
            {Role: "user", Content: `Who is James Kirk?`},
        },
        Options: options,
    }

    // answering the first question
    finalAnswer, err := completion.ChatStream(ollamaUrl, query,
        func(answer llm.Answer) error {
            fmt.Print(answer.Message.Content)
            return nil
        })

    if err != nil {
        log.Fatal("😡:", err)
    }

    query = llm.Query{
        Model: model,
        Messages: []llm.Message{
            {Role: "system", Content: systemContent},
            {Role: "system", Content: finalAnswer.Message.Content}, // add the previous answer
            {Role: "user", Content: `Who is his best friend?`},
        },
        Options: options,
    }

    fmt.Println() // add a new line

    // answering the second question
    _, err = completion.ChatStream(ollamaUrl, query,
        func(answer llm.Answer) error {
            fmt.Print(answer.Message.Content)
            return nil
        })

    if err != nil {
        log.Fatal("😡:", err)
    }
}

You should get two answers like these:

James T. Kirk, a major character in the Star Trek universe, 
was a renowned captain of the USS Enterprise and one of the most 
respected officers on his crew. 
He had a deep sense of responsibility for the lives he led, 
always putting the needs of his team first.

The best friend of James T. Kirk was Deck Officer Spock, 
who also held the rank of captain in Starfleet. 
Their relationship was a testament to their bond and 
mutual respect for each other's skills and abilities.

Chat with context

Sometimes, the LLM does not know the answer because he does not have the information. So, you can use the same principle used in the previous example to give it more context.

Everybody knows that the best pizza in the world is the pineapple pizza, but TinyLlama does not know that. So you need to help him by giving him examples:

package main

import (
    "fmt"
    "log"

    "github.com/parakeet-nest/parakeet/completion"
    "github.com/parakeet-nest/parakeet/llm"
    "github.com/parakeet-nest/parakeet/enums/option"
)

func main() {
    ollamaUrl := "http://localhost:11434"
    model := "tinydolphin"

    systemContent := `You are an expert on the pizza topic? 
    Based on the given examples and your knowledge, 
    give the more accurate answer.`

    contextContent := `
    QUESTION: What is the best pizza of the world?
    ANSWER: The pineapple pizza is the best pizza of the world for the following reasons:
    - Sweet and Savory Fusion.
    - Textural Delight.
    - Balanced Flavors.
    - Tropical Twist.
    `

    options := llm.SetOptions(map[string]interface{}{
        option.Temperature: 0.5,
    })

    query := llm.Query{
        Model: model,
        Messages: []llm.Message{
            {Role: "system", Content: systemContent},
            {Role: "system", Content: contextContent},
            {Role: "user", Content: `What is the best pizza of the world?`},
        },
        Options: options,
    }

    _, err := completion.ChatStream(ollamaUrl, query,
        func(answer llm.Answer) error {
            fmt.Print(answer.Message.Content)
            return nil
        })

    if err != nil {
        log.Fatal("😡:", err)
    }
}

You should get an answer like this one:

The best pizza of the world is undoubtedly the Pineapple Pizza, 
a delightful fusion of sweet and savory flavors 
that perfectly balances out the tanginess of the tropical pineapple 
with the richness of the baked goods. 
It's not only delectable to eat but also a testament to the artistry of 
the pizzeria owner.

🤣

That's it for this little introduction to Parakeet. You will find the source code of the examples of this blog post here: https://github.com/parakeet-nest/blog-post-samples/tree/main/2024-05-22

You will find more examples in the Parakeet project: https://github.com/parakeet-nest/parakeet

What next?

Next time, we will speak about how to do some RAG, simply with Parakeet. Stay tuned.

Parakeet, an easy way to create GenAI applications with Ollama and Golang

Table of contents