Ollama
Contents
Ollama runs large language models locally. Rye can talk to a running Ollama server to generate text, chat, and produce vector embeddings.
This package needs a build with -tags "b_contrib,b_ollama" and a running Ollama server (by default http://localhost:11434, overridable with the OLLAMA_HOST environment variable).
Create a client
; Uses OLLAMA_HOST or defaults to localhost:11434
client: ollama
; Or point at a specific server
client: ollama\url "http://localhost:11434"
Generate text
client: ollama
response: client .Generate "llama2" "The sky is"
print response
Chat
; Simple question
client: ollama
answer: client .Chat "llama2" "What is the capital of France?"
print answer
; With conversation history (system prompt + turns)
messages: [
dict { "role" "system" "content" "You are a helpful assistant." }
dict { "role" "user" "content" "Hello!" }
]
answer: client .Chat\messages "llama2" messages
; Stream the reply as it arrives
client .Chat\stream "llama2" "Tell me a story" { .print }
Embeddings
Embeddings turn text into vectors, useful for search and similarity.
client: ollama
; One text
v: client .Embed "bge-m3" "Hello, world!"
; Several texts at once, returned as a block of vectors
vs: client .Embed\many "bge-m3" { "Hello" "World" "Test" }
Manage models
client: ollama
; List installed models
client .List-models |for { -> "name" |print }
; Show details about a model
info: client .Show-model "llama2"
print info -> "parameters"
; Download a model
client .Pull-model "mistral"
Common chat models are llama2, mistral, codellama, and phi; common embedding models are bge-m3, nomic-embed-text, and all-minilm.