...
| Code Block | ||
|---|---|---|
| ||
models: {
"My CustomOpen Source Model": {
endpoint: "httpshttp:///myserver127.0.0.1:11434/v1",
apiFormatmodel: "completionllama2",
modelapiKey: "Llama-2-70b-chat-hf-function-calling-v2",
apiKey: "......"secret" // requred, even if unused
}
} |
The above configuration is based on the assumption that the model's API follows the format commonly used by the OpenAI Chat Completion API. This applies, for example, if you're hosting an open-source large language model locally with a tool like Ollama.
...