...
| Code Block | ||
|---|---|---|
| ||
models: {
"My Custom Open Source Model": {
endpoint: "httpshttp:///myserver127.0.0.1:11434/v1",
apiFormatmodel: "completionllama2",
modelapiKey: "Llama-2-70b-chat-hf-function-calling-v2secret", // requred, even if unused
apiKey stream: "......"true
}
} |
The above configuration is based on the assumption that the model's API follows the format commonly used by the OpenAI Chat Completion API. This applies, for example, if you're hosting an open-source large language model locally with a tool like Ollama.
...