If you’re hosting a model on your own server, use the endpoint property to specify the location of the model.
For example:
models: {
"My Custom Model": {
endpoint: "https://myserver",
apiFormat: "completion",
model: "Llama-2-70b-chat-hf-function-calling-v2",
apiKey: "......"
}
}