Available models are configured in your config.js file. Add or modify the “models” property to add new large language models. For example:
models: {
"GPT-4 Turbo": {
provider: "openai",
model: "gpt-4-1106-preview",
apiKey: "sk-....."
},
"GPT-3.5 Turbo": {
provider: "openai",
model: "gpt-3.5-turbo-1106",
apiKey: "sk-....."
},
"Gemini Pro": {
provider: "google",
model: "gemini-pro",
apiKey: "AI......"
},
"Azure OpenAI GPT-3.5": {
provider: "openai",
cloud: "azure",
model: "azure-gpt-35-0613",
resource: "profound-openai",
apiKey: "......",
apiVersion: "2023-12-01-preview"
},
"Mistral Medium with Streaming": {
provider: "mistral",
model: "mistral-medium",
stream: true,
apiKey: "......"
},
"Mistral 7B via Anyscale": {
endpoint: "https://api.endpoints.anyscale.com/v1",
apiFormat: "completion",
model: "mistralai/Mistral-7B-Instruct-v0.1",
apiKey: "esecret_......"
},
"Llama 2 70B": {
endpoint: "https://myserver",
apiFormat: "completion",
model: "Llama-2-70b-chat-hf-function-calling-v2",
apiKey: "esecret_......"
},
"Claude 2 Streaming": {
provider: "anthropic",
model: "claude-2.1",
apiKey: "sk-ant-......",
stream: true
},
"AWS Bedrock - Claude 2": {
provider: "anthropic",
cloud: "aws",
model: "claude-2.1",
accessKeyId: "......",
secretAccessKey: "......"
}
}
Model Properties
provider
When this property is specified (e.g. “openai”, “google”, “mistral”, “anthropic”), the endpoint and apiFormat properties assume default values for each respective provider.
cloud
The model may be hosted with a cloud provider, such as “azure” or “aws”, which may require additional authentication. If this is the case for the model you’re trying to work with, use this property to specify the cloud provider.
endpoint
Use this property to specify a custom endpoint or API URL. This may be necessary for custom, open-source, and/or inhouse models.
resource
For Azure OpenAI, you can specify the the resource name in place of the full endpoint.
awsAccessKey
When the “aws” cloud is used, this property specifies the AWS access key ID.
awsSecretKey
When the “aws” cloud is used, this property specifies the AWS secret access key.
awsRegion
When the “aws” cloud is used, this property specifies the AWS region.
model
The provider or endpoint may expose multiple models. This property identifies the specific model to use.
apiFormat
Use this property to override the default format of how the data is exchanged with the model. Providers may support different formats. For example, OpenAI offers “completion” and “assistant” API.
useMessages
The default value for this property is true. The property indicates whether the model provider’s API supports a list of conversation messages. Set this property to false if the model API cannot process conversation messages as a list and the information will be sent to the API as one text prompt containing all the messages.
apiKey
This property contains the API key or secret key associated with the respective model. These keys are used for authentication when making requests to the API.
apiVersion
The API you’re using may require a version. Use this property to set the appropriate API version.
defaultAssistantId
This property specifies the default assistant ID for use with the “assistant” API format. When using the “openai” provider, the “assistant” API format is the default. This format requires configuring an assistant object on the OpenAI platform.
If the property is missing while employing the “assistant” API format from OpenAI, Profound AI will automatically set up an assistant object. Additionally, a message containing the defaultAssistantId will be logged to the server. You should then update your config.js file with this defaultAssistantId. This update ensures no extra processing is needed to locate or create the assistant object in subsequent operations.
stream
Set this Boolean property to true to indicate that you want the model API to stream its output as it is generated rather than sending the output all at once. Beware that some model API may not have this capability.