Available models are configured in your config.js file. Add or modify the “models” property to add new large language models. For example:

models: {
  "GPT-4 Turbo": {
    provider: "openai",
    model: "gpt-4-1106-preview",
    apiKey: "sk-....."
  },
  "GPT-3.5 Turbo": {
    provider: "openai",
    model: "gpt-3.5-turbo-1106",
    apiKey: "sk-....."
  },
  "Gemini Pro": {
    provider: "google",
    model: "gemini-pro",
    apiKey: "AI......"
  },
  "Azure OpenAI GPT-3.5": {
    provider: "azure",
    model: "azure-gpt-35-0613",
    resource: "profound-openai",
    apiKey: "......",
    apiVersion: "2023-12-01-preview"
  },  
  "Mistral Medium with Streaming": {
    provider: "mistral",
    model: "mistral-medium",
    stream: true,
    apiKey: "......"
  },
  "Mistral 7B via Anyscale": {
    endpoint: "https://api.endpoints.anyscale.com/v1",
    apiFormat: "completion",
    model: "mistralai/Mistral-7B-Instruct-v0.1",
    apiKey: "esecret_......"
  },
  "Llama 2 70B": {
    endpoint: "https://myserver",
    apiFormat: "completion",
    model: "Llama-2-70b-chat-hf-function-calling-v2",
    apiKey: "esecret_......"
  },
  "Claude 2 Streaming": {
    provider: "anthropic",
    model: "claude-2.1",
    apiKey: "sk-ant-......",
    stream: true
  }  
}

Model Properties

provider

When this property is specified (e.g. “openai”, “google”, “mistral”, “anthropic”), the endpoint and apiFormat properties assume default values for each respective provider.

cloud

The model may be hosted with a cloud provider, such as “azure” or “aws”, which may require additional authentication. If this is the case for the model you’re trying to work with, use this property to specify the cloud provider.

endpoint

Use this property to specify a custom endpoint or API URL. This may be necessary for custom, open-source, and/or inhouse models.

model

The provider or endpoint may expose multiple models. This property identifies the specific model to use.

resource

For Azure OpenAI, you can specify the the resource name in place of the full endpoint.

apiFormat

Use this property to override the default format of how the data is exchanged with the model. Providers may support different formats. For example, OpenAI offers “completion” and “assistant” API.

apiKey

This property contains the API key or secret key associated with the respective model. These keys are used for authentication when making requests to the API.

apiVersion

The API you’re using may require a version. Use this property to set the appropriate API version.

defaultAssistantId

This property specifies the default assistant ID for use with the “assistant” API format. When using the “openai” provider, the “assistant” API format is the default. This format requires configuring an assistant object on the OpenAI platform.

If the property is missing while employing the “assistant” API format from OpenAI, Profound AI will automatically set up an assistant object. Additionally, a message containing the defaultAssistantId will be logged to the server. You should then update your config.js file with this defaultAssistantId. This update ensures no extra processing is needed to locate or create the assistant object in subsequent operations.

stream

Set this Boolean property to true to indicate that you want the model API to stream its output as it is generated rather than sending the output all at once. Beware that some model API may not have this capability.

OpenAI Models

To setup an OpenAI model, you will need an API key. Follow these steps to create the key:

Go to the OpenAI website.
Click Sign Up or Sign In in the top-right corner. You may need to provide payment details for API usage. See OpenAI pricing for details.
Go the API keys section.
Create a new secret key (you can provide an identifiable secret key name, such as “Profound AI Key”).
Copy the secret key (it generally starts with “sk-”), and paste it into your config.js file assigning it to the “apiKey” property.

Google Models

To setup a model from Google, visit https://ai.google.dev/ and select Get API key.

Copy the API key into into your config.js file assigning it to the “apiKey” property.

Prior to using models from Google, you must install the following prerequisites:

Node.js v18+
GoogleGenerativeAI package - use command below to install:

npm install @google/generative-ai

Azure Cognitive Services (Azure OpenAI)

To work with models through the Azure OpenAI service, follow these steps:

Go to https://portal.azure.com and sign in.
Search and select the Azure OpenAI service.
Click Create a resource and follow the steps to create the resource.
Go to the resource you created and select Keys and Endpoint.
1. Copy those items into your config.js file as endpoint and apiKey properties of your model.
2. Alternatively, instead of the endpoint property, you can also set the resource property based on the resource name you chose. With this property set, the endpoint will be constructed automatically.
3. Set the provider property to “azure”.
Go to Manage Deployments.
Select Create Deployment.
Select desired model and model version; then give it a name. Supply this name as the model property in your config.js file. Ensure the model and version you select supports function calling. It may take a few minutes before the model is fully deployed and you are ready to try it.

Anthropic Models

Use the Anthropic Console online to generate an API key and copy it into your config.js file assigning it to the “apiKey” property.

Prior to using models from Anthropic, use the command below to install the Anthropic SDK npm package:

npm install @anthropic-ai/sdk

Model Configuration