Hugging Face Models

These instructions walk you through setting up a hosted model on Hugging Face.

Log into your account at huggingface.co. If you don’t have an account, create one.

Find the model you’re looking to use. For best results with Profound AI, the model should support function calling. A foundation model that doesn’t support function calling can be fine tuned for function calling.

Select Deploy and follow the process to create an Inference endpoint. Copy the endpoint URL to your model configuration.

Go to your Hugging Face profile Settings, and select Access Tokens. Create a new access token and copy it to the apiKey property of your model configuration.

For example:

"HuggingFace model": {
  provider: "huggingface",
  endpoint: "......",
  model: "Trelis/CodeLlama-70b-Instruct-hf-function-calling-v3",
  apiKey: "......"
}