/
Hugging Face Models

Hugging Face Models

  • Verified

    • See Model Configuration for a full list of available model properties.

    Overview

    Use the procedure outlined below to configure and use Hugging Face models within Profound AI.

    Steps

    1. Log into your account at huggingface.co.

      1. If you don’t have an account, create one.

    2. Find the model you’re looking to use.

      1. For best results with Profound AI, the model should support function calling.

      2. A foundation model that doesn’t support function calling can be fine tuned for function calling.

    3. Select Deploy and follow the process to create an Inference endpoint.

      1. Copy the endpoint URL to your model configuration.

    4. Go to your Hugging Face profile settings and select Access Tokens.

    5. Create a new access token and copy it to the apiKey property of your model configuration.

    Example

    The example below shows a sample configuration for the Trelis/CodeLlama-70b-Instruct-hf-function-calling-v3 Hugging Face model:

    "HuggingFace model": { provider: "huggingface", endpoint: "......", model: "Trelis/CodeLlama-70b-Instruct-hf-function-calling-v3", apiKey: "......" }