Wildcard

Many model platforms (e.g. HuggingFace) or frameworks (e.g. LocalAI) natively supports hundreds of models with a unified API. For those providers, TaskingAI integrates with their unifed API and allows you to use any model from their platform. And this method is called Wildcard.

You may check the Providers and Models page to find supported wildcard models and get the model_schema_id.

How to use Wildcard

Specifying desired model

Wildcard itself is a model in TaskingAI. Wildcard models usually have a model_schema_id in the format of <provider_id>/wildcard.

To specify which specific model you want to use from that provider, you need to provide the model identifier to the provider_model_id field in the body of the request.

For example, to use the llama-7b-chat model from Llama-API using wildcard, you can use the following request body:

{
    "model_schema_id": "llama_api/wildcard",  
    "provider_model_id": "llama-7b-chat",
    ... other parameters in body
}

Set up credentials

To access the models provided by the LLM providers through TaskingAI, your API Key toward your chosen provider is required. All credentials should be passed under the credentials field in the body of the request.

Here's an example of passing credentials for accessing Llama-API models:

{
    "credentials": {
        "LLAMA_API_API_KEY": "{{LLAMA_API_API_KEY}}"
    },
    ... other parameters in body
}

Chat completion

The chat completion task is the same as the basic chat completion task. You can refer to Basic Usages for more information.

Here's an example of a chat completion task using the mistralai/mixtral-8x7b-instruct-v0.1 model from Replicate-API:

curl --location 'http://127.0.0.1:8000/v1/chat_completion' \
--header 'Content-Type: application/json' \
--data '{
    "model_schema_id": "replicate/wildcard",
    "provider_model_id": "mistralai/mixtral-8x7b-instruct-v0.1",
    "messages": [
        {
            "role": "user",
            "content": "Hello, nice to meet you, what is your name"
        }
    ],
    "stream": false,
    "credentials": {
        "REPLICATE_API_KEY": "YOUR_REPLICATE_API_KEY"
    },
    "configs": {
        "temperature": 0.8
    }
}'

Text embedding

The text embedding task is the same as the basic text embedding task. You can refer to Text Embedding for more information.

Here's an example of a text embedding task using the BAAI/bge-m3 model from Hugging Face, with our HuggingFace Inference Endpoint (Dedicated):

curl --location 'http://127.0.0.1:8000/v1/text_embedding' \
--header 'Content-Type: application/json' \
--data '{
    "model_schema_id": "hugging_face_inference_endpoint/wildcard",
    "provider_model_id": "BAAI/bge-m3",
    "input": ["Hello, how are you recently", "Good, thanks for asking."],
    "properties": {
        "max_batch_size": 100,
        "input_token_limit": 512,
        "embedding_size": 384
    },
    "credentials": {
        "HUGGING_FACE_API_KEY": "YOUR_HUGGING_FACE_API_KEY",
        "HUGGING_INFERENCE_ENDPOINT_URL":"YOUR_HUGGING_INFERENCE_ENDPOINT_URL"
    }
}'

Note : embedding_size in properties is required, and has to match with the embedding result.