Basic Usages

This section will cover the most basic usage of using TaskingAI-Inference to perform chat completion tasks. For more advanced usage, please refer to Streaming, Function Calls, and Wildcard.

info

For a detailed API reference, please refer to API Reference.

Specify the model of your choice

TaskingAI-Inference has integrated a wide range of models from different providers. To specify the model you want to use, you need to provide the model_schema_id in the body of the request. If you are going to use a wildcard model, please refer to Wildcard.

Here's an example to specify using the 'GPT-4' model from OpenAI:

{
    ...
    "model_schema_id": "openai/gpt-4",
    ... other parameters in body
}

To find a full list of supported models, and get the model_schema_id of your desired model, please check the Providers and Models page.

Set up credentials

To access the models provided by the LLM providers through TaskingAI, your API Key toward your chosen provider is required. All credentials should be passed under the credentials field in the body of the request.

Here's an example of passing credentials for accessing OpenAI models:

{
    "credentials": {
        "OPENAI_API_KEY": "{{OPENAI_API_KEY}}"
    },
    ... other parameters in body
}

For a full list of credentials required for each provider, please refer to Credentials. Note: you don't need to pass all the credentials on the table, only the one required by your chosen provider is enough.

Messages

Messages are the input for the chat completion task. It is a list of messages that you want to send to the model. The most simple example of Messages consists of only one user message:

{
    "messages": [
        {
            "role": "user",
            "content": "Hello, what is your name"
        }
    ],
    ... other parameters in body
}

In this example, Messages only contains one user message. But that is not enough in many other scenario.

Every item in Messages is a message object with two attributes: role and content, where content is the actual text of the message. Role, however, is a special attribute that tells the model who is sending the message. It can be one of the following values: user (messages sent by user), assistant (messages generated by the assistant), system (system prompt set by developer), function (triggered by assistant, implicating it intends to call a function).

A more complicated Messages can be like this:

{
    "messages": [
            {
                "role": "system",
                "content": "You are a assistant to answer questions about fun facts of numbers."
            },
            {
                "role": "user",
                "content": "What are the fun facts about 88?"
            },
            {
                "role": "assistant",
                "content": "88 is the length of a standard playing card in millimeters."
            },
            {
                "role": "user",
                "content": "What are the fun facts about 33?"
            }
    ],
    ... other parameters in body
}

Response

Here's an example of response:

{
    "status": "success",
    "data": {
        "object": "ChatCompletion",
        "finish_reason": "stop",
        "message": {
            "content": "Hello there! I'm an artificial intelligence language model. ",
            "role": "assistant",
            "function_calls": null
        },
        "created_timestamp": 1706690034620
    }
}

The ChatCompletion object of the response has the following fields:

• object: Indicating the object type ChatCompletion.
• finish_reason: The reason why the task is finished. Can be one of the following values: stop for normal cases, length for exceeding model's output token limit, function_calls for model deciding to call a function, error for errors, unknown for unknown reasons.
• message: The Message Object
  • role: The entity that generates this message. Expecting Assistant as this is the response from assistant.
  • content: The actual response message from assistant
  • function_calls: Usually null, but when model decides to call functions, this will contain a list of ChatCompletionFunctionCall object. See advanced usages for more details.
• created_timestamp: A UNIX int (13-digits) representing the creation time of the response in milliseconds.