Chat Completion

Chat completion API takes a list of message prompts as input and generates a conversation response. In TaskingAI-Inference, more complicated chat completion features are also supported including message streaming and function calls.

Chat completion endpoint: /v1/chat_completion

Example

curl --location 'http://127.0.0.1:8000/v1/chat_completion' \
--header 'Content-Type: application/json' \
--data '{
    "model_schema_id": "openai/gpt-4",
    "messages": [
        {
            "role": "user",
            "content": "Hello, nice to meet you, what is your name"
        }
    ],
    "stream": false,
    "credentials": {
        "OPENAI_API_KEY": "YOUR_OPENAI_API_KEY"
    },
    "configs": {
        "temperature": 0.8
    }
}'