Overview
TaskingAI API integrates with a variety of Large Language Model (LLM) providers, offering developers access to a diverse suite of state-of-the-art AI models. These models, which are the backbone of your projects, enable you to build powerful AI applications. This document offers a comprehensive overview of the LLM providers and models currently supported by TaskingAI, and it also details the credentials needed to access their services.
Types of models
TaskingAI currently supports three types of models:
- Chat Completion Models: These models are designed to generate human-like responses to user queries, making them ideal for chatbots and conversational AI applications.
- Embedding Models: These models are used to convert text into numerical vectors, which can then be used for various tasks such as semantic search, clustering, and classification.
- Rerank Models: These models are used to rerank the chunks retrieved from the retrieval system.
Provider and Model List
In the following table, the first column lists provider names and provider_id, while the second column lists model_id. If you are looking for model_schema_id for a certain model, simply join the provider_id and model_id with a "/".
For example, to use the gpt-4 model from openai, you would use openai/gpt-4 as the model_schema_id. Similarly, if you want to use ai21/j2-mid-v1 from AWS Bedrock, you would use aws_bedrock/ai21/j2-mid-v1 as the model_schema_id.
| Provider (provider_id) | Chat Completion Models | Embedding Models | Rerank Models |
|---|---|---|---|
| AI21 (ai21) | j2-ultra | ai21_embedding | |
| Anthropic (anthropic) | claude-2.0, claude-2.1, claude-3-haiku, claude-3-opus, claude-3-sonnet, claude-3.5-sonnet, claude-instant-1.2, wildcard | wildcard | |
| AWS Bedrock (aws_bedrock) | ai21/j2-mid-v1, ai21/j2-ultra-v1, amazon/titan-text-express-v1, amazon/titan-text-lite-v1, anthropic/claude-instant-v1, anthropic/claude-v2, anthropic/claude-v2.1, cohere/command-light-text-v14, cohere/command-text-v14, meta/llama2-13b-chat-v1, meta/llama2-70b-chat-v1 | cohere/embed-english-v3, cohere/embed-multilingual-v3 | |
| Azure OpenAI (azure_openai) | gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-4, gpt-4-32k, gpt-4-turbo, gpt-4-vision | text-embedding-ada-002 | |
| BaiChuan (baichuan) | baichuan2-turbo, baichuan2-turbo-192k, baichuan3-turbo, baichuan3-turbo-128k, baichuan4 | baichuan-text-embedding | |
| Cohere AI (cohere) | command, command-light, command-light-nightly, command-nightly, command-r, command-r-plus | embed-english-light-v2.0, embed-english-light-v3.0, embed-english-v3.0, embed-multilingual-light-v3.0, embed-multilingual-v2.0, embed-multilingual-v3.0 | rerank-english-v2.0, rerank-english-v3.0, rerank-multilingual-v2.0, rerank-multilingual-v3.0 |
| Custom Host (custom_host) | openai-function-call, openai-tool-calls | openai-text-embedding | |
| DeepSeek (deepseek) | deepseek-chat, deepseek-coder | ||
| Google Gemini (google_gemini) | gemini-1.0-pro, gemini-1.0-pro-vision, gemini-1.5-flash, gemini-1.5-pro, wildcard | text-embedding-004, wildcard | |
| Groq (groq) | gemma-7b, llama3-70b, llama3-8b, mixtral-8x7b, wildcard | wildcard | |
| Hugging Face Inference API (Serverless) (hugging_face) | wildcard | wildcard | |
| Hugging Face Inference Endpoint (Dedicated) (hugging_face_inference_endpoint) | wildcard | wildcard | |
| Jina AI (jina) | jina-embeddings-v2-base-code, jina-embeddings-v2-base-de, jina-embeddings-v2-base-en, jina-embeddings-v2-base-es, jina-embeddings-v2-base-zh, jina-embeddings-v2-small-en | jina-colbert-v1-en, jina-reranker-v1-base-en, jina-reranker-v1-tiny-en, jina-reranker-v1-turbo-en | |
| Lepton AI (leptonai) | gemma_7b, jetmoe_8b_chat, llama2_13b, llama3_70b, llama3_8b, mistral_7b, mixtral_8x7b, nous_hermes_llama2_13b, openchat_3_5 | ||
| Llama API (llama_api) | wildcard | wildcard | |
| LM Studio (lm_studio) | wildcard | wildcard | |
| LocalAI (localai) | wildcard | wildcard | |
| Minimax (minimax) | abab5.5-chat, abab5.5s-chat, abab6-chat, abab6.5-chat, abab6.5g-chat, abab6.5s-chat | ||
| MistralAI (mistralai) | mistral-large-latest, mistral-medium-latest, mistral-small-latest, open-mistral-7b, open-mixtral-8x22b, open-mixtral-8x7b, wildcard | mistral-embed, wildcard | |
| Moonshot (moonshot) | moonshot-v1-128k, moonshot-v1-32k, moonshot-v1-8k | ||
| Ollama (ollama) | wildcard | wildcard | |
| OpenAI (openai) | gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-4, gpt-4-turbo, gpt-4-vision, gpt-4o, wildcard | text-embedding-3-large-1024, text-embedding-3-large-256, text-embedding-3-small-1536, text-embedding-3-small-512, text-embedding-ada-002, wildcard | |
| OpenRouter (openrouter) | wildcard | wildcard | |
| Reka (reka) | reka-core, reka-edge, reka-flash | ||
| Replicate (replicate) | wildcard | wildcard | |
| SenseTime (sensetime) | sensechat, sensechat-128k, sensechat-32k, sensechat-5, sensechat-functioncall, sensechat-turbo | nova-embedding-stable | |
| Together AI (togetherai) | meta-llama/Llama-2-70b-chat-hf, mistralai/Mistral-7B-Instruct-v0.1, mistralai/Mixtral-8x7B-Instruct-v0.1, togethercomputer/CodeLlama-34b-Instruct, wildcard | wildcard | |
| Tongyi (tongyi) | qwen-long, qwen-max, qwen-max-longcontext, qwen-plus, qwen-turbo | text-embedding-v1, text-embedding-v2 | |
| Volcengine (volcengine) | doubao-lite-128k, doubao-lite-32k, doubao-lite-4k, doubao-pro-128k, doubao-pro-32k, doubao-pro-4k | ||
| ERNIE (wenxin) | ernie-bot, ernie-bot-4, ernie-bot-8k, ernie-bot-turbo | ||
| Yi (yi) | yi-large, yi-large-rag, yi-large-turbo, yi-medium, yi-medium-200k, yi-spark, yi-vision | ||
| Zhipu AI (zhipu) | glm-3-turbo, glm-4, glm-4v | embedding-2 |
Credentials required to access models
To access the models provided by the LLM providers through TaskingAI, specific credentials are needed. The following table outlines the required credentials for each provider:
| Provider | Credentials | Description |
|---|---|---|
| AI21 | AI21_API_KEY (Required) | API key for AI21. |
| Anthropic | ANTHROPIC_API_KEY (Required) | Your Anthropic API Key for authentication. |
| ANTHROPIC_API_VERSION (Optional) | The version of the Anthropic API being used. | |
| AWS Bedrock | AWS_ACCESS_KEY_ID (Required) | Your AWS Bedrock Access Key ID. It can be found on your security credentials page. |
| AWS_SECRET_ACCESS_KEY (Required) | Your AWS Bedrock Secret Access Key. It can be found on your security credentials page. | |
| AWS_REGION (Required) | The AWS region where your AWS Bedrock model is hosted. | |
| Azure OpenAI | AZURE_OPENAI_RESOURCE_NAME (Required) | The name of your Azure OpenAI Resource. |
| AZURE_OPENAI_DEPLOYMENT_ID (Required) | The deployment name you chose when you deployed the model. | |
| AZURE_OPENAI_API_KEY (Required) | Your Azure OpenAI API Key for authentication. | |
| AZURE_OPENAI_API_VERSION (Optional) | (Optional) The Azure API version follows the YYYY-MM-DD format. | |
| BaiChuan | BAICHUAN_API_KEY (Required) | Your BaiChuan API Key for authentication. |
| Cohere AI | COHERE_API_KEY (Required) | Your Cohere API key for authentication. |
| Custom Host | CUSTOM_HOST_ENDPOINT_URL (Required) | The url to your model endpoint. |
| CUSTOM_HOST_MODEL_ID (Required) | The model id to use for your endpoint. | |
| CUSTOM_HOST_API_KEY (Required) | Your API Key for authentication. | |
| DeepSeek | DEEPSEEK_API_KEY (Required) | Your DeepSeek API Key for authentication. |
| Google Gemini | GOOGLE_GEMINI_API_KEY (Required) | Your Google Gemini API Key for authentication. |
| GOOGLE_GEMINI_API_VERSION (Optional) | The version of the Google Gemini API being used. | |
| Groq | GROQ_API_KEY (Required) | Your Groq API Key for authentication. |
| Hugging Face Inference API (Serverless) | HUGGING_FACE_API_KEY (Required) | Your Hugging Face API Token for authentication. |
| Hugging Face Inference Endpoint (Dedicated) | HUGGING_FACE_API_KEY (Required) | Your Hugging Face API Token for authentication. |
| HUGGING_INFERENCE_ENDPOINT_URL (Required) | Your Hugging Face Inference Endpoint URL. | |
| Jina AI | JINA_API_KEY (Required) | Your Jina API key for authentication. |
| Lepton AI | LEPTONAI_API_KEY (Required) | Your LeptonAI API Key for authentication. |
| Llama API | LLAMA_API_API_KEY (Required) | Your Llama API Token for authentication. Get it from https://www.llama-api.com/account/api-token |
| LM Studio | LM_STUDIO_HOST (Required) | Your LM Studio host URL |
| LocalAI | LOCALAI_HOST (Required) | Your LocalAI host URL |
| Minimax | MINIMAX_API_KEY (Required) | Your Minimax API Key for authentication. |
| MistralAI | MISTRAL_API_KEY (Required) | Your Mistral API Key for authentication. |
| Moonshot | MOONSHOT_API_KEY (Required) | Your Moonshot API Key for authentication. |
| Ollama | OLLAMA_HOST (Required) | Your Ollama host URL |
| OpenAI | OPENAI_API_KEY (Required) | Your OpenAI API Key for authentication. |
| OPENAI_ORG_ID (Optional) | (Optional) Your OpenAI Organization ID, which can be found on your organization settings page. | |
| OpenRouter | OPENROUTER_API_KEY (Required) | Your OpenRouter API Token for authentication. |
| Reka | REKA_API_KEY (Required) | Your Reka API Key for authentication. |
| Replicate | REPLICATE_API_KEY (Required) | Your Replicate API Token for authentication. |
| SenseTime | SENSETIME_ACCESS_KEY_ID (Required) | Your SenseTime Access Key ID. It can be found on your security credentials page. |
| SENSETIME_SECRET_ACCESS_KEY (Required) | Your SenseTime Secret Access Key. It can be found on your security credentials page. | |
| Together AI | TOGETHERAI_API_KEY (Required) | Your TogetherAI API Key for authentication. |
| Tongyi | TONGYI_API_KEY (Required) | Your Tongyi API Key for authentication. |
| Volcengine | VOLCENGINE_API_KEY (Required) | Your Volcengine API Key for authentication. |
| VOLCENGINE_ENDPOINT_ID (Required) | Your Volcengine Endpoint ID, which can be found on your console page. | |
| ERNIE | WENXIN_API_KEY (Required) | Your ERNIE Bot API Key for authentication. |
| WENXIN_SECRET_KEY (Required) | Your ERNIE Bot Secret Key for authentication. | |
| Yi | YI_API_KEY (Required) | Your Yi API Key for authentication. |
| Zhipu AI | ZHIPU_API_KEY (Required) | Your Zhipu API Key for authentication. |
info
All customer data is encrypted at rest with AES-256 and in transit with TLS 1.2.
Ensure that you have the correct credentials configured in your TaskingAI client to successfully connect and interact with the desired LLM provider. Proper credential management is essential for maintaining security and access control when using these AI models.
For detailed instructions on how to obtain and set up these credentials, please refer to the documentation provided by the respective LLM providers from the table above.