Overview
TaskingAI API integrates with a variety of Large Language Model (LLM) providers, offering developers access to a diverse suite of state-of-the-art AI models. These models, which are the backbone of your projects, enable you to build powerful AI applications. This document offers a comprehensive overview of the LLM providers and models currently supported by TaskingAI, and it also details the credentials needed to access their services.
Provider and Model List
In the following table, the first column lists provider names and provider_id, while the second column lists model_id. If you are looking for model_schema_id for a certain model, simply join the provider_id and model_id with a "/".
For example, to use the gpt-4 model from openai, you would use openai/gpt-4 as the model_schema_id. Similarly, if you want to use ai21/j2-mid-v1 from AWS Bedrock, you would use aws_bedrock/ai21/j2-mid-v1 as the model_schema_id.
Provider and Model List
In the following table, the first column lists provider names and provider_id, while the second column lists model_id. If you are looking for model_schema_id for a certain model, simply join the provider_id and model_id with a "/".
For example, to use the gpt-4 model from openai, you would use openai/gpt-4 as the model_schema_id. Similarly, if you want to use ai21/j2-mid-v1 from AWS Bedrock, you would use aws_bedrock/ai21/j2-mid-v1 as the model_schema_id.
| Provider (provider_id) | Chat Completion Models | Embedding Models |
|---|---|---|
| AI21 (ai21) | j2-ultra | ai21_embedding |
| Anthropic (anthropic) | claude-2.0, claude-2.1, claude-3-haiku, claude-3-opus, claude-3-sonnet, claude-instant-1.2, wildcard | wildcard |
| AWS Bedrock (aws_bedrock) | ai21/j2-mid-v1, ai21/j2-ultra-v1, amazon/titan-text-express-v1, amazon/titan-text-lite-v1, anthropic/claude-instant-v1, anthropic/claude-v2, anthropic/claude-v2.1, cohere/command-light-text-v14, cohere/command-text-v14, meta/llama2-13b-chat-v1, meta/llama2-70b-chat-v1 | cohere/embed-english-v3, cohere/embed-multilingual-v3 |
| Azure OpenAI (azure_openai) | gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-4, gpt-4-32k, gpt-4-turbo, gpt-4-vision | text-embedding-ada-002 |
| BaiChuan (baichuan) | baichuan2-turbo, baichuan2-turbo-192k | |
| Cohere AI (cohere) | command, command-light, command-light-nightly, command-nightly, command-r, command-r-plus | embed-english-light-v2.0, embed-english-light-v3.0, embed-english-v3.0, embed-multilingual-light-v3.0, embed-multilingual-v2.0, embed-multilingual-v3.0 |
| Custom Host (custom_host) | openai-function-call, openai-tool-calls | openai-text-embedding |
| Google Gemini (google_gemini) | gemini-1.5-pro, gemini-pro, gemini-pro-vision, wildcard | wildcard |
| Groq (groq) | gemma-7b, llama3-70b, llama3-8b, mixtral-8x7b, wildcard | wildcard |
| Hugging Face Inference API (Serverless) (hugging_face) | wildcard | wildcard |
| Hugging Face Inference Endpoint (Dedicated) (hugging_face_inference_endpoint) | wildcard | wildcard |
| Jina AI (jina) | jina-embeddings-v2-base-code, jina-embeddings-v2-base-de, jina-embeddings-v2-base-en, jina-embeddings-v2-base-zh, jina-embeddings-v2-small-en | |
| Llama API (llama_api) | wildcard | wildcard |
| LM Studio (lm_studio) | wildcard | wildcard |
| LocalAI (localai) | wildcard | wildcard |
| Minimax (minimax) | abab5.5-chat, abab5.5s-chat, abab6-chat, abab6.5-chat, abab6.5s-chat | |
| MistralAI (mistralai) | mistral-large-latest, mistral-medium-latest, mistral-small-latest, open-mistral-7b, open-mixtral-8x22b, open-mixtral-8x7b, wildcard | mistral-embed, wildcard |
| Moonshot (moonshot) | moonshot-v1-128k, moonshot-v1-32k, moonshot-v1-8k | |
| Ollama (ollama) | wildcard | wildcard |
| OpenAI (openai) | gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-4, gpt-4-turbo, gpt-4-vision, wildcard | text-embedding-3-large-1024, text-embedding-3-large-256, text-embedding-3-small-1536, text-embedding-3-small-512, text-embedding-ada-002, wildcard |
| Replicate (replicate) | wildcard | wildcard |
| Together AI (togetherai) | meta-llama/Llama-2-70b-chat-hf, mistralai/Mistral-7B-Instruct-v0.1, mistralai/Mixtral-8x7B-Instruct-v0.1, togethercomputer/CodeLlama-34b-Instruct, wildcard | wildcard |
| Tongyi (tongyi) | qwen-max, qwen-max-1201, qwen-max-longcontext, qwen-plus, qwen-turbo | |
| ERNIE (wenxin) | ernie-bot, ernie-bot-4, ernie-bot-8k, ernie-bot-turbo | |
| Yi (yi) | yi-34b-chat-0205, yi-34b-chat-200k | |
| Zhipu AI (zhipu) | glm-3-turbo, glm-4, glm-4v | embedding-2 |
Credentials required to access models
To access the models provided by the LLM providers through TaskingAI, specific credentials are needed. The following table outlines the required credentials for each provider:
Credentials required to access models
| Provider | Credentials | Description |
|---|---|---|
| AI21 | AI21_API_KEY (Required) | API key for AI21. |
| Anthropic | ANTHROPIC_API_KEY (Required) | Your Anthropic API Key for authentication. |
| ANTHROPIC_API_VERSION (Optional) | The version of the Anthropic API being used. | |
| AWS Bedrock | AWS_ACCESS_KEY_ID (Required) | Your AWS Bedrock Access Key ID. It can be found on your security credentials page. |
| AWS_SECRET_ACCESS_KEY (Required) | Your AWS Bedrock Secret Access Key. It can be found on your security credentials page. | |
| AWS_REGION (Required) | The AWS region where your AWS Bedrock model is hosted. | |
| Azure OpenAI | AZURE_OPENAI_RESOURCE_NAME (Required) | The name of your Azure OpenAI Resource. |
| AZURE_OPENAI_DEPLOYMENT_ID (Required) | The deployment name you chose when you deployed the model. | |
| AZURE_OPENAI_API_KEY (Required) | Your Azure OpenAI API Key for authentication. | |
| AZURE_OPENAI_API_VERSION (Optional) | (Optional) The Azure API version follows the YYYY-MM-DD format. | |
| BaiChuan | BAICHUAN_API_KEY (Required) | Your BaiChuan API Key for authentication. |
| Cohere AI | COHERE_API_KEY (Required) | Your Cohere API key for authentication. |
| Custom Host | CUSTOM_HOST_ENDPOINT_URL (Required) | The url to your model endpoint. |
| CUSTOM_HOST_MODEL_ID (Required) | The model id to use for your endpoint. | |
| CUSTOM_HOST_API_KEY (Required) | Your API Key for authentication. | |
| Google Gemini | GOOGLE_GEMINI_API_KEY (Required) | Your Google Gemini API Key for authentication. |
| GOOGLE_GEMINI_API_VERSION (Optional) | The version of the Google Gemini API being used. | |
| Groq | GROQ_API_KEY (Required) | Your Groq API Key for authentication. |
| Hugging Face Inference API (Serverless) | HUGGING_FACE_API_KEY (Required) | Your Hugging Face API Token for authentication. |
| Hugging Face Inference Endpoint (Dedicated) | HUGGING_FACE_API_KEY (Required) | Your Hugging Face API Token for authentication. |
| HUGGING_INFERENCE_ENDPOINT_URL (Required) | Your Hugging Face Inference Endpoint URL. | |
| Jina AI | JINA_API_KEY (Required) | Your Jina API key for authentication. |
| Llama API | LLAMA_API_API_KEY (Required) | Your Llama API Token for authentication |
| LM Studio | LM_STUDIO_HOST (Required) | Your LM Studio host URL |
| LocalAI | LOCALAI_HOST (Required) | Your LocalAI host URL |
| Minimax | MINIMAX_API_KEY (Required) | Your Minimax API Key for authentication. |
| MistralAI | MISTRAL_API_KEY (Required) | Your Mistral API Key for authentication. |
| Moonshot | MOONSHOT_API_KEY (Required) | Your Moonshot API Key for authentication. |
| Ollama | OLLAMA_HOST (Required) | Your Ollama host URL |
| OpenAI | OPENAI_API_KEY (Required) | Your OpenAI API Key for authentication. |
| OPENAI_ORG_ID (Optional) | (Optional) Your OpenAI Organization ID, which can be found on your organization settings page. | |
| Replicate | REPLICATE_API_KEY (Required) | Your Replicate API Token for authentication. |
| Together AI | TOGETHERAI_API_KEY (Required) | Your TogetherAI API Key for authentication. |
| Tongyi | TONGYI_API_KEY (Required) | Your Tongyi API Key for authentication. |
| ERNIE | WENXIN_API_KEY (Required) | Your ERNIE Bot API Key for authentication. |
| WENXIN_SECRET_KEY (Required) | Your ERNIE Bot Secret Key for authentication. | |
| Yi | YI_API_KEY (Required) | Your Yi API Key for authentication. |
| Zhipu AI | ZHIPU_API_KEY (Required) | Your Zhipu API Key for authentication. |
All customer data is encrypted at rest with AES-256 and in transit with TLS 1.2.
Ensure that you have the correct credentials configured in your TaskingAI client to successfully connect and interact with the desired LLM provider. Proper credential management is essential for maintaining security and access control when using these AI models.
For detailed instructions on how to obtain and set up these credentials, please refer to the documentation provided by the respective LLM providers from the table above.