Skip to main content

Groq

Groq is an AI solutions company delivering ultra-low latency inference with the first ever LPU Inference Engine.

Requisites​

To use Groq models, you need to have the following credential information.

Required credentials:

  • GROQ_API_KEY: Your Groq API Key for authentication.

Supported Models​

All models provided by Groq has the following properties:

  • Function call: Not supported
  • Streaming: Supported

All models accept the following extra configs when generating texts:

  • temperature
  • top_p
  • max_tokens
  • stop

Llama2 70B Chat​

Model schema id: groq/llama2-70b-chat

The LLaMA2-70b-chat model, developed by Meta, features a 4,096 token context window, showcasing excellence in natural language understanding and generation tasks.

Properties:

  • Input Token Limit: 4096

Mixtral 8x7B Instruct v0.1​

Model schema id: groq/mixtral-8x7b-instruct-v0.1

Mixtral-8x7B-instruct-v0.1 is a large language model with a 32,768 token context window, produced by Mistral AI, excelling in natural language understanding and generation tasks.

Properties:

  • Input Token Limit: 32768