Memory

In an assistant, memory refers to its ability to retain and use the context of an ongoing conversation. Memory enhances how the assistant interprets and responds to a sequence of messages, helping maintain coherence and relevance over multiple turns.

Context Memory Size

Context memory size is the maximum number of tokens (units of text) that an AI agent can retain when processing input and generating responses in a conversation. A larger context memory size improves the assistant’s performance in multi-turn conversations but also increases model inference costs.

note

A zero context memory size means the AI agent does not retain any prior conversation context and only responds to the most recent user message.