Skip to main content

Manage Records

In the architecture of the TaskingAI retrieval system, records play a pivotal role in populating collections, which serve as repositories for retrievable data. As the primary data source for these collections, records undergo a process of text extraction. This process involves pulling textual content from records, regardless of their original format, and preparing it for storage within collections. Text extraction is a critical step that ensures data is stored in a consistent and retrievable format. The system segments the extracted text into manageable, searchable units, facilitating efficient data retrieval and management within the TaskingAI framework.

There are three primary types of records: Pure Text, Text Files, and Web URLs:

  • Pure Text refers to straightforward, unformatted text data, typically used for direct text input or simple text-based data sources.
  • Text Files involve text extracted from various file formats such as PDF, TXT, and Word documents. This capability to process text from diverse file types makes the system robust and adaptable to various user requirements and data formats.
  • Web URLs allow users to extract text content from web pages and store it as a record in the system.

Manage records

Create a Record​

You can add a record to your collection by any of the following methods:

  1. Provide the text content directly.
  2. Provide the URL of a webpage, and TaskingAI will scrape the content.
  3. Upload a file, and TaskingAI will extract the text content. Supported file formats: .txt, .pdf, .docx, .md, and .html.

Creating in Console​

After setting up a collection, click the menu button on the right side of the collection, and select Record to enter the record management page. In the record management page, click the New Record button to select a record type and create a new record.

Create a record

Checking Record Status​

In some cases, the record status will remain as creating for a short time after initiating creation. Generally, after waiting for a few seconds, the record status will change to ready. When the record status changes to ready, it indicates that the text has been effectively split into smaller fragments, and embeddings of these chunks have been constructed. Only when in the ready status can the record chunks be retrieved in response to user queries for related information.

Update a Record​

To update an existing record with new text content, click the corresponding row in the table to open the record management modal. Click the Update button on the right side of the top bar, where you can make changes to the record metadata and content. Records created by uploading a file cannot be updated.

note

After a content update, all old text chunks associated with the record will be deleted and replaced with the new ones. If you want to retain the old text chunks, you can edit chunks directly instead of updating the record.