Answer Question from Data/Document Trigger actions

The Trigger actions "Answer question from document" and "Answer question from data" analyze text-based information in a PDF, image, list of data, or text to return an answer to a user provided question. Additionally if they return the parts of the input that were used to answer the question as references, this allows users to review the source used to generate the response if they want to.

Example use cases:

• Find quick solutions to machinery problems without having to manually dig through long instruction or maintenance manuals
• Extract the key information from an issue description by asking for a summary.

Inputs and outputs

The trigger actions have two inputs, Document/Data and Question, and two outputs, the Answer and the References.

Input: Input

This is the data which will be used as an information source for

Supported data types:

*Password protected PDFs are currently not supported.

Currently supported languages (for PDF’s and images):

• English
• French
• German
• Italian
• Portuguese
• Spanish

Input: Question

This is the question which will be answered based on the data.

Supported data type:

• Text

Currently supported languages:

• English
• French
• German
• Italian
• Portuguese
• Spanish

Output: Answer

This is the answer to the question that was identified by the trigger action based on the input.

Output: References

This is an array of the parts of the input that were used by the trigger action to identify the answer. This output depends on the selected data type for Input.

Input is File (PDF) or an Image

List of objects with properties “Page” and “Text”

• For PDFs, the items in the list will be the page number and the extracted text for each page that were used to answer the question
• For Images, the list will have only one entry with the page number 1 and the text detected on image

Input is List of any type

List of the same type as the Input

• The list will contain all records that were used to answer the question.

How it works and what to consider

At the heart of our "Answer question" trigger actions, we're using a smart way to find the most relevant answers from your documents. Here's how it works use the example of inputs which can have multiple references like a PDF (= multiple pages) or a list of records:

1. We take each page of your document / record you provide as input and create a unique 'fingerprint' for it. This helps us understand what's on each page.
2. When you ask a question, we identify which pages / records are most related to your question based on that fingerprint.
3. We then take your question and the relevant pages or record and provide that as input to the AI system to answer the question based on the pages.
4. The trigger action then returns the answer, and also the pages or records that were used for answering the question, so that you can review both.

One important thing to keep in mind is that while the system is very powerful, it has a limit to how much it can 'read' at once. This means that we can't always use all the information (e.g. pages from a PDF) to answer a question. We give it the most relevant pages up to the maximum it can consume.
As a result, some types of questions are better suited than others for the system. For instance, asking it to count things across a whole document might not give the correct results, if not all pages or records can be included.
Therefore, while our system is designed to be as accurate as possible, we do recommend having a human in the loop, especially for critical topics. This isn't because our system isn't reliable, but because we believe in the value of human judgment in complementing AI. This way, you can ensure the highest level of accuracy and context understanding.

Edge Cases

No input and/or no query provided

If no input or no query is provided to the trigger action, the App will show the following system error:
Your Input or Query is empty

This happens for all of the following cases:

• The input and/or query do not have a value assigned. This is equivalent to “null”.
• The query has an empty string assigned.

Question cannot be answered from provided input

The answer will be:
There is no information provided in the given context [...]

Limits

Currently the following limits exist for Copilot triggers. These limits are tracked on an Instance level. In a case where these limits have been exceeded, the trigger action will fail.

Answer Question from Document:
Monthly Limit: 10,000 Requests/Month
Rate Limit: 10 Requests/Minute

Answer Question from Data:
Monthly Limit: 100,000 Requests/Month
Rate Limit: 10 Requests/Minute