Answer Question from Data/Document Trigger actions
  • 16 May 2024
  • 5 Minutes to read
  • Contributors

Answer Question from Data/Document Trigger actions


Article summary

Open Beta

The Frontline Copilot trigger actions are currently in open beta. Reach out to copilot@tulip.co if you are interested in joining the beta.

Beta Disclaimer

Use of Beta Versions offered by Tulip is optional and is at your sole risk. Due to the untested/unproven nature of Beta versions, they are provided “as is,” “as available,” and without warranty. This means that any warranties, indemnities and other obligations of Tulip under any terms of service or MSA do not apply to Beta Versions.

  • Frontline Copilot is powerful! We will help to guide, but you are ultimately responsible for the impact of what you build.
  • We reserve the right to turn off any and all Copilot Triggers at any time if we notice something unexpected.
  • We reserve the right to limit Copilot Triggers during the course of the beta if and as needed.
  • You will not be charged for the usage of Copilot Triggers during the program. You will have three months after the conclusion of the program to decide to continue with the feature or not.
  • At some point in the future, we may implement usage based pricing for beta features.

The Trigger actions "Answer question from document" and "Answer question from data" analyze text-based information in a PDF, image, list of data, or text to return an answer to a user provided question. Additionally if they return the parts of the input that were used to answer the question as references, this allows users to review the source used to generate the response if they want to.

Example use cases:

  • Find quick solutions to machinery problems without having to manually dig through long instruction or maintenance manuals
  • Extract the key information from an issue description by asking for a summary.

Inputs and outputs

The trigger actions have two inputs, Document/Data and Question, and two outputs, the Answer and the References.

image.png

Input: Input

This is the data which will be used as an information source for

Supported data types:

Answer question from documentAnswer question from data
File (PDF*)Arrays
ImageText

*Password protected PDFs are currently not supported.

NOTES

"Answer Question" works best with text based data in the Input. For numeric data, we recommend storing the data in a Tulip Table and using Tulip Analytics.

Currently supported languages (for PDF’s and images):

  • English
  • French
  • German
  • Italian
  • Portuguese
  • Spanish

Input: Question

This is the question which will be answered based on the data.

Supported data type:

  • Text

Currently supported languages:

  • English
  • French
  • German
  • Italian
  • Portuguese
  • Spanish

Output: Answer

This is the answer to the question that was identified by the trigger action based on the input.

Supported data type
OutputText

Output: References

This is an array of the parts of the input that were used by the trigger action to identify the answer. This output depends on the selected data type for Input.

Input is File (PDF) or an Image

List of objects with properties “Page” and “Text”

  • For PDFs, the items in the list will be the page number and the extracted text for each page that were used to answer the question
  • For Images, the list will have only one entry with the page number 1 and the text detected on image

Input is List of any type

List of the same type as the Input

  • The list will contain all records that were used to answer the question.

How it works and what to consider

At the heart of our "Answer question" trigger actions, we're using a smart way to find the most relevant answers from your documents. Here's how it works use the example of inputs which can have multiple references like a PDF (= multiple pages) or a list of records:

  1. We take each page of your document / record you provide as input and create a unique 'fingerprint' for it. This helps us understand what's on each page.
  2. When you ask a question, we identify which pages / records are most related to your question based on that fingerprint.
  3. We then take your question and the relevant pages or record and provide that as input to the AI system to answer the question based on the pages.
  4. The trigger action then returns the answer, and also the pages or records that were used for answering the question, so that you can review both.

One important thing to keep in mind is that while the system is very powerful, it has a limit to how much it can 'read' at once. This means that we can't always use all the information (e.g. pages from a PDF) to answer a question. We give it the most relevant pages up to the maximum it can consume.
As a result, some types of questions are better suited than others for the system. For instance, asking it to count things across a whole document might not give the correct results, if not all pages or records can be included.
Therefore, while our system is designed to be as accurate as possible, we do recommend having a human in the loop, especially for critical topics. This isn't because our system isn't reliable, but because we believe in the value of human judgment in complementing AI. This way, you can ensure the highest level of accuracy and context understanding.

Edge Cases

No input and/or no query provided

If no input or no query is provided to the trigger action, the App will show the following system error:
Your Input or Query is empty

This happens for all of the following cases:

  • The input and/or query do not have a value assigned. This is equivalent to “null”.
  • The query has an empty string assigned.

Question cannot be answered from provided input

The answer will be:
There is no information provided in the given context [...]

Limits

Warning

The following languages are the only languages supported for documents: English, Spanish, Italian, Portuguese, French, German.

Currently the following limits exist for Copilot triggers. These limits are tracked on an Instance level. In a case where these limits have been exceeded, the trigger action will fail.

Answer Question from Document:
Monthly Limit: 10,000 Requests/Month
Rate Limit: 10 Requests/Minute

Answer Question from Data:
Monthly Limit: 100,000 Requests/Month
Rate Limit: 10 Requests/Minute

Was this article helpful?