MENU
    Answer Question from Data/Document Trigger actions
    • 16 May 2024
    • 5 Minutes to read
    • Contributors

    Answer Question from Data/Document Trigger actions


    Article summary

    Open Beta

    The Frontline Copilot trigger actions are currently in open beta. Reach out to copilot@tulip.co if you are interested in joining the beta.

    Beta Disclaimer

    Use of Beta Versions offered by Tulip is optional and is at your sole risk. Due to the untested/unproven nature of Beta versions, they are provided “as is,” “as available,” and without warranty. This means that any warranties, indemnities and other obligations of Tulip under any terms of service or MSA do not apply to Beta Versions.

    • Frontline Copilot is powerful! We will help to guide, but you are ultimately responsible for the impact of what you build.
    • We reserve the right to turn off any and all Copilot Triggers at any time if we notice something unexpected.
    • We reserve the right to limit Copilot Triggers during the course of the beta if and as needed.
    • You will not be charged for the usage of Copilot Triggers during the program. You will have three months after the conclusion of the program to decide to continue with the feature or not.
    • At some point in the future, we may implement usage based pricing for beta features.

    The Trigger actions "Answer question from document" and "Answer question from data" analyze text-based information in a PDF, image, list of data, or text to return an answer to a user provided question. Additionally if they return the parts of the input that were used to answer the question as references, this allows users to review the source used to generate the response if they want to.

    Example use cases:

    • Find quick solutions to machinery problems without having to manually dig through long instruction or maintenance manuals
    • Extract the key information from an issue description by asking for a summary.

    Inputs and outputs

    The trigger actions have two inputs, Document/Data and Question, and two outputs, the Answer and the References.

    image.png

    Input: Input

    This is the data which will be used as an information source for

    Supported data types:

    Answer question from documentAnswer question from data
    File (PDF*)Arrays
    ImageText

    *Password protected PDFs are currently not supported.

    NOTES

    "Answer Question" works best with text based data in the Input. For numeric data, we recommend storing the data in a Tulip Table and using Tulip Analytics.

    Currently supported languages (for PDF’s and images):

    • English
    • French
    • German
    • Italian
    • Portuguese
    • Spanish

    Input: Question

    This is the question which will be answered based on the data.

    Supported data type:

    • Text

    Currently supported languages:

    • English
    • French
    • German
    • Italian
    • Portuguese
    • Spanish

    Output: Answer

    This is the answer to the question that was identified by the trigger action based on the input.

    Supported data type
    OutputText

    Output: References

    This is an array of the parts of the input that were used by the trigger action to identify the answer. This output depends on the selected data type for Input.

    Input is File (PDF) or an Image

    List of objects with properties “Page” and “Text”

    • For PDFs, the items in the list will be the page number and the extracted text for each page that were used to answer the question
    • For Images, the list will have only one entry with the page number 1 and the text detected on image

    Input is List of any type

    List of the same type as the Input

    • The list will contain all records that were used to answer the question.

    How it works and what to consider

    At the heart of our "Answer question" trigger actions, we're using a smart way to find the most relevant answers from your documents. Here's how it works use the example of inputs which can have multiple references like a PDF (= multiple pages) or a list of records:

    1. We take each page of your document / record you provide as input and create a unique 'fingerprint' for it. This helps us understand what's on each page.
    2. When you ask a question, we identify which pages / records are most related to your question based on that fingerprint.
    3. We then take your question and the relevant pages or record and provide that as input to the AI system to answer the question based on the pages.
    4. The trigger action then returns the answer, and also the pages or records that were used for answering the question, so that you can review both.

    One important thing to keep in mind is that while the system is very powerful, it has a limit to how much it can 'read' at once. This means that we can't always use all the information (e.g. pages from a PDF) to answer a question. We give it the most relevant pages up to the maximum it can consume.
    As a result, some types of questions are better suited than others for the system. For instance, asking it to count things across a whole document might not give the correct results, if not all pages or records can be included.
    Therefore, while our system is designed to be as accurate as possible, we do recommend having a human in the loop, especially for critical topics. This isn't because our system isn't reliable, but because we believe in the value of human judgment in complementing AI. This way, you can ensure the highest level of accuracy and context understanding.

    Edge Cases

    No input and/or no query provided

    If no input or no query is provided to the trigger action, the App will show the following system error:
    Your Input or Query is empty

    This happens for all of the following cases:

    • The input and/or query do not have a value assigned. This is equivalent to “null”.
    • The query has an empty string assigned.

    Question cannot be answered from provided input

    The answer will be:
    There is no information provided in the given context [...]

    Limits

    Warning

    The following languages are the only languages supported for documents: English, Spanish, Italian, Portuguese, French, German.

    Currently the following limits exist for Copilot triggers. These limits are tracked on an Instance level. In a case where these limits have been exceeded, the trigger action will fail.

    Answer Question from Document:
    Monthly Limit: 10,000 Requests/Month
    Rate Limit: 10 Requests/Minute
    
    Answer Question from Data:
    Monthly Limit: 100,000 Requests/Month
    Rate Limit: 10 Requests/Minute
    Plain text

    Was this article helpful?