Beispiel-Prompts

Die Beispiel-Prompts werden verwendet, um textuelle Demonstrationsbeispiele für das In-Context-Learning zu generieren. Die Beispiele werden im Verzeichnis ufo/prompts/examples mit den folgenden Unterverzeichnissen gespeichert

Verzeichnis	Beschreibung
lite	Lite-Version von Demonstrationsbeispielen
non-visual	Beispiele für nicht-visuelle LLMs
visual	Beispiele für visuelle LLMs

Sie können den verwendeten Beispiel-Prompt in der Datei config.yaml konfigurieren. Weitere Informationen zur Konfigurationsdatei finden Sie hier.

Beispiel-Prompts

Nachfolgend finden Sie Beispiele für den HostAgent und den AppAgent

• HostAgent:

Request: |-
    Summarize and add all to do items on Microsoft To Do from the meeting notes email, and write a summary on the meeting_notes.docx.
Response:
    Observation: |-
        The current screenshot shows the Microsoft To Do application is visible, and outlook application and the meeting_notes.docx are available in the list of applications.
    Thought: |-
        The user request can be decomposed into three sub-tasks: (1) Summarize all to do items on Microsoft To Do from the meeting_notes email, (2) Add all to do items to Microsoft To Do, and (3) Write a summary on the meeting_notes.docx. I need to open the Microsoft To Do application to complete the first two sub-tasks.
        Each sub-task will be completed in individual applications sequentially.
    CurrentSubtask: |-
        Summarized all to do items from the meeting notes email in Outlook.
    Message:
        - (1) You need to first search for the meeting notes email in Outlook to summarize.
        - (2) Only summarize the to do items from the meeting notes email, without any redundant information.
    ControlLabel: |-
        16
    ControlText: |-
        Mail - Outlook - Jim
    Status: |-
        CONTINUE
    Plan:
        - Add all to do items previously summarized from the meeting notes email to one-by-one Microsoft To Do.
        - Write a summary about the meeting notes email on the meeting_notes.docx.
    Comment: |-
        I plan to first summarize all to do items from the meeting notes email in Outlook.
    Questions: []

• AppAgent:

Request: |-
    How many stars does the Imdiffusion repo have?
Sub-task: |-
    Google search for the Imdiffusion repo on github and summarize the number of stars the Imdiffusion repo page visually.
Response: 
    Observation: |-
      I observe that the Edge browser is visible in the screenshot, with the Google search page opened.
    Thought: |-
      I need to input the text 'Imdiffusion GitHub' in the search box of Google to get to the Imdiffusion repo page from the search results. The search box is usually in a type of ComboBox.
    ControlLabel: |-
      36
    ControlText: |-
      搜索
    Function: |-
      set_edit_text
    Args: 
      {"text": "Imdiffusion GitHub"}
    Status: |-
      CONTINUE
    Plan:
      - (1) After input 'Imdiffusion GitHub', click Google Search to search for the Imdiffusion repo on github.
      - (2) Once the searched results are visible, click the Imdiffusion repo Hyperlink in the searched results to open the repo page.
      - (3) Observing and summarize the number of stars the Imdiffusion repo page, and reply to the user request.
    Comment: |-
      I plan to use Google search for the Imdiffusion repo on github and summarize the number of stars the Imdiffusion repo page visually.
    SaveScreenshot:
      {"save": false, "reason": ""}
Tips: |-
    - The search box is usually in a type of ComboBox.
    - The number of stars of a Github repo page can be found in the repo page visually.

Diese Beispiele regeln das Ausgabeformat der Antwort des Agenten und bieten eine strukturierte Möglichkeit, Demonstrationsbeispiele für das In-Context-Learning zu generieren.