UI widgets

Widgets are structured payloads the assistant can attach to a message so your client renders real UI — suggestion chips, an answer form, a triage card — instead of parsing prose.

They are opt-in: set ui_widgets_enabled to true in settings. With the flag off, assistant messages contain text parts only.

Widget variants

UIWidget is a tagged union discriminated by type.

hints

{ "type": "hints", "hints": ["Yes", "No", "Not sure"] }

form

Each item carries a human-readable label and the allowed answer labels in display order.

{
  "type": "form",
  "items": [
    { "name": "Headache", "choices": ["Yes", "No", "Don't know"] }
  ]
}

recommendation_triage

{
  "type": "recommendation_triage",
  "triage_level": "consultation_24",
  "specialist": { "id": "sp_1", "name": "General practitioner" },
  "channel": "personal_visit"
}

recommendation_conditions

Condition rows use the same enrichment shape as Condition in the assessment: id, probability, name, common_name, and condition_details. serious_findings rows carry id, name, common_name, and parent_ids.

summary

Part-way through the interview — once the opening evidence has been gathered, before the diagnostic questions — the assistant presents everything recorded so far and asks the user to confirm or correct it. That message carries the summary widget.

{
  "type": "summary",
  "patient": "Female, 34 years old",
  "groups": [
    {
      "state": "present",
      "title": "Symptoms you reported",
      "items": [
        {
          "name": "Headache",
          "highlighted": false,
          "children": [{ "name": "Throbbing headache", "highlighted": true, "children": [] }]
        }
      ]
    },
    {
      "state": "absent",
      "title": "Symptoms you ruled out",
      "items": [{ "name": "Fever", "highlighted": false, "children": [] }]
    }
  ],
  "unmatched": {
    "title": "Things I cannot assess",
    "note": "I could not match these to anything I know. Mention them to a clinician.",
    "items": [{ "name": "strange taste in my mouth", "highlighted": false }]
  }
}

It is display-only. Unlike form and hints the summary carries no interaction contract — the user replies in free text. The actionable affordances arrive as a separate hints widget on the same message.

It is cumulative, and it repeats. The widget is rebuilt from the full current state every time it is emitted: once when the step opens, with kind: "intro", and again on each later turn that does not close the step, with kind: "follow_up". Replace the card you rendered rather than appending to it. highlighted marks what the latest turn recorded — including a correction that resolved to something already listed — which is what makes a re-render worth showing. Nothing is highlighted on the first render. The step is entered once per conversation, so the widget stops appearing once the user confirms.

One case skips the re-render: when the user signals they want to change something but not yet what, the assistant asks what to change and leaves the card as it stands.

Groups are localized and ordered. state runs present, absent, unknown, in that order, and empty groups are omitted. title is display-ready localized copy and does not mirror the key — in English the unknown group is titled Not sure — so branch on state, never on title. A finding the user refused to answer is folded into unknown rather than dropped.

items nest one level deep: a finding whose parent concept is in the same group is nested under it, and deeper chains are flattened to that single level. Do not write a renderer that expects arbitrary depth — but do read children, or you will silently drop findings.

unmatched appears only when the user reported something the engine could not match to a known concept, and carries those findings as the user phrased them, deduplicated, in the order they were raised. They are outside what the assessment covers, which is why they sit apart and carry their own localized note — render them, so the user can still raise them with a clinician.

read_back

The confirmation card the assistant emits when the user volunteers information that was not asked for. Grouping follows summary — same state values, same order, empty groups omitted — with one difference: items are always flat. A read-back covers one message's new findings, and one message cannot produce a parent and its child, so children does not exist here. highlighted is always false.

{
  "type": "read_back",
  "groups": [
    {
      "state": "present",
      "title": "Symptoms you reported",
      "items": [{ "name": "Dizziness", "highlighted": false }]
    }
  ]
}

See Read-backs for when one is emitted and what it arrives with.

Where widgets appear

A widget reaches your client in up to two places:

1. In content_parts on the completed assistant message, as a ui_widget part. This is durable — it is in blocking turn responses, in conversation.assistant_message.completed, and in conversation history.
2. As a transient conversation.assistant_message.ui_widget event during streaming, so you can render structured UI before the message finishes.

{
  "type": "conversation.assistant_message.ui_widget",
  "assistant_message": {
    "id": "6f605890-ff28-4b6d-a17f-d410f9faf558",
    "turn_id": "a1b2c3d4-0000-0000-0000-000000000000",
    "widget": { "type": "hints", "hints": ["Yes", "No", "Not sure"] }
  }
}

Client rules

Widgets may arrive out of order relative to text. One or more widget events may land before, between, or after the text deltas of the same assistant_message.id — a recommendation triage card commonly precedes the intro text. Only the completed event carries the final, correctly ordered content_parts.

Reconcile after a reconnect. Transient events emitted between snapshot creation and live re-subscription are lost. Reconcile from the next durable boundary — conversation.turn.completed, conversation.turn.interrupted, or conversation.turn.failed — and from a refreshed snapshot. If the first transient you receive after reconnecting is not at the start of a new turn, ignore further transients for that in-flight turn until a terminal turn event.

Degrade gracefully. A client that ignores widgets entirely still shows a coherent conversation: the accompanying text parts stand on their own.