social-app/docs/protocols/ui/ui-schema.md

# UI Schema Protocol

> **NOTE**: This document is the single source of truth. All implementations must follow this specification.

## Overview

A generic UI schema for rendering tool/agent execution results. Designed for AI Agent / Tool ecosystem with extensibility.

**Design Philosophy**: Keep only "primitive components + layout containers". Frontend only needs to recursively render the root layout tree.

## Version

- **Current**: `2.0`
- **Status**: Active

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│  UiSchemaRenderer (root)                                    │
│  - version / locale / status / theme                        │
│  - meta (protocol-level metadata)                          │
│  - root (UiLayoutNode - stack or grid)                     │
├─────────────────────────────────────────────────────────────┤
│  Rendering Flow:                                            │
│  1. Backend returns UiSchemaRenderer with root layout      │
│  2. Frontend recursively renders root layout tree         │
│  3. Layout nodes (stack/grid) contain children             │
│  4. Primitive nodes (text/icon/button/etc) are leaves       │
└─────────────────────────────────────────────────────────────┘
```

---

## Data Types

### UiStatus

```typescript
type UiStatus = 'info' | 'success' | 'warning' | 'error' | 'pending';
```

### IconSource

```typescript
type IconSource = 'icon' | 'emoji' | 'url';
```

### TextFormat

```typescript
type TextFormat = 'plain' | 'markdown';
```

### TextRole

```typescript
type TextRole = 'title' | 'subtitle' | 'body' | 'caption' | 'code';
```

### ButtonStyle

```typescript
type ButtonStyle = 'primary' | 'secondary' | 'ghost' | 'danger';
```

### LayoutDirection

```typescript
type LayoutDirection = 'vertical' | 'horizontal';
```

### LayoutAppearance

```typescript
type LayoutAppearance = 'plain' | 'card' | 'section';
```

### LayoutAlign

```typescript
type LayoutAlign = 'start' | 'center' | 'end' | 'stretch';
```

### LayoutJustify

```typescript
type LayoutJustify = 'start' | 'center' | 'end' | 'space-between';
```

### RendererTheme

```typescript
type RendererTheme = 'default' | 'light' | 'dark';
```

---

## Root Structure

```typescript
interface UiSchemaRenderer {
  version: string;           // "2.0"
  locale: string;           // "zh-CN"
  status: UiStatus;
  theme: RendererTheme;
  
  // Protocol-level metadata (not rendered)
  meta?: {
    requestId?: string;
    toolId?: string;
    traceId?: string;
    userId?: string;
  };
  
  // Root layout node
  root: UiLayoutNode;
}
```

---

## Node Types

### Primitive Components

#### 1. Text Node

```typescript
interface UiTextNode extends UiBaseNode {
  type: 'text';
  content: string;
  format: TextFormat;        // 'plain' | 'markdown'
  role: TextRole;           // 'title' | 'subtitle' | 'body' | 'caption' | 'code'
  status?: UiStatus;
  maxLines?: number;
  visible?: boolean;
}
```

#### 2. Icon Node

```typescript
interface UiIconNode extends UiBaseNode {
  type: 'icon';
  source: IconSource;       // 'icon' | 'emoji' | 'url'
  value: string;
  color?: string;
  size?: number;
  visible?: boolean;
}
```

#### 3. Badge Node

```typescript
interface UiBadgeNode extends UiBaseNode {
  type: 'badge';
  label: string;
  status: UiStatus;
  visible?: boolean;
}
```

#### 4. Button Node

```typescript
interface UiButtonNode extends UiBaseNode {
  type: 'button';
  label: string;
  style: ButtonStyle;
  disabled?: boolean;
  icon?: UiIconSpec;
  action: UiActionPayload;
  visible?: boolean;
}
```

#### 5. Key-Value Node

```typescript
interface UiKvNode extends UiBaseNode {
  type: 'kv';
  items: UiKvItem[];
  columns?: number;
  visible?: boolean;
}

interface UiKvItem {
  key: string;
  label?: string;
  value: any;
  copyable?: boolean;
}
```

#### 6. Divider Node

```typescript
interface UiDividerNode extends UiBaseNode {
  type: 'divider';
  inset?: number;
  visible?: boolean;
}
```

### Layout Containers

#### 7. Stack Node

```typescript
interface UiStackNode extends UiBaseNode {
  type: 'stack';
  direction: LayoutDirection;   // 'vertical' | 'horizontal'
  gap?: number;
  appearance: LayoutAppearance; // 'plain' | 'card' | 'section'
  status?: UiStatus;
  align?: LayoutAlign;
  justify?: LayoutJustify;
  wrap?: boolean;
  children: UiNode[];
  visible?: boolean;
}
```

#### 8. Grid Node

```typescript
interface UiGridNode extends UiBaseNode {
  type: 'grid';
  columns: number;
  gap?: number;
  appearance: LayoutAppearance;
  status?: UiStatus;
  children: UiNode[];
  visible?: boolean;
}
```

### Base Node

```typescript
interface UiBaseNode {
  id?: string;
  visible?: boolean;
}
```

### Node Union

```typescript
type UiNode = 
  | UiTextNode
  | UiIconNode
  | UiBadgeNode
  | UiButtonNode
  | UiKvNode
  | UiDividerNode
  | UiStackNode
  | UiGridNode;

type UiLayoutNode = UiStackNode | UiGridNode;
```

---

## Action Payloads

```typescript
type UiActionPayload = 
  | NavigateAction
  | UrlAction
  | EventAction
  | ToolAction
  | CopyAction
  | PayloadAction;

// Navigation action
interface NavigateAction {
  type: 'navigation';
  path: string;
  params?: Record<string, any>;
}

// Navigation Contract (current implementation constraint)
// 1) path MUST be an internal app route and MUST be fully materialized
//    (e.g. '/todo/123', not '/todo/:id').
// 2) path MUST NOT include query string or fragment.
// 3) params, when provided, is treated as query params only.
// 4) params values MUST be scalar (string | number | boolean).
// 5) Backend MUST generate path from route catalog
//    `backend/src/core/config/static/route/frontend_routes.yaml`.

// URL action
interface UrlAction {
  type: 'url';
  url: string;
  target?: '_self' | '_blank';
}

// Event action
interface EventAction {
  type: 'event';
  event: string;
  payload?: Record<string, any>;
}

// Tool action
interface ToolAction {
  type: 'tool';
  toolId: string;
  params?: Record<string, any>;
}

// Copy action
interface CopyAction {
  type: 'copy';
  content: string;
  successMessage?: string;
}

// Payload action
interface PayloadAction {
  type: 'payload';
  payload: Record<string, any>;
  submitTo?: string;
}
```

---

## Icon Specification

```typescript
interface UiIconSpec {
  source: IconSource;
  value: string;
  color?: string;
  size?: number;
}
```

---

## Usage Patterns---

## JSON Examples

### Example 1: Simple Text

```json
{
  "version": "2.0",
  "locale": "zh-CN",
  "status": "success",
  "theme": "default",
  "root": {
    "type": "stack",
    "direction": "vertical",
    "gap": 12,
    "appearance": "plain",
    "children": [
      {
        "type": "text",
        "content": "操作成功",
        "role": "title"
      },
      {
        "type": "text",
        "content": "您的事项已创建完成",
        "role": "body"
      }
    ]
  }
}
```

### Example 2: Card with Actions

```json
{
  "version": "2.0",
  "locale": "zh-CN",
  "status": "success",
  "theme": "default",
  "root": {
    "type": "stack",
    "direction": "vertical",
    "gap": 16,
    "appearance": "card",
    "children": [
      {
        "type": "text",
        "content": "日程已创建",
        "role": "title"
      },
      {
        "type": "kv",
        "items": [
          { "key": "title", "label": "主题", "value": "Q1 规划会议" },
          { "key": "time", "label": "时间", "value": "2026-03-15 14:00" }
        ],
        "columns": 2
      },
      {
        "type": "stack",
        "direction": "horizontal",
        "gap": 8,
        "children": [
          {
            "type": "button",
            "label": "查看详情",
            "style": "primary",
            "action": { "type": "navigation", "path": "/calendar/evt_abc123" }
          },
          {
            "type": "button",
            "label": "删除",
            "style": "danger",
            "action": { "type": "tool", "toolId": "calendar.delete", "params": { "eventId": "evt_abc123" } }
          }
        ]
      }
    ]
  }
}
```

### Example 3: Error Status Panel

```json
{
  "version": "2.0",
  "locale": "zh-CN",
  "status": "error",
  "theme": "default",
  "root": {
    "type": "stack",
    "direction": "vertical",
    "gap": 12,
    "appearance": "card",
    "status": "error",
    "children": [
      {
        "type": "stack",
        "direction": "horizontal",
        "gap": 8,
        "align": "center",
        "justify": "space-between",
        "children": [
          {
            "type": "text",
            "content": "删除失败",
            "role": "title"
          },
          {
            "type": "badge",
            "label": "ERROR",
            "status": "error"
          }
        ]
      },
      {
        "type": "text",
        "content": "您没有权限执行此操作",
        "role": "body",
        "status": "error"
      },
      {
        "type": "stack",
        "direction": "horizontal",
        "gap": 8,
        "children": [
          {
            "type": "button",
            "label": "重试",
            "style": "primary",
            "action": { "type": "tool", "toolId": "user.delete", "params": { "userId": "u1" } }
          },
          {
            "type": "button",
            "label": "联系管理员",
            "style": "secondary",
            "action": { "type": "url", "url": "mailto:admin@example.com" }
          }
        ]
      }
    ]
  }
}
```

### Example 4: Grid Layout

```json
{
  "version": "2.0",
  "locale": "zh-CN",
  "status": "info",
  "theme": "default",
  "root": {
    "type": "grid",
    "columns": 3,
    "gap": 16,
    "appearance": "plain",
    "children": [
      {
        "type": "card",
        "children": [
          { "type": "text", "content": "今日订单", "role": "title" },
          { "type": "text", "content": "128", "role": "subtitle" }
        ]
      },
      {
        "type": "card",
        "children": [
          { "type": "text", "content": "待处理", "role": "title" },
          { "type": "text", "content": "24", "role": "subtitle", "status": "warning" }
        ]
      },
      {
        "type": "card",
        "children": [
          { "type": "text", "content": "总收入", "role": "title" },
          { "type": "text", "content": "¥8,640", "role": "subtitle", "status": "success" }
        ]
      }
    ]
  }
}
```

### Example 5: Section Layout

```json
{
  "version": "2.0",
  "locale": "zh-CN",
  "status": "success",
  "theme": "default",
  "root": {
    "type": "stack",
    "direction": "vertical",
    "gap": 24,
    "appearance": "plain",
    "children": [
      {
        "type": "stack",
        "direction": "vertical",
        "gap": 12,
        "appearance": "section",
        "children": [
          { "type": "text", "content": "基本信息", "role": "title" },
          { "type": "kv", "items": [
            { "key": "name", "label": "姓名", "value": "张三" },
            { "key": "email", "label": "邮箱", "value": "zhangsan@example.com", "copyable": true }
          ]}
        ]
      },
      {
        "type": "stack",
        "direction": "vertical",
        "gap": 12,
        "appearance": "section",
        "children": [
          { "type": "text", "content": "账户设置", "role": "title" },
          { "type": "button", "label": "修改密码", "style": "secondary", "action": { "type": "navigation", "path": "/settings/password" } },
          { "type": "button", "label": "退出登录", "style": "ghost", "action": { "type": "event", "event": "logout" } }
        ]
      }
    ]
  }
}
```

---


---

## UiHints (Descriptive UI)

### Overview

UiHints is a **descriptive** UI representation designed for AI agents to express UI intent with minimal token cost. It describes **what to show**, not **how to render**.

The `ui_compiler` transforms UiHints into UiSchemaRenderer for frontend rendering.

### Design Principles

1. **Descriptive not Rendered**: Express content intent, not visual instructions
2. **Minimal Token Cost**: Simple structure, semantic field names
3. **Composable**: Supports nested sections and mixed content
4. **Compilable**: Mechanical transformation to UiSchemaRenderer
5. **Lossless**: Main content fields in hints are preserved in renderer

### Intent Types

Intent is a **weak hint** - it only affects default layout style, not field presence.

| Intent | Default Layout |
|--------|----------------|
| `message` | plain |
| `data` | card |
| `list` | plain |
| `status` | card |
| `form` | section |
| `mixed` | card |

### UiHints Payload

```typescript
interface UiHintsPayload {
  version: string;           // "2.1"
  intent: UiHintIntent;      // Primary display intent (weak hint)
  status: UiStatus;          // Overall status

  title?: string;            // Top-level title
  description?: string;       // Top-level description
  body?: string;             // Top-level main body text
  bodyFormat?: "plain" | "markdown";  // Body text format

  items?: UiHintKvItem[];   // Top-level key-value items
  listItems?: UiHintListItem[];  // Top-level list items
  sections?: UiHintSection[];    // Grouped sections
  actions?: UiHintAction[];     // Top-level actions
  icon?: UiHintIcon;         // Top-level icon
  meta?: Record<string, any>;    // Extra meta
}

interface UiHintSection {
  title?: string;
  description?: string;
  icon?: UiHintIcon;
  content?: string;
  contentFormat?: "plain" | "markdown";
  items?: UiHintKvItem[];
  listItems?: UiHintListItem[];
  actions?: UiHintAction[];
}

interface UiHintListItem {
  id?: string;
  title: string;
  subtitle?: string;
  description?: string;
  icon?: UiHintIcon;
  status?: UiHintStatus;
  actions?: UiHintAction[];
}

interface UiHintKvItem {
  key: string;
  label?: string;
  value?: any;
  copyable?: boolean;
}

interface UiHintAction {
  label: string;
  style?: "primary" | "secondary" | "ghost" | "danger";
  disabled?: boolean;
  action: UiHintActionTarget;
}

type UiHintActionTarget = 
  | { type: "navigation"; path: string; params?: Record<string, any> }
  | { type: "url"; url: string; target?: "_self" | "_blank" }
  | { type: "event"; event: string; payload?: Record<string, any> }
  | { type: "tool"; toolId: string; params?: Record<string, any> }
  | { type: "copy"; content: string; successMessage?: string }
  | { type: "payload"; payload: Record<string, any>; submitTo?: string };

interface UiHintIcon {
  source: "icon" | "emoji" | "url";
  value: string;
  color?: string;
  size?: number;
}
```

### Compilation Flow

```
Agent Output (UiHints 2.1)
        │
        ▼
    ui_compiler
        │
        ▼
UiSchemaRenderer (for frontend rendering)
```

### Example

**Input (UiHints)**:
```json
{
  "intent": "status",
  "status": "success",
  "title": "日程已创建",
  "body": "本次创建已成功完成。",
  "items": [
    {"key": "title", "label": "主题", "value": "Q1 规划会议"},
    {"key": "time", "label": "时间", "value": "2026-03-15 14:00"}
  ],
  "actions": [
    {"label": "查看详情", "style": "primary", "action": {"type": "navigation", "path": "/calendar/evt_123"}},
    {"label": "删除", "style": "danger", "action": {"type": "tool", "toolId": "calendar.delete", "params": {"eventId": "evt_123"}}}
  ]
}
```

**Output (UiSchemaRenderer)**:
```json
{
  "version": "2.0",
  "locale": "zh-CN",
  "status": "success",
  "theme": "default",
  "root": {
    "type": "stack",
    "appearance": "card",
    "status": "success",
    "children": [
      {
        "type": "stack",
        "direction": "horizontal",
        "gap": 8,
        "children": [
          {"type": "text", "content": "日程已创建", "role": "title"},
          {"type": "badge", "label": "SUCCESS", "status": "success"}
        ],
        "justify": "space-between",
        "align": "center"
      },
      {"type": "text", "content": "本次创建已成功完成。", "role": "body"},
      {"type": "kv", "items": [...]},
      {
        "type": "stack",
        "direction": "horizontal",
        "gap": 8,
        "children": [
          {"type": "button", "label": "查看详情", "style": "primary", ...},
          {"type": "button", "label": "删除", "style": "danger", ...}
        ]
      }
    ]
  }
}
```

### Python Implementation

- `schemas.agent.ui_hints.UiHintsPayload` - Descriptive UI model (v2.1)
- `core.agentscope.runtime.ui_compiler.compile(hints)` - Compile to UiSchemaRenderer