social-app/docs/protocols/agent-chat-messages.md

Agent Chat Messages Protocol

Note

: This document is the single source of truth. All implementations must follow this specification.

Overview

Chat messages in agent conversations with metadata for tracking and telemetry.

Version

• Current: 1.0
• Status: Active

---

Message Metadata

interface AgentChatMessageMetadata {
  run_id?: string;       // Unique run identifier
  stage?: string;       // Processing stage (e.g., "intent", "execution")
  latency_ms?: number;  // Processing latency in milliseconds
  message_id?: string;  // Message identifier
  [key: string]: any;   // Additional custom fields allowed
}

---

Database Field

Field	Type	Description
metadata	jsonb	Message metadata including run_id, stage, latency_ms, message_id

---

JSON Examples

Basic Message Metadata

{
  "run_id": "run_1773286460762"
}

Stage Completion Metadata

{
  "run_id": "run_1773286460762",
  "stage": "intent",
  "latency_ms": 2610,
  "message_id": "intent-run_1773286460762"
}

Tool Execution Metadata

{
  "run_id": "run_1773287162123",
  "stage": "tool_execution",
  "latency_ms": 1500,
  "message_id": "tool_run_abc123",
  "tool_name": "calendar_create_event"
}

---

User Message Attachments

Overview

When a user sends a message with binary attachments (e.g., images), the frontend uploads the file to storage first, then sends a signed URL to the backend. The backend parses the signed URL to extract storage metadata and persists it with the message.

Flow

Frontend                          Backend
────────────────────────────────────────────────────────────────
1. Upload file
   POST /api/v1/agent/attachments
   ──────────────────────────────>
   <──────────────────────────────
   {bucket, path, mime_type, url: signed_url}

2. Send message with binary block
   POST /api/v1/agent/run
   content: [
     {type: "text", text: "..."},
     {type: "binary", mimeType: "image/jpeg", url: signed_url}
   ]
   ──────────────────────────────>

3. Backend parses signed URL
   parse_signed_url(url) → {bucket, path}

4. Persist to database
   metadata.user_message_attachments = {bucket, path, mime_type}

5. Return history (GET /history)
   <──────────────────────────────
   messages: [{
     role: "user",
     content: [
       {type: "text", text: "..."},
       {type: "binary", mimeType: "image/jpeg", url: new_signed_url}
     ]
   }]

Signed URL Format

Supabase signed URL format:

https://{project}.supabase.co/storage/v1/object/sign/{bucket}/{path}?token={jwt}

Backend parses to extract:

• bucket: URL path segment after /sign/
• path: Remaining path after bucket

Metadata Schema

interface UserMessageAttachments {
  bucket: string;      // Storage bucket name
  path: string;        // Object storage path
  mime_type: string;  // MIME type (e.g., "image/jpeg")
}

interface AgentChatMessageMetadata {
  // ... existing fields
  user_message_attachments?: UserMessageAttachments;
}

Database Storage

Field	Type	Description
metadata	jsonb	Contains user_message_attachments with bucket, path, mime_type

Example

Request (POST /run):

{
  "threadId": "thread-123",
  "runId": "run-456",
  "messages": [
    {
      "id": "msg-1",
      "role": "user",
      "content": [
        {"type": "text", "text": "帮我看看这张图"},
        {
          "type": "binary",
          "mimeType": "image/jpeg",
          "url": "https://xxx.supabase.co/storage/v1/object/sign/agent-files/agent-inputs/u/t/r/img.jpg?token=xxx"
        }
      ]
    }
  ]
}

Stored Metadata:

{
  "user_message_attachments": {
    "bucket": "agent-files",
    "path": "agent-inputs/u/t/r/img.jpg",
    "mime_type": "image/jpeg"
  }
}

History Response (GET /history):

{
  "messages": [
    {
      "id": "msg-1",
      "role": "user",
      "content": [
        {"type": "text", "text": "帮我看看这张图"},
        {
          "type": "binary",
          "mimeType": "image/jpeg",
          "url": "https://xxx.supabase.co/storage/v1/object/sign/agent-files/agent-inputs/u/t/r/img.jpg?token=yyy"
        }
      ]
    }
  ]
}