refactor: 重构 AgentScope 运行时模块并优化前端附件展示

2026-03-13 15:42:01 +08:00
parent a10a2db27a
commit 4c10929498
28 changed files with 1494 additions and 2163 deletions
@@ -67,3 +67,134 @@ interface AgentChatMessageMetadata {
  "tool_name": "calendar_create_event"
 }
 ```
+
+---
+
+## User Message Attachments
+
+### Overview
+
+When a user sends a message with binary attachments (e.g., images), the frontend uploads the file to storage first, then sends a signed URL to the backend. The backend parses the signed URL to extract storage metadata and persists it with the message.
+
+### Flow
+
+```
+Frontend                          Backend
+────────────────────────────────────────────────────────────────
+1. Upload file
+   POST /api/v1/agent/attachments
+   ──────────────────────────────>
+   <──────────────────────────────
+   {bucket, path, mime_type, url: signed_url}
+
+2. Send message with binary block
+   POST /api/v1/agent/run
+   content: [
+     {type: "text", text: "..."},
+     {type: "binary", mimeType: "image/jpeg", url: signed_url}
+   ]
+   ──────────────────────────────>
+
+3. Backend parses signed URL
+   parse_signed_url(url) → {bucket, path}
+
+4. Persist to database
+   metadata.user_message_attachments = {bucket, path, mime_type}
+
+5. Return history (GET /history)
+   <──────────────────────────────
+   messages: [{
+     role: "user",
+     content: [
+       {type: "text", text: "..."},
+       {type: "binary", mimeType: "image/jpeg", url: new_signed_url}
+     ]
+   }]
+```
+
+### Signed URL Format
+
+Supabase signed URL format:
+```
+https://{project}.supabase.co/storage/v1/object/sign/{bucket}/{path}?token={jwt}
+```
+
+Backend parses to extract:
+- `bucket`: URL path segment after `/sign/`
+- `path`: Remaining path after bucket
+
+### Metadata Schema
+
+```typescript
+interface UserMessageAttachments {
+  bucket: string;      // Storage bucket name
+  path: string;        // Object storage path
+  mime_type: string;  // MIME type (e.g., "image/jpeg")
+}
+
+interface AgentChatMessageMetadata {
+  // ... existing fields
+  user_message_attachments?: UserMessageAttachments;
+}
+```
+
+### Database Storage
+
+| Field | Type | Description |
+|-------|------|-------------|
+| metadata | jsonb | Contains user_message_attachments with bucket, path, mime_type |
+
+### Example
+
+**Request (POST /run):**
+```json
+{
+  "threadId": "thread-123",
+  "runId": "run-456",
+  "messages": [
+    {
+      "id": "msg-1",
+      "role": "user",
+      "content": [
+        {"type": "text", "text": "帮我看看这张图"},
+        {
+          "type": "binary",
+          "mimeType": "image/jpeg",
+          "url": "https://xxx.supabase.co/storage/v1/object/sign/agent-files/agent-inputs/u/t/r/img.jpg?token=xxx"
+        }
+      ]
+    }
+  ]
+}
+```
+
+**Stored Metadata:**
+```json
+{
+  "user_message_attachments": {
+    "bucket": "agent-files",
+    "path": "agent-inputs/u/t/r/img.jpg",
+    "mime_type": "image/jpeg"
+  }
+}
+```
+
+**History Response (GET /history):**
+```json
+{
+  "messages": [
+    {
+      "id": "msg-1",
+      "role": "user",
+      "content": [
+        {"type": "text", "text": "帮我看看这张图"},
+        {
+          "type": "binary",
+          "mimeType": "image/jpeg",
+          "url": "https://xxx.supabase.co/storage/v1/object/sign/agent-files/agent-inputs/u/t/r/img.jpg?token=yyy"
+        }
+      ]
+    }
+  ]
+}
+```