chore: checkpoint current backend/runtime changes
This commit is contained in:
@@ -0,0 +1,260 @@
|
||||
# Supabase 统一服务生命周期设计(优化版)
|
||||
|
||||
**Date:** 2026-03-06
|
||||
**Status:** Draft
|
||||
|
||||
---
|
||||
|
||||
## 0. Intake Contract
|
||||
|
||||
- Objective: 将 Supabase 客户端纳入统一服务生命周期管理,避免每次请求重复创建客户端。
|
||||
- Deliverable: 新增 `SupabaseService`,并基于 `service_interface.py` 的 `ServiceRegistry` 提供统一初始化/关闭路径,完成 auth 侧迁移。
|
||||
- Constraints:
|
||||
- 保持现有 `core.config.settings` 的配置读取行为不变。
|
||||
- 不引入 `os.environ` 直接读取。
|
||||
- 不改变现有 API 语义。
|
||||
- Verification target:
|
||||
- 通过单元测试证明 Supabase 服务初始化、关闭、健康检查行为。
|
||||
- 通过应用启动测试证明统一初始化流程可用。
|
||||
- 通过 auth 相关测试证明迁移后业务行为一致。
|
||||
|
||||
---
|
||||
|
||||
## 1. 复杂度与风险分级
|
||||
|
||||
- Complexity: `S2`
|
||||
- 原因:涉及多文件改造(`services/base`、`app.py`、`v1/auth`、测试)。
|
||||
- Risk Tier: `L1`
|
||||
- 原因:涉及应用启动链路和认证网关依赖,但不改变对外接口契约。
|
||||
|
||||
L1 Gate 要求:执行 `refactor-cleaner` 审视冗余与结构风险(`code-reviewer` 可选)。
|
||||
|
||||
---
|
||||
|
||||
## 2. 现状与问题
|
||||
|
||||
### 2.1 当前现状
|
||||
|
||||
- `SupabaseAuthGateway` 在 `__init__` 内直接 `create_client(...)`,每次实例化都会创建 anon/admin 客户端。
|
||||
- `get_auth_service()` 当前每次请求都会 new `SupabaseAuthGateway()`,导致客户端重复构造。
|
||||
- `ServiceRegistry` 已存在,但目前主要用于注册,应用启动仍是手写逐个初始化。
|
||||
|
||||
### 2.2 核心问题
|
||||
|
||||
1. 生命周期不统一:Supabase 没有接入应用启动/关闭的统一管理。
|
||||
2. 初始化代码重复趋势:服务增多后,`app.py` 的 lifespan 会继续膨胀。
|
||||
3. 网关构造时机风险:若在应用未初始化阶段取客户端,可能抛运行时异常。
|
||||
|
||||
---
|
||||
|
||||
## 3. 优化设计(推荐方案)
|
||||
|
||||
### 3.1 方案摘要
|
||||
|
||||
在 `service_interface.py` 基础上新增统一生命周期函数,按服务名列表批量初始化/关闭;`app.py` 仅声明服务顺序,减少样板代码。Supabase 使用 `config.supabase` 作为默认配置来源,保持 settings 行为一致。
|
||||
|
||||
### 3.2 目标文件结构
|
||||
|
||||
```text
|
||||
backend/src/services/base/
|
||||
├── __init__.py
|
||||
├── service_interface.py # 扩展:统一生命周期函数
|
||||
├── redis.py
|
||||
└── supabase.py # 新增
|
||||
```
|
||||
|
||||
### 3.3 service_interface 统一初始化能力(新增)
|
||||
|
||||
在 `service_interface.py` 新增以下函数(建议命名):
|
||||
|
||||
- `resolve_registered_services(service_names: list[str]) -> list[BaseServiceProvider]`
|
||||
- `initialize_registered_services(service_names: list[str]) -> tuple[bool, list[BaseServiceProvider]]`
|
||||
- `close_registered_services(services: list[BaseServiceProvider]) -> bool`
|
||||
|
||||
约束与行为:
|
||||
|
||||
1. 初始化按 `service_names` 顺序执行。
|
||||
2. 任一服务初始化失败时:
|
||||
- 返回 `False`。
|
||||
- 对已成功初始化的服务按逆序执行关闭回滚。
|
||||
3. 关闭按逆序执行,最大化依赖安全性。
|
||||
4. 日志必须包含失败服务名和错误摘要。
|
||||
|
||||
这样 `app.py` 只需声明:
|
||||
|
||||
```python
|
||||
SERVICE_STARTUP_ORDER = ["redis", "supabase"]
|
||||
```
|
||||
|
||||
并调用统一函数,减少重复初始化样板。
|
||||
|
||||
### 3.4 SupabaseService 设计
|
||||
|
||||
`supabase.py` 关键点:
|
||||
|
||||
- 继承 `BaseServiceProvider`。
|
||||
- 构造函数签名:
|
||||
- `def __init__(self, settings: SupabaseSettings | None = None) -> None`
|
||||
- 默认 `settings or config.supabase`,确保与当前配置源一致。
|
||||
- `initialize()`:创建 anon/admin 两个 client,失败返回 `False`。
|
||||
- `close()`:
|
||||
- 清空 `_client`、`_admin_client`。
|
||||
- `self._set_initialized(False)`。
|
||||
- `health_check()`:
|
||||
- 必须进行至少一个轻量真实请求验证,不仅检查本地对象存在。
|
||||
- 返回结构与 `RedisService.health_check()`风格一致(`status + details`)。
|
||||
|
||||
注册方式:
|
||||
|
||||
```python
|
||||
supabase_service: SupabaseService = register_service_instance(
|
||||
"supabase", SupabaseService()
|
||||
)
|
||||
```
|
||||
|
||||
### 3.5 app.py 改造
|
||||
|
||||
当前手写 `redis_service.initialize()` 改为调用统一初始化函数。
|
||||
|
||||
目标行为:
|
||||
|
||||
1. 启动阶段:
|
||||
- 调用 `initialize_registered_services(["redis", "supabase"])`。
|
||||
- 失败则 `raise RuntimeError("Service initialization failed")`。
|
||||
2. 关闭阶段:
|
||||
- 调用 `close_registered_services(initialized_services)`。
|
||||
|
||||
### 3.6 AuthGateway 迁移策略(避免构造时机问题)
|
||||
|
||||
不建议在 `SupabaseAuthGateway.__init__` 里立即绑定 client;改为按需获取:
|
||||
|
||||
- 保留网关对象轻量化。
|
||||
- 在每个业务方法内部通过 `supabase_service.get_client()` / `get_admin_client()` 取实例。
|
||||
|
||||
优点:
|
||||
|
||||
1. 避免模块导入或依赖构建阶段误触未初始化 client。
|
||||
2. 对 `users/dependencies.py` 中全局缓存 gateway 的场景更安全。
|
||||
3. 不改变业务层接口。
|
||||
|
||||
---
|
||||
|
||||
## 4. 配置与兼容性保证
|
||||
|
||||
### 4.1 settings/config 行为不变
|
||||
|
||||
迁移后依然通过 `core.config.settings.config.supabase` 读取:
|
||||
|
||||
- `url`
|
||||
- `anon_key`
|
||||
- `service_role_key`
|
||||
- `jwt_secret`(JWT 校验现有逻辑继续使用)
|
||||
|
||||
### 4.2 环境变量兼容
|
||||
|
||||
由于 `Settings` + `env_nested_delimiter` 机制不变,现有环境变量命名与 `.env` 内容无需修改。
|
||||
|
||||
### 4.3 对现有代码影响
|
||||
|
||||
- API 层 schema/路由不变。
|
||||
- 认证行为不变。
|
||||
- 仅优化客户端生命周期与启动流程。
|
||||
|
||||
---
|
||||
|
||||
## 5. 实施计划(可执行)
|
||||
|
||||
### Task 1: 扩展统一生命周期接口
|
||||
|
||||
**Files**
|
||||
- Modify: `backend/src/services/base/service_interface.py`
|
||||
- Test: `backend/tests/unit/services/base/test_service_interface.py`(新增)
|
||||
|
||||
**Steps**
|
||||
1. 写失败测试:初始化顺序、失败回滚、关闭逆序。
|
||||
2. 实现生命周期函数。
|
||||
3. 跑单测确认通过。
|
||||
|
||||
### Task 2: 新增 SupabaseService
|
||||
|
||||
**Files**
|
||||
- Create: `backend/src/services/base/supabase.py`
|
||||
- Modify: `backend/src/services/base/__init__.py`
|
||||
- Test: `backend/tests/unit/services/base/test_supabase.py`
|
||||
|
||||
**Steps**
|
||||
1. 写失败测试(init success/fail、close、health_check)。
|
||||
2. 实现 `SupabaseService` 与实例注册。
|
||||
3. 跑单测。
|
||||
|
||||
### Task 3: 接入 app lifespan 统一初始化
|
||||
|
||||
**Files**
|
||||
- Modify: `backend/src/app.py`
|
||||
- Test: `backend/tests/integration/test_app_lifespan.py`(新增或扩展)
|
||||
|
||||
**Steps**
|
||||
1. 写失败测试(supabase init fail 时应用启动失败)。
|
||||
2. 替换手写初始化为统一函数。
|
||||
3. 跑集成测试。
|
||||
|
||||
### Task 4: 迁移 AuthGateway 获取 client 方式
|
||||
|
||||
**Files**
|
||||
- Modify: `backend/src/v1/auth/gateway.py`
|
||||
- Optional Modify: `backend/src/v1/auth/dependencies.py`
|
||||
- Optional Modify: `backend/src/v1/users/dependencies.py`
|
||||
- Test: `backend/tests/unit/v1/auth/test_gateway.py`(扩展)
|
||||
|
||||
**Steps**
|
||||
1. 写失败测试(未初始化时错误、初始化后正常调用)。
|
||||
2. 改为方法内按需取 client。
|
||||
3. 跑 auth 相关单测。
|
||||
|
||||
### Task 5: 全量验证与门禁
|
||||
|
||||
**Commands**
|
||||
- `uv run ruff check backend/src backend/tests`
|
||||
- `uv run basedpyright`
|
||||
- `uv run pytest backend/tests/unit/services/base -q`
|
||||
- `uv run pytest backend/tests/unit/v1/auth -q`
|
||||
- `uv run pytest backend/tests/integration -q`
|
||||
|
||||
输出要求:记录每条命令 pass/fail 与关键摘要。
|
||||
|
||||
---
|
||||
|
||||
## 6. 验收标准(更新)
|
||||
|
||||
- [ ] `SupabaseService` 继承 `BaseServiceProvider` 并注册到 `ServiceRegistry`
|
||||
- [ ] `service_interface.py` 提供统一初始化/关闭函数
|
||||
- [ ] `app.py` 通过统一函数初始化 `redis + supabase`
|
||||
- [ ] Supabase 配置读取仍仅来自 `core.config.settings.config`
|
||||
- [ ] `auth/gateway.py` 不再在 `__init__` 新建客户端
|
||||
- [ ] 初始化失败具备回滚关闭逻辑
|
||||
- [ ] 单元/集成测试覆盖核心迁移路径并通过
|
||||
|
||||
---
|
||||
|
||||
## 7. 风险与缓解
|
||||
|
||||
| 风险 | 级别 | 缓解 |
|
||||
|---|---|---|
|
||||
| 统一初始化函数引入顺序错误 | 中 | 显式 `SERVICE_STARTUP_ORDER` + 顺序测试 |
|
||||
| Supabase 健康检查误报 | 中 | 使用真实轻量请求,不只做对象检查 |
|
||||
| gateway 与生命周期耦合导致运行时错误 | 中 | 改为方法内按需取 client,并覆盖未初始化测试 |
|
||||
| 迁移影响现有 auth 行为 | 中 | 保持 service 接口不变,补充回归测试 |
|
||||
|
||||
---
|
||||
|
||||
## 8. 完成定义(Completion Contract)
|
||||
|
||||
1. Complexity: `S2`
|
||||
2. Risk Tier: `L1`
|
||||
3. Gates:
|
||||
- 必需:`refactor-cleaner`
|
||||
- 可选:`code-reviewer`(建议在合并前执行)
|
||||
4. Verification evidence:
|
||||
- 提供 lint/typecheck/unit/integration 命令结果
|
||||
5. Remaining risks/follow-ups:
|
||||
- 若后续新增第三方服务,沿用 `ServiceRegistry + 统一生命周期函数` 接入,不再在 `app.py` 手写初始化。
|
||||
@@ -0,0 +1,359 @@
|
||||
# Celery To Taskiq One-Shot Migration Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** 在当前早期项目中一次性移除 Celery,并以 Taskiq 替换异步任务基础设施,保持 agent runtime 行为不变。
|
||||
|
||||
**Architecture:** 复用现有 `AgentService -> QueueClientLike` 抽象,仅替换基础设施层实现(任务声明、入队调用、worker 启动、配置与依赖)。保持 Redis 作为 broker/result 存储与事件流通道,避免改动业务服务层语义。
|
||||
|
||||
**Tech Stack:** FastAPI, Taskiq, taskiq-redis, Redis, pytest, uv
|
||||
|
||||
---
|
||||
|
||||
### Task 1: 依赖与配置切换(先 RED 后 GREEN)
|
||||
|
||||
**Files:**
|
||||
- Modify: `pyproject.toml`
|
||||
- Modify: `backend/src/core/config/settings.py`
|
||||
- Test: `backend/tests/unit/core/config/test_taskiq_settings.py` (new)
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
from core.config.settings import Settings
|
||||
|
||||
|
||||
def test_taskiq_uses_redis_url_by_default() -> None:
|
||||
settings = Settings()
|
||||
assert settings.taskiq_broker_url.startswith("redis://")
|
||||
|
||||
|
||||
def test_taskiq_queue_default_value() -> None:
|
||||
settings = Settings()
|
||||
assert settings.taskiq.default_queue == "default"
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/config/test_taskiq_settings.py -v`
|
||||
Expected: FAIL(`taskiq_broker_url` / `taskiq` 字段不存在)
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
class TaskiqSettings(BaseModel):
|
||||
broker_url: str | None = None
|
||||
result_backend_url: str | None = None
|
||||
default_queue: str = "default"
|
||||
|
||||
class Settings(BaseSettings):
|
||||
taskiq: TaskiqSettings = TaskiqSettings()
|
||||
|
||||
@computed_field
|
||||
@property
|
||||
def taskiq_broker_url(self) -> str:
|
||||
return self.taskiq.broker_url or self.redis.url
|
||||
|
||||
@computed_field
|
||||
@property
|
||||
def taskiq_result_backend_url(self) -> str:
|
||||
return self.taskiq.result_backend_url or self.redis.url
|
||||
```
|
||||
|
||||
`pyproject.toml` 同步变更:
|
||||
- 删除 `celery>=...`
|
||||
- 增加 `taskiq>=...`
|
||||
- 增加 `taskiq-redis>=...`
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/config/test_taskiq_settings.py -v`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add pyproject.toml backend/src/core/config/settings.py backend/tests/unit/core/config/test_taskiq_settings.py
|
||||
git commit -m "refactor(queue): replace celery config with taskiq settings"
|
||||
```
|
||||
|
||||
### Task 2: 新建 Taskiq broker 与 worker 启动入口
|
||||
|
||||
**Files:**
|
||||
- Create: `backend/src/core/taskiq/app.py`
|
||||
- Create: `backend/tests/unit/core/taskiq/test_app.py`
|
||||
- Delete: `backend/src/core/celery/app.py`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
from core.taskiq.app import broker
|
||||
|
||||
|
||||
def test_taskiq_broker_is_configured() -> None:
|
||||
assert broker is not None
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/taskiq/test_app.py -v`
|
||||
Expected: FAIL(模块不存在)
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
from taskiq_redis import ListQueueBroker, RedisAsyncResultBackend
|
||||
from core.config.settings import config
|
||||
|
||||
broker = ListQueueBroker(url=config.taskiq_broker_url).with_result_backend(
|
||||
RedisAsyncResultBackend(redis_url=config.taskiq_result_backend_url)
|
||||
)
|
||||
```
|
||||
|
||||
说明:若当前 `taskiq-redis` 版本 API 名称有差异,以该版本官方 API 为准做等价实现。
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/taskiq/test_app.py -v`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add backend/src/core/taskiq/app.py backend/tests/unit/core/taskiq/test_app.py backend/src/core/celery/app.py
|
||||
git commit -m "feat(queue): add taskiq broker app and remove celery app"
|
||||
```
|
||||
|
||||
### Task 3: 迁移任务定义(Celery task -> Taskiq task)
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/src/core/agent/infrastructure/queue/tasks.py`
|
||||
- Test: `backend/tests/unit/core/agent/infrastructure/queue/test_tasks.py` (new)
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
from core.agent.infrastructure.queue.tasks import run_agent_task
|
||||
|
||||
|
||||
async def test_run_agent_task_invalid_command_raises() -> None:
|
||||
try:
|
||||
await run_agent_task({"command": "unknown", "session_id": "00000000-0000-0000-0000-000000000001"})
|
||||
raise AssertionError("expected ValueError")
|
||||
except ValueError as exc:
|
||||
assert "invalid command type" in str(exc)
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/agent/infrastructure/queue/test_tasks.py -v`
|
||||
Expected: FAIL(测试文件不存在或导入失败)
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
from core.taskiq.app import broker
|
||||
|
||||
@broker.task(task_name="tasks.agent.run_command")
|
||||
async def run_command_task(command: dict[str, Any]) -> dict[str, object]:
|
||||
return await run_agent_task(command)
|
||||
```
|
||||
|
||||
并移除:
|
||||
- `from core.celery.app import celery_app`
|
||||
- `@celery_app.task(...)`
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/agent/infrastructure/queue/test_tasks.py -v`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add backend/src/core/agent/infrastructure/queue/tasks.py backend/tests/unit/core/agent/infrastructure/queue/test_tasks.py
|
||||
git commit -m "refactor(agent): migrate run command task to taskiq"
|
||||
```
|
||||
|
||||
### Task 4: 迁移 API 入队客户端(.delay -> .kiq)
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/src/v1/agent/dependencies.py`
|
||||
- Test: `backend/tests/unit/v1/agent/test_dependencies_queue.py` (new)
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
class _FakeTask:
|
||||
async def kiq(self, payload: dict[str, object]):
|
||||
class _Result:
|
||||
task_id = "task-123"
|
||||
return _Result()
|
||||
|
||||
|
||||
async def test_enqueue_returns_task_id(monkeypatch):
|
||||
from v1.agent.dependencies import CeleryQueueClient
|
||||
client = CeleryQueueClient() # 迁移后应重命名为 TaskiqQueueClient
|
||||
monkeypatch.setattr("v1.agent.dependencies.run_command_task", _FakeTask())
|
||||
task_id = await client.enqueue(command={"command": "run"}, dedup_key=None)
|
||||
assert task_id == "task-123"
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/v1/agent/test_dependencies_queue.py -v`
|
||||
Expected: FAIL(类型/方法不匹配)
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
class TaskiqQueueClient:
|
||||
async def enqueue(self, *, command: dict[str, object], dedup_key: str | None) -> str:
|
||||
payload = dict(command)
|
||||
if dedup_key:
|
||||
payload["dedup_key"] = dedup_key
|
||||
|
||||
result = await run_command_task.kiq(payload)
|
||||
task_id = str(result.task_id)
|
||||
return task_id
|
||||
```
|
||||
|
||||
并替换 DI:
|
||||
|
||||
```python
|
||||
queue=TaskiqQueueClient()
|
||||
```
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/v1/agent/test_dependencies_queue.py -v`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add backend/src/v1/agent/dependencies.py backend/tests/unit/v1/agent/test_dependencies_queue.py
|
||||
git commit -m "refactor(api): switch agent enqueue client from celery to taskiq"
|
||||
```
|
||||
|
||||
### Task 5: 运维脚本与日志测试清理(一次性删除 Celery)
|
||||
|
||||
**Files:**
|
||||
- Modify: `infra/scripts/app.sh`
|
||||
- Delete: `backend/tests/unit/test_celery_logging.py`
|
||||
- Modify/Create: `backend/tests/unit/core/logging/test_taskiq_logging.py` (if taskiq logging hook implemented)
|
||||
- Modify: `backend/src/core/logging/__init__.py`(移除 celery logging export)
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
def test_worker_command_uses_taskiq() -> None:
|
||||
content = Path("infra/scripts/app.sh").read_text()
|
||||
assert "uv run taskiq worker" in content
|
||||
assert "uv run celery" not in content
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/logging/test_taskiq_logging.py -v`
|
||||
Expected: FAIL(脚本仍含 celery)
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
`infra/scripts/app.sh` worker 命令替换为 Taskiq worker,例如:
|
||||
|
||||
```bash
|
||||
uv run taskiq worker core.taskiq.app:broker core.agent.infrastructure.queue.tasks
|
||||
```
|
||||
|
||||
删除所有 celery 进程清理匹配:
|
||||
|
||||
```bash
|
||||
pgrep -f "taskiq.*worker"
|
||||
pkill -f "taskiq.*worker"
|
||||
```
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/logging/test_taskiq_logging.py -v`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add infra/scripts/app.sh backend/src/core/logging/__init__.py backend/tests/unit/core/logging/test_taskiq_logging.py backend/tests/unit/test_celery_logging.py
|
||||
git commit -m "chore(infra): replace celery worker scripts and remove celery-specific tests"
|
||||
```
|
||||
|
||||
### Task 6: 全量引用清理与回归验证
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/runtime/runtime-runbook.md`
|
||||
- Modify: 其他引用 Celery 的运行文档(按 `rg` 结果逐个更新)
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# 用命令断言替代代码测试
|
||||
# rg -n "celery" backend/src infra/scripts docs/runtime pyproject.toml
|
||||
```
|
||||
|
||||
**Step 2: Run check to verify it fails**
|
||||
|
||||
Run: `rg -n "celery" backend/src infra/scripts docs/runtime pyproject.toml`
|
||||
Expected: 仍有旧引用
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- 删除/替换剩余 Celery 代码、文档、配置。
|
||||
- 保留历史变更记录中的 Celery 字样(如 bugs 归档)可接受,但运行路径必须为 0 引用。
|
||||
|
||||
**Step 4: Run verification suite**
|
||||
|
||||
Run:
|
||||
- `uv run pytest backend/tests/unit -q`
|
||||
- `uv run pytest backend/tests/integration -q`
|
||||
- `uv run pytest backend/tests/e2e -q`(如环境不满足,记录原因)
|
||||
- `uv run ruff check backend/src backend/tests`
|
||||
- `uv run basedpyright`
|
||||
- `rg -n "celery" backend/src infra/scripts pyproject.toml`
|
||||
|
||||
Expected:
|
||||
- 测试与静态检查通过
|
||||
- 运行路径无 Celery 引用
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/runtime/runtime-runbook.md pyproject.toml backend/src infra/scripts backend/tests
|
||||
git commit -m "refactor(queue): complete one-shot migration from celery to taskiq"
|
||||
```
|
||||
|
||||
### Task 7: L1 Review Gates 与交付确认
|
||||
|
||||
**Files:**
|
||||
- No code changes required by default
|
||||
|
||||
**Step 1: Run required L1 gate (`refactor-cleaner`)**
|
||||
|
||||
Run: 使用 `refactor-cleaner` 审查迁移后冗余代码、死引用、命名一致性。
|
||||
Expected: 无阻断问题。
|
||||
|
||||
**Step 2: Optional `code-reviewer` (recommended for infra switch)**
|
||||
|
||||
Run: 使用 `code-reviewer` 聚焦任务丢失、重复消费、幂等锁逻辑。
|
||||
Expected: 无 CRITICAL/HIGH 问题。
|
||||
|
||||
**Step 3: Final evidence report**
|
||||
|
||||
输出内容必须包含:
|
||||
- 执行命令列表
|
||||
- 每条命令 PASS/FAIL
|
||||
- 若有无法执行项(如 e2e 环境),给出原因与人工验证步骤
|
||||
|
||||
**Step 4: Commit review notes (optional)**
|
||||
|
||||
```bash
|
||||
git add docs/plans/2026-03-06-taskiq-migration.md
|
||||
git commit -m "docs(plan): taskiq one-shot migration execution checklist"
|
||||
```
|
||||
@@ -1,144 +0,0 @@
|
||||
# Agent LLM Config Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** 将 `system_agents.config` 中的 `temperature` / `max_tokens` 以受约束方式加载到运行时,并在调用 LiteLLM 时按需透传。
|
||||
|
||||
**Architecture:** 在应用层 `RunService` 读取模型选择时同步读取并校验 `SystemAgents.config`;将校验后的 `SystemAgentLLMConfig` 传入 `CrewAIRuntime`;由 runtime 将配置转交给 LiteLLM client,client 仅在值非 `None` 时向 `completion()` 传参,避免不必要的 provider 兼容风险。
|
||||
|
||||
**Tech Stack:** FastAPI, SQLAlchemy (async), Pydantic v2, LiteLLM, pytest
|
||||
|
||||
---
|
||||
|
||||
## 背景与修正点
|
||||
|
||||
- 当前真实调用链为:`RunService._load_agent_model_selection()` -> `create_runtime()` -> `CrewAIRuntime.execute()` -> `run_completion()`,并非 `load_stage_models()`。
|
||||
- `SystemAgentLLMConfig` 已存在:`backend/src/core/agent/domain/system_agent_config.py`。
|
||||
- `system_agents.config` 目前在初始化 YAML 侧有约束,但运行时 DB 读取仍需二次校验,防止脏数据绕过。
|
||||
|
||||
## 规则约束
|
||||
|
||||
- 严格 TDD:先写失败测试,再做实现。
|
||||
- Python 命令统一使用 `uv run ...`。
|
||||
- 仅做增量改动,不回滚或覆盖与本任务无关的已有变更。
|
||||
|
||||
## 字段映射与透传策略
|
||||
|
||||
| 配置字段 | LiteLLM 参数 | 规则 |
|
||||
|---|---|---|
|
||||
| `temperature` | `temperature` | `None` 不透传;非空直接透传 |
|
||||
| `max_tokens` | `max_tokens` | `None` 不透传;非空直接透传 |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: 应用层加载并校验 Agent LLM Config
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/src/core/agent/application/run_service.py`
|
||||
- Test: `backend/tests/unit/core/agent/test_run_resume_service.py`
|
||||
|
||||
**Step 1: 写失败测试(RED)**
|
||||
|
||||
新增单测覆盖以下行为:
|
||||
1. `_load_agent_model_selection()` 返回三元组:`(model_code, provider_name, llm_config)`。
|
||||
2. 当 DB `config` 为 `{}` 时,`llm_config.temperature/max_tokens` 为 `None`。
|
||||
3. 当 DB `config` 含非法值(如 `temperature=3`)时抛 `ValueError`。
|
||||
|
||||
**Step 2: 运行测试确认失败**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/agent/test_run_resume_service.py -q`
|
||||
Expected: 新增断言失败(返回值结构/异常行为不匹配)。
|
||||
|
||||
**Step 3: 最小实现(GREEN)**
|
||||
|
||||
在 `run_service.py`:
|
||||
1. 查询 `SystemAgents.config`。
|
||||
2. 用 `SystemAgentLLMConfig.model_validate(config or {})` 校验。
|
||||
3. 将 `_load_agent_model_selection()` 改为返回三元组。
|
||||
4. 在 `run()` 中把 `llm_config` 传递到 `create_runtime(...)`。
|
||||
|
||||
**Step 4: 运行测试确认通过**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/agent/test_run_resume_service.py -q`
|
||||
Expected: PASS。
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Runtime 与 LiteLLM Client 支持可选参数透传
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/src/core/agent/infrastructure/crewai/factory.py`
|
||||
- Modify: `backend/src/core/agent/infrastructure/crewai/runtime.py`
|
||||
- Modify: `backend/src/core/agent/infrastructure/litellm/client.py`
|
||||
- Test: `backend/tests/unit/core/agent/test_crewai_runtime.py`
|
||||
|
||||
**Step 1: 写失败测试(RED)**
|
||||
|
||||
在 `test_crewai_runtime.py` 增加用例:
|
||||
1. 传入 `temperature/max_tokens` 时,`run_completion` 收到对应参数。
|
||||
2. 参数为 `None` 时,不应被透传到 LiteLLM。
|
||||
|
||||
必要时新增 `backend/tests/unit/core/agent/test_litellm_client.py`,单测 `run_completion` 的 kwargs 组装逻辑。
|
||||
|
||||
**Step 2: 运行测试确认失败**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/agent/test_crewai_runtime.py -q`
|
||||
Expected: 新增断言失败(参数未透传或未过滤 `None`)。
|
||||
|
||||
**Step 3: 最小实现(GREEN)**
|
||||
|
||||
1. `create_runtime()` 增加 `llm_config` 参数并传给 `CrewAIRuntime`。
|
||||
2. `CrewAIRuntime` 保存 `llm_config`,执行时调用:
|
||||
- `run_completion(..., temperature=llm_config.temperature, max_tokens=llm_config.max_tokens)`
|
||||
3. `run_completion()` 改为支持可选 `temperature/max_tokens`,内部仅在非 `None` 时加入 kwargs 再调用 `completion()`。
|
||||
|
||||
**Step 4: 运行测试确认通过**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/agent/test_crewai_runtime.py -q`
|
||||
Expected: PASS。
|
||||
|
||||
---
|
||||
|
||||
### Task 3: 初始化数据补齐与回归验证
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/src/core/config/static/database/system_agents.yaml`
|
||||
- Modify: `backend/src/core/config/initial/init_data.py`(如需补充类型兜底)
|
||||
- Test: `backend/tests/unit/core/agent/test_run_resume_service.py`
|
||||
|
||||
**Step 1: 写失败测试(RED)**
|
||||
|
||||
补充断言:YAML 读取后 `config` 可为空或包含 `max_tokens: null`,初始化逻辑不会报错,且生成结构符合 `SystemAgentLLMConfig`。
|
||||
|
||||
**Step 2: 运行测试确认失败**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/agent/test_run_resume_service.py -q`
|
||||
Expected: 新增断言失败。
|
||||
|
||||
**Step 3: 最小实现(GREEN)**
|
||||
|
||||
1. 在 `system_agents.yaml` 为各 agent 配置显式补充 `max_tokens: null`。
|
||||
2. `init_data.py` 保持 `config: SystemAgentLLMConfig | None = None`,写库时统一序列化为 dict。
|
||||
|
||||
**Step 4: 运行测试确认通过**
|
||||
|
||||
Run: `uv run pytest backend/tests/unit/core/agent/test_run_resume_service.py -q`
|
||||
Expected: PASS。
|
||||
|
||||
---
|
||||
|
||||
## 最终验证
|
||||
|
||||
1. `uv run pytest backend/tests/unit/core/agent/test_run_resume_service.py backend/tests/unit/core/agent/test_crewai_runtime.py -q`
|
||||
2. `uv run pytest backend/tests/integration/core/agent/test_queue_run_resume.py -q`
|
||||
3. `uv run ruff check backend/src backend/tests`
|
||||
4. `uv run basedpyright`
|
||||
|
||||
预期:全部通过;若集成测试依赖本地 DB 状态导致跳过/失败,需记录原因并给出手工验证步骤。
|
||||
|
||||
## 完成标准
|
||||
|
||||
- `RunService` 从 DB 读取并校验 `config`。
|
||||
- runtime 到 LiteLLM 链路支持 `temperature/max_tokens` 可选透传。
|
||||
- `None` 不透传。
|
||||
- 单测与相关集成测试通过,并给出命令级证据。
|
||||
@@ -69,7 +69,7 @@ docker compose --env-file .env -f infra/docker/docker-compose.yml exec -T db \
|
||||
### 启动应用进程
|
||||
|
||||
```bash
|
||||
bash infra/scripts/app-up.sh
|
||||
bash infra/scripts/app.sh start
|
||||
```
|
||||
|
||||
该脚本会在 tmux `social-dev` 会话中拉起:
|
||||
@@ -172,6 +172,7 @@ curl -sS "${WEB_BASE_URL}/api/v1/profile/me" \
|
||||
- 症状:队列堆积,任务长时间 pending。
|
||||
- 定位:检查 `worker-*` tmux 窗口和对应日志文件。
|
||||
- 修复:重启 tmux 会话,确认并发配置与队列名(critical/default/bulk)。
|
||||
- 说明:Taskiq 路径当前仅消费 `SOCIAL_WORKER__GROUPS__*__CONCURRENCY`,旧 Celery 参数(prefetch/time_limit 等)已废弃。
|
||||
|
||||
### 2.1) Agent Runtime run/resume 事件不闭环
|
||||
|
||||
@@ -179,7 +180,7 @@ curl -sS "${WEB_BASE_URL}/api/v1/profile/me" \
|
||||
- 定位步骤:
|
||||
|
||||
```bash
|
||||
# 1) 检查 celery worker 是否消费 agent 任务
|
||||
# 1) 检查 taskiq worker 是否消费 agent 任务
|
||||
grep -E "tasks\.agent\.run_command|RUN_STARTED|RUN_FINISHED|RUN_ERROR" logs/worker-default.log
|
||||
|
||||
# 2) 检查 API SSE 事件读取(带 Last-Event-ID)
|
||||
@@ -192,7 +193,7 @@ docker compose --env-file .env -f infra/docker/docker-compose.yml exec -T redis
|
||||
```
|
||||
|
||||
- 修复建议:
|
||||
- 若 worker 无消费:重启 `worker-default` 窗口并确认 `core.agent.infrastructure.queue.tasks` 已被 Celery include。
|
||||
- 若 worker 无消费:重启 `worker-default` 窗口并确认 `core.agent.infrastructure.queue.tasks` 已被 Taskiq worker 加载。
|
||||
- 若 worker 有事件但 API 无输出:排查 Redis stream 前缀配置与 session_id 是否一致。
|
||||
- 若出现 `RUN_ERROR`:按 error_id 回查后端日志,不在 API/SSE 中暴露敏感上下文。
|
||||
|
||||
@@ -270,4 +271,4 @@ docker compose --env-file .env -f infra/docker/docker-compose.yml up -d --force-
|
||||
| 2026-02-28 | 邀请码功能:新增 invite_codes 表、profiles.referred_by,注册时可选填邀请码并记录邀请关系 |
|
||||
| 2026-03-02 | 文档整理:修正 auth 端点名称(/verifications)、补充 profile 路由文档、修复 L2/L3 验证命令 |
|
||||
| 2026-03-02 | 修正 bootstrap 命令:init-job 需要使用 `uv run python -m core.runtime.cli bootstrap` |
|
||||
| 2026-03-05 | 新增 Agent Runtime run/resume/events 运维排障流程(Celery + Redis + Last-Event-ID) |
|
||||
| 2026-03-05 | 新增 Agent Runtime run/resume/events 运维排障流程(Taskiq + Redis + Last-Event-ID) |
|
||||
|
||||
Reference in New Issue
Block a user