refactor: align backend layout and supabase infra
Consolidate backend modules/tests under the backend package while syncing Supabase compose/env config and related plans.
This commit is contained in:
@@ -0,0 +1,108 @@
|
||||
# Plan: Base Service for Redis and Qdrant
|
||||
|
||||
**Date:** 2026-02-05
|
||||
**Author:** AI Assistant
|
||||
**Status:** Draft
|
||||
|
||||
## Overview
|
||||
|
||||
Create a reusable base service module under `backend/src/services/base` that standardizes Redis and Qdrant client creation, lifecycle management, and error handling. Align the design with the DIVA-backend equivalent (once provided) and integrate configuration through existing `SOCIAL_REDIS__*` and `SOCIAL_QDRANT__*` settings.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional
|
||||
- [ ] Provide a base service abstraction that exposes Redis and Qdrant clients to other services.
|
||||
- [ ] Use async client implementations compatible with FastAPI async execution.
|
||||
- [ ] Support connection lifecycle hooks (initialize, health check, close).
|
||||
- [ ] Centralize error handling and translate connection failures to consistent HTTP errors.
|
||||
- [ ] Mirror DIVA-backend base service features and naming conventions where applicable.
|
||||
|
||||
### Non-Functional
|
||||
- [ ] Performance: reuse client instances; avoid per-request connection creation.
|
||||
- [ ] Security: never log secrets (API keys/passwords); enforce TLS settings when enabled.
|
||||
- [ ] Reliability: implement timeouts and retry policy where supported by client libraries.
|
||||
|
||||
## Technical Approach
|
||||
|
||||
Introduce a `services/base` package that provides a small, composable base class plus Redis/Qdrant client factories. Configuration will be sourced from `core/config/settings.py` using the existing `.env` keys. The base service will accept injected clients to keep testability high and avoid global state, while a module-level factory will handle creation and cleanup.
|
||||
|
||||
### Key Decisions
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| Use async Redis and Qdrant clients | Matches FastAPI async usage and avoids blocking the event loop. |
|
||||
| Constructor injection with factories | Keeps services testable and avoids hidden global state. |
|
||||
| Centralized error mapping in base service | Ensures consistent HTTP 503 responses and logging. |
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: DIVA-backend Parity Review (1-2 hours)
|
||||
1. Locate DIVA-backend base service module (path or repo) and document its responsibilities, public API, and lifecycle behavior.
|
||||
2. Produce a parity checklist to map DIVA behaviors to this repo (naming, error types, retry policy, health checks).
|
||||
|
||||
### Phase 2: Configuration and Client Factories (3 hours)
|
||||
1. Add `RedisSettings` and `QdrantSettings` sections to `backend/src/core/config/settings.py` using existing `SOCIAL_REDIS__*` and `SOCIAL_QDRANT__*` env keys.
|
||||
2. Create `backend/src/services/base/redis_client.py` and `backend/src/services/base/qdrant_client.py` with async client factory functions and close helpers.
|
||||
3. Add structured logging for client initialization, connection failures, and shutdown paths.
|
||||
|
||||
### Phase 3: Base Service Class (3 hours)
|
||||
1. Create `backend/src/services/base/service.py` with a `BaseService` that accepts optional Redis/Qdrant clients (dependency injection).
|
||||
2. Add helper methods (e.g., `require_redis()`, `require_qdrant()`) that raise HTTP 503 on unavailable clients.
|
||||
3. Define error translation utilities for Redis/Qdrant exceptions with consistent messages and logging.
|
||||
|
||||
### Phase 4: Tests (TDD) and Minimal Integration (4 hours)
|
||||
1. Unit tests for settings parsing and default values (RED/GREEN).
|
||||
2. Unit tests for base service behavior: missing client errors, exception mapping, and logging context.
|
||||
3. Integration tests using running Redis/Qdrant containers to verify client factories can connect and execute a simple command.
|
||||
4. E2E test that exercises a minimal endpoint using the base service (e.g., `/health/infra`), or record an explicit exception if no API integration is allowed.
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| backend/src/core/config/settings.py | Add Redis/Qdrant settings models and defaults. |
|
||||
| backend/src/app.py | (If needed) register startup/shutdown hooks for client lifecycle. |
|
||||
| backend/src/v1/router.py | (If needed) add an infra health endpoint to support E2E. |
|
||||
|
||||
## Files to Create
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| backend/src/services/base/__init__.py | Package export surface for base services. |
|
||||
| backend/src/services/base/service.py | Base service class for Redis/Qdrant access. |
|
||||
| backend/src/services/base/redis_client.py | Redis client factory and teardown helpers. |
|
||||
| backend/src/services/base/qdrant_client.py | Qdrant client factory and teardown helpers. |
|
||||
| backend/tests/unit/services/base/test_service.py | Unit tests for base service error handling. |
|
||||
| backend/tests/unit/services/base/test_clients.py | Unit tests for client factory behavior. |
|
||||
| backend/tests/integration/services/base/test_clients.py | Integration tests with Redis/Qdrant containers. |
|
||||
| backend/tests/e2e/test_infra_health.py | E2E test for an endpoint using base service. |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- [ ] `redis` (async client) for Redis connectivity.
|
||||
- [ ] `qdrant-client` for Qdrant connectivity (async/GRPC as configured).
|
||||
- [ ] No additional infra services required (Redis/Qdrant already in Docker compose).
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit Tests:** Base service behavior, missing client errors, exception translation, settings parsing.
|
||||
- **Integration Tests:** Connect to Redis and Qdrant, run minimal ping/health operations.
|
||||
- **E2E Tests:** Call a minimal endpoint that uses the base service to validate wiring and error handling.
|
||||
|
||||
## Risks & Mitigations
|
||||
|
||||
| Risk | Impact | Likelihood | Mitigation |
|
||||
|------|--------|------------|------------|
|
||||
| DIVA-backend module not available | Medium | High | Add a parity checklist and update plan once module location is provided. |
|
||||
| Client library mismatch (sync vs async) | Medium | Medium | Select async-supported libraries and verify compatibility in unit tests. |
|
||||
| Lack of API integration for E2E | High | Medium | Add a minimal infra health endpoint or record a documented exception. |
|
||||
| Connection config mismatches | Medium | Medium | Validate settings with integration tests and mirror `.env.example`. |
|
||||
|
||||
## Estimated Effort
|
||||
|
||||
| Phase | Effort |
|
||||
|-------|--------|
|
||||
| Phase 1 | 2 hours |
|
||||
| Phase 2 | 3 hours |
|
||||
| Phase 3 | 3 hours |
|
||||
| Phase 4 | 4 hours |
|
||||
| **Total** | **12 hours** |
|
||||
@@ -0,0 +1,143 @@
|
||||
# Plan: Env Config Refactor
|
||||
|
||||
**Date:** 2026-02-05
|
||||
**Author:** AI Assistant
|
||||
**Status:** Draft
|
||||
|
||||
## Overview
|
||||
|
||||
对 `.env` / `.env.example` 与 `backend/src/core/config/settings.py` 做一次一致性重构,消除同一含义的重复配置来源(例如 `DATABASE_URL` 与分段口令、host/port 与完整 URL)。目标是明确一组规范化环境变量,确保后端仅通过 Settings 读取,并兼顾 `infra/docker/docker-compose.yml` 的现有依赖。
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional
|
||||
- [ ] 提出“规范化环境变量”清单(canonical set),覆盖后端与 Supabase 本地栈的关键配置。
|
||||
- [ ] 定义 Settings 的读取与推导策略(优先级、默认值、派生字段)。
|
||||
- [ ] 给出 `.env` / `.env.example` 的迁移步骤与兼容策略。
|
||||
- [ ] 兼容 `infra/docker/docker-compose.yml` 使用的变量(保证 compose 不被破坏)。
|
||||
|
||||
### Non-Functional
|
||||
- [ ] Performance: Settings 解析不增加明显启动耗时
|
||||
- [ ] Security: 不在仓库中暴露真实密钥;对后端使用的数据库 URL 与密钥来源保持单一可信源
|
||||
|
||||
## Technical Approach
|
||||
|
||||
以“后端设置单一来源 + docker-compose 继续使用 Supabase 变量”为原则:
|
||||
- 后端只接受 `SOCIAL_DATABASE_URL` 与必要的 Supabase 访问变量(`public_url/anon_key/service_role_key/jwt_secret`)。
|
||||
- Supabase stack 继续使用 `SOCIAL_SUPABASE__*` 变量,保持 compose 模板稳定。
|
||||
- 通过 Settings 做派生字段(例如 `supabase.url`)与兼容性读入(可选旧字段,设置弃用期)。
|
||||
|
||||
### Key Decisions
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| 保留 `SOCIAL_DATABASE_URL` 作为后端唯一数据库连接来源 | 避免与分段 `POSTGRES_*` 产生冲突,清晰配置入口 |
|
||||
| Supabase stack 变量继续使用 `SOCIAL_SUPABASE__*` | docker-compose 已广泛引用,改动成本高 |
|
||||
| Settings 允许短期兼容旧字段 | 保障迁移期间部署安全,减少切换风险 |
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Inventory & Mapping (2 hours)
|
||||
1. 盘点 `.env` / `.env.example` 与 `settings.py` 的变量差异,标注重复与冲突字段。
|
||||
2. 输出 canonical env vars 列表与映射关系表(旧 -> 新)。
|
||||
|
||||
### Phase 2: Settings Refactor (3 hours)
|
||||
1. 在 `settings.py` 中实现新的读取优先级与派生字段。
|
||||
2. 为旧字段加兼容读取与弃用注记(仅内存兼容,不继续写入)。
|
||||
|
||||
### Phase 3: Env Templates Update (2 hours)
|
||||
1. 更新 `.env.example` 为 canonical 变量,并标注“后端使用/compose 使用”。
|
||||
2. 更新 `.env`(本地开发用)以匹配新模板。
|
||||
|
||||
### Phase 4: Validation & Docs (2 hours)
|
||||
1. 本地启动 docker-compose,验证 Supabase stack 与后端连接正常。
|
||||
2. 写简要迁移说明(README 或 docs/ 中短节)。
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `.env.example` | 替换为 canonical 变量,移除重复字段 |
|
||||
| `.env` | 与模板对齐,移除重复字段 |
|
||||
| `backend/src/core/config/settings.py` | 调整 Settings 读取与派生策略 |
|
||||
| `infra/docker/docker-compose.yml` | 仅在必要时新增兼容映射变量 |
|
||||
| `README.md` 或 `docs/*` | 增加迁移说明 |
|
||||
|
||||
## Files to Create
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `docs/plans/PLAN-env-config-refactor-2026-02-05.md` | 规划文档 |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- [ ] 无新增第三方依赖
|
||||
|
||||
## Proposed Canonical Env Vars
|
||||
|
||||
### Backend (Settings)
|
||||
- `SOCIAL_DATABASE_URL` (required) — 后端数据库连接(唯一来源)
|
||||
- `SOCIAL_SUPABASE__PUBLIC_URL` — Supabase 公网/本地外部访问 URL
|
||||
- `SOCIAL_SUPABASE__API_EXTERNAL_URL` — Supabase Auth 回调外部 URL
|
||||
- `SOCIAL_SUPABASE__ANON_KEY` — 前端/匿名访问 key
|
||||
- `SOCIAL_SUPABASE__SERVICE_ROLE_KEY` — 后端服务角色 key
|
||||
- `SOCIAL_SUPABASE__JWT_SECRET` — JWT 验证密钥(后端验证用)
|
||||
|
||||
### Supabase Stack (docker-compose)
|
||||
- `SOCIAL_SUPABASE__POSTGRES_HOST`
|
||||
- `SOCIAL_SUPABASE__POSTGRES_PORT`
|
||||
- `SOCIAL_SUPABASE__POSTGRES_DB`
|
||||
- `SOCIAL_SUPABASE__POSTGRES_PASSWORD`
|
||||
- `SOCIAL_SUPABASE__KONG_HTTP_PORT`
|
||||
- `SOCIAL_SUPABASE__KONG_HTTPS_PORT`
|
||||
- `SOCIAL_SUPABASE__SITE_URL`
|
||||
- `SOCIAL_SUPABASE__JWT_SECRET`
|
||||
- `SOCIAL_SUPABASE__ANON_KEY`
|
||||
- `SOCIAL_SUPABASE__SERVICE_ROLE_KEY`
|
||||
- 其余 `SOCIAL_SUPABASE__*` 保持现状(Logflare、SMTP、Pooler 等)
|
||||
|
||||
## Mapping Strategy
|
||||
|
||||
1. **Backend DB**
|
||||
- Only: `SOCIAL_DATABASE_URL`
|
||||
- Deprecated: `SOCIAL_SUPABASE__POSTGRES_*`(后端不再拼接)
|
||||
|
||||
2. **Supabase URL**
|
||||
- Primary: `SOCIAL_SUPABASE__PUBLIC_URL`
|
||||
- Fallback: `SOCIAL_SUPABASE__API_EXTERNAL_URL`
|
||||
- Settings 中 `supabase.url` 由上述字段派生
|
||||
|
||||
3. **Docker Compose**
|
||||
- 继续读 `SOCIAL_SUPABASE__POSTGRES_*`
|
||||
- 不引入 `SOCIAL_DATABASE_URL` 到 compose,以免混淆职责
|
||||
|
||||
## Migration Steps
|
||||
|
||||
1. 在 `settings.py` 中增加兼容逻辑:若 `SOCIAL_DATABASE_URL` 不存在,可临时从 `SOCIAL_SUPABASE__POSTGRES_*` 组装(同时记录弃用)。
|
||||
2. 更新 `.env.example`:只保留 canonical 变量并标注用途。
|
||||
3. 更新 `.env`:移除重复字段,确保本地后端使用 `SOCIAL_DATABASE_URL`。
|
||||
4. 校验 compose:`infra/docker/docker-compose.yml` 不依赖被移除字段。
|
||||
5. 发布说明:提示下游用户迁移并在下个版本移除兼容逻辑。
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit Tests:** Settings 派生字段与优先级规则
|
||||
- **Integration Tests:** 后端连接 Supabase DB(使用 `SOCIAL_DATABASE_URL`)
|
||||
- **E2E Tests:** 关键登录/读写流程(确认 JWT 与 Service Role 配置无误)
|
||||
|
||||
## Risks & Mitigations
|
||||
|
||||
| Risk | Impact | Likelihood | Mitigation |
|
||||
|------|--------|------------|------------|
|
||||
| compose 变量被误删导致 Supabase 启动失败 | High | Medium | 迁移前后对照 `docker-compose.yml`,保留全部 `SOCIAL_SUPABASE__*` 依赖 |
|
||||
| 后端数据库连接断开 | High | Medium | 兼容旧字段,先引入新变量再切换 |
|
||||
| 开发环境 `.env` 未更新 | Medium | High | 更新模板并在 README 明确迁移步骤 |
|
||||
|
||||
## Estimated Effort
|
||||
|
||||
| Phase | Effort |
|
||||
|-------|--------|
|
||||
| Phase 1 | 2 hours |
|
||||
| Phase 2 | 3 hours |
|
||||
| Phase 3 | 2 hours |
|
||||
| Phase 4 | 2 hours |
|
||||
| **Total** | **9 hours** |
|
||||
@@ -74,21 +74,21 @@
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| api/src/core/config/settings.py | 扩展日志相关配置模型 |
|
||||
| backend/src/core/config/settings.py | 扩展日志相关配置模型 |
|
||||
|
||||
## Files to Create
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| api/src/core/logging/__init__.py | 模块导出与初始化入口 |
|
||||
| api/src/core/logging/config.py | dictConfig 构建与环境配置 |
|
||||
| api/src/core/logging/formatters.py | JSON formatter 与字段规范 |
|
||||
| api/src/core/logging/handlers.py | 文件、控制台、错误 handler |
|
||||
| api/src/core/logging/filters.py | 等级过滤、敏感字段脱敏 |
|
||||
| api/src/core/logging/context.py | contextvars 绑定与获取 |
|
||||
| api/src/core/logging/middleware.py | FastAPI 请求中间件 |
|
||||
| api/src/core/logging/celery.py | Celery 日志信号集成 |
|
||||
| api/src/core/logging/examples.py | 使用示例(可选) |
|
||||
| backend/src/core/logging/__init__.py | 模块导出与初始化入口 |
|
||||
| backend/src/core/logging/config.py | dictConfig 构建与环境配置 |
|
||||
| backend/src/core/logging/formatters.py | JSON formatter 与字段规范 |
|
||||
| backend/src/core/logging/handlers.py | 文件、控制台、错误 handler |
|
||||
| backend/src/core/logging/filters.py | 等级过滤、敏感字段脱敏 |
|
||||
| backend/src/core/logging/context.py | contextvars 绑定与获取 |
|
||||
| backend/src/core/logging/middleware.py | FastAPI 请求中间件 |
|
||||
| backend/src/core/logging/celery.py | Celery 日志信号集成 |
|
||||
| backend/src/core/logging/examples.py | 使用示例(可选) |
|
||||
|
||||
## Dependencies
|
||||
|
||||
@@ -126,6 +126,35 @@ logger.info("user login", extra={"user_id": "u_123"})
|
||||
- **Integration Tests:** FastAPI 中间件注入的 request_id 与错误分离写入
|
||||
- **E2E Tests:** 关键流程触发错误,验证 error 日志输出与轮转
|
||||
|
||||
## Test Database 约定
|
||||
|
||||
### Supabase 组件能力范围
|
||||
|
||||
- Supabase 的 Auth/Storage/Realtime 等组件是独立服务,默认指向同一个主数据库。
|
||||
- 单独创建一个“测试数据库”(Postgres database)并不会自动获得这些组件的能力,除非显式为这些服务配置新的数据库连接。
|
||||
- 因此,“测试数据库”默认只具备纯 Postgres 能力;Supabase 组件能力仍然作用在主数据库上。
|
||||
|
||||
### 对测试的影响
|
||||
|
||||
- **只走直连数据库的测试**(如通过 SQLAlchemy/psycopg 直连)不会受影响。
|
||||
- **依赖 Supabase 组件的测试**(例如通过 Auth/Storage/Realtime API)会仍然落到主数据库,可能导致:
|
||||
- 测试数据污染主库
|
||||
- 并发测试互相干扰
|
||||
- 若需要 Supabase 组件也“指向测试数据库”,需要启动一套独立 Supabase 栈或重新配置各服务连接(通常不建议在同一栈内动态切换)。
|
||||
|
||||
### 环境变量与自动创建
|
||||
|
||||
- 建议为“测试数据库”提供独立环境变量(仅测试环境读取),例如:
|
||||
- `SOCIAL_TEST_DATABASE__HOST`
|
||||
- `SOCIAL_TEST_DATABASE__PORT`
|
||||
- `SOCIAL_TEST_DATABASE__NAME`
|
||||
- `SOCIAL_TEST_DATABASE__USER`
|
||||
- `SOCIAL_TEST_DATABASE__PASSWORD`
|
||||
- 若使用 Docker 启动 Postgres,建议在容器初始化阶段自动创建测试数据库(避免手动创建):
|
||||
- 通过 `docker-entrypoint-initdb.d` 的 init SQL 脚本创建测试数据库与权限
|
||||
- 保证容器重建后自动恢复测试数据库
|
||||
- 若使用独立 Supabase 栈做测试,测试环境变量应指向该栈的数据库与服务端口。
|
||||
|
||||
## Risks & Mitigations
|
||||
|
||||
| Risk | Impact | Likelihood | Mitigation |
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
# Plan: Merge Supabase Compose and Base Services
|
||||
|
||||
**Date:** 2026-02-05
|
||||
**Author:** AI Assistant
|
||||
**Status:** Draft
|
||||
|
||||
## Overview
|
||||
|
||||
Integrate Supabase Docker services into the project's `infra/docker/docker-compose.yml` and align all environment variables with the project's `.env` conventions. Add reusable BaseRepository and BaseService abstractions (soft-delete filtering and auth/user validation) and refactor profile/auth services to use them, with full TDD coverage.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional
|
||||
- [ ] Merge Supabase Docker Compose services into `infra/docker/docker-compose.yml` using project `.env` variable names.
|
||||
- [ ] Update `.env.example` to include all required Supabase compose variables.
|
||||
- [ ] Implement BaseRepository with standard soft-delete filtering (excludes `deleted_at` rows by default).
|
||||
- [ ] Implement BaseService with shared auth/user validation helpers.
|
||||
- [ ] Refactor profile repository/service and auth service to use BaseRepository/BaseService.
|
||||
- [ ] Add unit, integration, and E2E tests following TDD.
|
||||
|
||||
### Non-Functional
|
||||
- [ ] Performance: keep repository queries indexed and avoid extra round-trips.
|
||||
- [ ] Security: validate user identity consistently; no secrets in repo; no bypass of auth checks.
|
||||
- [ ] Compatibility: keep Supabase config compatible with existing `Settings` and `.env` prefixes.
|
||||
|
||||
## Technical Approach
|
||||
|
||||
Introduce small, reusable base classes in `backend/src/core` for repository and service concerns, then refactor profile and auth modules to leverage them. Merge the Supabase compose services from the official template into `infra/docker/docker-compose.yml`, mapping variables to `SOCIAL_SUPABASE__*` and related infra keys already used in `backend/src/core/config/settings.py`.
|
||||
|
||||
### Key Decisions
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| BaseRepository provides a `base_select()` or `apply_soft_delete_filter()` | Avoid duplicated `deleted_at` filters and enforce consistent behavior. |
|
||||
| BaseService handles user validation helpers | Keeps auth checks consistent across services and reduces duplicated error handling. |
|
||||
| Compose variables aligned to `SOCIAL_*` prefixes | Matches existing settings resolution and simplifies local/dev parity. |
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Compose Merge and Env Alignment (3 hours)
|
||||
1. Identify the Supabase Docker Compose template to merge (official Supabase Docker template) and list required services and env vars.
|
||||
2. Merge Supabase services into `infra/docker/docker-compose.yml`, keeping existing Redis/Qdrant services intact and aligning ports/volumes.
|
||||
3. Map Supabase compose env variables to project `.env` names (e.g., `SOCIAL_SUPABASE__*`, `SOCIAL_INFRA__*` where needed).
|
||||
4. Update `.env.example` with all required Supabase-related variables, keeping comments updated for local vs. cloud usage.
|
||||
5. Add/adjust docker compose healthchecks or depends_on as needed for startup ordering.
|
||||
|
||||
### Phase 2: BaseRepository and BaseService (4 hours)
|
||||
1. Add `backend/src/core/db/repository.py` (or `backend/src/core/repository/base.py`) with a BaseRepository that applies `SoftDeleteMixin` filters by default.
|
||||
2. Add `backend/src/core/services/base.py` with BaseService helpers for current user validation (e.g., `require_user`, `require_user_id`).
|
||||
3. Add unit tests for BaseRepository soft delete filtering and BaseService auth validation (TDD red/green).
|
||||
|
||||
### Phase 3: Refactor Profile/Auth (4 hours)
|
||||
1. Refactor `backend/src/v1/profile/repository.py` to inherit from BaseRepository and remove duplicated `deleted_at` logic.
|
||||
2. Refactor `backend/src/v1/profile/service.py` to inherit from BaseService and use shared validation helpers where applicable.
|
||||
3. Refactor `backend/src/v1/auth/service.py` to adopt BaseService helpers for user validation (where applicable) and keep gateway contract unchanged.
|
||||
4. Update unit tests for profile and auth services to reflect base class usage and ensure behavior unchanged.
|
||||
|
||||
### Phase 4: Integration/E2E Tests and Hardening (4 hours)
|
||||
1. Add integration tests for repository soft delete behavior using SQLAlchemy session fixtures.
|
||||
2. Add or update E2E tests for profile flow to ensure auth/user validation still enforced.
|
||||
3. Run coverage check (80%+), fix gaps, and verify CI pre-commit tooling passes.
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| infra/docker/docker-compose.yml | Merge Supabase services; map env vars to `SOCIAL_*`. |
|
||||
| .env.example | Add Supabase compose variables and update comments. |
|
||||
| backend/src/v1/profile/repository.py | Inherit BaseRepository; simplify soft delete filtering. |
|
||||
| backend/src/v1/profile/service.py | Inherit BaseService; use shared validation helpers. |
|
||||
| backend/src/v1/auth/service.py | Use BaseService helpers where applicable. |
|
||||
| backend/tests/unit/v1/profile/* | Update tests for BaseRepository/BaseService. |
|
||||
| backend/tests/unit/v1/auth/* | Update tests for base service helpers (if needed). |
|
||||
| backend/tests/integration/* | Add/adjust tests for soft delete filtering. |
|
||||
| backend/tests/e2e/* | Update/extend critical auth/profile flow tests. |
|
||||
|
||||
## Files to Create
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| backend/src/core/db/repository.py | BaseRepository with soft-delete filtering. |
|
||||
| backend/src/core/services/base.py | BaseService with auth/user validation helpers. |
|
||||
| backend/tests/unit/core/db/test_base_repository.py | Unit tests for soft delete filters. |
|
||||
| backend/tests/unit/core/services/test_base_service.py | Unit tests for auth/user validation. |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- [ ] Supabase official Docker Compose template (source of services/env vars).
|
||||
- [ ] No new Python dependencies expected.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit Tests:** BaseRepository soft-delete filter logic; BaseService user validation helpers; updated profile/auth service behavior.
|
||||
- **Integration Tests:** SQLAlchemy queries exclude soft-deleted rows; profile endpoints still return expected responses.
|
||||
- **E2E Tests:** Critical profile flow with authenticated user; verify unauthorized access remains blocked.
|
||||
|
||||
## Risks & Mitigations
|
||||
|
||||
| Risk | Impact | Likelihood | Mitigation |
|
||||
|------|--------|------------|------------|
|
||||
| Missing or outdated Supabase compose template | Medium | Medium | Pin to official template version and document source in plan. |
|
||||
| Env var mismatches break local auth or DB connections | High | Medium | Add validation checklist and update `.env.example` with exact mappings. |
|
||||
| BaseRepository changes alter query behavior | Medium | Medium | Add unit/integration tests and verify no regressions. |
|
||||
| Auth validation refactor introduces regressions | High | Low | TDD with unit + E2E tests; keep behavior parity. |
|
||||
|
||||
## Estimated Effort
|
||||
|
||||
| Phase | Effort |
|
||||
|-------|--------|
|
||||
| Phase 1 | 3 hours |
|
||||
| Phase 2 | 4 hours |
|
||||
| Phase 3 | 4 hours |
|
||||
| Phase 4 | 4 hours |
|
||||
| **Total** | **15 hours** |
|
||||
@@ -0,0 +1,148 @@
|
||||
# 测试数据隔离方案(Supabase + Python 后端)
|
||||
|
||||
## 背景现状
|
||||
|
||||
- 后端在 `backend/src/core/config/settings.py` 使用 `SOCIAL_DATABASE__*` 生成 `database_url`。
|
||||
- 本地 Supabase 通过 `supabase-db` 容器提供 Postgres,宿主端口由 `SOCIAL_DATABASE__PORT` 控制(默认映射到容器 5432)。
|
||||
- 注意:`supabase-pooler` 的 5432 仅用于连接池;测试与迁移应直连 `supabase-db` 的宿主端口。
|
||||
- 单元数据库测试目前使用 SQLite 内存库(见 `tests/unit/database/*`),不影响开发库。
|
||||
- 真实 Postgres 的集成/E2E 当前未统一隔离策略;当开始接入真实 DB 时,需要按本文方案隔离与清理。
|
||||
|
||||
## 目标
|
||||
|
||||
- 测试过程不污染开发数据。
|
||||
- 测试可重复、可并行、可在本地与 CI 稳定运行。
|
||||
- 变更成本可控,优先在现有架构上落地。
|
||||
|
||||
## 结论(适配本项目)
|
||||
|
||||
采用“事务回滚 + 独立测试数据库”的混合策略:
|
||||
|
||||
- 默认测试使用事务回滚,快速、零污染(适用于单连接/单事务场景)。
|
||||
- 需要真实提交、并发或触发器行为的测试使用独立测试数据库。
|
||||
|
||||
## 方案设计
|
||||
|
||||
### A. 事务回滚(默认)
|
||||
|
||||
适用:单元测试、绝大多数集成测试(当这些测试连接真实 Postgres 时)。
|
||||
|
||||
核心思路:
|
||||
|
||||
- 每个测试在事务中运行。
|
||||
- 测试结束自动回滚。
|
||||
|
||||
优点:
|
||||
|
||||
- 速度快,无需新增数据库。
|
||||
- 测试间完全隔离。
|
||||
|
||||
限制:
|
||||
|
||||
- 无法验证真实 COMMIT 结果。
|
||||
- 并发、多连接事务隔离测试不准确。
|
||||
|
||||
### B. 独立测试数据库(E2E/并发)
|
||||
|
||||
适用:E2E、并发、触发器、LISTEN/NOTIFY 等需要真实提交的场景。
|
||||
|
||||
核心思路:
|
||||
|
||||
- 在现有 Supabase Postgres 实例中创建独立数据库。
|
||||
- 测试使用专用 `SOCIAL_DATABASE__NAME` 连接,端口/账号/密码来自 `SOCIAL_DATABASE__*`。
|
||||
- 测试前应用迁移,测试后清理。
|
||||
|
||||
优点:
|
||||
|
||||
- 行为最接近真实环境。
|
||||
- 与开发数据完全隔离。
|
||||
|
||||
成本:
|
||||
|
||||
- 需要迁移与清理策略。
|
||||
|
||||
## 与现有测试模块的衔接
|
||||
|
||||
- `tests/unit/database/*` 已使用 SQLite 内存库,无需改造。
|
||||
- 未来若 `tests/integration/*` 或 `tests/e2e/*` 连接真实 Postgres,应切换到本文的测试库策略。
|
||||
- 使用 `SOCIAL_DATABASE__NAME=postgres_test` 启动测试,以避免污染开发库。
|
||||
|
||||
## 实施步骤(与项目当前结构对齐)
|
||||
|
||||
### 1) 创建独立测试数据库
|
||||
|
||||
在本地 Supabase 容器中创建测试库:
|
||||
|
||||
```bash
|
||||
docker exec -e PGPASSWORD="$SOCIAL_DATABASE__PASSWORD" supabase-db \
|
||||
psql -U "$SOCIAL_DATABASE__USER" -c "CREATE DATABASE postgres_test;"
|
||||
```
|
||||
|
||||
说明:
|
||||
|
||||
- 容器名为 `supabase-db`(已在 `infra/docker` 运行)。
|
||||
- 数据库名建议 `postgres_test`,与 `.env` 的 `SOCIAL_DATABASE__NAME=postgres` 区分。
|
||||
|
||||
### 2) 运行迁移到测试库
|
||||
|
||||
使用测试环境变量指向测试库后,应用 Alembic 迁移:
|
||||
|
||||
```bash
|
||||
SOCIAL_RUNTIME__ENVIRONMENT=test \
|
||||
SOCIAL_DATABASE__NAME=postgres_test \
|
||||
uv run alembic upgrade head
|
||||
```
|
||||
|
||||
说明:
|
||||
|
||||
- 执行位置:`/home/qzl/Code/social-app/backend`。
|
||||
- 仍使用当前 `.env` 中的 `SOCIAL_DATABASE__HOST` 与 `SOCIAL_DATABASE__PORT`。
|
||||
|
||||
### 3) 事务回滚测试(默认)
|
||||
|
||||
测试执行时注入事务回滚机制:
|
||||
|
||||
- 在测试会话层创建单连接事务。
|
||||
- 对每个测试用例使用 SAVEPOINT(或嵌套事务)。
|
||||
- 测试结束回滚到 SAVEPOINT。
|
||||
|
||||
这套策略可保持速度与隔离性,同时不需要额外数据库。
|
||||
|
||||
### 4) 独立测试数据库执行(E2E/并发)
|
||||
|
||||
对于需要真实提交的测试,使用测试库运行:
|
||||
|
||||
```bash
|
||||
SOCIAL_RUNTIME__ENVIRONMENT=test \
|
||||
SOCIAL_DATABASE__NAME=postgres_test \
|
||||
uv run pytest tests/e2e
|
||||
```
|
||||
|
||||
清理策略(二选一):
|
||||
|
||||
- 小规模测试:TRUNCATE public schema 的业务表(不影响 `auth` 等系统 schema)。
|
||||
- 大规模测试:`DROP DATABASE postgres_test;` 后重建并迁移。
|
||||
|
||||
### 5) 本地/CI 统一策略
|
||||
|
||||
- 本地默认:事务回滚。
|
||||
- CI:独立测试库(保证完全隔离、无隐式依赖)。
|
||||
|
||||
## 风险与规避
|
||||
|
||||
- 不要在清理时操作 `auth`、`storage` 等 Supabase 系统 schema。
|
||||
- E2E 使用独立数据库,避免与开发数据交叉。
|
||||
- 迁移必须由 Alembic 统一维护,禁止手动改库。
|
||||
|
||||
## 落地检查清单
|
||||
|
||||
- [ ] 已创建 `postgres_test` 数据库。
|
||||
- [ ] 测试库迁移已应用。
|
||||
- [ ] 事务回滚测试已接入(默认路径)。
|
||||
- [ ] E2E 使用测试库运行。
|
||||
- [ ] 清理策略执行脚本可复用。
|
||||
|
||||
## 备注
|
||||
|
||||
本方案基于当前项目的 Supabase 本地 Docker 结构与后端配置方式(`SOCIAL_DATABASE__*`)。
|
||||
无需变更 Supabase 组件,优先在测试层完成隔离与清理。
|
||||
Reference in New Issue
Block a user