qzl
ad06fe7de4
Consolidate backend modules/tests under the backend package while syncing Supabase compose/env config and related plans.
109 lines
6.0 KiB
Markdown
109 lines
6.0 KiB
Markdown
# Plan: Base Service for Redis and Qdrant
|
|
|
|
**Date:** 2026-02-05
|
|
**Author:** AI Assistant
|
|
**Status:** Draft
|
|
|
|
## Overview
|
|
|
|
Create a reusable base service module under `backend/src/services/base` that standardizes Redis and Qdrant client creation, lifecycle management, and error handling. Align the design with the DIVA-backend equivalent (once provided) and integrate configuration through existing `SOCIAL_REDIS__*` and `SOCIAL_QDRANT__*` settings.
|
|
|
|
## Requirements
|
|
|
|
### Functional
|
|
- [ ] Provide a base service abstraction that exposes Redis and Qdrant clients to other services.
|
|
- [ ] Use async client implementations compatible with FastAPI async execution.
|
|
- [ ] Support connection lifecycle hooks (initialize, health check, close).
|
|
- [ ] Centralize error handling and translate connection failures to consistent HTTP errors.
|
|
- [ ] Mirror DIVA-backend base service features and naming conventions where applicable.
|
|
|
|
### Non-Functional
|
|
- [ ] Performance: reuse client instances; avoid per-request connection creation.
|
|
- [ ] Security: never log secrets (API keys/passwords); enforce TLS settings when enabled.
|
|
- [ ] Reliability: implement timeouts and retry policy where supported by client libraries.
|
|
|
|
## Technical Approach
|
|
|
|
Introduce a `services/base` package that provides a small, composable base class plus Redis/Qdrant client factories. Configuration will be sourced from `core/config/settings.py` using the existing `.env` keys. The base service will accept injected clients to keep testability high and avoid global state, while a module-level factory will handle creation and cleanup.
|
|
|
|
### Key Decisions
|
|
| Decision | Rationale |
|
|
|----------|-----------|
|
|
| Use async Redis and Qdrant clients | Matches FastAPI async usage and avoids blocking the event loop. |
|
|
| Constructor injection with factories | Keeps services testable and avoids hidden global state. |
|
|
| Centralized error mapping in base service | Ensures consistent HTTP 503 responses and logging. |
|
|
|
|
## Implementation Steps
|
|
|
|
### Phase 1: DIVA-backend Parity Review (1-2 hours)
|
|
1. Locate DIVA-backend base service module (path or repo) and document its responsibilities, public API, and lifecycle behavior.
|
|
2. Produce a parity checklist to map DIVA behaviors to this repo (naming, error types, retry policy, health checks).
|
|
|
|
### Phase 2: Configuration and Client Factories (3 hours)
|
|
1. Add `RedisSettings` and `QdrantSettings` sections to `backend/src/core/config/settings.py` using existing `SOCIAL_REDIS__*` and `SOCIAL_QDRANT__*` env keys.
|
|
2. Create `backend/src/services/base/redis_client.py` and `backend/src/services/base/qdrant_client.py` with async client factory functions and close helpers.
|
|
3. Add structured logging for client initialization, connection failures, and shutdown paths.
|
|
|
|
### Phase 3: Base Service Class (3 hours)
|
|
1. Create `backend/src/services/base/service.py` with a `BaseService` that accepts optional Redis/Qdrant clients (dependency injection).
|
|
2. Add helper methods (e.g., `require_redis()`, `require_qdrant()`) that raise HTTP 503 on unavailable clients.
|
|
3. Define error translation utilities for Redis/Qdrant exceptions with consistent messages and logging.
|
|
|
|
### Phase 4: Tests (TDD) and Minimal Integration (4 hours)
|
|
1. Unit tests for settings parsing and default values (RED/GREEN).
|
|
2. Unit tests for base service behavior: missing client errors, exception mapping, and logging context.
|
|
3. Integration tests using running Redis/Qdrant containers to verify client factories can connect and execute a simple command.
|
|
4. E2E test that exercises a minimal endpoint using the base service (e.g., `/health/infra`), or record an explicit exception if no API integration is allowed.
|
|
|
|
## Files to Modify
|
|
|
|
| File | Changes |
|
|
|------|---------|
|
|
| backend/src/core/config/settings.py | Add Redis/Qdrant settings models and defaults. |
|
|
| backend/src/app.py | (If needed) register startup/shutdown hooks for client lifecycle. |
|
|
| backend/src/v1/router.py | (If needed) add an infra health endpoint to support E2E. |
|
|
|
|
## Files to Create
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| backend/src/services/base/__init__.py | Package export surface for base services. |
|
|
| backend/src/services/base/service.py | Base service class for Redis/Qdrant access. |
|
|
| backend/src/services/base/redis_client.py | Redis client factory and teardown helpers. |
|
|
| backend/src/services/base/qdrant_client.py | Qdrant client factory and teardown helpers. |
|
|
| backend/tests/unit/services/base/test_service.py | Unit tests for base service error handling. |
|
|
| backend/tests/unit/services/base/test_clients.py | Unit tests for client factory behavior. |
|
|
| backend/tests/integration/services/base/test_clients.py | Integration tests with Redis/Qdrant containers. |
|
|
| backend/tests/e2e/test_infra_health.py | E2E test for an endpoint using base service. |
|
|
|
|
## Dependencies
|
|
|
|
- [ ] `redis` (async client) for Redis connectivity.
|
|
- [ ] `qdrant-client` for Qdrant connectivity (async/GRPC as configured).
|
|
- [ ] No additional infra services required (Redis/Qdrant already in Docker compose).
|
|
|
|
## Testing Strategy
|
|
|
|
- **Unit Tests:** Base service behavior, missing client errors, exception translation, settings parsing.
|
|
- **Integration Tests:** Connect to Redis and Qdrant, run minimal ping/health operations.
|
|
- **E2E Tests:** Call a minimal endpoint that uses the base service to validate wiring and error handling.
|
|
|
|
## Risks & Mitigations
|
|
|
|
| Risk | Impact | Likelihood | Mitigation |
|
|
|------|--------|------------|------------|
|
|
| DIVA-backend module not available | Medium | High | Add a parity checklist and update plan once module location is provided. |
|
|
| Client library mismatch (sync vs async) | Medium | Medium | Select async-supported libraries and verify compatibility in unit tests. |
|
|
| Lack of API integration for E2E | High | Medium | Add a minimal infra health endpoint or record a documented exception. |
|
|
| Connection config mismatches | Medium | Medium | Validate settings with integration tests and mirror `.env.example`. |
|
|
|
|
## Estimated Effort
|
|
|
|
| Phase | Effort |
|
|
|-------|--------|
|
|
| Phase 1 | 2 hours |
|
|
| Phase 2 | 3 hours |
|
|
| Phase 3 | 3 hours |
|
|
| Phase 4 | 4 hours |
|
|
| **Total** | **12 hours** |
|