diff --git a/docs/plans/2026-03-04-simplify-agent-architecture-design.md b/docs/plans/2026-03-04-simplify-agent-architecture-design.md new file mode 100644 index 0000000..a5cd637 --- /dev/null +++ b/docs/plans/2026-03-04-simplify-agent-architecture-design.md @@ -0,0 +1,199 @@ +# Agent Architecture Simplification Design + +**Date:** 2026-03-04 +**Status:** Approved +**Author:** AI Assistant + +## Overview + +Simplify the agent configuration architecture by removing the redundant `user_agents` table and renaming `user_agent_catalog` to `system_agents`. + +## Problem Statement + +Current architecture has redundant data: +- `user_agent_catalog`: System-level agent configurations (3 agent types for all users) +- `user_agents`: Per-user agent instances (copies catalog data for each user) + +Since every user has the same 3 agents with identical configurations (from catalog), maintaining `user_agents` table creates unnecessary complexity and data duplication. + +## Goals + +1. Remove `user_agents` table and related code +2. Rename `user_agent_catalog` to `system_agents` for clarity +3. Preserve ability for future user-level prompt customization via `profiles.settings` +4. Maintain backward compatibility in deployment process + +## Non-Goals + +- User-level agent configuration (LLM selection, temperature, etc.) +- User-level prompt customization implementation (deferred to future iteration) + +## Architecture Changes + +### Current Architecture + +``` +user_agent_catalog (system config) + ↓ (trigger copies for each new user) +user_agents (per-user instances) +``` + +### New Architecture + +``` +system_agents (shared by all users) +profiles.settings.agent_prompts (future: user-level prompts) +``` + +### Data Flow + +1. System startup: Load `system_agents` from YAML +2. User creation: No longer creates `user_agents` records +3. Runtime (future): Read from `system_agents` + merge with `profiles.settings.agent_prompts` + +## Database Migration + +### Changes + +1. **Delete `memories.agent_id` column** + - Remove foreign key `fk_memories_agent_id` + - Remove check constraint `chk_memory_type_agent_id` + - Remove index `ix_memories_agent_type_status` + - Drop column `agent_id` + +2. **Delete `user_agents` table** + - Remove all RLS policies + - Remove indexes: `ix_user_agents_agent_type`, `ix_user_agents_status` + - Remove foreign keys: `fk_user_agents_user_id`, `fk_user_agents_llm_id`, etc. + - Remove check constraint `chk_agent_type` + - Remove unique constraint `uq_user_agents_user_id_agent_type` + - Drop table + +3. **Rename `user_agent_catalog` → `system_agents`** + - Remove old RLS policies + - Rename table + - Rename constraints: `fk_user_agent_catalog_llm_id` → `fk_system_agents_llm_id` + - Rename check constraint: `chk_user_agent_catalog_status` → `chk_system_agents_status` + - Re-create RLS policies with new table name + +4. **Update trigger `create_profile_for_new_user()`** + - Remove logic that inserts into `user_agents` + - Initialize `profiles.settings.agent_prompts` with empty object + +5. **Update existing `profiles.settings`** + - Add `agent_prompts: {}` to all existing profiles + +### Downgrade Path + +- Re-create `user_agents` table with all constraints and indexes +- Restore `memories.agent_id` column and constraints +- Rename `system_agents` → `user_agent_catalog` +- Restore original trigger + +## Code Changes + +### Model Layer + +**Delete:** +- `backend/src/models/user_agents.py` + +**Rename:** +- `backend/src/models/user_agent_catalog.py` → `backend/src/models/system_agents.py` +- Class `UserAgentCatalog` → `SystemAgents` + +**Update:** +- `backend/src/models/__init__.py` - Update imports and exports + +### Configuration Layer + +**Rename:** +- `backend/src/core/config/static/database/user_agent_catalog.yaml` + → `backend/src/core/config/static/database/system_agents.yaml` + +**Update:** +- `backend/src/core/config/initial/init_data.py` + - `UserAgentCatalogSeed` → `SystemAgentsSeed` + - `UserAgentCatalogYaml` → `SystemAgentsYaml` + - Import from `models.system_agents` + - Path: `system_agents.yaml` + - Function: `initialize_user_agent_catalog()` → `initialize_system_agents()` + +### Future: Profile Settings Structure (Deferred) + +```json +{ + "agent_prompts": { + "INTENT_RECOGNITION": "custom prompt...", + "TASK_EXECUTION": "custom prompt...", + "RESULT_REPORTING": "custom prompt..." + } +} +``` + +## Testing Strategy + +### Migration Tests + +- Verify `user_agents` table is deleted +- Verify `system_agents` table exists with correct structure +- Verify trigger no longer creates `user_agents` records +- Verify `profiles.settings.agent_prompts` is initialized +- Verify downgrade path works correctly + +### Model Tests + +- Verify `SystemAgents` model CRUD operations +- Verify `Profile.settings` JSONB storage + +### Integration Tests + +- Verify `initialize_system_agents()` loads from YAML +- Verify data is correctly inserted into `system_agents` table + +## Deployment Considerations + +### Pre-deployment + +- Backup database (especially `user_agents` if any data exists) +- Confirm production `user_agents` table has no critical data + +### Deployment + +1. Run migration: `alembic upgrade head` +2. Verify migration success +3. Restart application services +4. Verify new user registration works without `user_agents` + +### Post-deployment + +- Monitor application logs for any references to deleted `user_agents` +- Verify agent-related functionality still works + +## Risks and Mitigations + +| Risk | Mitigation | +|------|-----------| +| Existing `user_agents` data loss | Backup before migration; data is redundant anyway | +| Code still references `user_agents` | Comprehensive code search and testing | +| Trigger fails on new user creation | Test migration thoroughly; include rollback plan | +| Future need for user-level config | Can add `agent_overrides` to `profiles.settings` | + +## Success Criteria + +- [ ] All tests pass +- [ ] Migration runs successfully (upgrade and downgrade) +- [ ] New user registration creates profile without `user_agents` records +- [ ] System agents are loaded from YAML correctly +- [ ] No references to `user_agents` remain in codebase + +## Timeline + +- Design: 2026-03-04 (Completed) +- Implementation: TBD +- Testing: TBD +- Deployment: TBD + +## References + +- Migration file: `backend/alembic/versions/YYYYMMDD_simplify_agent_architecture.py` +- Original catalog migration: `backend/alembic/versions/50ae013ce530_add_user_agent_catalog.py`