social-app/docs/plans/2026-03-04-simplify-agent-architecture-design.md

# Agent Architecture Simplification Design

**Date:** 2026-03-04
**Status:** Approved
**Author:** AI Assistant

## Overview

Simplify the agent configuration architecture by removing the redundant `user_agents` table and renaming `user_agent_catalog` to `system_agents`.

## Problem Statement

Current architecture has redundant data:
- `user_agent_catalog`: System-level agent configurations (3 agent types for all users)
- `user_agents`: Per-user agent instances (copies catalog data for each user)

Since every user has the same 3 agents with identical configurations (from catalog), maintaining `user_agents` table creates unnecessary complexity and data duplication.

## Goals

1. Remove `user_agents` table and related code
2. Rename `user_agent_catalog` to `system_agents` for clarity
3. Preserve ability for future user-level prompt customization via `profiles.settings`
4. Maintain backward compatibility in deployment process

## Non-Goals

- User-level agent configuration (LLM selection, temperature, etc.)
- User-level prompt customization implementation (deferred to future iteration)

## Architecture Changes

### Current Architecture

```
user_agent_catalog (system config)
    ↓ (trigger copies for each new user)
user_agents (per-user instances)
```

### New Architecture

```
system_agents (shared by all users)
profiles.settings.agent_prompts (future: user-level prompts)
```

### Data Flow

1. System startup: Load `system_agents` from YAML
2. User creation: No longer creates `user_agents` records
3. Runtime (future): Read from `system_agents` + merge with `profiles.settings.agent_prompts`

## Database Migration

### Changes

1. **Delete `memories.agent_id` column**
   - Remove foreign key `fk_memories_agent_id`
   - Remove check constraint `chk_memory_type_agent_id`
   - Remove index `ix_memories_agent_type_status`
   - Drop column `agent_id`

2. **Delete `user_agents` table**
   - Remove all RLS policies
   - Remove indexes: `ix_user_agents_agent_type`, `ix_user_agents_status`
   - Remove foreign keys: `fk_user_agents_user_id`, `fk_user_agents_llm_id`, etc.
   - Remove check constraint `chk_agent_type`
   - Remove unique constraint `uq_user_agents_user_id_agent_type`
   - Drop table

3. **Rename `user_agent_catalog` → `system_agents`**
   - Remove old RLS policies
   - Rename table
   - Rename constraints: `fk_user_agent_catalog_llm_id` → `fk_system_agents_llm_id`
   - Rename check constraint: `chk_user_agent_catalog_status` → `chk_system_agents_status`
   - Re-create RLS policies with new table name

4. **Update trigger `create_profile_for_new_user()`**
   - Remove logic that inserts into `user_agents`
   - Initialize `profiles.settings.agent_prompts` with empty object

5. **Update existing `profiles.settings`**
   - Add `agent_prompts: {}` to all existing profiles

### Downgrade Path

- Re-create `user_agents` table with all constraints and indexes
- Restore `memories.agent_id` column and constraints
- Rename `system_agents` → `user_agent_catalog`
- Restore original trigger

## Code Changes

### Model Layer

**Delete:**
- `backend/src/models/user_agents.py`

**Rename:**
- `backend/src/models/user_agent_catalog.py` → `backend/src/models/system_agents.py`
- Class `UserAgentCatalog` → `SystemAgents`

**Update:**
- `backend/src/models/__init__.py` - Update imports and exports

### Configuration Layer

**Rename:**
- `backend/src/core/config/static/database/user_agent_catalog.yaml`
  → `backend/src/core/config/static/database/system_agents.yaml`

**Update:**
- `backend/src/core/config/initial/init_data.py`
  - `UserAgentCatalogSeed` → `SystemAgentsSeed`
  - `UserAgentCatalogYaml` → `SystemAgentsYaml`
  - Import from `models.system_agents`
  - Path: `system_agents.yaml`
  - Function: `initialize_user_agent_catalog()` → `initialize_system_agents()`

### Future: Profile Settings Structure (Deferred)

```json
{
  "agent_prompts": {
    "INTENT_RECOGNITION": "custom prompt...",
    "TASK_EXECUTION": "custom prompt...",
    "RESULT_REPORTING": "custom prompt..."
  }
}
```

## Testing Strategy

### Migration Tests

- Verify `user_agents` table is deleted
- Verify `system_agents` table exists with correct structure
- Verify trigger no longer creates `user_agents` records
- Verify `profiles.settings.agent_prompts` is initialized
- Verify downgrade path works correctly

### Model Tests

- Verify `SystemAgents` model CRUD operations
- Verify `Profile.settings` JSONB storage

### Integration Tests

- Verify `initialize_system_agents()` loads from YAML
- Verify data is correctly inserted into `system_agents` table

## Deployment Considerations

### Pre-deployment

- Backup database (especially `user_agents` if any data exists)
- Confirm production `user_agents` table has no critical data

### Deployment

1. Run migration: `alembic upgrade head`
2. Verify migration success
3. Restart application services
4. Verify new user registration works without `user_agents`

### Post-deployment

- Monitor application logs for any references to deleted `user_agents`
- Verify agent-related functionality still works

## Risks and Mitigations

| Risk | Mitigation |
|------|-----------|
| Existing `user_agents` data loss | Backup before migration; data is redundant anyway |
| Code still references `user_agents` | Comprehensive code search and testing |
| Trigger fails on new user creation | Test migration thoroughly; include rollback plan |
| Future need for user-level config | Can add `agent_overrides` to `profiles.settings` |

## Success Criteria

- [ ] All tests pass
- [ ] Migration runs successfully (upgrade and downgrade)
- [ ] New user registration creates profile without `user_agents` records
- [ ] System agents are loaded from YAML correctly
- [ ] No references to `user_agents` remain in codebase

## Timeline

- Design: 2026-03-04 (Completed)
- Implementation: TBD
- Testing: TBD
- Deployment: TBD

## References

- Migration file: `backend/alembic/versions/YYYYMMDD_simplify_agent_architecture.py`
- Original catalog migration: `backend/alembic/versions/50ae013ce530_add_user_agent_catalog.py`