social-app/docs/plans/2026-03-04-simplify-agent-architecture.md

Agent Architecture Simplification Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Simplify agent configuration by removing redundant user_agents table and renaming user_agent_catalog to system_agents

Architecture: Delete user_agents table (including memories.agent_id dependency), rename user_agent_catalog to system_agents, update all references in code

Tech Stack: Python 3.11+, SQLAlchemy, Alembic, PostgreSQL

---

Prerequisites

• Current branch: dev
• No uncommitted changes
• Docker services running (Supabase local)

Task 1: Create Database Migration

Files:

• Create: backend/alembic/versions/20260304_simplify_agent_architecture.py

Step 1: Create migration file

Run: cd backend && uv run alembic revision -m "simplify_agent_architecture"

Expected: New migration file created with revision ID

Step 2: Write migration upgrade logic

Edit the generated migration file with this complete upgrade function:

def upgrade() -> None:
    # 1. Delete memories.agent_id dependencies
    op.drop_constraint("fk_memories_agent_id", "memories", type_="foreignkey")
    op.drop_constraint("chk_memory_type_agent_id", "memories", type_="check")
    op.execute("DROP INDEX IF EXISTS ix_memories_agent_type_status")
    op.drop_column("memories", "agent_id")

    # 2. Delete user_agents table
    _drop_rls("user_agents")
    
    op.drop_constraint("fk_user_agents_updated_by", "user_agents", type_="foreignkey")
    op.drop_constraint("fk_user_agents_created_by", "user_agents", type_="foreignkey")
    op.drop_constraint("fk_user_agents_llm_id", "user_agents", type_="foreignkey")
    op.drop_constraint("fk_user_agents_user_id", "user_agents", type_="foreignkey")
    op.drop_constraint("chk_agent_type", "user_agents", type_="check")
    op.drop_constraint("uq_user_agents_user_id_agent_type", "user_agents", type_="unique")
    
    op.execute("DROP INDEX IF EXISTS ix_user_agents_status")
    op.execute("DROP INDEX IF EXISTS ix_user_agents_agent_type")
    
    op.drop_table("user_agents")

    # 3. Rename user_agent_catalog to system_agents
    _drop_rls("user_agent_catalog")
    
    op.rename_table("user_agent_catalog", "system_agents")
    
    op.execute(
        "ALTER TABLE system_agents RENAME CONSTRAINT fk_user_agent_catalog_llm_id "
        "TO fk_system_agents_llm_id"
    )
    op.execute(
        "ALTER TABLE system_agents RENAME CONSTRAINT chk_user_agent_catalog_status "
        "TO chk_system_agents_status"
    )
    
    _enable_rls("system_agents")

    # 4. Update trigger
    op.execute("DROP TRIGGER IF EXISTS on_auth_user_created ON auth.users")
    op.execute("DROP FUNCTION IF EXISTS public.create_profile_for_new_user()")
    
    op.execute("""
        CREATE OR REPLACE FUNCTION public.create_profile_for_new_user()
        RETURNS trigger
        LANGUAGE plpgsql
        SECURITY DEFINER
        SET search_path = public
        AS $$
        BEGIN
            INSERT INTO public.profiles (id, username, avatar_url, bio, settings, created_at, updated_at)
            VALUES (
                NEW.id,
                COALESCE(
                    NEW.raw_user_meta_data ->> 'username',
                    split_part(NEW.email, '@', 1),
                    'user_' || substring(NEW.id::text, 1, 8)
                ),
                NULL,
                NULL,
                '{"agent_prompts": {}}'::jsonb,
                now(),
                now()
            )
            ON CONFLICT (id) DO NOTHING;
            
            RETURN NEW;
        END;
        $$
    """)
    
    op.execute("""
        CREATE TRIGGER on_auth_user_created
            AFTER INSERT ON auth.users
            FOR EACH ROW EXECUTE FUNCTION public.create_profile_for_new_user()
    """)

    # 5. Update existing profiles.settings
    op.execute("""
        UPDATE profiles 
        SET settings = jsonb_set(
            COALESCE(settings, '{}'::jsonb),
            '{agent_prompts}',
            '{}'::jsonb
        )
        WHERE NOT settings ? 'agent_prompts'
    """)

Step 3: Write migration downgrade logic

Add this complete downgrade function:

def downgrade() -> None:
    # 1. Revert trigger
    op.execute("DROP TRIGGER IF EXISTS on_auth_user_created ON auth.users")
    op.execute("DROP FUNCTION IF EXISTS public.create_profile_for_new_user()")
    
    op.execute("""
        CREATE OR REPLACE FUNCTION public.create_profile_for_new_user()
        RETURNS trigger
        LANGUAGE plpgsql
        SECURITY DEFINER
        SET search_path = public
        AS $$
        BEGIN
            INSERT INTO public.profiles (id, username, avatar_url, bio, settings, created_at, updated_at)
            VALUES (
                NEW.id,
                COALESCE(
                    NEW.raw_user_meta_data ->> 'username',
                    split_part(NEW.email, '@', 1),
                    'user_' || substring(NEW.id::text, 1, 8)
                ),
                NULL,
                NULL,
                '{}'::jsonb,
                now(),
                now()
            )
            ON CONFLICT (id) DO NOTHING;

            INSERT INTO public.user_agents (id, user_id, llm_id, agent_type, config, status, created_by, updated_by)
            SELECT 
                gen_random_uuid(),
                NEW.id,
                uac.llm_id,
                uac.agent_type,
                uac.config,
                uac.status,
                NEW.id,
                NEW.id
            FROM public.user_agent_catalog uac;
            
            RETURN NEW;
        END;
        $$
    """)
    
    op.execute("""
        CREATE TRIGGER on_auth_user_created
            AFTER INSERT ON auth.users
            FOR EACH ROW EXECUTE FUNCTION public.create_profile_for_new_user()
    """)

    # 2. Revert rename: system_agents -> user_agent_catalog
    _drop_rls("system_agents")
    
    op.rename_table("system_agents", "user_agent_catalog")
    
    op.execute(
        "ALTER TABLE user_agent_catalog RENAME CONSTRAINT fk_system_agents_llm_id "
        "TO fk_user_agent_catalog_llm_id"
    )
    op.execute(
        "ALTER TABLE user_agent_catalog RENAME CONSTRAINT chk_system_agents_status "
        "TO chk_user_agent_catalog_status"
    )
    
    _enable_rls("user_agent_catalog")

    # 3. Recreate user_agents table
    op.create_table(
        "user_agents",
        sa.Column("id", sa.UUID(), nullable=False),
        sa.Column("user_id", sa.UUID(), nullable=False),
        sa.Column("llm_id", sa.UUID(), nullable=False),
        sa.Column("agent_type", sa.String(length=20), nullable=False),
        sa.Column(
            "config",
            postgresql.JSONB(astext_type=sa.Text()),
            server_default="{}",
            nullable=False,
        ),
        sa.Column("status", sa.String(length=20), nullable=False),
        sa.Column("created_by", sa.UUID(), nullable=True),
        sa.Column("updated_by", sa.UUID(), nullable=True),
        sa.Column(
            "created_at",
            sa.DateTime(timezone=True),
            server_default=sa.text("now()"),
            nullable=False,
        ),
        sa.Column(
            "updated_at",
            sa.DateTime(timezone=True),
            server_default=sa.text("now()"),
            nullable=False,
        ),
        sa.Column("deleted_at", sa.DateTime(timezone=True), nullable=True),
        sa.PrimaryKeyConstraint("id"),
    )
    
    op.create_unique_constraint(
        "uq_user_agents_user_id_agent_type", 
        "user_agents", 
        ["user_id", "agent_type"]
    )
    
    op.execute(
        "CREATE INDEX ix_user_agents_agent_type ON user_agents (agent_type)"
    )
    op.execute(
        "CREATE INDEX ix_user_agents_status ON user_agents (status)"
    )
    
    op.execute(
        "ALTER TABLE user_agents ADD CONSTRAINT chk_agent_type "
        "CHECK (agent_type IN ('INTENT_RECOGNITION', 'TASK_EXECUTION', 'RESULT_REPORTING'))"
    )
    
    op.create_foreign_key(
        "fk_user_agents_user_id",
        "user_agents",
        "users",
        ["user_id"],
        ["id"],
        referent_schema="auth",
        ondelete="CASCADE",
    )
    op.create_foreign_key(
        "fk_user_agents_llm_id",
        "user_agents",
        "llms",
        ["llm_id"],
        ["id"],
        ondelete="RESTRICT",
    )
    op.create_foreign_key(
        "fk_user_agents_created_by",
        "user_agents",
        "users",
        ["created_by"],
        ["id"],
        referent_schema="auth",
        ondelete="SET NULL",
    )
    op.create_foreign_key(
        "fk_user_agents_updated_by",
        "user_agents",
        "users",
        ["updated_by"],
        ["id"],
        referent_schema="auth",
        ondelete="SET NULL",
    )
    
    _enable_rls("user_agents")

    # 4. Recreate memories.agent_id
    op.add_column(
        "memories",
        sa.Column("agent_id", sa.UUID(), nullable=True)
    )
    
    op.create_foreign_key(
        "fk_memories_agent_id",
        "memories",
        "user_agents",
        ["agent_id"],
        ["id"],
        ondelete="CASCADE",
    )
    
    op.execute(
        "CREATE INDEX ix_memories_agent_type_status ON memories (agent_id, memory_type, status)"
    )
    
    op.execute(
        "ALTER TABLE memories ADD CONSTRAINT chk_memory_type_agent_id "
        "CHECK ((memory_type = 'work' AND agent_id IS NOT NULL) OR "
        "(memory_type = 'user' AND agent_id IS NULL))"
    )

Step 4: Add helper functions

Add these helper functions at the end of the migration file:

def _enable_rls(table_name: str) -> None:
    for role in ["anon", "authenticated"]:
        for action in ["select", "insert", "update", "delete"]:
            op.execute(
                f"DROP POLICY IF EXISTS {role}_{action}_{table_name} ON {table_name}"
            )
    op.execute(f"ALTER TABLE {table_name} ENABLE ROW LEVEL SECURITY")
    for role in ["anon", "authenticated"]:
        op.execute(
            f"CREATE POLICY {role}_select_{table_name} ON {table_name} "
            f"FOR SELECT TO {role} USING (false)"
        )
        op.execute(
            f"CREATE POLICY {role}_insert_{table_name} ON {table_name} "
            f"FOR INSERT TO {role} WITH CHECK (false)"
        )
        op.execute(
            f"CREATE POLICY {role}_update_{table_name} ON {table_name} "
            f"FOR UPDATE TO {role} USING (false) WITH CHECK (false)"
        )
        op.execute(
            f"CREATE POLICY {role}_delete_{table_name} ON {table_name} "
            f"FOR DELETE TO {role} USING (false)"
        )


def _drop_rls(table_name: str) -> None:
    for role in ["anon", "authenticated"]:
        op.execute(f"DROP POLICY IF EXISTS {role}_delete_{table_name} ON {table_name}")
        op.execute(f"DROP POLICY IF EXISTS {role}_update_{table_name} ON {table_name}")
        op.execute(f"DROP POLICY IF EXISTS {role}_insert_{table_name} ON {table_name}")
        op.execute(f"DROP POLICY IF EXISTS {role}_select_{table_name} ON {table_name}")
    op.execute(f"ALTER TABLE {table_name} DISABLE ROW LEVEL SECURITY")

Step 5: Verify migration file

Check that all imports are correct:

from typing import Sequence, Union

from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql

Step 6: Commit migration

git add backend/alembic/versions/20260304_simplify_agent_architecture.py
git commit -m "feat(db): add migration to simplify agent architecture"

---

Task 2: Delete UserAgents Model

Files:

• Delete: backend/src/models/user_agents.py
• Modify: backend/src/models/__init__.py

Step 1: Remove import from models/__init__.py

Edit backend/src/models/__init__.py:

Remove these lines:

from models.user_agents import UserAgent

And remove "UserAgent" from __all__ list.

Step 2: Delete user_agents.py file

rm backend/src/models/user_agents.py

Step 3: Verify no other imports

Run: cd backend && grep -r "from models.user_agents" src/

Expected: No results (or only in __init__.py which we already fixed)

Step 4: Commit

git add backend/src/models/user_agents.py backend/src/models/__init__.py
git commit -m "refactor(models): remove UserAgents model"

---

Task 3: Rename UserAgentCatalog to SystemAgents

Files:

• Rename: backend/src/models/user_agent_catalog.py → backend/src/models/system_agents.py
• Modify: backend/src/models/__init__.py

Step 1: Rename model file

mv backend/src/models/user_agent_catalog.py backend/src/models/system_agents.py

Step 2: Update class name in system_agents.py

Edit backend/src/models/system_agents.py:

Change:

class UserAgentCatalog(TimestampMixin, Base):
    __tablename__: str = "user_agent_catalog"

To:

class SystemAgents(TimestampMixin, Base):
    __tablename__: str = "system_agents"

Step 3: Update imports in models/__init__.py

Edit backend/src/models/__init__.py:

Change:

from models.user_agent_catalog import UserAgentCatalog

To:

from models.system_agents import SystemAgents

And change "UserAgentCatalog" to "SystemAgents" in __all__ list.

Step 4: Commit

git add backend/src/models/
git commit -m "refactor(models): rename UserAgentCatalog to SystemAgents"

---

Task 4: Update Configuration Files

Files:

• Rename: backend/src/core/config/static/database/user_agent_catalog.yaml → backend/src/core/config/static/database/system_agents.yaml
• Modify: backend/src/core/config/initial/init_data.py

Step 1: Rename YAML file

mv backend/src/core/config/static/database/user_agent_catalog.yaml \
   backend/src/core/config/static/database/system_agents.yaml

Step 2: Update init_data.py imports

Edit backend/src/core/config/initial/init_data.py:

Change:

from models.user_agent_catalog import UserAgentCatalog

To:

from models.system_agents import SystemAgents

Step 3: Update Pydantic models

Change:

class UserAgentCatalogSeed(BaseModel):
    agent_type: str
    llm_model_code: str
    status: str
    config: dict[str, Any]


class UserAgentCatalogYaml(BaseModel):
    agents: list[UserAgentCatalogSeed]

To:

class SystemAgentsSeed(BaseModel):
    agent_type: str
    llm_model_code: str
    status: str
    config: dict[str, Any]


class SystemAgentsYaml(BaseModel):
    agents: list[SystemAgentsSeed]

Step 4: Update path function

Change:

def _default_user_agent_catalog_path() -> Path:
    return (
        Path(__file__).resolve().parents[1]
        / "static"
        / "database"
        / "user_agent_catalog.yaml"
    )

To:

def _default_system_agents_path() -> Path:
    return (
        Path(__file__).resolve().parents[1]
        / "static"
        / "database"
        / "system_agents.yaml"
    )

Step 5: Update load function

Change:

def load_user_agent_catalog(catalog_path: Path | None = None) -> dict[str, Any]:
    path = catalog_path or _default_user_agent_catalog_path()
    with path.open("r", encoding="utf-8") as file:
        loaded = yaml.safe_load(file) or {}
    if not isinstance(loaded, dict):
        raise ValueError(f"Invalid user agent catalog format: {path}")
    raw_agents = loaded.get("agents", [])
    if not isinstance(raw_agents, list):
        raise ValueError(f"Invalid user agent catalog agents section: {path}")
    try:
        parsed = UserAgentCatalogYaml.model_validate({"agents": list(raw_agents)})
    except ValidationError as exc:
        raise ValueError(f"Invalid user agent catalog data: {path}") from exc

    return parsed.model_dump()

To:

def load_system_agents(catalog_path: Path | None = None) -> dict[str, Any]:
    path = catalog_path or _default_system_agents_path()
    with path.open("r", encoding="utf-8") as file:
        loaded = yaml.safe_load(file) or {}
    if not isinstance(loaded, dict):
        raise ValueError(f"Invalid system agents format: {path}")
    raw_agents = loaded.get("agents", [])
    if not isinstance(raw_agents, list):
        raise ValueError(f"Invalid system agents agents section: {path}")
    try:
        parsed = SystemAgentsYaml.model_validate({"agents": list(raw_agents)})
    except ValidationError as exc:
        raise ValueError(f"Invalid system agents data: {path}") from exc

    return parsed.model_dump()

Step 6: Update upsert function

Change:

async def _upsert_user_agent_catalog(
    session: AsyncSession,
    *,
    agent_type: str,
    llm_id: uuid.UUID,
    status: str,
    config: dict[str, Any],
) -> None:
    result = await session.execute(
        select(UserAgentCatalog).where(UserAgentCatalog.agent_type == agent_type)
    )
    catalog_entry = result.scalar_one_or_none()

    if catalog_entry is None:
        session.add(
            UserAgentCatalog(
                agent_type=agent_type,
                llm_id=llm_id,
                status=status,
                config=config,
            )
        )
    else:
        catalog_entry.llm_id = llm_id
        catalog_entry.status = status
        catalog_entry.config = config

To:

async def _upsert_system_agents(
    session: AsyncSession,
    *,
    agent_type: str,
    llm_id: uuid.UUID,
    status: str,
    config: dict[str, Any],
) -> None:
    result = await session.execute(
        select(SystemAgents).where(SystemAgents.agent_type == agent_type)
    )
    catalog_entry = result.scalar_one_or_none()

    if catalog_entry is None:
        session.add(
            SystemAgents(
                agent_type=agent_type,
                llm_id=llm_id,
                status=status,
                config=config,
            )
        )
    else:
        catalog_entry.llm_id = llm_id
        catalog_entry.status = status
        catalog_entry.config = config

Step 7: Update initialize function

Change:

async def initialize_user_agent_catalog() -> None:
    """Initialize user agent catalog from YAML."""
    catalog = load_user_agent_catalog()

    async with AsyncSessionLocal() as session:
        async with session.begin():
            for agent in catalog["agents"]:
                result = await session.execute(
                    select(Llm).where(Llm.model_code == agent["llm_model_code"])
                )
                llm = result.scalar_one_or_none()
                if llm is None:
                    raise RuntimeError(
                        f"LLM model '{agent['llm_model_code']}' not found for agent type '{agent['agent_type']}'"
                    )

                await _upsert_user_agent_catalog(
                    session,
                    agent_type=agent["agent_type"],
                    llm_id=llm.id,
                    status=agent["status"],
                    config=agent["config"],
                )

    logger.info("Initialized user agent catalog")

To:

async def initialize_system_agents() -> None:
    """Initialize system agents from YAML."""
    catalog = load_system_agents()

    async with AsyncSessionLocal() as session:
        async with session.begin():
            for agent in catalog["agents"]:
                result = await session.execute(
                    select(Llm).where(Llm.model_code == agent["llm_model_code"])
                )
                llm = result.scalar_one_or_none()
                if llm is None:
                    raise RuntimeError(
                        f"LLM model '{agent['llm_model_code']}' not found for agent type '{agent['agent_type']}'"
                    )

                await _upsert_system_agents(
                    session,
                    agent_type=agent["agent_type"],
                    llm_id=llm.id,
                    status=agent["status"],
                    config=agent["config"],
                )

    logger.info("Initialized system agents")

Step 8: Update initialize_data function

Change:

async def initialize_data() -> bool:
    """Initialize bootstrap data."""
    await initialize_llm_catalog()
    await initialize_user_agent_catalog()

    return True

To:

async def initialize_data() -> bool:
    """Initialize bootstrap data."""
    await initialize_llm_catalog()
    await initialize_system_agents()

    return True

Step 9: Commit

git add backend/src/core/config/
git commit -m "refactor(config): rename user_agent_catalog to system_agents"

---

Task 5: Run Migration

Step 1: Run migration

cd backend && uv run alembic upgrade head

Expected: Migration runs successfully

Step 2: Verify tables

Connect to database and check:

• user_agents table should NOT exist
• system_agents table should exist
• memories.agent_id column should NOT exist

Step 3: Test downgrade (optional but recommended)

cd backend && uv run alembic downgrade -1

Expected: Previous migration restored

Step 4: Re-run upgrade

cd backend && uv run alembic upgrade head

Expected: Migration runs successfully again

---

Task 6: Run Tests and Linting

Step 1: Run type checking

cd backend && uv run basedpyright src/

Expected: No errors

Step 2: Run linting

cd backend && uv run ruff check src/

Expected: No errors

Step 3: Run tests

cd backend && uv run pytest tests/

Expected: All tests pass

Step 4: Fix any failures

If any tests fail due to UserAgent references, update them to use SystemAgents.

---

Task 7: Final Verification

Step 1: Search for any remaining references

cd backend && grep -r "user_agents" src/ --include="*.py"
cd backend && grep -r "UserAgent" src/ --include="*.py"

Expected: No results (except in migration files)

Step 2: Test new user registration

Start the backend server and register a new user. Verify:

• Profile is created
• No user_agents records are created
• profiles.settings contains agent_prompts: {}

Step 3: Commit final changes

git add .
git commit -m "feat: complete agent architecture simplification"

---

Success Criteria

• Migration runs successfully (upgrade and downgrade)
• No UserAgent model references in code
• SystemAgents model works correctly
• All tests pass
• Linting passes
• Type checking passes
• New user registration works without user_agents

Notes

• Keep the design document updated if any changes are made during implementation
• Test migration thoroughly before deploying to production
• Backup database before running migration in production