# Backend Development Rules

This document defines Python/FastAPI backend development constraints.

## Scope and Precedence

- This file applies to all changes under `backend/**`.
- It extends root routing rules in `AGENTS.md` and workspace global runtime rules.
- If rules conflict, follow stricter requirements.
- Keep backend-only rules here; do not duplicate them in root `AGENTS.md`.

## Python Environment

**MUST use uv for dependency management and virtual environment execution.**

- All Python commands: `uv run <command>`
- Add dependencies: `uv add <package>`
- All dependencies declared in `pyproject.toml`

## Code Quality Checks

**Git pre-commit hook enforces code quality before commit.**

Pre-commit hook automatically runs on backend/ directory:
- `ruff check` - code style and linting
- `basedpyright` - type checking with error level

If any error detected, commit is rejected. Fix errors before committing.
Do not bypass or weaken checks (no ignores, disables, or config relaxations). Resolve the underlying issues.

## Logging

**MUST use project logger for all runtime logging.**

- Use project logger from `backend/src/core/logging/*`
- Prohibit: print(), logging.info/warning/error directly
- Required: structured logging with context
- Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL

## HTTP API Standards

**MUST follow RESTful conventions and RFC 7807 for error responses.**

- Errors must use `application/problem+json` with RFC 7807 fields
- No custom response envelopes for HTTP APIs
- Request and response validation must use Pydantic models

## Environment Variables

**Backend env access MUST go through** `backend/src/core/config/settings.py`.

- Only use `Settings()` / `config` from `core.config.settings`
- Do not call `os.environ`, `os.getenv`, `dotenv`, or manual parsing in backend runtime code
- Tests can set env vars via `monkeypatch.setenv`, and should read values via `Settings()` unless the test is explicitly validating env plumbing
- Canonical principle: one source of truth per setting; no duplicate/derived env vars in backend code

## TDD Workflow

### Coverage Requirements

- Minimum coverage: 80%
- Required test types:
  - Unit: isolated functions, utilities, components
  - Integration: API endpoints, database operations
  - E2E: critical user flows (Playwright)

### Limited Exceptions

- Docs-only changes (README, comments, formatting) may skip integration/E2E
- Non-runtime config changes may skip E2E if no behavior changes
- Any runtime code change requires unit + integration + E2E
- If an exception is used, record the reason in the PR/test notes

### Mandatory TDD Workflow

1. Write tests (RED) - they must fail
2. Run tests - confirm failure
3. Implement minimal code (GREEN) - only to pass
4. Run tests - confirm success
5. Refactor (IMPROVE)
6. Verify coverage - target 80%+

### Enforcement

- Must use the `tdd-guide` agent for new features
- Do not write implementation before tests
- Do not lower coverage requirements
- Must include unit, integration, and E2E tests

## Code Style

### Immutability

**ALWAYS create new objects, NEVER mutate.**

```python
# WRONG: Mutation
def update_user(user, name):
    user["name"] = name
    return user

# CORRECT: Immutability
def update_user(user, name):
    return {**user, "name": name}
```

### File Organization

- Many small files over few large files
- 200-400 lines typical, 800 max per file
- Extract utilities from large components

### Error Handling

Always handle errors comprehensively:

```python
try:
    result = risky_operation()
    return result
except Exception as exc:
    logger.exception("Operation failed")
    raise RuntimeError("Detailed user-friendly message") from exc
```

## Security

### Mandatory Security Checks

Before ANY commit:
- [ ] No hardcoded secrets (API keys, passwords, tokens)
- [ ] All user inputs validated (use Pydantic)
- [ ] SQL injection prevention (parameterized queries)
- [ ] Authentication/authorization verified

### Secret Management

```python
# NEVER: Hardcoded secrets
api_key = "sk-proj-xxxxx"

# ALWAYS: Read through centralized settings
from core.config.settings import Settings

settings = Settings()
api_key = settings.openai_api_key
if not api_key:
    raise ValueError("OPENAI_API_KEY not configured in settings")
```

## Database Development Rules

### Architecture

- **Supabase**: authentication (JWT source of truth)
- **Backend**: business authorization (service layer)
- **SQLAlchemy ORM**: data access layer (async + asyncpg, service_role connection)

### Code Organization

Use `schemas / repository / service` pattern:
- `schemas.py` — Pydantic models
- `repository.py` — CRUD only, no auth, no commit (only flush), must receive session (never create session/engine)
- `service.py` — authorization + business logic + transaction boundary (must commit/rollback)
- `dependencies.py` — DI (`get_db`, `get_current_user`)

### Auth & Data Access

- Backend must verify JWT signature and expiration (not just decode)
- Extract `user_id` from JWT `sub` claim
- Backend connects with **service_role** (bypasses RLS)
- `owner_id` always derived from JWT, never from client
- Scope queries by owner/org; public access must be explicit
- service_role key is backend-only; never expose credentials
- Prohibit calling Supabase Admin API (service_role key) from repository/service layers

### Soft Delete

**Soft delete marks data as invisible, not cascade delete.**

- Use `deleted_at: datetime | None` column (via `SoftDeleteMixin`)
- **Query filtering**: Repository `_apply_soft_delete_filter()` auto-excludes deleted records
- **No automatic cascade**: Related data stays intact; visibility controlled by JOIN filtering
- **Cascade only for strong dependencies**: When parent deletion must invalidate children, implement in Service layer explicitly
- **Recovery**: Only restore the record itself; related data visibility restored automatically via queries
- **Unique constraints**: Use partial indexes excluding `deleted_at IS NOT NULL` to allow re-creation

```python
# Partial unique index in migration
op.execute("""
    CREATE UNIQUE INDEX ux_user_email 
    ON users(email) 
    WHERE deleted_at IS NULL
""")
```

### Migrations

- **Alembic is the single source of truth** for schema migrations
- ORM model changes → `alembic revision --autogenerate`
- Raw SQL (policies, triggers, functions) → `op.execute()`
- Migrations must be reversible; no reliance on generated IDs

### Enum Storage Convention

**Store enum names (strings), not integer values.**

- Use `VARCHAR(20)` + `CHECK` constraint in database
- Use Python `Enum` class with `str` base in code

```python
class AgentType(str, Enum):
    INTENT_RECOGNITION = "INTENT_RECOGNITION"
    TASK_EXECUTION = "TASK_EXECUTION"
    RESULT_REPORTING = "RESULT_REPORTING"
```

### RLS Policy

- Backend does not rely on RLS for correctness (uses service_role), but RLS is mandatory as a defensive boundary for tables in PostgREST-exposed schemas.
- **Mandatory default**: any new business table in `public` must enable RLS in the same Alembic migration.
- The same migration must create policies covering `SELECT/INSERT/UPDATE/DELETE` (minimum requirement).
- Recommended default policy set for `anon, authenticated`: deny all operations first, then open explicit access only when required.
- `alembic_version` must not be exposed to `anon` or `authenticated`.

#### Exemption Rule (strict)

- Exemptions are allowed only when a new `public` table is guaranteed not to be exposed to PostgREST clients.
- Exemptions must be explicit in the migration file with rationale and verification notes.
- If exposure is uncertain, do not exempt: enable defensive RLS by default.

#### Migration Checklist

- [ ] New `public` business table has `ALTER TABLE ... ENABLE ROW LEVEL SECURITY` in migration
- [ ] Policies for `SELECT/INSERT/UPDATE/DELETE` are present in migration
- [ ] Policy target roles are explicit (`anon`, `authenticated`, or both)
- [ ] Downgrade path is reversible and does not silently weaken intended production security
- [ ] Any exemption is documented with clear non-exposure evidence

## Backend Startup

**Always use `./infra/scripts/app.sh` to start/stop the backend.** Do not start uvicorn directly.
**Always use `./logs/*.log` to check the backend log output.**

## Agent Loop (AG-UI Protocol)

Agent loop functionality MUST follow the AG-UI protocol. **Use the `ag-ui` skill** for protocol reference and implementation guidance.

## Custom Tool Result Contract

Custom tool `ToolAgentOutput` MUST follow these rules:

- Use field name `result` only. Do not introduce or keep `result_summary` compatibility aliases.
- `metadata.tool_agent_output` is the canonical source for runtime observation and history replay.
- `tool_call_args` stores input snapshot only; avoid mixing execution output into `tool_call_args`.
- `result` stores output facts only; do not repeat input parameters already present in `tool_call_args`.
- `result` is for downstream agent reasoning and tool chaining, not for end-user presentation.
- For list/read tools, include multiple candidate records when needed (at least top matches) with stable identifiers and scheduling-critical fields.
- For write tools, include per-item operation outcomes and affected resource identifiers in `result`.
- Keep `result` concise, deterministic, and machine-oriented; avoid decorative wording and UI-style formatting.

## Multi-Agent Orchestration (AgentScope Framework)

Multi-agent orchestration MUST use the AgentScope framework. **Use the `agentscope-skill`** for framework reference and implementation guidance.

### Core Principles

- Use AgentScope for orchestrating multiple agents working together
- Define clear agent roles, stage responsibilities, and pipeline boundaries
- Leverage AgentScope built-in workflow and tool middleware mechanisms
- Follow AgentScope best practices for agent configuration

### Key Components

- **Agents**: Autonomous units with specific roles and goals
- **Tasks**: Stage-specific prompts and execution goals
- **Pipelines**: Ordered orchestration flow between agents
- **Tools**: Capabilities available to agents
- **Flows**: Workflow orchestration and state management

## Testing

### Real Database Tests

Tests requiring real Supabase operations MUST use environment variables:
- Define `TestSettings` in `settings.py` with nested configuration
- Access via `settings.test.email` / `settings.test.password`
- NEVER hardcode credentials in code
