perf: optimize web data resources
This commit is contained in:
ZL-Q
@@ -0,0 +1,383 @@
|
||||
# Web data interaction refactor plan
|
||||
|
||||
## Executive summary
|
||||
|
||||
The current web app has solid typed API helpers, but it does not yet have an application data layer. Most pages call backend endpoints directly inside local `useEffect` blocks, so route changes repeatedly reload the same user/profile/points/history data. This is especially visible when the backend is far away because every avoidable request becomes a user-visible wait.
|
||||
|
||||
The recommended refactor is to keep the UI unchanged and introduce a lightweight, project-owned data layer that centralizes:
|
||||
|
||||
* request in-flight dedupe
|
||||
* TTL cache and stale-while-revalidate behavior
|
||||
* explicit invalidation after writes
|
||||
* route-level prefetch
|
||||
* shared resource state for pages
|
||||
* optimistic local updates where safe
|
||||
|
||||
Do not start by sprinkling `localStorage` or ad hoc memoization into components. That would reduce some requests but preserve the deeper problem: every component still owns its own data lifecycle.
|
||||
|
||||
## Current architecture diagnosis
|
||||
|
||||
### 1. API helpers are typed, but not stateful
|
||||
|
||||
`web/src/lib/api.ts` provides typed functions such as `getUserProfile`, `getPointsBalance`, `getAgentHistory`, and `getNotifications`. Except for points, those functions always hit the backend.
|
||||
|
||||
This is clean as a transport layer, but incomplete as a frontend data layer.
|
||||
|
||||
### 2. Components own network lifecycle
|
||||
|
||||
Examples:
|
||||
|
||||
* `AppShell` fetches profile globally, but `SettingsPage`, `GeneralSettingsPage`, and `ProfileDetailPage` refetch profile.
|
||||
* `Dashboard` fetches points/history/unread count on mount.
|
||||
* `HistoryListPage` fetches history again even if dashboard just fetched it.
|
||||
* `StorePage` fetches packages on mount even though packages are stable config-style data.
|
||||
* `NotificationPage` owns its own list and mark-read updates without updating dashboard unread-count cache.
|
||||
|
||||
Each page is locally correct, but globally wasteful.
|
||||
|
||||
### 3. Loading states are page-local
|
||||
|
||||
Because data state is local to each page, the UI often transitions from a populated page to a blank/loading page even if relevant data was just fetched elsewhere. This makes high-latency backend access feel worse than it needs to.
|
||||
|
||||
### 4. Mutations do not have a shared invalidation model
|
||||
|
||||
Examples:
|
||||
|
||||
* `updateUserSettings` returns updated profile, but only the calling component updates local state.
|
||||
* `updateUserProfile` / `uploadAvatar` can leave `AppShell` profile stale unless context is manually updated.
|
||||
* `markNotificationRead` updates the notification page list, but dashboard unread count is not centrally invalidated.
|
||||
* `enqueueDivinationRun` should invalidate points and history after success, but that is not represented in a central policy.
|
||||
|
||||
## Target architecture
|
||||
|
||||
### Layer 1: transport stays simple
|
||||
|
||||
Keep `authFetch`, `authFetchRaw`, `apiUrl`, `api-routes`, and RFC7807 parsing as the low-level transport boundary.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
* auth refresh and 401 handling
|
||||
* JSON request/response parsing
|
||||
* SSE raw response support
|
||||
* no UI cache knowledge
|
||||
|
||||
### Layer 2: API functions remain typed commands
|
||||
|
||||
Keep `web/src/lib/api.ts` as the typed backend API command layer.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
* endpoint-specific payload construction
|
||||
* response types
|
||||
* backend data mapping helpers
|
||||
* no React hooks
|
||||
|
||||
Change:
|
||||
|
||||
* Remove ad hoc points-only cache from this file.
|
||||
* Move caching into a dedicated data/cache layer.
|
||||
|
||||
### Layer 3: new app data client
|
||||
|
||||
Add `web/src/lib/data-client.ts` or `web/src/lib/query-client.ts`.
|
||||
|
||||
Core capabilities:
|
||||
|
||||
```ts
|
||||
type CacheKey = readonly string[];
|
||||
|
||||
interface CacheEntry<T> {
|
||||
data?: T;
|
||||
error?: unknown;
|
||||
updatedAt: number;
|
||||
expiresAt: number;
|
||||
promise?: Promise<T>;
|
||||
}
|
||||
|
||||
interface QueryOptions<T> {
|
||||
key: CacheKey;
|
||||
ttlMs: number;
|
||||
fetcher: () => Promise<T>;
|
||||
staleWhileRevalidate?: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
Required operations:
|
||||
|
||||
* `query<T>(options): Promise<T>`: returns fresh data, dedupes in-flight request.
|
||||
* `peek<T>(key): T | undefined`: synchronous read for instant UI seeding.
|
||||
* `set<T>(key, data, ttlMs)`: update cache after mutations.
|
||||
* `invalidate(keyPrefix)`: remove or mark stale.
|
||||
* `prefetch(options)`: warm cache without forcing UI loading.
|
||||
* `clearAll()`: call on logout or terminal auth failure.
|
||||
|
||||
Why local helper first:
|
||||
|
||||
* Project currently has no query library.
|
||||
* The data model is small enough to solve with a typed helper.
|
||||
* Avoid adding dependency and architecture churn before measuring gains.
|
||||
|
||||
Future option:
|
||||
|
||||
* If data flows grow, migrate the same key model to TanStack Query later. The refactor should name cache keys and invalidation policies in a way that would map cleanly.
|
||||
|
||||
### Layer 4: resource-specific services
|
||||
|
||||
Add resource wrappers, for example:
|
||||
|
||||
* `web/src/lib/resources/profile.ts`
|
||||
* `web/src/lib/resources/points.ts`
|
||||
* `web/src/lib/resources/store.ts`
|
||||
* `web/src/lib/resources/history.ts`
|
||||
* `web/src/lib/resources/notifications.ts`
|
||||
|
||||
Each resource owns:
|
||||
|
||||
* cache key
|
||||
* TTL
|
||||
* fetch function
|
||||
* invalidation after writes
|
||||
* local patch helpers
|
||||
|
||||
Example:
|
||||
|
||||
```ts
|
||||
export const profileKey = ['profile'] as const;
|
||||
|
||||
export function getProfileResource() {
|
||||
return query({
|
||||
key: profileKey,
|
||||
ttlMs: 5 * 60_000,
|
||||
fetcher: getUserProfile,
|
||||
staleWhileRevalidate: true,
|
||||
});
|
||||
}
|
||||
|
||||
export async function updateProfileResource(input: UpdateProfileRequest) {
|
||||
const updated = await updateUserProfile(input);
|
||||
set(profileKey, updated, PROFILE_TTL);
|
||||
return updated;
|
||||
}
|
||||
```
|
||||
|
||||
### Layer 5: React provider and hooks
|
||||
|
||||
Add `AppDataProvider` under the authenticated app shell.
|
||||
|
||||
Hooks:
|
||||
|
||||
* `useResource(key, loader, options)`
|
||||
* `useProfile()`
|
||||
* `usePoints()`
|
||||
* `useHistorySummary()`
|
||||
* `useHistoryThread(threadId)`
|
||||
* `useNotifications(locale)`
|
||||
* `useUnreadCount()`
|
||||
|
||||
Hook behavior:
|
||||
|
||||
* seed UI from `peek()`
|
||||
* return cached data immediately when available
|
||||
* refresh in background when stale
|
||||
* expose `{ data, loading, refreshing, error, reload }`
|
||||
* never bypass auth 401 behavior
|
||||
|
||||
This keeps UI unchanged: pages still render the same cards/lists/forms, but data source shifts from local `useEffect` to shared hooks.
|
||||
|
||||
## Cache policy by data domain
|
||||
|
||||
| Domain | Key | TTL | Stale UI allowed | Invalidate on |
|
||||
| --- | --- | ---: | --- | --- |
|
||||
| Auth session | localStorage `meeyao_auth` | token expiry | no | logout, 401 refresh failure |
|
||||
| Profile/settings | `['profile']` | 5 min | yes | profile update, settings update, avatar upload, logout |
|
||||
| Points balance | `['points', 'balance']` | 30-60 sec | yes | divination run accepted/success, purchase, manual reload, logout |
|
||||
| Packages | `['points', 'packages']` | 30 min | yes | app reload, explicit admin/package changes if added later |
|
||||
| Dashboard history summary | `['history', 'summary']` | 60 sec | yes | divination run success, follow-up success, session delete |
|
||||
| History full list | `['history', 'list']` | 60 sec | yes | divination run success, follow-up success, session delete |
|
||||
| History thread | `['history', 'thread', threadId]` | 2-5 min | yes | follow-up success for same thread, session delete |
|
||||
| Notifications list | `['notifications', locale]` | 60 sec | yes | mark read/all read, new notification polling if added |
|
||||
| Unread count | `['notifications', 'unread-count']` | 30 sec | yes | mark read/all read, notification list refresh |
|
||||
|
||||
## Display-state model
|
||||
|
||||
Every page should distinguish:
|
||||
|
||||
* `loading`: no cached data yet and initial request is pending
|
||||
* `refreshing`: cached data is visible while background request updates it
|
||||
* `error`: no usable data, or background refresh failed
|
||||
* `stale`: data is visible but may be older than TTL
|
||||
|
||||
UI stays visually the same, but behavior improves:
|
||||
|
||||
* If cached data exists, do not show full-page loading.
|
||||
* Use existing skeleton/spinner only for first load.
|
||||
* On refresh failure, keep visible stale data and show a small non-blocking error only where useful.
|
||||
|
||||
## Route prefetch strategy
|
||||
|
||||
In `AppShell`, prefetch cheap likely-next data:
|
||||
|
||||
* After auth/profile load:
|
||||
* points balance
|
||||
* unread notification count
|
||||
* dashboard history summary
|
||||
* On sidebar hover/focus/click intent:
|
||||
* Store: packages + points
|
||||
* History: history list
|
||||
* Settings/Profile/General: profile
|
||||
* Divination manual/auto: points
|
||||
* Notifications: notification list + unread count
|
||||
|
||||
Prefetch must be bounded:
|
||||
|
||||
* only authenticated routes
|
||||
* no SSE prefetch
|
||||
* no mutation prefetch
|
||||
* no repeated prefetch while in-flight or fresh
|
||||
|
||||
## Mutation and invalidation policy
|
||||
|
||||
### Profile/settings
|
||||
|
||||
* `updateUserProfile` and `uploadAvatar`
|
||||
* set `['profile']` to returned profile
|
||||
* update `AppShell` user context immediately
|
||||
* `updateUserSettings`
|
||||
* set `['profile']` to returned profile
|
||||
* redirect language only after cache/context update
|
||||
|
||||
### Points/store
|
||||
|
||||
* Purchase flow or future payment success:
|
||||
* invalidate `['points', 'balance']`
|
||||
* Divination run:
|
||||
* optimistically mark points stale when run is accepted
|
||||
* invalidate points and history after run finishes
|
||||
|
||||
### History
|
||||
|
||||
* Divination run success:
|
||||
* set result in sessionStorage/router state as today
|
||||
* invalidate history summary/list
|
||||
* optionally seed `['history', 'thread', threadId]` if enough data exists
|
||||
* Follow-up success:
|
||||
* invalidate `['history', 'thread', threadId]`
|
||||
* invalidate summary/list
|
||||
|
||||
### Notifications
|
||||
|
||||
* Mark one read:
|
||||
* patch `['notifications', locale]`
|
||||
* decrement `['notifications', 'unread-count']` locally
|
||||
* Mark all read:
|
||||
* patch list to all read
|
||||
* set unread count to 0
|
||||
|
||||
## Implementation phases
|
||||
|
||||
### Phase 1: measurement and guardrails
|
||||
|
||||
Deliverables:
|
||||
|
||||
* Add a dev-only request logger around `authFetch`/`apiRequest`.
|
||||
* Capture before counts for:
|
||||
* login -> dashboard
|
||||
* dashboard -> settings -> profile -> general
|
||||
* dashboard -> manual/auto divination
|
||||
* dashboard -> history
|
||||
* dashboard -> store
|
||||
* Keep UI unchanged.
|
||||
|
||||
Acceptance:
|
||||
|
||||
* We can prove before/after request count improvements.
|
||||
|
||||
### Phase 2: data client foundation
|
||||
|
||||
Deliverables:
|
||||
|
||||
* Add typed cache/data client.
|
||||
* Move existing points TTL cache into the data client.
|
||||
* Add clear-on-logout/auth-failure hook.
|
||||
* Add unit-light self checks where possible, or targeted browser checks if test infra is absent.
|
||||
|
||||
Acceptance:
|
||||
|
||||
* In-flight duplicate calls collapse into one request.
|
||||
* Cache invalidation is explicit and inspectable.
|
||||
|
||||
### Phase 3: profile/settings context refactor
|
||||
|
||||
Deliverables:
|
||||
|
||||
* AppShell uses profile resource.
|
||||
* SettingsPage/ProfileDetailPage/GeneralSettingsPage use profile resource/context instead of unconditional fetch.
|
||||
* Mutations update profile cache and shell state.
|
||||
|
||||
Acceptance:
|
||||
|
||||
* Dashboard -> settings -> profile -> general should not issue repeated `GET /users/me/profile` while profile is fresh.
|
||||
|
||||
### Phase 4: points, packages, dashboard
|
||||
|
||||
Deliverables:
|
||||
|
||||
* Points uses data client TTL + in-flight dedupe.
|
||||
* Packages cached with long TTL.
|
||||
* Dashboard data uses shared resources.
|
||||
* Store/divination/settings reuse points cache.
|
||||
|
||||
Acceptance:
|
||||
|
||||
* Dashboard -> store -> divination does not repeatedly fetch points/packages while fresh.
|
||||
|
||||
### Phase 5: history and notifications
|
||||
|
||||
Deliverables:
|
||||
|
||||
* History list/summary/thread resources.
|
||||
* Dashboard and HistoryListPage share cached history.
|
||||
* Notification list/unread resources with local patch after mark-read.
|
||||
|
||||
Acceptance:
|
||||
|
||||
* Dashboard -> history reuses fresh history data.
|
||||
* Mark-read updates unread count without waiting for dashboard reload.
|
||||
|
||||
### Phase 6: perceived-performance polish
|
||||
|
||||
Deliverables:
|
||||
|
||||
* Replace full-page loading on cached routes with stale data + subtle refreshing state.
|
||||
* Add bounded route prefetch in AppShell.
|
||||
* Add regression browser checks for mobile/desktop.
|
||||
|
||||
Acceptance:
|
||||
|
||||
* UI remains visually unchanged at rest.
|
||||
* Route transitions feel instant when data is cached.
|
||||
|
||||
## Risks and mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
| --- | --- |
|
||||
| Stale user-visible data | Short TTL for volatile data; explicit invalidation after writes. |
|
||||
| Auth cache leaking between users | Clear all data cache on logout, missing auth, refresh failure, and user id change. |
|
||||
| Hidden background refresh errors | Keep stale data visible but expose reload/error state for first-load failures. |
|
||||
| Over-prefetch increasing traffic | Prefetch only small GET endpoints and rely on in-flight/fresh checks. |
|
||||
| Refactor touching too many pages at once | Ship by phases with request-count verification after each phase. |
|
||||
| UI drift | Do not alter markup/layout except replacing data source/loading conditions. |
|
||||
|
||||
## Recommended final architecture
|
||||
|
||||
```
|
||||
React Page
|
||||
-> useProfile/usePoints/useHistory/useNotifications
|
||||
-> Resource wrapper (keys, TTL, invalidation)
|
||||
-> DataClient (cache, in-flight dedupe, stale-while-revalidate)
|
||||
-> typed API functions in api.ts
|
||||
-> authFetch/apiRequest transport
|
||||
-> backend
|
||||
```
|
||||
|
||||
The core principle: components describe what data they need; resource wrappers decide when the backend actually needs to be called.
|
||||
Reference in New Issue
Block a user