Skip to content

feat(core): add QuotaManager with exponential backoff and persistence#34

Open
iceteaSA wants to merge 1 commit into
cortexkit:mainfrom
iceteaSA:feat/quota-manager
Open

feat(core): add QuotaManager with exponential backoff and persistence#34
iceteaSA wants to merge 1 commit into
cortexkit:mainfrom
iceteaSA:feat/quota-manager

Conversation

@iceteaSA
Copy link
Copy Markdown
Contributor

@iceteaSA iceteaSA commented May 21, 2026

Unified quota cache and API gateway for main + fallback quota state. All consumers share one QuotaManager instance for consistent caching.

Features:

  • Inflight deduplication prevents concurrent API calls
  • Exponential backoff (60s–15min) for 429/5xx errors with escalating retry counts
  • Persists main quota and backoff state to disk via callbacks
  • Cross-process file lock guard for quota API dedup
  • Seeds from persisted storage on construction (avoids cold-start API call)
  • Integrates with FallbackAccountManager for shared staleness checks
  • Captures storage path at init to prevent test config corruption
  • Request-count-based refresh trigger (refreshEveryNRequests)

Files:

  • packages/core/src/quota-manager.ts — new (391 lines)
  • packages/core/src/accounts.ts — adds backoff helpers, FallbackAccountManager integration
  • packages/opencode/src/index.ts — QuotaManager construction, persistence callbacks, routing
  • packages/opencode/src/tests/quota-manager.test.ts — new (294 lines)

Summary by cubic

Adds a unified QuotaManager for main and fallback quotas with shared in-memory cache, a serial API gate, exponential backoff, and on-disk persistence. Integrates it into packages/opencode for routing and the quotas view, with fail-closed behavior when the Usage API is backed off and no cached quota is available.

  • New Features

    • Single QuotaManager with inflight dedup, a serialized API gate, and a cross‑process file lock (main quota).
    • Exponential backoff for 429/5xx (60s up to 15m) with persisted state and callbacks (onMainQuotaFetched, onApiError); seeds backoff and main quota from disk on startup.
    • Request-count refresh trigger via quota.refreshEveryNRequests; staleness checks shared across processes.
    • Integrates with FallbackAccountManager: seeds from persisted fallback quotas, shared staleness, and gated refreshAllFallbacks.
    • packages/opencode: uses QuotaManager for routing and quotas view, persists main quota/backoff to disk, and adds fail‑closed behavior with 429 + retry-after when backed off and uncached; captures storage path at init.
  • Migration

    • @cortexkit/anthropic-auth-core now exports QuotaManager.
    • If using FallbackAccountManager, pass a shared QuotaManager to unify cache and staleness.
    • To persist main quota/backoff, wire QuotaManager callbacks to save quota.mainQuota, quota.mainQuotaCheckedAt, and quota.mainLastQuotaApiError.

Written for commit 5938687. Summary will update on new commits. Review in cubic

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 6 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread packages/opencode/src/index.ts
Comment thread packages/core/src/accounts.ts
@iceteaSA iceteaSA force-pushed the feat/quota-manager branch 2 times, most recently from 731431a to 7a47838 Compare May 22, 2026 16:52
Unified quota cache and API gateway for main + fallback quota state.
All consumers share one QuotaManager instance for consistent caching.

Features:
- Inflight deduplication prevents concurrent API calls
- Exponential backoff (60s-15min) for 429/5xx errors
- Persists main quota and backoff state to disk via callbacks
- Cross-process file lock guard for quota API dedup
- Seeds from persisted storage on construction
- Integrates with FallbackAccountManager for shared staleness
- Captures storage path at init to prevent test config corruption
- Request-count-based refresh trigger (refreshEveryNRequests)
@iceteaSA iceteaSA force-pushed the feat/quota-manager branch from 7a47838 to 5938687 Compare May 22, 2026 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant