myxce/.kiro/specs/rbac-persistence/brief.md

Brief: rbac-persistence

Problem

Cluster admins manage roles and permissions through the apiserver's REST API:

• create and delete roles;
• add permissions to a role;
• assign roles to users and remove them.

Those changes live only in apiserver memory. When the apiserver restarts, every change made through the API since startup silently reverts to whatever config/rbac.yaml says. There is no error, warning or audit trail. An admin who granted a user access (or, worse, revoked it) finds the change gone after a routine restart or upgrade. A revoked permission silently coming back is a security problem.

This matters more now:

• application-profile adds app:use/app:view grants, which admins will manage per profile.
• The job-sandbox initiative relies on RBAC to decide who may run which applications, including sandbox versus direct execution.

Current State

Documented in .kiro/steering/security.md as a known, deferred gap (flagged 2026-08-27 during the scheduler-snapshot-job-allocation persistence survey).

• Loading. store.Roles and store.UserRoles (apiserver/store/stores.go) are loaded from config/rbac.yaml at startup (LoadRBACConfig(), controllers/rbac_check.go).
• Changes made at runtime with no write-back:
  • RoleController.CreateRole / AddPermissionsForRole / DeleteRoleAndPermissions (controllers/roles.go:499, :540, :594);
  • store.AddRoleForUser / DeleteRoleForUser, called from the automatic role assignment in AuthController.Logon and from the user-role endpoints in controllers/users.go.
• Model. Permission strings are action:scope with domain types all, self, self_ug, queue (with values) and hostgroup (with values). cluster_admin bypasses checks through one early return.
• Failure class. The same save/restore field-completeness failure that persistence.md describes, but in the apiserver's in-memory RBAC store.
• HA. The apiserver is a separate process fed by controller push. Nothing covers how several apiserver instances would share RBAC state; the design must settle this.

Desired Outcome

• Every RBAC change accepted by the API is durable. It survives an apiserver restart, or it is rejected; never silently lost.
• Revocations are durable exactly like grants.
• Every RBAC change is audited: who changed what, when, and old versus new value.
• On startup, the effective RBAC state equals the last successfully persisted state. Conflicts between the shipped rbac.yaml defaults and API changes are resolved by a documented, predictable rule.
• If there are several apiserver instances, they converge on the same RBAC state (the design decides how).

Approach

Design decides. Candidates noted at discovery:

• (a) Write-through to rbac.yaml. Atomic write (temp file plus rename), preserving the builtin roles. Simple, but a single node's file, and it merges badly with shipped defaults.
• (b) Store RBAC state in the controller's replicated state. A new cluster entity persisted through the WAL and snapshots. HA-consistent and shared by every apiserver, but a much larger HA/persistence change.
• (c) A small embedded store (for example SQLite or bbolt) in the apiserver. Durable, but local to the node.
• (d) Declare API changes runtime-only and reject them. Make rbac.yaml (or xceadmin-managed config) the only source, and turn the mutating endpoints into errors or read-only. The smallest change, but it removes a capability.

Scope

• In:
  • Durability of every mutating RBAC API: roles, permissions, user-role assignments, and automatic role assignment at login.
  • The audit trail.
  • The rule that reconciles shipped defaults with API changes at startup.
  • Multi-instance consistency (whatever the design chooses).
  • Tests, including restart round-trips of grants and revocations.
• Out:
  • New permission types (application-profile owns app:*).
  • Auth backends and JWT.
  • Changing the middleware chain or the cluster_admin bypass.

Boundary Candidates

• RBAC state storage and load (the persistence mechanism).
• Mutating-API write path (durability, error on failure).
• Audit log.
• Reconciling defaults with runtime changes.

Out of Boundary

• The permission model's semantics (actions, domain types).
• Profile, queue and other resource-specific permission checks.

Upstream / Downstream

• Upstream: the existing RBAC store and APIs. If approach (b) is chosen, the controller WAL/snapshot machinery.
• Downstream: application-profile (durable app:use grants) and any future RBAC-governed feature.

Existing Spec Touchpoints

• Extends: none (closes a gap recorded in security.md).
• Adjacent: application-profile, admin-config-tool (if RBAC moves into xceadmin-managed config).

Constraints

• Middleware chain unchanged: WhitelistURL → WhitelistIP → RateLimit → JWT → CORS → RBAC.
• Fail closed. If a change cannot be persisted, the API returns an error and the in-memory state is left unchanged.
• Never log secrets (security.md).
• If approach (b): HA/persistence rules, and the full 3-phase workflow.
• When done, update security.md to remove the "Known gap" note.