docs(review): project and spec review: conflicts, code gaps, priorities
27 cross-spec conflicts among the 30 pending specs (5 high), code gaps against the five product goals and the no-shared-storage use case (silent failures such as unenforced queue/host close and REST submit dropping job limits, cwd, env and file transfers; inter-component and apiserver security; file staging; monitoring and accounting; scheduling policy; operations), and a tiered priority list (spec hygiene, P0 defects, P1 foundation specs, P2 features, P3 initiatives). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
a68aac1a2b
commit
94472ee3e7
1 changed files with 363 additions and 0 deletions
363
.kiro/reviews/project-gap-review-2026-10-10.md
Normal file
363
.kiro/reviews/project-gap-review-2026-10-10.md
Normal file
|
|
@ -0,0 +1,363 @@
|
|||
# XCE Project and Spec Review: Conflicts, Gaps and Priorities (2026-10-10)
|
||||
|
||||
Scope: every pending spec under `.kiro/specs/` (30 specs that are not
|
||||
implemented), the steering rules, and the code base against the product goals
|
||||
in `.kiro/steering/product.md`. The findings come from four parallel
|
||||
investigations (spec conflicts; job lifecycle and hosts; scheduling;
|
||||
security, HA and operations) plus direct checks of the most consequential
|
||||
claims (marked *verified*). Paths are relative to `src/` unless they start
|
||||
with `.kiro/`.
|
||||
|
||||
## 1. Summary
|
||||
|
||||
- **Specs:** 62 specs are implemented or complete, 2 superseded, 30 pending
|
||||
(19 brief only, 5 initialized, 3 requirements generated, 1 design
|
||||
generated, 1 brief, 1 decomposed umbrella). The pending set has **27
|
||||
cross-spec problems: 5 high, 15 medium, 7 low** (section 2).
|
||||
- **Code:** HA and persistence (product goal 5) are mature: no open bugs
|
||||
(all 564 bug records are closed), and phase P2.5 is done for workers,
|
||||
scheduler and apiserver. The biggest gaps are elsewhere:
|
||||
- several **shipped features silently do nothing**: queue close, host
|
||||
close, job limits, cwd, env and file transfer submitted through REST, the
|
||||
user's signal choice;
|
||||
- **security** between components and in the apiserver;
|
||||
- the **no-shared-storage** use case (file staging, output, checkpoint);
|
||||
- **job monitoring** (exit status, pend reasons and history never reach
|
||||
users);
|
||||
- **scheduling policy** depth (no reservation, backfill, fairshare,
|
||||
preemption, priority, windows);
|
||||
- **operations** (metrics, health, docs, backup).
|
||||
- **Priorities** (section 4): fix the silent defects and the security
|
||||
basics first (P0), then the foundation specs many others depend on (P1),
|
||||
then feature specs (P2), then the new initiatives (P3). Before any
|
||||
further spec generation, resolve the 5 high spec conflicts (section 4,
|
||||
"Spec hygiene").
|
||||
|
||||
## 2. Spec conflicts
|
||||
|
||||
Severity: H = must be resolved before the affected specs go further;
|
||||
M = resolve in the affected spec's next phase; L = cleanup.
|
||||
|
||||
### High
|
||||
1. **Scheduler reconnect: xce-node-manager vs component-failover-adoption
|
||||
(implemented) and component-rejoin-resync.** The node-manager design
|
||||
(Req 6.3; design.md:400, :12) forbids the scheduler from rejoining any
|
||||
controller other than the one that started it and removes the reconnect
|
||||
paths. component-failover-adoption (implementation-complete, Req 2/7)
|
||||
makes the scheduler reconnect across the candidate list.
|
||||
component-rejoin-resync still has scheduler reconnect cost in scope.
|
||||
*Resolve:* in the node-manager revision, either retire the scheduler half
|
||||
of failover-adoption (recorded in that spec) or keep reconnect only to
|
||||
the same controller; drop the scheduler from component-rejoin-resync.
|
||||
2. **Who replaces a lost or failed service replica.** service-supervision
|
||||
defaults "rerunnable after host loss" to yes; host-failure-job-recovery
|
||||
requeues rerunnable jobs; service-replica-set submits a replacement when
|
||||
a replica ends. Together: N+1 replicas, or a replica that never ends so
|
||||
the replica set never reacts. *Resolve:* replicas of a replica-set
|
||||
service are never rerunnable and never self-requeue; the replica set
|
||||
always replaces them; requeue applies to standalone service jobs only.
|
||||
Record it in application-profile's service section and job-config-merge.
|
||||
3. **application-profile brief still contradicts its own update and
|
||||
job-config-merge.** "Resolution at submit time" and "Approach" still say
|
||||
effective values are resolved and persisted at submit; the 2026-10-09
|
||||
update and job-config-merge say live merge, derived values never
|
||||
persisted; roadmap.md:64 says the old thing. *Resolve:* rewrite those
|
||||
sections and the roadmap line before `/kiro-spec-requirements
|
||||
application-profile`; settle whether dispatch freezes execution values on
|
||||
the job (then field-completeness applies).
|
||||
4. **Pend-reason codes have no owner.** New codes come from
|
||||
job-time-window, job-dependency, project-config, job-config-merge,
|
||||
provisioning-demand, parjob-placement and service-replica-set; several
|
||||
carry parameters (time, project, limit) that `JobPendReason`
|
||||
(mainReason/hostReasons/resReasons) cannot hold; job-time-window and
|
||||
job-dependency both claim the skipped-job publish path;
|
||||
job-status-detail-display lists only the three existing codes and puts
|
||||
"deciding what reasons are" out of scope. *Resolve:* one owner (the first
|
||||
spec to land, or a small shared spec) for the code registry, the
|
||||
parameter field and the skipped-job publish path; the display table is
|
||||
extensible and each producing spec adds its rows; every new reason must
|
||||
be stable across sessions or it defeats pend-reason-push-dedup.
|
||||
5. **resource-distribution-mapping is blocked, but four specs build on it**
|
||||
(elastic-shared-resources, provisioning-demand for licenses,
|
||||
service-endpoint ports, project-guarantee). research.md says "do not
|
||||
generate requirements until a scope is chosen". *Resolve:* record the
|
||||
option-2 (shared pool) decision, generate requirements, and confirm it
|
||||
also covers per-replica port numbers.
|
||||
|
||||
### Medium
|
||||
6. job-config-merge treats queue cpu/mem/run limits as hard limits that are
|
||||
"wired through the merge", but job-config-layering.md says the queue never
|
||||
owns limits enforced on the job. Also undefined: a job with no runtime in
|
||||
a queue that has a maximum, and services with no run limit. Define "no
|
||||
value vs ceiling" and the service case.
|
||||
7. parjob-placement's host-count limit is a placement bound decided by the
|
||||
planner, not a submit-time value check; the layering rule has no such
|
||||
case. Add a third case (queue placement bounds applied by the planner) or
|
||||
require a declared host count.
|
||||
8. job-time-window uses the job's own run limit for its unmeetable-window
|
||||
rule; after job-config-merge the effective limit can come from the
|
||||
profile and change. Read the effective limit; add an "estimate exceeds
|
||||
effective limit" reason.
|
||||
9. Deadline, run-limit and user kills vs the service restart loop: neither
|
||||
spec says whether they are terminal. Make them terminal and define how
|
||||
the restart loop and deadline timer share the worker helper.
|
||||
10. scheduler-preemption needs a revision before design: released memory of
|
||||
SIGSTOP'd jobs vs resource-accounting.md, suspended service replicas
|
||||
failing liveness, a stale "dispatch-commit not designed" note, state
|
||||
naming (SSUSP/USUSP), and project-guarantee needing kill/requeue reclaim
|
||||
that v1 excludes.
|
||||
11. Requeue semantics across host-failure-job-recovery, job-dependency,
|
||||
job-dependency-insight and controller-job-cleanup: which states are
|
||||
requeueable; dependency verdicts for a requeued or unknown parent;
|
||||
cleanup must not remove a job about to be requeued.
|
||||
12. controller-job-cleanup vs job-dependency race: a dependent submitted
|
||||
between the sweep's selection and the removal fact references a removed
|
||||
parent; re-check at commit on the primary.
|
||||
13. service-replica-set rules run in the apply seam on every node, but only
|
||||
the primary may append facts; backoff and rollout deadlines are timers.
|
||||
Fire rules only on the primary after commit; use trigger-only one-shots
|
||||
handing off to the slow lane.
|
||||
14. Provisioning overlap: host templates (elastic-hosts, provisioning-demand,
|
||||
resource-provisioner), expected/orphan instances (elastic-hosts and
|
||||
resource-provisioner), idle times (three specs). Assign: templates and
|
||||
join matching to elastic-hosts; request/instance lifecycle, orphans and
|
||||
reclaim policy to resource-provisioner; idle detection only to
|
||||
provisioning-demand.
|
||||
15. Preemption and provisioning-demand both react to allocation failure;
|
||||
order undefined (preempt first, count demand only when no victim set).
|
||||
16. RBAC scope types spread over application-profile (`app`), project-config
|
||||
(`project`), service-replica-set; rbac-persistence's model lists only the
|
||||
old scopes; security.md lists another set; validation of unknown names
|
||||
differs; project-config's roadmap entry says no dependencies but relies
|
||||
on rbac-persistence.
|
||||
17. The scheduler-framework umbrella plans a project-aware picker and
|
||||
project priority; project-config excludes project influence on ordering;
|
||||
the project-config brief and roadmap still mention project priority;
|
||||
`queue->project_name` exists in code (`scheduler/include/sched/types.h:115`).
|
||||
18. scheduler-allocation Req 4 AC1 ("plan shape is a pure function of
|
||||
maxTasks") is not yet amended; parjob-placement, provisioning-demand and
|
||||
job-resize depend on the change.
|
||||
19. xce-node-manager's new cross-spec items 4 and 5 need a channel to the
|
||||
controller it does not have (local socket only). Route through the local
|
||||
worker (it reports "restarting"; it exits with a do-not-restart code when
|
||||
decommissioned).
|
||||
20. Liveness probes for sandboxed jobs would run as root outside the unit;
|
||||
job-sandbox-execution must offer "probe inside the unit" and "restart
|
||||
without stopping the unit".
|
||||
|
||||
### Low
|
||||
21. Nobody owns a general `jobs modify` (job-time-window, job-dependency,
|
||||
project-config, application-profile all assume one).
|
||||
22. service-replica-set's cross-job spread rule has no scheduler owner.
|
||||
23. Missing or unlisted dependencies: peer-authentication-hardening and
|
||||
lsf-config-migration have no spec; edges service-supervision →
|
||||
job-config-merge, host-failure → application-profile, job-resize →
|
||||
job-config-merge are not in the roadmap.
|
||||
24. Stale statements: roadmap.md:9 ("next: design scheduler-allocation"),
|
||||
component-rejoin-resync ("failover-adoption in progress"),
|
||||
scheduler-alter-pend-reasons ("preason-equality 0/8"), commit-batch-window
|
||||
premise predates scheduler-dispatch-commit; pend-reason-push-dedup,
|
||||
commit-batch-window and xce-node-manager lack roadmap entries.
|
||||
25. job-config-merge: recordings must carry profile/queue/project config
|
||||
changes so the simulator reproduces live re-evaluation.
|
||||
26. elastic-hosts templates include dispatch windows, which the layering rule
|
||||
gives to queues.
|
||||
27. Service objects and dependency walks may reference jobs already removed
|
||||
by cleanup; define the display.
|
||||
|
||||
Checked without conflict: pre/post-exec ownership; queue `res_req`
|
||||
deprecation; project membership vs RBAC; delaymgr non-blocking in pending
|
||||
specs (except #13); field completeness statements; the pend-reason display
|
||||
chain; job-dependency/cleanup `job_has_dependents()` interface; no live
|
||||
references to the removed service-head-agent; sandbox single vs multinode
|
||||
boundaries; job-time-window vs preemption.
|
||||
|
||||
## 3. Code gaps against the product goals
|
||||
|
||||
### Goal 1: Job lifecycle management
|
||||
| Sev | Gap | Evidence |
|
||||
|---|---|---|
|
||||
| H | **REST submit drops most job settings** *(verified)*: `modelJobToSubmit` never sets `exdata`, `exechome` or `execcwd`, so `--cwd`, `--env`, file transfers, interactive mode, every `*_limit` (incl. `-W` run limit), `-i/-o/-e`, asked hosts and begin/end time never reach the worker | `frontend/apiserver/backend/convert.go:323-375` |
|
||||
| H | File-transfer key mismatch *(verified)*: apiserver writes `transfer_files`, worker parses `transfer_file` | `controllers/jobs.go:1923` vs `worker/src/xcecd_exdata.c:115` |
|
||||
| H | User signal ignored (JSON string parsed as a signal name, falls back to TERM/TSTP/CONT); ForceKill and checkpoint kill/method dropped | `controllers/jobs.go:2260-2277`; `backend/clib.go:697-705,746-762` |
|
||||
| H | Kill/stop/resume refused unless the host status is exactly 0: no control of jobs on a closed or dead host | `controller/job.c:2145-2156` |
|
||||
| H | Only kill, stop, resume, checkpoint, restart exist; REQUEUE, MIG, MODIFY, SWITCH, TOP/BOT are enum values only | `include/xce/types.h:614-627`; `controllers/jobs.go:2367-2384`; `controller/job.c:2171-2222` |
|
||||
| H | Restart from checkpoint can never work (`ChkpntInfo` never filled); checkpoint hard-wired to `/shared/ckpt` | `controllers/jobs.go:934-970`; `worker/src/xcecd_worker.c:24` |
|
||||
| H | Monitoring fields never filled: pend reason, exit status, pids, usage per host, reserve time, exec uid, checkpoint info; exit code and fail reason are FIXMEs in the controller | `models/jobs.go:528-571`; `backend/convert_pure.go:284-395`; `controller/apiserver.c:248,283,318` |
|
||||
| H | No job history/accounting: finished jobs vanish from REST after 3600 s; no accounting file | `include/config.h:85`; `controller/persist.c:8736`; `backend/handler.go:83` |
|
||||
| H | Jobs on a dead host stay RUN; TRANSIENT/UNKWN never set | `controller/sched.c:632`; `controller/apiserver.c:509`; `persist.c:8725` |
|
||||
| M | Job control on a multi-host job goes only to the first host | `controller/job.c:2126-2131` |
|
||||
| M | SSUSP never set; no load-based suspend | `controller/worker.c:409-411` |
|
||||
| M | xcecli lacks requeue, modify, signal, switch, top/bottom | `xcecli/cmd/jobs/jobs.go:25-35` |
|
||||
| M | `xcecd_virtual` restart loses its jobs (no recovery), one concurrent job only | `worker/src/xcecd.c:511-555` |
|
||||
|
||||
### Use case: no-shared-storage HPC
|
||||
| Sev | Gap | Evidence |
|
||||
|---|---|---|
|
||||
| H | Stage-in/stage-out not implemented in the worker (`transfer_info` parsed, never used) | `worker/src/xcecd_exdata.c:114-175`; `xcecd_worker.c:505,774-810` |
|
||||
| H | Remote-to-local copy is a placeholder (empty MD5, nothing copied; shared package globals) | `controllers/jobs.go:34-42,1455-1493,2979-2995` |
|
||||
| H | Uploads land on the apiserver host (`/home/<user>/apiserver_uploads`); workers cannot fetch them; file DELETE disabled | `controllers/files.go:40,144-155`; `router/routers.go:150-153` |
|
||||
| M | No result download/list in xcecli; output in `/tmp` on the exec host, lost with the host; user `-o/-e` never reach the worker | `xcecli/common/request/request.go:260`; `worker/include/job_output_path.h:24-41` |
|
||||
|
||||
### Goal 2: Queue and scheduling
|
||||
| Sev | Gap | Evidence |
|
||||
|---|---|---|
|
||||
| H | **Queue close/inactive not enforced** *(verified)*: `sched_queue_is_ready()` and `sched_job_is_ready()` are `return 1` stubs | `scheduler/src/core.c:646-649` |
|
||||
| H | **Host close not enforced** *(verified)*: the scheduler never checks CLOSED / CLOSED_BY_ADMIN; parjob bad-host check empty | `scheduler/src` (no reference); `plugins/parjob_allocator.c:40-48` |
|
||||
| H | Queue user/host lists not enforced (not in the scheduler's queue struct) | `include/sched/types.h:101-121` |
|
||||
| H | Dispatch and run windows not enforced | `xceadmin/.../mapping.py:397-398`; no scheduler use |
|
||||
| H | No reservation/backfill: big parallel jobs starve (picker skips a blocked job and lets smaller ones pass) | `plugins/builtin_fcfs.c:964-979`; `include/sched/types.h:166-169` |
|
||||
| H | No preemption (decision type unreachable), no fairshare, no job priority (hard-coded 0), no begin time or dependencies | `driver.c:923-936`; `schedobj.c:214`; `core_events.c:353` |
|
||||
| H | No `order[]` host sorting (not in the grammar) | `lib/section.peg:36-65`; `host_iter.c:72,126` |
|
||||
| M | span/same/affinity parsed then ignored; rusage duration/decay ignored | `include/resreq.h:233-237`; `alloc_runner.c:429` |
|
||||
| M | Consumable flag ignored (possible double count of `mem`; inferred, not run) | `resource.c:163,178`; `configs/resources.yaml:11-16` |
|
||||
| M | Shared/global resources cannot be declared from YAML; no GPU support; `sched_stop` thresholds never checked | `mapping.py:372-377,91-95,396` |
|
||||
| M | Project and queue groups never loaded | `scheduler.c:1144-1145`; `core.c:184-185` |
|
||||
| M | **Simulator does not load user/host groups**: limits that name groups can behave differently in replay than live (driver-equivalence violation) | `core_events.c:846-850` vs `scheduler.c:1144` |
|
||||
| M | A job moved to another queue takes the queue's priority, new jobs get 0, so moved jobs jump ahead | `core_events.c:1360`; `schedobj.c:684` |
|
||||
| L | Many queue YAML fields dropped or ignored (description, nice, fairshare, limits, runtimeest, reservation, preemption, job_starter, attrs, accept_interval, sched_delay) | `mapping.py:85-97`; `core_events.c:380-381` |
|
||||
| L | Per-dispatch linear walk of every queue; large parallel plans cost about maxTasks × hosts | `builtin_fcfs.c:375-400`; `parjob_allocator.c:165-200` |
|
||||
|
||||
Works well: queue-priority FCFS ordering, select filtering with a per-session
|
||||
host cache, compound and alternative resreqs, min/max task ranges
|
||||
(bug-1931), user/queue/project/host limits, the unsatisfiable-request cache,
|
||||
and recorder/simulator/REPL on one core.
|
||||
|
||||
### Goal 3: Cluster and host management
|
||||
| Sev | Gap | Evidence |
|
||||
|---|---|---|
|
||||
| H | LOCKEDU, LOCKEDW, BUSY never set; CLOSED only by an admin (never when full) | `controller/worker.c:345,1024` |
|
||||
| H | Host fields never filled: host group, job counts, topology, sched_stop, cores/threads, dispatch window; `ip_addr`/`real_load` commented out | `models/hosts.go:41-90`; `backend/convert_pure.go:297-311,492` |
|
||||
| M | Host control: open/close only (TAG/UNTAG return -1); no drain, remove or decommission; HIDDEN never set | `controller/worker.c:1013-1029`; `controllers/hosts.go:411-418` |
|
||||
| M | No host-group/rack topology (`cluster_build_groups` is a stub) | `controller/cluster.c:1513-1517` |
|
||||
|
||||
### Goal 4: Multi-tenant RBAC and pluggable auth
|
||||
| Sev | Gap | Evidence |
|
||||
|---|---|---|
|
||||
| H | LDAP and OAuth2 are stubs ("not yet implemented"), selectable without warning; PAM real (Linux + CGO) | `backend/auth/ldap.go:9-11`, `oauth2.go:9-11`, `init.go:33-36` |
|
||||
| H | Config-file passwords plaintext, compared with `!=`; shipped `admin/admin`, `user1/user1` | `config_file.go:60`; `config/auth.yaml:12-21` |
|
||||
| H | Logout does not revoke tokens (deletes a file never read); no refresh | `controllers/auth.go:69-95`; `middleware/validatehandler.go:101-116` |
|
||||
| H | Default JWT secret `xce-api-key` accepted when `XCE_JWT_SECRET` is unset | `config/app.yaml:17`; `utils/config.go:315-320` |
|
||||
| H | Shipped IP whitelist allows everyone; `X-Forwarded-For` trusted (no `SetTrustedProxies`), so client IP is spoofable | `config/app.yaml:50-51`; `router/routers.go:19` |
|
||||
| H | RBAC runtime changes in memory only (rbac-persistence is a brief) | `store/stores.go:64-93` |
|
||||
| M | `RequirePermission` and IP whitelist write an error but do not abort; `ReturnError` never aborts | `rbac_check.go:119-122`; `whitelist.go:130-132`; `response.go:30-37` |
|
||||
| M | Scope checks spread over 13 handlers; `users.go` discards errors and compares only `!= all`; a narrower `rbac:control` grant may allow self-assigning `cluster_admin` (latent, needs verification) | `controllers/users.go:41,183,404-412,455-462` |
|
||||
| M | Rate limit per endpoint only (no per-user/IP, no login brute-force limit); no audit log; CORS after JWT | `middleware/rate_limit.go:16,65-85` |
|
||||
| M | JWT algorithm not pinned in validation; nil-pointer on `auth.yaml` without `config_file` | `validatehandler.go:102-104`; `config_file.go:39` |
|
||||
| Fixed | `self_ug` acted as `all` (bug-1934), project filter rejected (bug-1935), limit scope swap (bug-1933) | commits 6ecd418d, 4a38bb3c |
|
||||
|
||||
### Goal 5 and HA phases
|
||||
| Sev | Gap | Evidence |
|
||||
|---|---|---|
|
||||
| H | **No certificate verification anywhere**: any TLS client without a certificate can send any opcode; `validate_host()` returns 1; `upgradeConfig`/`schedAdmin` check only "is primary"; `OP_replConfig` gives the full config to any peer | `lib/netio.c:1235-1269`; `myapp.c:116`; `controller/join.c:13-16`; `cluster.c:948,1004` |
|
||||
| H | Remote tasks of multi-host jobs run as root (limits never set for task contexts) | `worker/src/controller_connector.c:1028` |
|
||||
| M | P2.5 done for worker/scheduler/apiserver (libxce-failover, component-failover-adoption, worker-rejoin-resync); remaining: component-rejoin-resync (full state on rejoin), stale reads (no read leases; apiserver cache refresh 300 s), no step-down on quorum loss | roadmap.md:113,119,120 |
|
||||
| M | xce-node-manager design only (no tasks/code); needs requirements revision | roadmap.md |
|
||||
|
||||
### Operations (P3–P4)
|
||||
| Sev | Gap | Evidence |
|
||||
|---|---|---|
|
||||
| H | No metrics or health endpoints (only `/ping`) | `router/routers.go:53` |
|
||||
| H | No shipped process supervision (no systemd units/containers); real worker needs systemd/dbus, so multi-host tests rely on `xcecd_virtual` | `docs/deployment.md:287` |
|
||||
| M | Thin docs (`docs/xce.md` 138 lines, `deployment.md` 302 lines; no admin guide for auth/RBAC/LDAP/PAM/backup/upgrade; a stale contradiction at deployment.md:291 vs :105) | `src/docs/` |
|
||||
| M | No backup/restore tooling (`snapdump` todo, no config diff/rollback/history); no daemon log rotation | roadmap.md:140; deployment.md:108 |
|
||||
| L | Packaging: Nix flake + AppImage, all-or-nothing build | `flake.nix:70-104` |
|
||||
|
||||
### Bugs
|
||||
All 564 bug records are closed (550 fixed, 8 cannot-reproduce, 5 wontfix,
|
||||
1 duplicate). The defects above marked as silent failures are **not yet
|
||||
logged** as bug records; the security items are tracked only as roadmap
|
||||
items (peer-authentication-hardening, rbac-persistence).
|
||||
|
||||
## 4. Priorities for the remaining work
|
||||
|
||||
Order inside each tier is the suggested order.
|
||||
|
||||
### Spec hygiene (before any further spec generation; small, documentation only)
|
||||
- Resolve conflicts H1–H5: node-manager vs failover-adoption; replica
|
||||
replacement owner; rewrite the application-profile brief's stale
|
||||
sections; assign the pend-reason registry owner; record the
|
||||
resource-distribution-mapping option-2 decision.
|
||||
- Fix the stale roadmap statements (L24) and add missing dependency edges
|
||||
(L23); drop "project priority" from project-config (M17).
|
||||
|
||||
### P0: defects in shipped features and security basics (direct fixes as bugs, no spec)
|
||||
1. REST submit drops exdata/cwd/env/limits/files, and the `transfer_file`
|
||||
key mismatch (goal 1; silent data loss of user intent).
|
||||
2. Queue close/inactive and host close not enforced (goal 2/3; admin
|
||||
controls silently ignored).
|
||||
3. Job control: user signal ignored, ForceKill dropped, control refused on
|
||||
non-OK hosts, multi-host control only to the first host.
|
||||
4. Simulator does not load user/host groups (driver-equivalence rule).
|
||||
5. apiserver security basics: refuse the default JWT secret, token
|
||||
revocation on logout, hashed passwords and no shipped defaults,
|
||||
`SetTrustedProxies`, abort on middleware failures, pin the JWT
|
||||
algorithm, verify the `rbac:control` escalation path, the `auth.yaml`
|
||||
nil-pointer.
|
||||
6. Moved jobs taking the queue's priority (ordering bug).
|
||||
|
||||
### P1: foundation specs (unblock many others or close core product gaps)
|
||||
1. **peer-authentication-hardening** (promote from roadmap to a spec): TLS
|
||||
certificate verification, `validate_host`, role-based opcode allowlists,
|
||||
remote tasks not as root. Highest security gap.
|
||||
2. **host-failure-job-recovery**: dead-host jobs (batch gap; prerequisite
|
||||
for services and elastic hosts).
|
||||
3. **job-status-detail-display** (requirements generated), extended with
|
||||
exit status/reason, pids and usage per host, plus the pend-reason
|
||||
registry (H4).
|
||||
4. **New: job-file-staging** for the no-shared-storage use case: stage-in
|
||||
and stage-out on the worker, upload/download/list, output retrieval
|
||||
after the host is gone, checkpoint without shared storage.
|
||||
5. **New: job-accounting-history**: durable finished-job records and usage
|
||||
history beyond the 3600 s retention (feeds bacct-like views and project
|
||||
accounting).
|
||||
6. **New: queue-policy-enforcement**: queue user/host lists, dispatch/run
|
||||
windows, `sched_stop` thresholds, auto CLOSED/BUSY host states, and the
|
||||
queue fields dropped by xceadmin that remain queue-owned (per
|
||||
job-config-layering).
|
||||
7. **job-config-merge** (initialized) and **resource-distribution-mapping**
|
||||
(decision then requirements): both are dependencies of most P2/P3 specs.
|
||||
8. **rbac-persistence**: runtime RBAC changes survive restart (project and
|
||||
app scopes build on it).
|
||||
9. **New: observability**: health/readiness endpoints, metrics for
|
||||
controller, scheduler, apiserver and workers.
|
||||
10. **xce-node-manager**: requirements revision (H1, M19, cross-spec
|
||||
items), then tasks; ships process supervision.
|
||||
|
||||
### P2: scheduling and feature depth
|
||||
1. **New: slot-reservation-backfill**: reserve slots for blocked big jobs
|
||||
and backfill around them (starvation today).
|
||||
2. **scheduler-preemption** (revise requirements first, M10).
|
||||
3. **parjob-placement** (span[ptile/hosts/block], allowed task counts),
|
||||
plus host `order[]` sorting and rusage duration (new small items or part
|
||||
of it).
|
||||
4. **application-profile**, **project-config** (requirements generated),
|
||||
**job-time-window**, **job-dependency** and **job-dependency-insight**,
|
||||
**controller-job-cleanup**, **scheduler-alter-pend-reasons**,
|
||||
**pend-reason-push-dedup**, **commit-batch-window** (re-check premise).
|
||||
5. **New: job-priority-fairshare**: user job priority, top/bottom, fairshare
|
||||
by user/project.
|
||||
6. Host management: drain/remove/decommission, topology, host fields
|
||||
(fold into elastic-hosts or a small spec).
|
||||
7. Docs and backup/restore tooling (snapdump, walcompare, config history).
|
||||
|
||||
### P3: new initiatives
|
||||
- Long-running services: service-supervision, service-replica-set,
|
||||
service-endpoint.
|
||||
- Auto provisioning: provisioning-demand, resource-provisioner,
|
||||
elastic-hosts, elastic-shared-resources.
|
||||
- Sandbox execution: job-sandbox-execution, job-sandbox-multinode.
|
||||
- project-guarantee, project-delegated-membership, job-resize,
|
||||
lsf-config-migration, component-rejoin-resync (only if measured).
|
||||
|
||||
## 5. Suggested next steps
|
||||
1. Log the P0 defects as bug records (`buglog`) and fix them one by one
|
||||
with tests.
|
||||
2. Do the spec-hygiene edits (H1–H5, L23–L24, M17) in one documentation
|
||||
commit.
|
||||
3. Promote peer-authentication-hardening to a spec (`/kiro-spec-init`), then
|
||||
host-failure-job-recovery.
|
||||
4. Run `/kiro-discovery` for the new P1/P2 specs named above (job-file-staging,
|
||||
job-accounting-history, queue-policy-enforcement, observability,
|
||||
slot-reservation-backfill, job-priority-fairshare) to add them to the
|
||||
roadmap with briefs.
|
||||
Loading…
Add table
Add a link
Reference in a new issue