lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(scheduler-core): define replay stream envelope schema (task 7.1)

Browse source 
Adds ReplayCluster/ReplayHeader/SnapshotEntry/ReplayEvent/
SessionSummary/ReplaySession to controller.capnp, additively, first
task of Phase 3 (Foundation: Replay Wire Format).

Requirements.md's AC 20 capnp sketch isn't self-contained -- ReplayEvent's
union references JobStateChangeEvent/JobUpdateEvent (task 7.2's job),
SnapshotEntry.data/ReplayDecision's union reference SnapshotData/
DispatchDecision/PendDecision/PreemptDecision (task 7.3's job), none
of which exist yet. Resolved by verifying capnp's real constraints
against the toolchain directly: SnapshotData is an empty {} placeholder
(task 7.3 replaces its mapname'd C struct wholesale when adding the
real ~11 fields), ReplayDecision holds only {ts} (task 7.3 appends a
body union at new ordinals), and ReplayEvent's union holds only the 9
members with pre-existing backing types (Conf4Sched/SchedJobData/
WorkerData/ResourceData/ResValList/QueueConf/SchedJobRusages/Void),
deferring jobStateChange/jobUpdate to task 7.2 as new ordinals --
capnp requires strictly sequential ordinals with no holes, so this
split is compiler-mandated, not stylistic.

Round-trip test added to test_e2e_capnp.c. Full project build (all
binaries sharing this schema) and 1075/1075 controller_unit_test
verified clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019dw5WnN3DXStaEaPYfACU1
This commit is contained in:
Rongsong Shen 2026-08-04 09:38:13 +00:00
parent d39ffc1b4e
commit 68a6196517
Signed by: rshen
SSH key fingerprint: SHA256:8Vf2pZ3h6rJa7EhGK+WaAQ9Mta/apZvteW6g9U/WKKk
4 changed files with 407 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

105
src/schema/controller.capnp
View file

@ -977,3 +977,108 @@ struct CtrlHelloReply $C.mapname("ctrl_hello_reply_t") {
needsSnapshot @1 :Bool $C.mapname("needs_snapshot");
configTag @2 :UInt64 $C.mapname("config_tag");
}
# ── Replay stream envelope schema (scheduler-core Group 7 task 7.1,
# Req 5 AC 20) ────────────────────────────────────────────────────────────
# `.replay` file format: a single stream of consecutive capnp messages --
# ReplayHeader (msg 1) -> SnapshotEntry (msg 2, MUST be the first data
# message) -> ReplayEvent/ReplaySession cycles -> periodic SnapshotEntry
# checkpoints -> final ReplayEvent{endSession}. See requirements.md Req 5
# AC 20 for the full framing prose; this schema captures only the message
# *shapes* -- the file-level reader/writer that produces/consumes the
# stream is task 9+, entirely out of Group 7's (schema-only) scope.
#
# Struct-ownership split across Group 7 tasks, so a fresh implementer on
# 7.2/7.3/7.4 knows exactly what to add and where. Every later addition is a
# NEW ordinal on a struct THIS task already created -- nothing below is ever
# renumbered or restructured by a later task:
# - task 7.1 (this task): ReplayCluster, ReplayHeader, SnapshotData
# (zero-field placeholder), SnapshotEntry, ReplayEvent (9 members, all
# reusing pre-existing structs), SessionSummary (fully real),
# ReplayDecision (ts-only stub), ReplaySession.
# - task 7.2: adds the new JobStateChangeEvent + JobUpdateEvent structs,
# then adds two new union members (jobStateChange, jobUpdate) to
# ReplayEvent's `body` union at new ordinals (10, 11) -- purely additive.
# - task 7.3: adds SnapshotData's real 11 fields (Req 5 AC 21) starting at
# ordinal @0 -- replacing this task's zero-field placeholder wholesale,
# both in this schema and in typemap.h's sched_snapshot_data_t (which
# has no capnp-visible fields yet, so nothing there is "renumbered"
# either). task 7.3 also adds DispatchDecision/PendDecision/
# PreemptVictim/PreemptDecision, then adds a `body :union { dispatch @1:
# DispatchDecision; pend @2: PendDecision; preempt @3: PreemptDecision;
# }` to ReplayDecision -- purely additive new ordinals on the ts@0
# struct this task creates (see the JobControl/JoinData `:union` group
# precedent above: a union group name never consumes its own ordinal,
# only its member variants do, so this is safe to bolt on later).
struct ReplayCluster $C.mapname("replay_cluster_t") {
name @0: Text;
controllers @1: List(Text) $C.maplistcount("n_controllers"); # ["host1:9991", "host2:9992", "host3:9993"]
recordingHost @2: Text $C.mapname("recording_host"); # controller hostname that made the recording
}
struct ReplayHeader $C.mapname("replay_header_t") {
recorderVersion @0: Text $C.mapname("recorder_version"); # scheduler binary version that wrote this file
configHash @1: Data $C.mapname("config_hash"); # sha256 of xce_conf4sched_t
cluster @2: ReplayCluster;
recordingStartTs @3: UInt64 $C.mapname("recording_start_ts"); # logical time recording started (ns)
recordingStartWall @4: Text $C.mapname("recording_start_wall"); # wall-clock time recording started (ISO 8601)
schedPolicy @5: Text $C.mapname("sched_policy"); # active picker plugin name (e.g. "builtin_fcfs")
}
# Placeholder -- task 7.3 replaces this with the full snapshot schema
# (Req 5 AC 21): config, jobs, queues, workers, resources, resvals, users,
# projects, prngSeed, internalState, internalVersion (see design.md's Wire
# Protocol > ReplayFormat section for the already-committed 11-field
# layout). Deliberately left with ZERO capnp fields here (instead of a
# placeholder scalar) so task 7.3 is free to start numbering its real
# fields at ordinal @0 exactly as design.md specifies, with no renumbering.
struct SnapshotData $C.mapname("sched_snapshot_data_t") {
}
struct SnapshotEntry $C.mapname("replay_snapshot_entry_t") {
ts @0: UInt64; # logical time of snapshot
data @1: SnapshotData; # full core state
}
# ReplayEvent.body reuses 7 pre-existing wire structs verbatim (Conf4Sched,
# SchedJobData, WorkerData, ResourceData, ResValList x2, QueueConf,
# SchedJobRusages -- verified present in controller.capnp, research.md) plus
# a Void end-of-recording marker. jobStateChange/jobUpdate (task 7.2) are
# added later as new union members at ordinals 10/11, once
# JobStateChangeEvent/JobUpdateEvent exist.
struct ReplayEvent $C.mapname("replay_event_t") {
ts @0: UInt64; # logical time
body :union $C.mapuniontag("body_which") {
configPush @1: Conf4Sched $C.mapname("config_push");
jobsPush @2: SchedJobData $C.mapname("jobs_push");
workersPush @3: WorkerData $C.mapname("workers_push");
resourcesPush @4: ResourceData $C.mapname("resources_push");
resvalsFull @5: ResValList $C.mapname("resvals_full");
resvalsDelta @6: ResValList $C.mapname("resvals_delta");
queueUpdate @7: QueueConf $C.mapname("queue_update");
jobRusage @8: SchedJobRusages $C.mapname("job_rusage");
endSession @9: Void; # recording stop marker
}
}
struct SessionSummary $C.mapname("session_summary_t") {
durationNs @0: UInt64 $C.mapname("duration_ns");
nDecisions @1: UInt32 $C.mapname("n_decisions");
nDispatched @2: UInt32 $C.mapname("n_dispatched");
nPended @3: UInt32 $C.mapname("n_pended");
}
# Placeholder -- task 7.3 adds `body :union { dispatch @1: DispatchDecision;
# pend @2: PendDecision; preempt @3: PreemptDecision; }` here once those
# payload types exist (purely additive new ordinals 1-3; ts@0 untouched --
# see the JobControl/JoinData precedent noted above).
struct ReplayDecision $C.mapname("replay_decision_t") {
ts @0: UInt64;
}
struct ReplaySession $C.mapname("replay_session_t") {
beginRound @0: UInt64 $C.mapname("begin_round"); # logical time (ts value)
decisions @1: List(ReplayDecision) $C.maplistcount("n_decisions");
summary @2: SessionSummary; # optional diagnostic metadata
}