fix(scheduler): sim resource over-commit, byte-unit display, sync job-state-at-capture
- bug-1799/1801: fix resInUse over-commit in sim replay -- sched_account_jobs's per-round re-derive pass now skips RECONCILING/UNAVAIL workers the same way core_reset_workers already does (bug-1751), closing the gap that let a worker's used slots exceed its own capacity across a resync replay - bug-1804: revise scheduler-sim-eager-reconciliation spec (user-directed) so a captured job's job->state is set to RUN synchronously at dispatch capture, mirroring sched_commit_decision's normal-mode semantics (bug-1624), instead of deferring until its separately-recorded JOB_STATE_CHANGE event replays - bug-1800: render bytes-valued CONSUMABLE resources (mem/swp/tmp) with the same human-scaled unit formatting already used for maxmem/maxswap/maxtmp - bug-1802/1803: add then revert a break-on-ANY-event breakpoint kind after live feedback that it over-fires inside busy sessions; replace it with a prescan-based "next non-dummy session" hint in info summary instead Full scheduler suite: 77/77 tests pass (asan profile). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLSNJhNoULvVcUhXEFafQC
This commit is contained in:
parent
ffca959ed9
commit
25fb5c115a
22 changed files with 1827 additions and 303 deletions
|
|
@ -1,62 +1,242 @@
|
|||
Following is the scheduler runs in simulation mode:
|
||||
|
||||
Simluator REPL output
|
||||
```
|
||||
rshen@pc13v ~/p/x/w/s/src (scheduler-framework)> ./build.dbg/scheduler/xce_scheduler -i -s /tmp/scheduler-replay-20260907-032446.replay
|
||||
[2026-09-07 11:25:55,060] [32922] [ WARN] [xce_sched_core_load_snapshot] xce_sched_core_load_snapshot failed to resolve job picker plugin (rc=-2); picker rebuild will be skipped for this load
|
||||
[sim] replay file contains 33 sessions
|
||||
[sim] loaded: 2 jobs, snapshot @ ts=1788751486917208975 (2026-09-07T03:24:46Z)
|
||||
rshen@pc13v ~/p/x/w/s/src (scheduler-framework)> ./build.dbg/scheduler/xce_scheduler -i -s /tmp/scheduler-replay-20260911-064117.replay
|
||||
[2026-09-11 14:49:54,696] [801857] [ WARN] [xce_sched_core_load_snapshot] xce_sched_core_load_snapshot failed to resolve job picker plugin (rc=-2); picker rebuild will be skipped for this load
|
||||
[sim] replay file contains 420 sessions
|
||||
[sim] loaded: 0 jobs, snapshot @ ts=1789108877564564549 (2026-09-11T06:41:17Z)
|
||||
[sim] (jobs above reflect this snapshot only -- step/continue to replay later events)
|
||||
[sim] ready. type 'help' for commands.
|
||||
(sim) info summary
|
||||
[sim] summary: 2 jobs (0 pending, 1 running), 1 workers (1 online), 4 queues, 33 resources
|
||||
(sim) info jobs
|
||||
[sim] jobs:
|
||||
0-14-1788751409-0 DONE xce
|
||||
0-14-1788751435-0 RUN xce
|
||||
[sim] info jobs: 2 job(s)
|
||||
(sim) info worker pc13v
|
||||
[sim] summary: 0 jobs (0 pending, 0 running), 1 workers (1 online), 4 queues, 33 resources
|
||||
(sim) step
|
||||
(sim session=1 t=2026-09-11T06:41:17Z) step
|
||||
(sim session=2 t=2026-09-11T06:41:18Z) step
|
||||
(sim session=3 t=2026-09-11T06:41:19Z) step
|
||||
(sim session=4 t=2026-09-11T06:41:20Z) step
|
||||
(sim session=5 t=2026-09-11T06:41:21Z) step
|
||||
(sim session=6 t=2026-09-11T06:41:22Z) step
|
||||
(sim session=7 t=2026-09-11T06:41:23Z) step
|
||||
[sim] t=1789108884567768744 (2026-09-11T06:41:24Z) session=8
|
||||
[sim] event: RESVALS_DELTA n=11
|
||||
|
||||
- how can we step to the next session which is not dummy(something occurs)? or how can I known what's the next non dummy session? for above I just want to goto session 8 directly
|
||||
|
||||
for session 40:
|
||||
|
||||
(sim session=39 t=2026-09-11T06:41:55Z) step
|
||||
[sim] t=1789108916579881546 (2026-09-11T06:41:56Z) session=40
|
||||
[sim] event: JOB_PUSH njobs=10 jobids=[0-15-1789108916-0[0-9]]
|
||||
[sim] DISPATCH jobid=0-15-1789108916-0[0]
|
||||
[sim] DISPATCH jobid=0-15-1789108916-0[1]
|
||||
[sim] DISPATCH jobid=0-15-1789108916-0[2]
|
||||
[sim] DISPATCH jobid=0-15-1789108916-0[3]
|
||||
[sim] DISPATCH jobid=0-15-1789108916-0[4]
|
||||
[sim] PEND jobid=0-15-1789108916-0[5]
|
||||
[sim] PEND jobid=0-15-1789108916-0[6]
|
||||
[sim] PEND jobid=0-15-1789108916-0[7]
|
||||
[sim] PEND jobid=0-15-1789108916-0[8]
|
||||
[sim] PEND jobid=0-15-1789108916-0[9]
|
||||
(sim session=40 t=2026-09-11T06:41:56Z) info worker pc13v
|
||||
[sim] worker pc13v: status=online
|
||||
[sim] -- Consumable/Capacity --
|
||||
[sim] resource slots: value=22.00 used=2.00
|
||||
[sim] resource mem: value=25749303296.00 used=0.00
|
||||
[sim] resource swp: value=16384.00 used=0.00
|
||||
[sim] resource tmp: value=541460271104.00 used=0.00
|
||||
[sim] resource slots: value=22.00 used=0.00
|
||||
[sim] resource mem: value=20684476416.00 used=0.00
|
||||
[sim] resource swp: value=13132.17 used=0.00
|
||||
[sim] resource tmp: value=576636968960.00 used=0.00
|
||||
[sim] -- Load Metrics --
|
||||
[sim] resource r1m: value=0.16
|
||||
[sim] resource r15m: value=1.27
|
||||
[sim] resource r15s: value=0.16
|
||||
[sim] resource ut: value=1.19
|
||||
[sim] resource pg: value=129.07
|
||||
[sim] resource r1m: value=2.30
|
||||
[sim] resource r15m: value=4.67
|
||||
[sim] resource r15s: value=2.30
|
||||
[sim] resource ut: value=1.16
|
||||
[sim] resource pg: value=263.47
|
||||
[sim] resource ls: value=1
|
||||
[sim] resource it: value=57.83
|
||||
[sim] resource it: value=3135.20
|
||||
[sim] resource io: value=0
|
||||
[sim] resource cpuf: value=1
|
||||
[sim] resource maxcpus: value=22
|
||||
[sim] resource maxmem: value=33182072832.000000 (30.90 GB)
|
||||
[sim] resource maxswap: value=10000.000000 (9.77 KB)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^I think maxswap is not 10 KB
|
||||
[sim] resource maxtmp: value=2147483648.000000 (2.00 GB)
|
||||
[sim] resource maxswap: value=17179865088.000000 (16.00 GB)
|
||||
[sim] resource maxtmp: value=1005921779712.000000 (936.84 GB)
|
||||
[sim] resource ndisks: value=1
|
||||
[sim] -- Static Hardware Attributes --
|
||||
[sim] resource type: value=x86_64-linux-gnu
|
||||
[sim] resource model: value=x86_64-linux-gnu
|
||||
[sim] assigned jobs: 0-14-1788751435-0
|
||||
(sim)
|
||||
```
|
||||
[sim] assigned jobs: 0-15-1789108916-0[0-4]*4
|
||||
|
||||
xce_test output
|
||||
```
|
||||
xce/client>workerlist
|
||||
HostName Status HostType HostModel CPUfactor MaxCPU MaxMem MaxSwap MaxTmp NDisks
|
||||
pc13v OK x86_64-linux-gnu x86_64-linux-gnu 1.000000 22 30.90 GB 1.00 GB 2.00 GB 1
|
||||
```
|
||||
- The used resources value of slots is still 0.00? as we have dispatched 5 jobs?
|
||||
|
||||
Issues:
|
||||
- resource maxswap value is not correct
|
||||
Can we improve resource usage value display?
|
||||
|
||||
Improvement:
|
||||
- It will be better to show number of slots allocated in the worker node, so following will be better?
|
||||
```
|
||||
[sim] assigned jobs: 0-14-1788751435-0*2
|
||||
^ indicate 2 slots
|
||||
```
|
||||
(sim session=48 t=2026-09-11T06:42:07Z) info worker pc13v
|
||||
[sim] worker pc13v: status=online
|
||||
[sim] -- Consumable/Capacity --
|
||||
[sim] resource slots: value=22.00 used=20.00
|
||||
[sim] resource mem: value=20684476416.00 used=87960930222080.00 <<<can we use GB/MB/...>>>
|
||||
[sim] resource swp: value=13132.17 used=0.00 <<same as above>>
|
||||
[sim] resource tmp: value=576636968960.00 used=0.00 <<same as above>>
|
||||
[sim] -- Load Metrics --
|
||||
[sim] resource r1m: value=2.30
|
||||
[sim] resource r15m: value=4.67
|
||||
[sim] resource r15s: value=2.30
|
||||
[sim] resource ut: value=1.16
|
||||
[sim] resource pg: value=263.47
|
||||
[sim] resource ls: value=1
|
||||
[sim] resource it: value=3135.20
|
||||
[sim] resource io: value=0
|
||||
[sim] resource cpuf: value=1
|
||||
[sim] resource maxcpus: value=22
|
||||
[sim] resource maxmem: value=33182072832.000000 (30.90 GB)
|
||||
[sim] resource maxswap: value=17179865088.000000 (16.00 GB)
|
||||
[sim] resource maxtmp: value=1005921779712.000000 (936.84 GB)
|
||||
[sim] resource ndisks: value=1
|
||||
[sim] -- Static Hardware Attributes --
|
||||
[sim] resource type: value=x86_64-linux-gnu
|
||||
[sim] resource model: value=x86_64-linux-gnu
|
||||
[sim] assigned jobs: 0-15-1789108916-0[0-4]*4
|
||||
|
||||
Something wrong in session 98, the count value of slots used is inconsistent with assigned jobs in pc13v?
|
||||
|
||||
[sim] t=1789109023245433276 (2026-09-11T06:43:43Z) session=98
|
||||
[sim] event: JOB_UPDATE jobid=0-15-1789108916-0[5]
|
||||
[sim] event: JOB_UPDATE jobid=0-15-1789108916-0[6]
|
||||
[sim] event: WORKERS_PUSH nworkers=1 names=[pc13v]
|
||||
[sim] event: JOB_PUSH njobs=20 jobids=[0-15-1789108916-0[0-9],0-15-1789108940-0[0-9]]
|
||||
[sim] event: WORKERS_PUSH nworkers=1 names=[pc13v]
|
||||
[sim] event: JOB_PUSH njobs=20 jobids=[0-15-1789108916-0[0-9],0-15-1789108940-0[0-9]]
|
||||
[sim] event: JOB_UPDATE jobid=0-15-1789108916-0[9]
|
||||
[sim] event: JOB_UPDATE jobid=0-15-1789108940-0[0]
|
||||
[sim] event: JOB_RUSAGE jobid=0-15-1789108940-0[0]
|
||||
[sim] event: JOB_UPDATE jobid=0-15-1789108940-0[1]
|
||||
[sim] event: JOB_RUSAGE jobid=0-15-1789108940-0[1]
|
||||
[sim] event: JOB_STATE_CHANGE jobid=0-15-1789108916-0[5] state=RUN
|
||||
[sim] event: JOB_STATE_CHANGE jobid=0-15-1789108916-0[6] state=RUN
|
||||
[sim] event: JOB_STATE_CHANGE jobid=0-15-1789108916-0[7] state=RUN
|
||||
[sim] event: JOB_STATE_CHANGE jobid=0-15-1789108916-0[8] state=RUN
|
||||
[sim] event: JOB_STATE_CHANGE jobid=0-15-1789108916-0[9] state=RUN
|
||||
[2026-09-11 06:43:43,245] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[0] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,245] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[1] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,246] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[2] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,246] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[3] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,246] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[4] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,246] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[5] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,246] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[6] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,246] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[7] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,246] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: resync claims RUN for jobid 0-15-1789108916-0[7] with no allocation data -- refusing to merge a phantom RUN (existing state 0x1 kept)
|
||||
[2026-09-11 06:43:43,246] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[8] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[9] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[0] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[1] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[2] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[3] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[4] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[5] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[6] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[7] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[8] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,247] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[9] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[0] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[1] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[2] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[3] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[4] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[5] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[6] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[7] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: resync claims RUN for jobid 0-15-1789108916-0[7] with no allocation data -- refusing to merge a phantom RUN (existing state 0x1 kept)
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[8] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,248] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: resync claims RUN for jobid 0-15-1789108916-0[8] with no allocation data -- refusing to merge a phantom RUN (existing state 0x1 kept)
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108916-0[9] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[0] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[1] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[2] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[3] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[4] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[5] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[6] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[7] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[8] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[2026-09-11 06:43:43,249] [801857] [ WARN] [ xce_core_push_events] xce_core_push_events JOBS_PUSH: sched_add_job failed for jobid 0-15-1789108940-0[9] (rc=-17), merging state into existing tracked job instead of discarding
|
||||
[sim] DISPATCH jobid=0-15-1789108940-0[2]
|
||||
[sim] DISPATCH jobid=0-15-1789108940-0[3]
|
||||
[sim] DISPATCH jobid=0-15-1789108940-0[4]
|
||||
[sim] DISPATCH jobid=0-15-1789108940-0[5]
|
||||
[sim] PEND jobid=0-15-1789108940-0[6]
|
||||
[sim] PEND jobid=0-15-1789108940-0[7]
|
||||
[sim] PEND jobid=0-15-1789108940-0[8]
|
||||
[sim] PEND jobid=0-15-1789108940-0[9]
|
||||
(sim session=98 t=2026-09-11T06:43:43Z) info worker pc13v
|
||||
[sim] worker pc13v: status=online
|
||||
[sim] -- Consumable/Capacity --
|
||||
[sim] resource slots: value=22.00 used=22.00
|
||||
[sim] resource mem: value=20555202560.00 used=35184372088832.00
|
||||
[sim] resource swp: value=13132.59 used=0.00
|
||||
[sim] resource tmp: value=576442761216.00 used=0.00
|
||||
[sim] -- Load Metrics --
|
||||
[sim] resource r1m: value=0.84
|
||||
[sim] resource r15m: value=4.24
|
||||
[sim] resource r15s: value=0.84
|
||||
[sim] resource ut: value=3.52
|
||||
[sim] resource pg: value=1752
|
||||
[sim] resource ls: value=1
|
||||
[sim] resource it: value=3136.95
|
||||
[sim] resource io: value=0
|
||||
[sim] resource cpuf: value=1
|
||||
[sim] resource maxcpus: value=22
|
||||
[sim] resource maxmem: value=33182072832.000000 (30.90 GB)
|
||||
[sim] resource maxswap: value=17179865088.000000 (16.00 GB)
|
||||
[sim] resource maxtmp: value=1005921779712.000000 (936.84 GB)
|
||||
[sim] resource ndisks: value=1
|
||||
[sim] -- Static Hardware Attributes --
|
||||
[sim] resource type: value=x86_64-linux-gnu
|
||||
[sim] resource model: value=x86_64-linux-gnu
|
||||
[sim] assigned job: jobid=0-15-1789108916-0[5]*4
|
||||
[sim] assigned job: jobid=0-15-1789108916-0[6]*4
|
||||
[sim] assigned job: jobid=0-15-1789108916-0[7]*4
|
||||
[sim] assigned job: jobid=0-15-1789108916-0[8]*4
|
||||
[sim] assigned job: jobid=0-15-1789108916-0[9]*4
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[0]
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[1]
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[2]
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[3]
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[4]
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[5]
|
||||
(sim session=98 t=2026-09-11T06:43:54Z)
|
||||
|
||||
- it seems the scheduler has broken max slots constraint? Investigate why? check next session, the used slots get to 26
|
||||
|
||||
(sim session=99 t=2026-09-11T06:43:54Z) info worker pc13v
|
||||
[sim] worker pc13v: status=online
|
||||
[sim] -- Consumable/Capacity --
|
||||
[sim] resource slots: value=22.00 used=26.00
|
||||
[sim] resource mem: value=20555202560.00 used=35184372088832.00
|
||||
[sim] resource swp: value=13132.59 used=0.00
|
||||
[sim] resource tmp: value=576442761216.00 used=0.00
|
||||
[sim] -- Load Metrics --
|
||||
[sim] resource r1m: value=0.84
|
||||
[sim] resource r15m: value=4.24
|
||||
[sim] resource r15s: value=0.84
|
||||
[sim] resource ut: value=3.52
|
||||
[sim] resource pg: value=1752
|
||||
[sim] resource ls: value=1
|
||||
[sim] resource it: value=3136.95
|
||||
[sim] resource io: value=0
|
||||
[sim] resource cpuf: value=1
|
||||
[sim] resource maxcpus: value=22
|
||||
[sim] resource maxmem: value=33182072832.000000 (30.90 GB)
|
||||
[sim] resource maxswap: value=17179865088.000000 (16.00 GB)
|
||||
[sim] resource maxtmp: value=1005921779712.000000 (936.84 GB)
|
||||
[sim] resource ndisks: value=1
|
||||
[sim] -- Static Hardware Attributes --
|
||||
[sim] resource type: value=x86_64-linux-gnu
|
||||
[sim] resource model: value=x86_64-linux-gnu
|
||||
[sim] assigned job: jobid=0-15-1789108916-0[5]*4
|
||||
[sim] assigned job: jobid=0-15-1789108916-0[6]*4
|
||||
[sim] assigned job: jobid=0-15-1789108916-0[7]*4
|
||||
[sim] assigned job: jobid=0-15-1789108916-0[8]*4
|
||||
[sim] assigned job: jobid=0-15-1789108916-0[9]*4
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[0]
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[1]
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[2]
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[3]
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[4]
|
||||
[sim] assigned job: jobid=0-15-1789108940-0[5]
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue