lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(scheduler): cache hits keep the full pend reason; sim uses the cache (bug-1932)

Browse source 
The session-level unsatisfiable-resreq cache stored only the exhausted
attempt's main reason (an int). The allocators record shortfalls as
per-host/per-resource entries and leave main_reason at 0, so every cache
hit in the live scheduler pended its job with an empty reason. The
simulator registered no cache at all (its table lived on xce_scheduler_t),
so it searched every identical pending job again and its traces differed
from production.

- The cache table moves to xce_sched_core_t; cache_lookup/insert/clear and
  the hook adapters take the core, with no cast to a driver struct.
  sched_init() builds it, cache_clear() rebuilds it per session,
  xce_sched_core_fini() destroys it.
- A cache entry is a deep copy of the attempt's full xce_job_preason_t
  plus the job that proved the request unsatisfiable (core->alloc_jobid,
  set by sched_allocator()). A hit moves a copy into ctx->reason; a miss
  leaves ctx->reason untouched.
- main.c passes the same three adapters to the simulator through new
  sched_sim_args_t fields; sched_sim_driver_start() sets them on the sim
  core (plain field writes: white-box tests include sim.c alone).
- Decision trace: a hit renders "no plan generated: skipped, same request
  as <job> proved unsatisfiable earlier this session" (new
  XCE_TRACE_NOPLAN_KNOWN_UNSAT via sched_trace_known_unsat()); the outcome
  shows the restored reasons.
- sim_verify keeps its main-only PEND compare (old recordings carry
  main-only hits); comments and the scheduler-sim-live-allocation design
  note updated.

Tests: full-reason assertions in the cache dispatch tests (RED before:
"a cache hit must restore the original attempt's FULL pend reason"),
test_alloc_cache.c ported to the new API (deep copy, clear frees, -EEXIST
frees), a sim-registered clear test, a renderer test, and a sim e2e
scenario with two identical unsatisfiable jobs (204, 205) where 205 is
skipped with 204's full reasons (RED before: no skip line).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rongsong Shen 2026-10-09 08:02:18 +00:00
parent cff3304f5d
commit 0317c7af98
Signed by untrusted user: rshen
GPG key ID: 80533536B0EB651C
27 changed files with 982 additions and 737 deletions
Download patch file Download diff file Expand all files Collapse all files

519
src/scheduler/testsuite/test_alloc_cache.c
View file

@ -1,7 +1,7 @@
/*
* test_alloc_cache.c - unit test for the session-level unsatisfiable-
* resreq cache (scheduler-allocation spec, task 2.2, Req 7.1, 7.3, 7.4,
* 7.5).
* 7.5; bug-1932).
*
* White-box test TU: #includes ../src/alloc_cache.c directly (same
* convention test_host_iter.c/test_job_update.c/test_handlers.c already
@ -9,14 +9,12 @@
* cache_key_hash_/cache_key_cmp_/cache_key_assign_/cache_key_destroy_ and
* cache_lookup()/cache_insert()/cache_clear() implementations, not a stub.
*
* alloc_cache.c only depends on kv.h (libxce, already linked) and
* scheduler.h/sched/types.h (header-only struct definitions) -- it never
* touches xce_scheduler_t.core, so this fixture never calls
* xce_sched_core_init(); a bare memset()'d xce_scheduler_t with its own
* unsatisfiable_cache field constructed directly (mirroring sched_init()'s
* own construction call, scheduler.c) is sufficient.
* bug-1932: the cache lives on xce_sched_core_t (both drivers own one), so
* the fixture is a bare memset()'d core whose table cache_clear() builds --
* the same call both drivers' per-session clear hook makes. core.c is not
* linked here, so the fixture destroys the table itself.
*
* Covers, per this task's Observable line:
* Covers:
* - same-key-twice HIT (not just insert-then-miss-on-a-different-key,
* which the cache_key_cmp_() inverted-polarity bug already found and
* fixed during design review -- requirements.md's "New types (Req 7)"
@ -25,10 +23,10 @@
* - a lookup differing in any single field (each of the four resreq
* fingerprint fields, and each of the three identity fields) is a
* miss.
* - reason-category round-trip: insert with category X, hit later
* returns X -- tested with two distinct category values to rule out a
* trivial always-returns-0 implementation.
* - clear() empties all entries.
* - bug-1932: a hit returns the FULL stored pend reason (main, per-host
* and per-resource entries) and the job that proved it, as a deep copy
* independent of both the inserted original and later lookups.
* - clear() empties all entries (and frees them: ASAN).
*/
#include <errno.h>
#include <stdint.h>
@ -37,6 +35,7 @@
#include <unity/unity.h>
#include "preason.h"
#include "sched/alloc_cache.h"
#include "scheduler.h"
@ -51,28 +50,20 @@ void tearDown(void) { return; }
/* Fixture helpers */
/* ---------------------------------------------------------------------- */
/*
* init_sched - a bare xce_scheduler_t with only unsatisfiable_cache
* constructed (this cache never touches sched->core). Mirrors sched_init()'s
* own construction call (scheduler.c) exactly, so the fixture and
* production code can never silently drift.
*/
static void init_sched(xce_scheduler_t* sched)
/* init_core - a bare core with only its cache table built, by the same
* cache_clear() call the drivers' per-session hook makes. */
static void init_core(xce_sched_core_t* core)
{
kv_params_t params = { .nslots = XCE_UNSAT_CACHE_NSLOTS,
.keyops = &cache_key_ops,
.datakiller = NULL };
memset(sched, 0, sizeof(*sched));
sched->unsatisfiable_cache = makeInstance(&kvtab_type, &params);
TEST_ASSERT_NOT_NULL(sched->unsatisfiable_cache);
memset(core, 0, sizeof(*core));
cache_clear(core);
TEST_ASSERT_NOT_NULL(core->unsatisfiable_cache);
}
static void fini_sched(xce_scheduler_t* sched)
static void fini_core(xce_sched_core_t* core)
{
if (sched->unsatisfiable_cache != NULL) {
destroy(sched->unsatisfiable_cache);
sched->unsatisfiable_cache = NULL;
if (core->unsatisfiable_cache != NULL) {
destroy(core->unsatisfiable_cache);
core->unsatisfiable_cache = NULL;
}
}
@ -105,6 +96,22 @@ static void init_primary_obj(xce_sched_obj_t* obj, xce_primary_schedobj_t* prima
obj->project = (char*)project;
}
/* A pend reason with every kind of entry: main `main_reason`, one host
* reason and two resource reasons (one host-scoped, one cluster-wide). */
static void make_reasons(xce_job_preason_t* r, int main_reason,
const char* host)
{
memset(r, 0, sizeof(*r));
TEST_ASSERT_EQUAL_INT(0, add_job_preason(r, main_reason, NULL, NULL));
TEST_ASSERT_TRUE(add_job_preason(r, PREASON_MATCH_FAIL, (char*)host, NULL)
> 0);
TEST_ASSERT_TRUE(
add_job_preason(r, PREASON_NO_RESOURCE, (char*)host, (char*)"slots")
> 0);
TEST_ASSERT_TRUE(
add_job_preason(r, PREASON_NO_RESOURCE, NULL, (char*)"license") > 0);
}
/* Baseline fixture fingerprint + identity, reused (with one field varied
* at a time) across the miss tests below. Distinct string literals (not
* merely distinct content) are used for the identity fields so pointer
@ -115,6 +122,7 @@ static void init_primary_obj(xce_sched_obj_t* obj, xce_primary_schedobj_t* prima
#define BASE_MAX_TASKS 4
#define BASE_SELECT_NODE_ID 100u
#define BASE_RUSAGE_HASH ((uint64_t)0xdeadbeefULL)
#define BASE_JOBID ((uint64_t)4242ULL)
static const char BASE_USER[] = "alice";
static const char BASE_QUEUE[] = "normal";
static const char BASE_PROJECT[] = "proj1";
@ -128,21 +136,24 @@ static const char BASE_PROJECT[] = "proj1";
* from day one would not fail any test that doesn't specifically assert a
* hit"): insert once, then look up an INDEPENDENTLY constructed key with
* every field identical (including pointer-identical identity fields) --
* must be a hit, returning the exact category given at insert time. A
* cache_key_cmp_() with the earlier, inverted boolean polarity would fail
* must be a hit, returning the exact reasons and job given at insert time.
* A cache_key_cmp_() with the earlier, inverted boolean polarity would fail
* this test (every lookup a false miss) while still passing a weaker
* insert-then-miss-on-a-different-key round-trip.
*/
static void test_same_key_twice_is_a_hit(void)
{
xce_scheduler_t sched;
xce_sched_obj_t obj1, obj2;
xce_primary_schedobj_t primary1, primary2;
int category = 42;
int out = -1;
int rc;
xce_sched_core_t core;
xce_sched_obj_t obj1, obj2;
xce_primary_schedobj_t primary1, primary2;
xce_job_preason_t in;
xce_job_preason_t out;
uint64_t jobid = 0;
int rc;
init_sched(&sched);
init_core(&core);
make_reasons(&in, 0, "host1");
memset(&out, 0, sizeof(out));
init_primary_obj(&obj1, &primary1, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
@ -151,56 +162,73 @@ static void test_same_key_twice_is_a_hit(void)
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
BASE_QUEUE, BASE_PROJECT);
rc = cache_insert(&sched, &obj1, category);
rc = cache_insert(&core, &obj1, &in, BASE_JOBID);
TEST_ASSERT_EQUAL_INT(0, rc);
rc = cache_lookup(&sched, &obj2, &out);
rc = cache_lookup(&core, &obj2, &out, &jobid);
TEST_ASSERT_EQUAL_INT(1, rc);
TEST_ASSERT_EQUAL_INT(category, out);
TEST_ASSERT_EQUAL_UINT64(BASE_JOBID, jobid);
TEST_ASSERT_TRUE_MESSAGE(
job_preason_eq(&in, &out, NULL),
"bug-1932: a hit must restore the full stored pend reason -- main, "
"per-host and per-resource entries -- not just the main reason");
TEST_ASSERT_EQUAL_INT(1, out.nhost_reasons);
TEST_ASSERT_EQUAL_INT(2, out.nres_reasons);
fini_sched(&sched);
erase_job_preason(&in);
erase_job_preason(&out);
fini_core(&core);
}
/*
* A lookup before any insert is a plain miss.
* A lookup before any insert is a plain miss, and leaves `out` alone.
*/
static void test_lookup_before_any_insert_is_a_miss(void)
{
xce_scheduler_t sched;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
int out = -1;
int rc;
xce_sched_core_t core;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
xce_job_preason_t out;
uint64_t jobid = 7;
int rc;
init_sched(&sched);
init_core(&core);
memset(&out, 0, sizeof(out));
out.main_reason = 9;
init_primary_obj(&obj, &primary, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
BASE_QUEUE, BASE_PROJECT);
rc = cache_lookup(&sched, &obj, &out);
rc = cache_lookup(&core, &obj, &out, &jobid);
TEST_ASSERT_EQUAL_INT(0, rc);
TEST_ASSERT_EQUAL_INT(9, out.main_reason);
TEST_ASSERT_EQUAL_UINT64(7, jobid);
fini_sched(&sched);
fini_core(&core);
}
/*
* differing_field_misses - shared body for "a lookup differing in exactly
* one field is a miss": insert the base key with `category`, then look up
* a second key built from the given (possibly-varied) fields -- must miss.
* assert_variant_misses - shared body for "a lookup differing in exactly
* one field is a miss": insert the base key, then look up a second key
* built from the given (possibly-varied) fields -- must miss.
*/
static void assert_variant_misses(int min_tasks, int max_tasks,
uint32_t select_node_id,
uint64_t rusage_hash, const char* user,
const char* queue, const char* project)
{
xce_scheduler_t sched;
xce_sched_obj_t base_obj, variant_obj;
xce_primary_schedobj_t base_primary, variant_primary;
int out = -1;
int rc;
xce_sched_core_t core;
xce_sched_obj_t base_obj, variant_obj;
xce_primary_schedobj_t base_primary, variant_primary;
xce_job_preason_t in;
xce_job_preason_t out;
uint64_t jobid = 0;
int rc;
init_sched(&sched);
init_core(&core);
make_reasons(&in, 0, "host1");
memset(&out, 0, sizeof(out));
init_primary_obj(&base_obj, &base_primary, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
@ -208,13 +236,14 @@ static void assert_variant_misses(int min_tasks, int max_tasks,
init_primary_obj(&variant_obj, &variant_primary, min_tasks, max_tasks,
select_node_id, rusage_hash, user, queue, project);
rc = cache_insert(&sched, &base_obj, 7);
rc = cache_insert(&core, &base_obj, &in, BASE_JOBID);
TEST_ASSERT_EQUAL_INT(0, rc);
rc = cache_lookup(&sched, &variant_obj, &out);
rc = cache_lookup(&core, &variant_obj, &out, &jobid);
TEST_ASSERT_EQUAL_INT(0, rc);
fini_sched(&sched);
erase_job_preason(&in);
fini_core(&core);
}
static void test_miss_on_differing_min_tasks(void)
@ -273,63 +302,128 @@ static void test_miss_on_differing_project(void)
}
/*
* Req 7.5: a cache hit reports the SAME reason category the original
* miss's exhausted attempt stored -- tested with two distinct category
* values (on two distinct keys, since re-inserting the identical key
* would hit kvtab_add()'s own -EEXIST contract) to rule out a trivial
* always-returns-0 (or always-returns-the-first-inserted-value)
* implementation.
* Req 7.5 / bug-1932: two distinct keys keep their own reasons and jobs --
* rules out an implementation returning the first-inserted (or a fixed)
* value for every hit.
*/
static void test_reason_category_round_trip_two_distinct_categories(void)
static void test_two_entries_keep_their_own_reasons_and_jobs(void)
{
xce_scheduler_t sched;
xce_sched_obj_t obj_a, obj_b;
xce_primary_schedobj_t primary_a, primary_b;
static const char user_b[] = "carol";
int out_a = -1, out_b = -1;
int rc;
xce_sched_core_t core;
xce_sched_obj_t obj_a, obj_b;
xce_primary_schedobj_t primary_a, primary_b;
static const char user_b[] = "carol";
xce_job_preason_t in_a, in_b, out;
uint64_t jobid = 0;
init_sched(&sched);
init_core(&core);
make_reasons(&in_a, 0, "host-a");
make_reasons(&in_b, PREASON_BAD_RESREQ, "host-b");
memset(&out, 0, sizeof(out));
init_primary_obj(&obj_a, &primary_a, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
BASE_QUEUE, BASE_PROJECT);
/* Distinct key (differs only in identity) so this doesn't collide with
* obj_a's own insert. */
init_primary_obj(&obj_b, &primary_b, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, user_b,
BASE_QUEUE, BASE_PROJECT);
rc = cache_insert(&sched, &obj_a, 11);
TEST_ASSERT_EQUAL_INT(0, rc);
rc = cache_insert(&sched, &obj_b, 99);
TEST_ASSERT_EQUAL_INT(0, rc);
TEST_ASSERT_EQUAL_INT(0, cache_insert(&core, &obj_a, &in_a, 11));
TEST_ASSERT_EQUAL_INT(0, cache_insert(&core, &obj_b, &in_b, 99));
rc = cache_lookup(&sched, &obj_a, &out_a);
TEST_ASSERT_EQUAL_INT(1, rc);
TEST_ASSERT_EQUAL_INT(11, out_a);
TEST_ASSERT_EQUAL_INT(1, cache_lookup(&core, &obj_a, &out, &jobid));
TEST_ASSERT_EQUAL_UINT64(11, jobid);
TEST_ASSERT_TRUE(job_preason_eq(&in_a, &out, NULL));
rc = cache_lookup(&sched, &obj_b, &out_b);
TEST_ASSERT_EQUAL_INT(1, rc);
TEST_ASSERT_EQUAL_INT(99, out_b);
/* a hit replaces whatever `out` held: no merge with a's entries */
TEST_ASSERT_EQUAL_INT(1, cache_lookup(&core, &obj_b, &out, &jobid));
TEST_ASSERT_EQUAL_UINT64(99, jobid);
TEST_ASSERT_TRUE(job_preason_eq(&in_b, &out, NULL));
fini_sched(&sched);
erase_job_preason(&in_a);
erase_job_preason(&in_b);
erase_job_preason(&out);
fini_core(&core);
}
/*
* cache_clear() empties every entry: insert two distinct keys, clear, then
* confirm BOTH now miss (not just the one inserted last).
* bug-1932: the cache owns a deep copy. Erasing (freeing) the inserted
* original, and erasing a previous lookup's output, must leave the stored
* entry intact -- the original attempt's reason is freed when its decision
* is published, long before a later job's lookup.
*/
static void test_entry_is_a_deep_copy(void)
{
xce_sched_core_t core;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
xce_job_preason_t in, expect, out;
uint64_t jobid = 0;
init_core(&core);
make_reasons(&in, 0, "host1");
make_reasons(&expect, 0, "host1");
memset(&out, 0, sizeof(out));
init_primary_obj(&obj, &primary, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
BASE_QUEUE, BASE_PROJECT);
TEST_ASSERT_EQUAL_INT(0, cache_insert(&core, &obj, &in, BASE_JOBID));
erase_job_preason(&in);
TEST_ASSERT_EQUAL_INT(1, cache_lookup(&core, &obj, &out, &jobid));
TEST_ASSERT_TRUE(job_preason_eq(&expect, &out, NULL));
erase_job_preason(&out);
TEST_ASSERT_EQUAL_INT(1, cache_lookup(&core, &obj, &out, &jobid));
TEST_ASSERT_TRUE(job_preason_eq(&expect, &out, NULL));
erase_job_preason(&expect);
erase_job_preason(&out);
fini_core(&core);
}
/*
* A NULL reason (an attempt that pended without recording any) is stored
* as an empty reason; a hit then reports it as empty, not as garbage.
*/
static void test_null_reasons_stored_as_empty(void)
{
xce_sched_core_t core;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
xce_job_preason_t empty, out;
uint64_t jobid = 0;
init_core(&core);
memset(&empty, 0, sizeof(empty));
make_reasons(&out, 0, "stale");
init_primary_obj(&obj, &primary, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
BASE_QUEUE, BASE_PROJECT);
TEST_ASSERT_EQUAL_INT(0, cache_insert(&core, &obj, NULL, BASE_JOBID));
TEST_ASSERT_EQUAL_INT(1, cache_lookup(&core, &obj, &out, &jobid));
TEST_ASSERT_TRUE(job_preason_eq(&empty, &out, NULL));
fini_core(&core);
}
/*
* cache_clear() empties all entries (freeing them -- ASAN reports a leak
* otherwise) and leaves a usable table.
*/
static void test_clear_empties_all_entries(void)
{
xce_scheduler_t sched;
xce_sched_obj_t obj_a, obj_b;
xce_primary_schedobj_t primary_a, primary_b;
static const char user_b[] = "dave";
int out = -1;
int rc;
xce_sched_core_t core;
xce_sched_obj_t obj_a, obj_b;
xce_primary_schedobj_t primary_a, primary_b;
static const char user_b[] = "dave";
xce_job_preason_t in, out;
uint64_t jobid = 0;
init_sched(&sched);
init_core(&core);
make_reasons(&in, 0, "host1");
memset(&out, 0, sizeof(out));
init_primary_obj(&obj_a, &primary_a, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
@ -338,37 +432,42 @@ static void test_clear_empties_all_entries(void)
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, user_b,
BASE_QUEUE, BASE_PROJECT);
TEST_ASSERT_EQUAL_INT(0, cache_insert(&sched, &obj_a, 1));
TEST_ASSERT_EQUAL_INT(0, cache_insert(&sched, &obj_b, 2));
TEST_ASSERT_EQUAL_INT(0, cache_insert(&core, &obj_a, &in, 1));
TEST_ASSERT_EQUAL_INT(0, cache_insert(&core, &obj_b, &in, 2));
/* Sanity: both are hits before clear. */
TEST_ASSERT_EQUAL_INT(1, cache_lookup(&sched, &obj_a, &out));
TEST_ASSERT_EQUAL_INT(1, cache_lookup(&sched, &obj_b, &out));
TEST_ASSERT_EQUAL_INT(1, cache_lookup(&core, &obj_a, &out, &jobid));
TEST_ASSERT_EQUAL_INT(1, cache_lookup(&core, &obj_b, &out, &jobid));
cache_clear(&sched);
TEST_ASSERT_NOT_NULL(sched.unsatisfiable_cache);
cache_clear(&core);
TEST_ASSERT_NOT_NULL(core.unsatisfiable_cache);
rc = cache_lookup(&sched, &obj_a, &out);
TEST_ASSERT_EQUAL_INT(0, rc);
rc = cache_lookup(&sched, &obj_b, &out);
TEST_ASSERT_EQUAL_INT(0, rc);
TEST_ASSERT_EQUAL_INT(0, cache_lookup(&core, &obj_a, &out, &jobid));
TEST_ASSERT_EQUAL_INT(0, cache_lookup(&core, &obj_b, &out, &jobid));
fini_sched(&sched);
erase_job_preason(&in);
erase_job_preason(&out);
fini_core(&core);
}
/*
* Re-inserting the identical key without an intervening clear propagates
* kvtab_add()'s own -EEXIST contract -- exercising cache_insert()'s error
* path.
* path, which must free the rejected copy (ASAN) and keep the first entry.
*/
static void test_insert_duplicate_key_returns_eexist(void)
{
xce_scheduler_t sched;
xce_sched_obj_t obj1, obj2;
xce_primary_schedobj_t primary1, primary2;
int rc;
xce_sched_core_t core;
xce_sched_obj_t obj1, obj2;
xce_primary_schedobj_t primary1, primary2;
xce_job_preason_t in1, in2, out;
uint64_t jobid = 0;
int rc;
init_sched(&sched);
init_core(&core);
make_reasons(&in1, 0, "host1");
make_reasons(&in2, 0, "host2");
memset(&out, 0, sizeof(out));
init_primary_obj(&obj1, &primary1, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
@ -377,26 +476,39 @@ static void test_insert_duplicate_key_returns_eexist(void)
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
BASE_QUEUE, BASE_PROJECT);
TEST_ASSERT_EQUAL_INT(0, cache_insert(&sched, &obj1, 5));
rc = cache_insert(&sched, &obj2, 6);
TEST_ASSERT_EQUAL_INT(0, cache_insert(&core, &obj1, &in1, 5));
rc = cache_insert(&core, &obj2, &in2, 6);
TEST_ASSERT_EQUAL_INT(-EEXIST, rc);
fini_sched(&sched);
TEST_ASSERT_EQUAL_INT(1, cache_lookup(&core, &obj1, &out, &jobid));
TEST_ASSERT_EQUAL_UINT64(5, jobid);
TEST_ASSERT_TRUE(job_preason_eq(&in1, &out, NULL));
erase_job_preason(&in1);
erase_job_preason(&in2);
erase_job_preason(&out);
fini_core(&core);
}
/*
* Defensive argument checking: NULL sched/obj/out_reason_category, and a
* non-primary tag object, are all rejected rather than crashing.
* Defensive argument checking: NULL core/obj/out/out_jobid, and a
* non-primary tag object, are all rejected rather than crashing. A core
* whose table was never built is a plain miss on lookup and an error on
* insert.
*/
static void test_argument_validation(void)
{
xce_scheduler_t sched;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
xce_sched_obj_t non_primary_obj;
int out = -1;
xce_sched_core_t core;
xce_sched_core_t bare;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
xce_sched_obj_t non_primary_obj;
xce_job_preason_t out;
uint64_t jobid = 0;
init_sched(&sched);
init_core(&core);
memset(&bare, 0, sizeof(bare));
memset(&out, 0, sizeof(out));
init_primary_obj(&obj, &primary, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
BASE_QUEUE, BASE_PROJECT);
@ -404,35 +516,37 @@ static void test_argument_validation(void)
memset(&non_primary_obj, 0, sizeof(non_primary_obj));
non_primary_obj.tag = XCE_RESREQ_PRIMARY + 1; /* anything != PRIMARY */
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_lookup(NULL, &obj, &out));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_lookup(&sched, NULL, &out));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_lookup(&sched, &obj, NULL));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_lookup(NULL, &obj, &out, &jobid));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_lookup(&core, NULL, &out, &jobid));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_lookup(&core, &obj, NULL, &jobid));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_lookup(&core, &obj, &out, NULL));
TEST_ASSERT_EQUAL_INT(-EINVAL,
cache_lookup(&sched, &non_primary_obj, &out));
cache_lookup(&core, &non_primary_obj, &out, &jobid));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_insert(NULL, &obj, 1));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_insert(&sched, NULL, 1));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_insert(&sched, &non_primary_obj, 1));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_insert(NULL, &obj, NULL, 1));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_insert(&core, NULL, NULL, 1));
TEST_ASSERT_EQUAL_INT(-EINVAL,
cache_insert(&core, &non_primary_obj, NULL, 1));
TEST_ASSERT_EQUAL_INT(0, cache_lookup(&bare, &obj, &out, &jobid));
TEST_ASSERT_EQUAL_INT(-EINVAL, cache_insert(&bare, &obj, NULL, 1));
/* cache_clear(NULL) must not crash. */
cache_clear(NULL);
fini_sched(&sched);
fini_core(&core);
}
/* ---------------------------------------------------------------------- */
/* Task 1.1 (scheduler-sim-live-allocation spec, Req 5.2): the */
/* unsatisfiable-cache callback seam. */
/* The unsatisfiable-cache callback seam (scheduler-sim-live-allocation */
/* Req 5.2; bug-1932: both drivers register it). */
/* ---------------------------------------------------------------------- */
/*
* A freshly-zeroed xce_sched_core_t (the same memset() default
* xce_sched_core_init() applies -- core.c is not linked into this
* white-box TU, so this test zeroes a bare instance directly rather than
* calling that function) leaves BOTH new callback fields NULL. This is
* the exact state every simulation-mode xce_sched_core_t is in forever --
* sim_loop.c never registers either callback (Req 5 AC 17 "Simulation
* isolation").
* white-box TU) leaves BOTH callback fields NULL: a core whose driver
* registered no cache computes every attempt fresh.
*/
static void test_seam_fields_default_null_on_zeroed_core(void)
{
@ -442,116 +556,93 @@ static void test_seam_fields_default_null_on_zeroed_core(void)
TEST_ASSERT_NULL(core.unsatisfiable_lookup_cb);
TEST_ASSERT_NULL(core.unsatisfiable_populate_cb);
TEST_ASSERT_NULL(core.unsatisfiable_cache);
}
/*
* THE Observable this task's own TDD note names: "with both callbacks
* left NULL, the allocation dispatch chokepoint always computes fresh
* with no crash and no incorrect skip." The real chokepoint
* (sched_make_primary_alloc_(), scheduler.c) is migrated to actually
* consult this seam by a LATER task (tasks.md 2.2, boundary "Core-Safe
* Allocation Chain, Unsatisfiable-Cache Callback Seam") -- this task adds
* only the seam itself, not that call site. This test proves the seam's
* own NULL-safety contract in isolation, using the identical NULL-check
* shape design.md's own invariant specifies for that future call site
* ("the allocation chain ... only ever NULL-checks and calls these"):
* with both fields NULL, a NULL-checked consult never dereferences either
* function pointer (no crash), is never mistaken for a cache HIT (no
* incorrect skip -- `hit` stays 0, meaning "compute fresh"), and a
* NULL-checked populate attempt is silently skipped, not a fatal error.
* With both callbacks left NULL, a NULL-checked consult never dereferences
* either function pointer (no crash), is never mistaken for a cache HIT
* (no incorrect skip), and a NULL-checked populate attempt is silently
* skipped -- the same NULL-check shape sched_make_primary_alloc_()
* (scheduler.c) uses.
*/
static void test_null_callbacks_mean_always_compute_fresh(void)
{
xce_sched_core_t core;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
int category = -1;
int hit = -1; /* poison: neither 0 nor 1 yet */
xce_sched_core_t core;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
xce_job_preason_t out;
uint64_t jobid = 0;
int hit = -1; /* poison: neither 0 nor 1 yet */
memset(&core, 0, sizeof(core));
memset(&out, 0, sizeof(out));
init_primary_obj(&obj, &primary, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
BASE_QUEUE, BASE_PROJECT);
/* Prove the guard's PRECONDITION explicitly, immediately before relying
* on it below -- not just that the end result happens to be "compute
* fresh", but that the NULL-ness this test's two guarded calls depend
* on genuinely holds at the exact point each guard is evaluated. A
* design that accidentally called through a non-NULL stale/garbage
* pointer here (e.g. an uninitialized field instead of a true NULL)
* could still coincidentally return 0/not crash on some platforms;
* asserting NULL directly, right where each guard fires, rules that
* out. */
TEST_ASSERT_NULL(core.unsatisfiable_lookup_cb);
if (core.unsatisfiable_lookup_cb != NULL) {
hit = core.unsatisfiable_lookup_cb(&core, &obj, &category);
hit = core.unsatisfiable_lookup_cb(&core, &obj, &out, &jobid);
} else {
hit = 0; /* no cache registered => always a miss => compute fresh */
}
TEST_ASSERT_EQUAL_INT(0, hit);
/* A NULL-checked populate attempt must be a true no-op, not a crash --
* same "assert the guard's own precondition, right at the guard"
* discipline as the lookup check just above. */
TEST_ASSERT_NULL(core.unsatisfiable_populate_cb);
if (core.unsatisfiable_populate_cb != NULL) {
core.unsatisfiable_populate_cb(&core, &obj, 7);
core.unsatisfiable_populate_cb(&core, &obj, NULL, 7);
}
}
/*
* Normal-mode observable (this task's own note): "a normal-mode test
* confirming the cache is still consulted/populated exactly as before
* this change." xce_alloc_cache_lookup_cb()/xce_alloc_cache_populate_cb()
* (alloc_cache.c) are the two adapters main.c registers on the real
* xce_scheduler_t's core (xce_sched_core_t core is xce_scheduler_t's
* first member, scheduler.h -- the same pointer-to-first-member idiom
* xce_alloc_cache_clear_cb() already uses). This test calls them through
* core.unsatisfiable_lookup_cb/unsatisfiable_populate_cb exactly as a
* normal-mode driver would, and confirms the result is identical to
* calling cache_lookup()/cache_insert() directly -- proving the adapters
* are a transparent passthrough, not a behavior change.
* xce_alloc_cache_lookup_cb()/xce_alloc_cache_populate_cb() (alloc_cache.c)
* are the adapters both drivers register. Called through the core's seam
* exactly as sched_make_primary_alloc_() does, they must be a transparent
* passthrough to cache_lookup()/cache_insert() on that same core -- no
* cast to a driver struct, so they work on any core (bug-1932).
*/
static void test_lookup_and_populate_cb_adapters_match_direct_cache_calls(void)
{
xce_scheduler_t sched;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
int out_direct = -1;
int out_via_cb = -1;
int rc;
xce_sched_core_t core;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
xce_job_preason_t in, out_direct, out_via_cb;
uint64_t jobid = 0;
int rc;
init_sched(&sched);
sched.core.unsatisfiable_lookup_cb = xce_alloc_cache_lookup_cb;
sched.core.unsatisfiable_populate_cb = xce_alloc_cache_populate_cb;
init_core(&core);
core.unsatisfiable_lookup_cb = xce_alloc_cache_lookup_cb;
core.unsatisfiable_populate_cb = xce_alloc_cache_populate_cb;
make_reasons(&in, 0, "host1");
memset(&out_direct, 0, sizeof(out_direct));
memset(&out_via_cb, 0, sizeof(out_via_cb));
init_primary_obj(&obj, &primary, BASE_MIN_TASKS, BASE_MAX_TASKS,
BASE_SELECT_NODE_ID, BASE_RUSAGE_HASH, BASE_USER,
BASE_QUEUE, BASE_PROJECT);
/* Miss before any insert, via the callback -- same as a direct
* cache_lookup() call (test_lookup_before_any_insert_is_a_miss
* above). */
TEST_ASSERT_NOT_NULL(sched.core.unsatisfiable_lookup_cb);
rc = sched.core.unsatisfiable_lookup_cb(&(sched.core), &obj, &out_via_cb);
rc = core.unsatisfiable_lookup_cb(&core, &obj, &out_via_cb, &jobid);
TEST_ASSERT_EQUAL_INT(0, rc);
/* Populate via the callback... */
TEST_ASSERT_NOT_NULL(sched.core.unsatisfiable_populate_cb);
sched.core.unsatisfiable_populate_cb(&(sched.core), &obj, 55);
core.unsatisfiable_populate_cb(&core, &obj, &in, 55);
/* ...then confirm a DIRECT cache_lookup() call sees the SAME entry --
* the adapter genuinely populated the real cache, not a shadow copy. */
rc = cache_lookup(&sched, &obj, &out_direct);
/* the adapter populated the real cache, not a shadow copy */
rc = cache_lookup(&core, &obj, &out_direct, &jobid);
TEST_ASSERT_EQUAL_INT(1, rc);
TEST_ASSERT_EQUAL_INT(55, out_direct);
TEST_ASSERT_EQUAL_UINT64(55, jobid);
TEST_ASSERT_TRUE(job_preason_eq(&in, &out_direct, NULL));
/* And the callback-based lookup now also reports the same hit. */
rc = sched.core.unsatisfiable_lookup_cb(&(sched.core), &obj, &out_via_cb);
jobid = 0;
rc = core.unsatisfiable_lookup_cb(&core, &obj, &out_via_cb, &jobid);
TEST_ASSERT_EQUAL_INT(1, rc);
TEST_ASSERT_EQUAL_INT(55, out_via_cb);
TEST_ASSERT_EQUAL_UINT64(55, jobid);
TEST_ASSERT_TRUE(job_preason_eq(&in, &out_via_cb, NULL));
fini_sched(&sched);
erase_job_preason(&in);
erase_job_preason(&out_direct);
erase_job_preason(&out_via_cb);
fini_core(&core);
}
int main(void)
@ -566,7 +657,9 @@ int main(void)
RUN_TEST(test_miss_on_differing_user);
RUN_TEST(test_miss_on_differing_queue);
RUN_TEST(test_miss_on_differing_project);
RUN_TEST(test_reason_category_round_trip_two_distinct_categories);
RUN_TEST(test_two_entries_keep_their_own_reasons_and_jobs);
RUN_TEST(test_entry_is_a_deep_copy);
RUN_TEST(test_null_reasons_stored_as_empty);
RUN_TEST(test_clear_empties_all_entries);
RUN_TEST(test_insert_duplicate_key_returns_eexist);
RUN_TEST(test_argument_validation);

86
src/scheduler/testsuite/test_core_session_cache_clear.c
View file

@ -18,19 +18,19 @@
* if non-NULL, and this fixture explicitly registers the REAL production
* adapter (alloc_cache.c's xce_alloc_cache_clear_cb() -- see that
* function's own doc comment for why it lives in alloc_cache.c rather
* than driver.c) on init_sched()'s xce_scheduler_t, the same call main.c
* makes in production.
* than driver.c) on init_core()'s core, the same call both drivers make
* (bug-1932).
*
* This file still does NOT use the white-box #include convention most of
* this test suite uses for core_session.c; its meson.build entry still
* lists core.c/core_clock.c/core_session.c/decision.c/alloc_cache.c as
* genuinely separate compiled sources so this fixture links against the
* REAL cache_clear(xce_scheduler_t*) (alloc_cache.c) that its registered
* REAL cache_clear() (alloc_cache.c) that its registered
* hook calls into -- not a stub.
*
* ── What this proves ─────────────────────────────────────────────────────
* A job's fingerprint+identity is inserted into a REAL, live
* xce_scheduler_t.unsatisfiable_cache (cache_insert(), simulating what
* core->unsatisfiable_cache (cache_insert(), simulating what
* task 5.1's allocator-exhaustion path would have done); a lookup for the
* identical key is confirmed a HIT; the REAL xce_core_begin_session()
* (core_session.c's public entry point, exercising the real
@ -87,36 +87,28 @@ void tearDown(void) { return; }
/* ---------------------------------------------------------------------- */
/*
* init_sched - a real xce_scheduler_t with BOTH core (xce_sched_core_init(),
* core.c) and unsatisfiable_cache (mirroring sched_init()'s own
* construction call, scheduler.c, and test_alloc_cache.c's own
* init_sched()) constructed -- everything core_begin_session_internal()
* touches, without sched_init()'s heavier networking/threading setup
* (this test never calls sched_init() at all). Also registers the REAL
* production cache_clear_cb (round 2, xce_alloc_cache_clear_cb() --
* alloc_cache.c, linked into this exact test binary, see this file's own
* header comment) -- the same one-time-at-startup registration main.c
* performs in production (xce_core_set_cache_clear_cb(), right alongside
* xce_core_set_decision_cb()), so this fixture's own registration is not
* a stand-in wrapper but the identical call production makes.
* init_core - a real core (xce_sched_core_init(), core.c) with the REAL
* production cache_clear_cb registered (xce_alloc_cache_clear_cb(),
* alloc_cache.c, linked into this exact test binary) -- the same
* one-time-at-startup registration both drivers make (main.c for normal
* mode, sched_sim_driver_start() for simulation, bug-1932). The cache
* table lives on the core (bug-1932); cache_clear() builds it here, the
* same call the hook makes at every session start.
*/
static void init_sched(xce_scheduler_t* sched)
static void init_core(xce_sched_core_t* core)
{
kv_params_t params = { .nslots = XCE_UNSAT_CACHE_NSLOTS,
.keyops = &cache_key_ops,
.datakiller = NULL };
int rc;
int rc;
memset(sched, 0, sizeof(*sched));
memset(core, 0, sizeof(*core));
rc = xce_sched_core_init(&(sched->core));
rc = xce_sched_core_init(core);
TEST_ASSERT_EQUAL_INT(0, rc);
rc = xce_core_set_cache_clear_cb(&(sched->core), xce_alloc_cache_clear_cb);
rc = xce_core_set_cache_clear_cb(core, xce_alloc_cache_clear_cb);
TEST_ASSERT_EQUAL_INT(0, rc);
sched->unsatisfiable_cache = makeInstance(&kvtab_type, &params);
TEST_ASSERT_NOT_NULL(sched->unsatisfiable_cache);
cache_clear(core);
TEST_ASSERT_NOT_NULL(core->unsatisfiable_cache);
}
/*
@ -157,40 +149,40 @@ static void init_primary_obj(xce_sched_obj_t* obj,
*/
static void test_begin_session_clears_previously_cached_entry(void)
{
xce_scheduler_t sched;
xce_sched_core_t core;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
int category = 7;
int out = -1;
xce_job_preason_t out;
uint64_t jobid = 0;
int rc;
init_sched(&sched);
init_core(&core);
memset(&out, 0, sizeof(out));
init_primary_obj(&obj, &primary, 1, 4, 100u, 0xdeadbeefULL, "alice",
"normal", "proj1");
rc = cache_insert(&sched, &obj, category);
rc = cache_insert(&core, &obj, NULL, 7);
TEST_ASSERT_EQUAL_INT(0, rc);
/* Baseline: proves the fixture/insert are sound before relying on them
* below -- same "prove the harness works first" discipline
* test_core_session.c's own worker-reset tests already follow. */
rc = cache_lookup(&sched, &obj, &out);
rc = cache_lookup(&core, &obj, &out, &jobid);
TEST_ASSERT_EQUAL_INT(1, rc);
TEST_ASSERT_EQUAL_INT(category, out);
TEST_ASSERT_EQUAL_UINT64(7, jobid);
rc = xce_core_begin_session(&(sched.core));
rc = xce_core_begin_session(&core);
TEST_ASSERT_EQUAL_INT(0, rc);
out = -1;
rc = cache_lookup(&sched, &obj, &out);
rc = cache_lookup(&core, &obj, &out, &jobid);
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, rc,
"Req 7.2: a job cached as unsatisfiable in one session must be "
"re-evaluated fresh (a cache MISS), not immediately pended from "
"the cache, in the next session -- core_begin_session_internal() "
"must clear xce_scheduler_t.unsatisfiable_cache");
"must clear the core's unsatisfiable_cache");
rc = xce_core_end_session(&(sched.core));
rc = xce_core_end_session(&core);
TEST_ASSERT_EQUAL_INT(0, rc);
}
@ -203,28 +195,30 @@ static void test_begin_session_clears_previously_cached_entry(void)
*/
static void test_cache_still_usable_after_begin_session_clear(void)
{
xce_scheduler_t sched;
xce_sched_core_t core;
xce_sched_obj_t obj;
xce_primary_schedobj_t primary;
int out = -1;
xce_job_preason_t out;
uint64_t jobid = 0;
int rc;
init_sched(&sched);
init_core(&core);
memset(&out, 0, sizeof(out));
rc = xce_core_begin_session(&(sched.core));
rc = xce_core_begin_session(&core);
TEST_ASSERT_EQUAL_INT(0, rc);
init_primary_obj(&obj, &primary, 2, 8, 200u, 0xfeedfaceULL, "bob", "batch",
"proj2");
rc = cache_insert(&sched, &obj, 3);
rc = cache_insert(&core, &obj, NULL, 3);
TEST_ASSERT_EQUAL_INT(0, rc);
rc = cache_lookup(&sched, &obj, &out);
rc = cache_lookup(&core, &obj, &out, &jobid);
TEST_ASSERT_EQUAL_INT(1, rc);
TEST_ASSERT_EQUAL_INT(3, out);
TEST_ASSERT_EQUAL_UINT64(3, jobid);
rc = xce_core_end_session(&(sched.core));
rc = xce_core_end_session(&core);
TEST_ASSERT_EQUAL_INT(0, rc);
}

40
src/scheduler/testsuite/test_maxtasks_wiring_ingest.c
View file

@ -1065,26 +1065,28 @@ test_resource_request_update_never_changes_dispatched_task_count(void)
static void
test_differing_task_counts_do_not_collide_in_unsatisfiable_cache(void)
{
xce_scheduler_t sched;
xce_queue_t* queue;
xce_core_event_t queue_event;
xce_job_t* job_a;
xce_job_t* job_b;
xce_job_t* job_ptrs[2];
xce_core_event_t job_event;
xce_sched_job_t* sjob_a;
xce_sched_job_t* sjob_b;
int out_reason = -1;
int rc;
xce_scheduler_t sched;
xce_queue_t* queue;
xce_core_event_t queue_event;
xce_job_t* job_a;
xce_job_t* job_b;
xce_job_t* job_ptrs[2];
xce_core_event_t job_event;
xce_sched_job_t* sjob_a;
xce_sched_job_t* sjob_b;
xce_job_preason_t out_reason;
uint64_t out_jobid = 0;
int rc;
memset(&out_reason, 0, sizeof(out_reason));
memset(&sched, 0, sizeof(sched));
rc = xce_sched_core_init(&(sched.core));
TEST_ASSERT_TRUE_MESSAGE(rc >= 0, "fixture: xce_sched_core_init() must "
"succeed");
cache_clear(&sched);
cache_clear(&(sched.core));
TEST_ASSERT_NOT_NULL_MESSAGE(
sched.unsatisfiable_cache,
sched.core.unsatisfiable_cache,
"fixture: cache_clear() must construct a real unsatisfiable_cache "
"table (mirrors the real per-session construction, main.c's "
"xce_core_set_cache_clear_cb() registration)");
@ -1147,24 +1149,26 @@ test_differing_task_counts_do_not_collide_in_unsatisfiable_cache(void)
/* ── The Req 5.4 proof: an unsatisfiable outcome cached for job A
* must NOT produce a cache hit for job B. ──────────────────────────── */
rc = cache_insert(&sched, sjob_a->schedobj, 7 /* arbitrary category */);
rc = cache_insert(&(sched.core), sjob_a->schedobj, NULL,
7 /* arbitrary job */);
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, rc, "fixture: cache_insert() for job A must succeed");
rc = cache_lookup(&sched, sjob_b->schedobj, &out_reason);
rc = cache_lookup(&(sched.core), sjob_b->schedobj, &out_reason, &out_jobid);
TEST_ASSERT_EQUAL_INT_MESSAGE(
0, rc,
"Req 5.4: job B (a different requested task count, otherwise "
"identical) must NOT hit job A's cached unsatisfiable outcome");
/* Sanity: the cache mechanism itself genuinely works -- job A's OWN
* key IS a hit, with the SAME reason category given at insert time.
* key IS a hit, with the SAME job given at insert time.
* Without this, the miss above could just as easily mean the cache is
* dead altogether, not that it correctly distinguishes task counts. */
rc = cache_lookup(&sched, sjob_a->schedobj, &out_reason);
rc = cache_lookup(&(sched.core), sjob_a->schedobj, &out_reason, &out_jobid);
TEST_ASSERT_EQUAL_INT_MESSAGE(
1, rc, "sanity: job A's own key must hit its own cached entry");
TEST_ASSERT_EQUAL_INT(7, out_reason);
TEST_ASSERT_EQUAL_UINT64(7, out_jobid);
erase_job_preason(&out_reason);
}
/*

66
src/scheduler/testsuite/test_repl_trace_e2e.c
View file

@ -23,8 +23,9 @@
* - extra.replay, generated per run, for what the baseline lacks: a
* satisfied compound requirement (201), a compound requirement that
* fails at its first member (202), a job array with two elements
* (0-0-0-32768[1] and [2]) and a host-selection mismatch (203 selects
* mem>20000, which only w02 has).
* (0-0-0-32768[1] and [2]), a host-selection mismatch (203 selects
* mem>20000, which only w02 has) and two identical unsatisfiable jobs
* (204, 205; bug-1932).
*
* `break DECISION jobid=105` (a raw packed decimal id) names 0-0-0-105;
* since bug-1926 the dashed form 0-0-0-105 names it too.
@ -42,6 +43,10 @@
* selection mismatch, host-limit rejection, shortfall next to a limit,
* `info trace` for an array, an element and an unknown job, and the
* file equal to the inline console traces.
* 4. bug-1932 (extra replay): 205's request equals 204's, which a full
* search proved unsatisfiable earlier in the same session; 205 pends
* from the unsatisfiable-request cache with no search, its trace names
* 204, and it pends with 204's full reasons (per-host included).
*/
#include <stdbool.h>
#include <stdio.h>
@ -665,11 +670,68 @@ static void test_compound_array_selection_and_info_trace(void)
run_done(&r);
}
/* ---------------------------------------------------------------------- */
/* 4. A request already proved unsatisfiable this session (bug-1932) */
/* ---------------------------------------------------------------------- */
/* The outcome section of a trace block: from its " outcome: " line on. */
static const char* outcome_of(const char* block)
{
const char* o = strstr(block, "\n outcome: ");
TEST_ASSERT_NOT_NULL_MESSAGE(o, block);
return o;
}
static void test_known_unsatisfiable_request_skipped_with_full_reasons(void)
{
run_t r;
char* b204;
char* b205;
run_repl(&r, EXTRA_REPLAY,
"trace on job 0-0-0-204 0-0-0-205 level full\n"
"step\n"
"step\n"
"step\n"
"quit\n");
/* Session 3: nothing else runs, so 204 passes the cluster-wide check
* and every host rejects it for mem: the reason is per-host only. */
b204 = block_of(r.out, "[trace] job 0-0-0-204 session 3");
IN_ORDER(b204, "\n cluster-wide check passed\n",
"\n host w01 rejected (per-host check): resource "
"shortfall\n",
"\n host w02 rejected (per-host check): resource "
"shortfall\n",
"\n outcome: PEND\n", "\n host w01: ");
/* 205: no check and no plan, the trace names the job that proved the
* request unsatisfiable, and the pend reasons are 204's, in full. */
b205 = block_of(r.out, "[trace] job 0-0-0-205 session 3");
IN_ORDER(b205,
"\n no plan generated: skipped, same request as 0-0-0-204 "
"proved unsatisfiable earlier this session\n",
"\n outcome: PEND\n");
TEST_ASSERT_NULL_MESSAGE(strstr(b205, "cluster-wide check"), b205);
TEST_ASSERT_NULL_MESSAGE(strstr(b205, "\n cluster-wide:"), b205);
TEST_ASSERT_NULL_MESSAGE(strstr(b205, " plan 1 "), b205);
TEST_ASSERT_EQUAL_STRING_MESSAGE(
outcome_of(b204), outcome_of(b205),
"bug-1932: a cache hit must pend with the full reasons of the job "
"that proved the request unsatisfiable");
free(b204);
free(b205);
run_done(&r);
}
int main(void)
{
UNITY_BEGIN();
RUN_TEST(test_step_mode_full_all_jobs_and_file_copy);
RUN_TEST(test_continue_breakpoint_summary_and_info_trace);
RUN_TEST(test_compound_array_selection_and_info_trace);
RUN_TEST(test_known_unsatisfiable_request_skipped_with_full_reasons);
return UNITY_END();
}

12
src/scheduler/testsuite/test_repl_trace_e2e_fixture.c
View file

@ -386,6 +386,9 @@ static uint64_t array_elem(uint32_t k)
* 203 selects mem>20000 (w01 mismatches), pended by the
* cluster-wide limit (6 of 6 in use)
* 3000 after 201 and both elements finish: 202 and 203 are tried again
* 4000 after 202 and 203 finish: 204 and 205 (identical requests, more
* mem than any host has; bug-1932) pend in every session -- 204
* by a full search, 205 by the unsatisfiable-request cache
*/
void write_extra_replay(const char* dest)
{
@ -393,7 +396,7 @@ void write_extra_replay(const char* dest)
char path[PATH_MAX];
FILE* f;
xce_worker_t* workers[2];
xce_job_t* jobs[5];
xce_job_t* jobs[7];
snprintf(record_dir, sizeof(record_dir), "/tmp/sched_tre2e_gen_XXXXXX");
TEST_ASSERT_NOT_NULL(mkdtemp(record_dir));
@ -420,7 +423,9 @@ void write_extra_replay(const char* dest)
1, 1);
jobs[4] = make_push_job(203, 1104, "select[mem>20000] rusage[mem=512]",
"alice", 1, 1);
append_jobs(f, 1003ULL, jobs, 5);
jobs[5] = make_push_job(204, 1105, "rusage[mem=999999]", "carol", 1, 1);
jobs[6] = make_push_job(205, 1106, "rusage[mem=999999]", "carol", 1, 1);
append_jobs(f, 1003ULL, jobs, 7);
append_session(f, 2000ULL);
append_state_change(f, 2500ULL, 201, JOB_STAT_RUN, JOB_STAT_DONE);
@ -429,6 +434,9 @@ void write_extra_replay(const char* dest)
append_state_change(f, 2700ULL, array_elem(2), JOB_STAT_RUN,
JOB_STAT_DONE);
append_session(f, 3000ULL);
append_state_change(f, 3500ULL, 202, JOB_STAT_RUN, JOB_STAT_DONE);
append_state_change(f, 3600ULL, 203, JOB_STAT_RUN, JOB_STAT_DONE);
append_session(f, 4000ULL);
append_end_of_stream(f);
TEST_ASSERT_EQUAL_INT(0, fclose(f));

31
src/scheduler/testsuite/test_scheduler_cache_dispatch.c
View file

@ -173,22 +173,24 @@ int __wrap_sched_meet_host_selection(xce_sched_core_t* core,
static int g_lookup_cb_calls = 0;
static int g_populate_cb_calls = 0;
static int spy_unsatisfiable_lookup_cb(xce_sched_core_t* core,
static int spy_unsatisfiable_lookup_cb(xce_sched_core_t* core,
xce_sched_obj_t* obj,
int* out_reason_category)
xce_job_preason_t* out_reasons,
uint64_t* out_jobid)
{
g_lookup_cb_calls++;
return xce_alloc_cache_lookup_cb(core, obj, out_reason_category);
return xce_alloc_cache_lookup_cb(core, obj, out_reasons, out_jobid);
}
static void spy_unsatisfiable_populate_cb(xce_sched_core_t* core,
xce_sched_obj_t* obj,
int reason_category)
static void spy_unsatisfiable_populate_cb(xce_sched_core_t* core,
xce_sched_obj_t* obj,
const xce_job_preason_t* reasons,
uint64_t jobid)
{
g_populate_cb_calls++;
xce_alloc_cache_populate_cb(core, obj, reason_category);
xce_alloc_cache_populate_cb(core, obj, reasons, jobid);
}
void setUp(void) { return; }
@ -451,6 +453,21 @@ static void test_standalone_primary_cache_hit_skips_allocator(void)
decision2->u.reason->reasons->main_reason,
"a cache hit must attribute the SAME pend reason category as the "
"original unsatisfiable determination (Req 7.5)");
/* bug-1932: job 1's real cause is per-host (the worker's slots
* shortfall), with main reason 0; a hit that keeps only the main
* reason pends job 2 with an EMPTY reason. */
TEST_ASSERT_TRUE_MESSAGE(
(decision1->u.reason->reasons->nhost_reasons
+ decision1->u.reason->reasons->nres_reasons)
> 0,
"fixture: job 1's exhausted attempt must record a per-host or "
"per-resource reason (its one worker has no free slots)");
TEST_ASSERT_TRUE_MESSAGE(
job_preason_eq(decision1->u.reason->reasons,
decision2->u.reason->reasons, NULL),
"bug-1932: a cache hit must restore the original attempt's FULL "
"pend reason (main, per-host and per-resource), not only its main "
"reason");
TEST_ASSERT_EQUAL_INT_MESSAGE(
2, g_lookup_cb_calls,
"job 2's cache hit must also be consulted through core->"

21
src/scheduler/testsuite/test_scheduler_cache_dispatch_compound.c
View file

@ -132,22 +132,24 @@ int __wrap_sched_meet_host_selection(xce_sched_core_t* core,
static int g_lookup_cb_calls = 0;
static int g_populate_cb_calls = 0;
static int spy_unsatisfiable_lookup_cb(xce_sched_core_t* core,
static int spy_unsatisfiable_lookup_cb(xce_sched_core_t* core,
xce_sched_obj_t* obj,
int* out_reason_category)
xce_job_preason_t* out_reasons,
uint64_t* out_jobid)
{
g_lookup_cb_calls++;
return xce_alloc_cache_lookup_cb(core, obj, out_reason_category);
return xce_alloc_cache_lookup_cb(core, obj, out_reasons, out_jobid);
}
static void spy_unsatisfiable_populate_cb(xce_sched_core_t* core,
xce_sched_obj_t* obj,
int reason_category)
static void spy_unsatisfiable_populate_cb(xce_sched_core_t* core,
xce_sched_obj_t* obj,
const xce_job_preason_t* reasons,
uint64_t jobid)
{
g_populate_cb_calls++;
xce_alloc_cache_populate_cb(core, obj, reason_category);
xce_alloc_cache_populate_cb(core, obj, reasons, jobid);
}
void setUp(void) { return; }
@ -412,6 +414,11 @@ static void test_compound_nested_primary_cache_hit_skips_allocator(void)
"a cache hit must attribute the SAME pend reason category as the "
"original unsatisfiable determination (Req 7.5), even across a "
"compound-nested primary");
TEST_ASSERT_TRUE_MESSAGE(
job_preason_eq(decisionA->u.reason->reasons,
decisionB->u.reason->reasons, NULL),
"bug-1932: a cache hit must restore the original attempt's FULL "
"pend reason, even across a compound-nested primary");
TEST_ASSERT_EQUAL_INT_MESSAGE(
2, g_lookup_cb_calls,
"job B's cache hit must also be consulted through core->"

58
src/scheduler/testsuite/test_sim_cache_clear_isolation.c
View file

@ -37,6 +37,13 @@
* REAL xce_core_begin_session() (core_session.c's public entry point),
* and asserts alt_events_file is bit-for-bit unchanged across the call.
*
* bug-1932 update: the table now lives on xce_sched_core_t and the
* simulator registers the same hook as normal mode (from its
* sched_sim_args_t). The NULL-hook test below still guards the no-hook
* path; test_sim_registered_cache_clear_uses_the_core_table guards the
* registered one: clearing builds the core's table and leaves
* alt_events_file alone.
*
* ── Link technique ───────────────────────────────────────────────────────
* Same technique as the sibling test_core_session_cache_clear.c (round 1):
* this fixture's meson.build entry lists core.c/core_clock.c/
@ -107,10 +114,9 @@ static void test_sim_begin_session_never_corrupts_alt_events_file(void)
TEST_ASSERT_NOT_NULL(f);
sim.alt_events_file = f;
/* Deliberately NOT calling xce_core_set_cache_clear_cb() here --
* mirrors sim_loop.c exactly, which never registers this hook (Req 5
* AC 17 "Simulation isolation": sim mode has no cache to clear). This
* is the crux of the regression: before round 2's fix,
/* Deliberately NOT calling xce_core_set_cache_clear_cb() here -- the
* no-hook path (a core whose driver registered no cache). This is the
* crux of the regression: before round 2's fix,
* core_begin_session_internal() called cache_clear() unconditionally,
* so linking alloc_cache.c's real, strong cache_clear(xce_scheduler_t*)
* into this binary was enough to corrupt alt_events_file below even
@ -132,9 +138,53 @@ static void test_sim_begin_session_never_corrupts_alt_events_file(void)
fclose(f);
}
/*
* bug-1932: the simulator now registers the SAME cache hooks as normal mode
* (sched_sim_driver_start(), from the driver's sched_sim_args_t). That is
* safe only because the cache table lives on xce_sched_core_t and no hook
* casts the core to a driver struct: clearing at session start must build
* the core's own table and still leave every sim-only field alone.
*/
static void test_sim_registered_cache_clear_uses_the_core_table(void)
{
xce_sched_sim_driver_t sim;
FILE* f;
int rc;
memset(&sim, 0, sizeof(sim));
rc = xce_sched_core_init(&(sim.core));
TEST_ASSERT_EQUAL_INT(0, rc);
f = tmpfile();
TEST_ASSERT_NOT_NULL(f);
sim.alt_events_file = f;
rc = xce_core_set_cache_clear_cb(&(sim.core), xce_alloc_cache_clear_cb);
TEST_ASSERT_EQUAL_INT(0, rc);
rc = xce_core_begin_session(&(sim.core));
TEST_ASSERT_EQUAL_INT(0, rc);
TEST_ASSERT_EQUAL_PTR_MESSAGE(
f, sim.alt_events_file,
"the registered cache_clear_cb must clear the core's own table, "
"never reinterpret a simulation driver field");
TEST_ASSERT_NOT_NULL_MESSAGE(
sim.core.unsatisfiable_cache,
"bug-1932: a session start must build the core's cache table");
rc = xce_core_end_session(&(sim.core));
TEST_ASSERT_EQUAL_INT(0, rc);
fclose(f);
/* core fini needs the driver-owned pend_updates queues this fixture
* never builds; free the one thing this test made */
destroy(sim.core.unsatisfiable_cache);
}
int main(void)
{
UNITY_BEGIN();
RUN_TEST(test_sim_begin_session_never_corrupts_alt_events_file);
RUN_TEST(test_sim_registered_cache_clear_uses_the_core_table);
return UNITY_END();
}

7
src/scheduler/testsuite/test_sim_verify.c
View file

@ -733,11 +733,8 @@ static void test_pend_decisions_differing_main_reason_mismatch(void)
/*
* test_pend_decisions_differing_reason_detail_still_match - bug-1845
* regression: the recorded (expected) side represents a live cache-hit
* PEND (core.h's documented "simulation isolation" -- sim/replay never
* registers core->unsatisfiable_lookup_cb, so it can never itself produce
* this shape, only ever receive it from a recorded normal-mode run) --
* main_reason set, zero host/resource reason detail
* (sched_pend_from_cached_reason_(), scheduler.c, never adds any). The
* PEND recorded before bug-1932 -- main_reason set, zero host/resource
* reason detail (the pre-fix cache stored only the main reason). The
* live (actual) side represents replay's own full per-host computation for
* the SAME job/round -- same main_reason, but with real host- and
* resource-based reason detail attached (sched_try_alloc_on_host_(),

46
src/scheduler/testsuite/test_trace_render.c
View file

@ -1021,6 +1021,51 @@ static void test_no_decision_pend_without_reason_keeps_hierarchy(void)
sched_trace_rec_free(rec);
}
/* bug-1932: a request the unsatisfiable-request cache already holds is
* skipped; the line names the job whose search proved it, or says so
* without a name when that job is unknown (0). */
static void test_known_unsatisfiable_names_source_job(void)
{
static const char* const rn[] = { "slots" };
static const float rv[] = { 1.0f };
xce_sched_trace_rec_t* rec;
fx_primary_t p;
char* out;
rec = new_rec(XCE_TRACE_LEVEL_FULL, XCE_TRACE_LINE_BOUND_DEFAULT,
XCE_TRACE_REC_DECISION);
TEST_ASSERT_NOT_NULL(rec);
set_selection(rec);
set_glimits(rec, 0, NULL, NULL);
make_primary(&p, 1, 1, 1, rn, rv, NULL);
sched_trace_path_push(rec, XCE_TRACE_PATH_SINGLE, 1, 1);
sched_trace_resreq_begin(rec, &p.p, 0);
/* the hit returns before the plan search: no search end follows */
sched_trace_known_unsat(rec, (uint64_t)204);
sched_trace_known_unsat(rec, 0);
sched_trace_resreq_end(rec, 0, 0);
sched_trace_path_pop(rec);
sched_trace_rec_finish(rec, XCE_TRACE_REC_PEND_NO_REASON, 0);
TEST_ASSERT_EQUAL_INT(1, rec->counters.reqs_none_generated);
out = render(rec, XCE_REPL_UNITS_RAW);
assert_golden(
"[trace] job 1-2-1000-42 session 7 position 3 level full\n"
SEL_LINE NO_LIMITS_LINE
" requirement single tasks 1..1 requests slots=1\n"
" no plan generated: skipped, same request as 0-0-0-204 proved "
"unsatisfiable earlier this session\n"
" no plan generated: skipped, same request proved unsatisfiable "
"earlier this session\n"
" single requirement not satisfied\n"
" no decision: job pended without any pend reason\n",
out);
free(out);
sched_trace_rec_free(rec);
}
static void test_reconstructed_record(void)
{
xce_sched_trace_rec_t* rec;
@ -1318,6 +1363,7 @@ int main(void)
RUN_TEST(test_no_decision_no_schedobj);
RUN_TEST(test_no_decision_internal_error_keeps_hierarchy);
RUN_TEST(test_no_decision_pend_without_reason_keeps_hierarchy);
RUN_TEST(test_known_unsatisfiable_names_source_job);
RUN_TEST(test_reconstructed_record);
RUN_TEST(test_selection_and_limits_unavailable);
RUN_TEST(test_entry_kinds_and_edge_cases);