docs(scheduler-agent): update design doc with cluster verification results

- Design doc v1.2: Phase 0-1 cluster verification status (read + submit OK,
  control HTTP 500 open bug-1490), bug table for whitelist/job_status fixes,
  mcp_operator queue:control note.
- Root + frontend CLAUDE.md: reflect verified state and apiserver bugs found.
- wolf memory/anatomy/buglog: log deployment session, bug-1489/bug-1490.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
lei.chen 2026-08-01 09:54:49 +00:00
parent e36ed63ca0
commit 9fe14c1512
6 changed files with 135 additions and 30 deletions

View file

@ -1199,6 +1199,25 @@
## src/frontend/apiserver/config/
- `auth.yaml` — Authentication provider: config_file | pam | ldap | oauth2 (~187 tok)
- `rbac.yaml` — Roles/permissions; includes mcp_readonly (Phase 0) + mcp_operator (Phase 1, queue:submit + job:control) (~1.4k tok)
## src/frontend/schedulerclient/
- `client.go` — Client interface: 6 read + 4 write (SubmitJob/CancelJob/HoldJob/ReleaseJob) methods (~0.5k tok)
- `types.go` — URM types (URMJob/URMQueue/URMNode/JobFilter/LogOpts/Page/Metrics) (~1k tok)
- `httpclient.go` — resty v3 read implementation (GET, apiEnvelope, mbToBytes=1e6) (~3.5k tok)
- `write.go` — Phase 1: JobSpec, SubmitJobResult, JobControlRequest, doPost (HTTP 200 + envelope-code), SubmitJob/Cancel/Hold/Release impl (~3k tok)
- `statusmap.go` — Status string → URM mapping functions (~0.9k tok)
- `write_test.go` — httptest TDD for write paths (~2.5k tok)
- `statusmap_test.go` — Table-driven status mapping tests (~0.9k tok)
## src/frontend/mcp-server/
- `cmd/main.go` — MCP stdio entry; registers 10 tools (6 read + 4 write), env XCE_API_URL/XCE_API_KEY (~2k tok)
- `internal/app.go` — Tool handlers wiring schedulerclient (read + write) (~3k tok)
- `internal/templates.go` — JobTemplate registry + general_serial (通用串行脚本) + parseMemoryToBytes/parseWalltimeSeconds (~3k tok)
- `internal/app_test.go` — fake-client handler TDD (~2.5k tok)
- `internal/templates_test.go` — Template/registry/parser TDD (~2k tok)
## src/frontend/xcecli/cmd/auth/

View file

@ -14507,5 +14507,42 @@
],
"occurrences": 1,
"last_seen": "2026-08-01T03:30:00Z"
},
{
"id": "bug-1489",
"timestamp": "2026-08-01",
"error_message": "POST /xce/v1/apikeys -> 403 \"URL is not allowed\": WhitelistURLMiddleware blocks it",
"file": "src/frontend/apiserver/middleware/whitelist.go",
"root_cause": "Phase 0 added apikeys route (routers.go:82) but never added /xce/v1/apikeys to WhitelistPOSTURL, so the URL whitelist middleware rejects it before RBAC",
"fix": "Added \"/xce/v1/apikeys\" to WhitelistPOSTURL in whitelist.go",
"tags": [
"apiserver",
"whitelist",
"apikeys",
"phase0"
],
"related_bugs": [],
"occurrences": 1,
"last_seen": "2026-08-01"
},
{
"id": "bug-1490",
"timestamp": "2026-08-01",
"error_message": "POST /xce/v1/jobs/<id>/control action=kill/stop/resume returns 202 with \"job not found\" even though the job exists in store.Jobs (list shows it)",
"file": "src/frontend/apiserver/controllers/jobs.go",
"root_cause": "Chain of issues in apiserver job-control path. (1) job_status int64 assertion broke control (fixed: normalizeJobStatus in jobs.go). (2) After that fix, control now returns HTTP 500 \u2014 request never reaches the controller (controller log shows no control event). Likely a backend call to controller blocking/erroring inside apiserver. Not yet root-caused.",
"fix": "PARTIAL: jobs.go normalizeJobStatus() deployed. Control still returns HTTP 500 \u2014 open.",
"tags": [
"apiserver",
"job-control",
"http500",
"phase1-verify",
"open"
],
"related_bugs": [
"bug-1489"
],
"occurrences": 2,
"last_seen": "2026-08-01"
}
]

View file

@ -7265,3 +7265,13 @@
| 02:32 | Session end: 35 writes across 11 files (_live_e2e_common.py, test_e2e_live_push.py, test_e2e_live_ha_push.py, manual_ha_deploy.sh, test_repro_bug1485.py) | 30 reads | ~366672 tok |
| 03:04 | Merged origin/working (64 commits: config-upgrade-snapshot feature + HA/repl bug fixes), deduped capnpc override — kept xce.nix fetchFromGitHub pin (bug-918 canonical fix), dropped flake.nix ghfast.top overlay | src/flake.nix, src/nixSupport/xce.nix | merged clean, 0 conflicts; nix files parse OK | ~1200 |
| 03:50 | User rule: never push to Rongsong's branch (origin/working) without explicit permission; we only sync his code into our own branch. Written to Claude memory (xce-push-policy-rongsong) + cerebrum User Preferences | .wolf/cerebrum.md, /home/lchen/.claude/projects/-home-lchen-code/memory/ | persisted both stores | ~400 |
| 04:44 | Phase 1: schedulerclient write API (SubmitJob/CancelJob/HoldJob/ReleaseJob + JobSpec + doPost) TDD green | src/frontend/schedulerclient/write.go, write_test.go, client.go | 33+12 tests pass | ~2k tok |
| 04:48 | Phase 1: mcp-server write tools + templates (general_serial, dry_run review mode) + fix read-tool annotations | src/frontend/mcp-server/internal/templates.go, app.go, app_test.go, templates_test.go, cmd/main.go | go vet/build/test green | ~3k tok |
| 04:49 | Phase 1: mcp_operator RBAC role (queue:submit + job:control, domain all) TDD green | src/frontend/apiserver/config/rbac.yaml, controllers/rbac_mcp_operator_test.go | tests pass | ~0.5k tok |
| 04:50 | Phase 1 build verify: schedulerclient/mcp-server vet+build+test green, apiserver controllers/store green, make test green, mcp-server binary builds (12MB) | all modules | pass | ~0.5k tok |
| 04:53 | Phase 1 docs: updated 设计文档 v1.1 (§4.4.2 write tools, §4.5.3 mcp_operator, §5.2 done, §6.2 layout, §10 progress, §12.2 gotchas) + root/frontend CLAUDE.md + anatomy | docs/Scheduler-Agent-设计文档.md, CLAUDE.md, src/frontend/CLAUDE.md, .wolf/anatomy.md | done | ~1k tok |
| 06:16 | AppImage built: xce-x86_64.AppImage 280500768B, single-pass mksquashfs (relative AppRun/mountroot sources + -no-strip deps + -p entrypoint) | /home/lchen/xce-appimage/package.sh + xce-x86_64.AppImage | OK, layout matches old | ~2k tok |
| 06:33 | Stopped+disabled old XCE systemd (t1 scheduler+workerd, t2 workerd) via sudo | t1/t2 systemd | old daemons killed | ~0.5k tok |
| 06:53 | Synced /opt/newxce store (99 paths) + run.sh t1→t2,t3 (rsync, 929MB); cluster.bin built (sha 044f1b3a) + certs + workdir on t1 | t1/t2/t3 /opt/newxce | done | ~1k tok |
| 08:59 | HA 3-node election deadlock (voted_for term residue + no-pool standbys) → switched to t1 single-node controller (omit --peers) per user; now CTRL_PRIMARY term=1 | t1 /opt/newxce | primary OK | ~1k tok |
| 09:01 | Cluster up on t1: controller(9990, CTRL_PRIMARY) + apiserver(8080, RBAC OK) + scheduler + worker; t2/t3 workers launched | t1/t2/t3 | all daemons running | ~1k tok |

View file

@ -49,7 +49,7 @@ For details on any topic below, read the corresponding steering file in `.kiro/s
- `src/worker/` — See `src/worker/CLAUDE.md`
- `src/scheduler/` — See `src/scheduler/CLAUDE.md`
- `src/lib/` — See `src/lib/CLAUDE.md`
- `src/frontend/schedulerclient/`, `src/frontend/mcp-server/` — Phase 0 已完成(2026-07-26)。`schedulerclient` 是独立 Go library 提供只读 Client interface + resty HTTP 实现(33 单测)。`mcp-server` 是 MCP stdio 二进制(`github.com/mark3labs/mcp-go@v0.57.0`),通过 `schedulerclient` 调用 `apiserver` REST API,暴露 6 个只读 MCP Tool。Pure Go, `CGO_ENABLED=0`。总体架构与分阶段计划见根目录 `CLAUDE.md`("Unified Scheduler Agent Framework")。
- `src/frontend/schedulerclient/`, `src/frontend/mcp-server/` — **Phase 0-1 已完成**(Phase 0: 2026-07-26,Phase 1: 2026-08-01)。`schedulerclient` 是独立 Go library,提供读 + 写 Client(6 只读 + 4 写:SubmitJob/CancelJob/HoldJob/ReleaseJob,resty HTTP 实现,33+12 单测)。`mcp-server` 是 MCP stdio 二进制(`github.com/mark3labs/mcp-go@v0.57.0`),通过 `schedulerclient` 调用 `apiserver` REST API,暴露 10 个 MCP Tool(6 只读 + 4 写,写工具含 `dry_run` review 模式 + `general_serial` 作业模板 registry)。Pure Go, `CGO_ENABLED=0`。Phase 1 需 `mcp_operator` RBAC 角色(`config/rbac.yaml`)。总体架构与分阶段计划见 `docs/Scheduler-Agent-设计文档.md`。**集群验证(2026-08-01)**:t1 单节点 controller 集群上,读操作 + submit 已通过 mcp_operator API key 验证;hold/release/cancel 因 apiserver control 链路预存 bug(HTTP 500)待修(bug-1490)。
# Agentic SDLC and Spec-Driven Development

View file

@ -1,9 +1,9 @@
# Unified Scheduler Agent Framework — 设计文档
> **项目代号**:scheduler-agent
> **文档版本**:v1.0
> **文档版本**:v1.2
> **最近更新**:2026-08-01
> **文档状态**:实施中(Phase 0 完成,待集群验证)
> **文档状态**:实施中(Phase 0-1 代码完成;集群验证:读 + submit ✅,control 有 apiserver 预存 bug 待修)
---
@ -234,13 +234,18 @@ Phase 2+: Go 主体 + LSF 侧 Python Sidecar(UDS/gRPC 通信)
#### 4.4.2 写工具(默认 require confirm)
| Tool | 说明 |
|------|------|
| `scheduler_submit(job_spec)` | 提交作业,使用格式化参数或模板名称 |
| `scheduler_cancel(id)` | 取消作业 |
| `scheduler_hold(id)` | 暂停止作业 |
| `scheduler_release(id)` | 恢复暂停的作业 |
| `scheduler_modify(id, changes)` | 修改运行中作业的参数 |
**Phase 1 已实现(2026-08-01)**:
| Tool | 说明 | confirm 机制 |
|------|------|--------------|
| `scheduler_submit(template, command, ...)` | 提交作业,支持模板(`general_serial` 通用串行脚本)+ 格式化参数;`dry_run=true` 返回 review card | `dry_run` review 模式 + destructive 注解 |
| `scheduler_cancel(id)` | 取消作业(→ action `kill`,默认 SIGTERM) | destructive 注解,客户端确认 |
| `scheduler_hold(id)` | 暂停止作业(→ action `stop`,默认 SIGTSTP) | destructive 注解,客户端确认 |
| `scheduler_release(id)` | 恢复暂停的作业(→ action `resume`,默认 SIGCONT) | destructive 注解,客户端确认 |
> **Phase 1 未做**:`scheduler_modify(id, changes)` —— apiserver 当前无 modify 端点(job control 仅支持 kill/stop/resume/checkpoint),需新增后端端点,推迟。
>
> **confirm 实现说明**:MCP 协议(mcp-go v0.57.0)无原生 `requiresConfirmation` 字段,只有 `ToolAnnotation{ReadOnlyHint, DestructiveHint}`。实现采用两层:① `scheduler_submit` 用 `dry_run` 参数作为协议内 review 门(先返回 resolved JobSpec 供用户确认,再以 `dry_run=false` 真正提交);② cancel/hold/release 标记 `DestructiveHint: true`,由 MCP 客户端(Claude Code)在执行前弹出确认。这与设计 §4.5"写操作默认 require confirm"一致。
#### 4.4.3 高级工具
@ -278,8 +283,9 @@ Phase 2+: Go 主体 + LSF 侧 Python Sidecar(UDS/gRPC 通信)
- **mcp-server 用服务账号 API Key**(`xce_ak_` 前缀)调用 apiserver,而非透传终端用户 JWT
- API Key 不做 IP 绑定(面向长期运行的自动化客户端,源 IP 不稳定)
- Phase 0 用 `mcp_readonly` 角色(仅 `:view` 权限)
- 写操作阶段引入 `mcp_operator` 等带写权限的角色
- Phase 0 用 `mcp_readonly` 角色(仅 `:view` 权限)—— ✅ 已实施(2026-07-26)
- **Phase 1 引入 `mcp_operator` 角色** —— ✅ 已实施(2026-08-01):5 个 `*:view`(domain all)+ `queue:submit` + `job:control`(domain all)。覆盖 `scheduler_submit` 和 cancel/hold/release 的全部写端点权限。**不含** host:control / cluster:control / rbac:* / file:* / user:*——MCP 服务账号仍不能管理集群配置或其他用户/角色。
- 生成方式:管理员用 `POST /xce/v1/apikeys`(`rbac:control`,仅 cluster_admin)创建,`role_name: "mcp_operator"`。handler 同时注册 key 和 `AddRoleForUser(username, role)`。
### 4.6 事件流设计
@ -331,12 +337,13 @@ C Controller ──OPCode_push──→ apiserver ──SSE──→ mcp-server
**输入**:`schedulerclient` Go package(只读接口)
**输出**:可运行的 MCP Server,连接 Claude Code 后可以用自然语言查集群状态
### 5.2 Phase 1 — 作业提交 + 模板(+2 天)
### 5.2 Phase 1 — 作业提交 + 模板(✅ 已完成代码,待集群验证;2026-08-01)
- Submit / Cancel / Hold / Release / Modify 写操作工具
- 覆盖团队最常用的 3-5 种作业模板
- 严格的参数校验 + `--review` 模式(先展示命令再执行)
- 安全性审查
- **Submit / Cancel / Hold / Release 写操作工具**(✅ 4 个已实现;Modify 因 apiserver 无端点推迟)
- 作业模板(✅ `general_serial` 通用串行脚本已实现;registry 机制支持后续扩展 GPU/array/dependency)
- 严格的参数校验 + review 模式(✅ `scheduler_submit` 的 `dry_run=true` 返回 resolved JobSpec 供用户确认)
- 安全性审查(✅ destructive 注解 + `mcp_operator` RBAC 角色)
- **待集群验证**:运行中的 apiserver + 生成 `mcp_operator` API Key + Claude Code MCP 配置(同 Phase 0 阻塞项)
### 5.3 Phase 2 — 监控 + 纠错 loop(+1 周)
@ -394,15 +401,20 @@ scheduler-agent/
```
myxce/src/frontend/
├── apiserver/ # Go REST API server(cgo 桥接 C backend)
│ └── config/rbac.yaml # 含 mcp_readonly(Phase 0)+ mcp_operator(Phase 1)角色
├── xcecli/ # Go CLI 工具
├── schedulerclient/ # ← Phase 0 ✅ scheduler-agent 只读 Go client
│ ├── client.go # Client interface
├── schedulerclient/ # ← Phase 0-1 ✅ scheduler-agent Go client(读 + 写)
│ ├── client.go # Client interface(6 只读 + 4 写方法)
│ ├── types.go # URMJob / URMQueue / URMNode
│ ├── statusmap.go # 状态映射(含单测)
│ └── httpclient.go # resty HTTP 实现
└── mcp-server/ # ← Phase 0 ✅ MCP Server 二进制
├── cmd/main.go # 入口,注册 6 个 MCP Tool,启动 stdio server
└── internal/app.go # Tool handlers 连接 schedulerclient
│ ├── httpclient.go # resty HTTP 实现(GET)
│ └── write.go # ← Phase 1 写接口:JobSpec / SubmitJobResult / doPost /
│ # SubmitJob / CancelJob / HoldJob / ReleaseJob(含单测)
└── mcp-server/ # ← Phase 0-1 ✅ MCP Server 二进制
├── cmd/main.go # 入口,注册 10 个 MCP Tool(6 只读 + 4 写),stdio server
└── internal/
├── app.go # Tool handlers 连接 schedulerclient(读 + 写)
└── templates.go # ← Phase 1 作业模板 registry + general_serial(含单测)
```
---
@ -501,6 +513,8 @@ Agent loop 跨多轮调度器轮次保持上下文:
### 10.1 已完成
**Phase 0(只读链路)**:
| 任务 | 内容 | 状态 |
|------|------|------|
| Task #1 | 架构与设计文档 | ✅ |
@ -508,19 +522,40 @@ Agent loop 跨多轮调度器轮次保持上下文:
| Task #3 | `schedulerclient` 只读客户端 | ✅ 33 单测 |
| Task #4 | `mcp-server` MCP 二进制 | ✅ 构建通过 |
**Phase 0 完整链路(代码层面就绪)**:
**Phase 1(写操作链路,2026-08-01)**:
| 任务 | 内容 | 状态 |
|------|------|------|
| Task #5 | `schedulerclient` 写接口(Submit/Cancel/Hold/Release + JobSpec + doPost) | ✅ 12 单测 |
| Task #6 | `mcp-server` 4 个写工具 + `general_serial` 模板 + `dry_run` review 模式 | ✅ 14 单测 |
| Task #7 | 修复 6 个只读工具注解(原被默认误标 destructive) | ✅ |
| Task #8 | `mcp_operator` RBAC 角色(`queue:submit` + `job:control`) | ✅ 测试全绿 |
**Phase 0-1 完整链路(代码层面就绪)**:
```
用户自然语言 → Claude Code → MCP stdio → mcp-server → schedulerclient → apiserver REST API → 调度器
(读:6 只读工具 + 写:4 写工具)
```
### 10.2 待办(阻塞项)
| # | 阻塞项 | 状态 | 责任方 |
|---|--------|------|--------|
| 1 | 运行中的 apiserver + C 编译环境 | 🚫 阻塞 | 用户准备集群 |
| 2 | 生成 API Key(`mcp_readonly` 角色) | 🚫 依赖 #1 | 用户/管理员 |
| 3 | 配置 Claude Code MCP | ✅ 拿到 key 后 1 分钟 | 用户 |
| 1 | 运行中的 apiserver + C 编译环境 | ✅ t1 单节点 controller + apiserver + scheduler + worker 已启动(2026-08-01) | 已完成 |
| 2 | 生成 API Key | ✅ `mcp_operator` key 已生成并验证读 + submit | 已完成 |
| 3 | 配置 Claude Code MCP | 🚫 拿到集群 API 后配置 | 用户 |
| 4 | Phase 1 端到端验证:提交→列表→hold→release→cancel | ⚠️ 部分完成:读 + submit ✅;**hold/release/cancel 返回 HTTP 500**(apiserver control 链路预存 bug,见 bug-1490) | 待修 apiserver |
### 10.2b 本次集群验证发现并修复的 apiserver bug(2026-08-01)
| Bug | 现象 | 修复 |
|-----|------|------|
| whitelist 缺 `/xce/v1/apikeys` | API Key 生成端点 403 "URL is not allowed" | `middleware/whitelist.go` 加 `/xce/v1/apikeys`(bug-1489) |
| control 的 `job_status` int64 断言 | control 返回 "Invalid job_status" | `controllers/jobs.go` 加 `normalizeJobStatus()` 兼容 int/int64/float64 |
| control 链路 500(未修) | hold/release/cancel 返回 500,请求未达 controller | 待排查(bug-1490) |
### 10.3 未决决策项
### 10.3 未决决策项
@ -559,6 +594,10 @@ Agent loop 跨多轮调度器轮次保持上下文:
### 12.2 关键实现细节备忘
- **认证**:apiserver 的 `GetUsernameFromToken`/`isValidToken` 已支持 `xce_ak_` 前缀,JWT 和 API Key 透明共存(27 处调用点零改动)
- **schedulerclient 内存单位**:假设 apiserver 返回 MB(HPC 惯例),转 bytes 存入 URM
- **schedulerclient 内存单位**:假设 apiserver 返回 MB(HPC 惯例),转 bytes 存入 URM(`mbToBytes = 1_000_000` decimal)
- **schedulerclient 认证**:Authorization header 直接传 token(不拼 Bearer,服务端会 TrimPrefix)
- **mcp-server 依赖**:`github.com/mark3labs/mcp-go@v0.57.0`,`replace schedulerclient => ../schedulerclient`
- **写端点 HTTP 状态陷阱(Phase 1)**:`ReturnSuccess` 恒返回 HTTP 200,逻辑状态在 envelope `code` 字段(CreateJob→201,ControlJob→202);只有 `ReturnError` 才把 HTTP 状态设为错误码。`doPost` 检查 `resp.StatusCode()==200` 再 `env.Code==expectedCode`,**不要**要求 HTTP 201/202
- **`run_limit` 分钟陷阱(Phase 1)**:apiserver `CustomInt` 解析裸整数为**分钟**(`minutes*60`),`"HH:MM"` 才是秒。schedulerclient 只发 `"HH:MM"` 字符串;模板 `parseWalltimeSeconds` 拒绝裸整数避免歧义
- **`mem_limit` wire 格式(Phase 1)**:`CustomLimit` 只接受 `"<number> <unit>"` 字符串(`"4096 MB"`),GB=1024MB 二进制;客户端通过 `formatMemory` 发 MB 字符串,模板解析用 decimal 与 `mbToBytes` 保持一致
- **confirm 无原生支持(Phase 1)**:mcp-go v0.57.0 无 `RequiresConfirmation`,仅 `ToolAnnotation` 且 `NewTool` 默认 `destructive=true`(Phase 0 只读工具因此被误标,已在 Phase 1 修复);写操作 confirm 用 `dry_run` review 参数 + destructive 注解实现

View file

@ -8,8 +8,8 @@ Go-based REST API server (Gin) and CLI tool (Cobra) that expose cluster manageme
|--------|------|-------------|
| `apiserver` | `src/frontend/apiserver/` | `cmd/main.go` |
| `xcecli` | `src/frontend/xcecli/` | `cmd/main.go` |
| `schedulerclient` (Phase 0, ✅ done) | `src/frontend/schedulerclient/` | library, no entry point — read-only Go client for the scheduler agent's MCP Server, calls `apiserver`'s REST API. `go vet`/`go build`/`go test` clean (33 tests). See module source: `types.go` (URM types), `client.go` (interface), `httpclient.go` (resty impl), `statusmap.go`+`_test.go` (TDD, status mapping). |
| `mcp-server` (Phase 0, ✅ done) | `src/frontend/mcp-server/` | `cmd/main.go` — read-only MCP Server exposing 6 tools (`scheduler_list_jobs`/`get`/`list_queues`/`list_nodes`/`job_logs`/`cluster_metrics`); pure Go, `CGO_ENABLED=0`, uses `schedulerclient` as API client, authenticates via `XCE_API_KEY` env var. Uses `github.com/mark3labs/mcp-go@v0.57.0` for MCP stdio transport. `go build`/`go vet` clean, binary ~11MB. |
| `schedulerclient` (Phase 0-1, ✅ done) | `src/frontend/schedulerclient/` | library, no entry point — read + write Go client for the scheduler agent's MCP Server, calls `apiserver`'s REST API. `go vet`/`go build`/`go test` clean (45 tests). Read: `types.go` (URM types), `client.go` (interface), `httpclient.go` (resty impl), `statusmap.go`+`_test.go` (TDD, status mapping). Write (Phase 1): `write.go`+`_test.go` — `JobSpec`, `SubmitJob`/`CancelJob`/`HoldJob`/`ReleaseJob`, `doPost` (HTTP 200 + envelope-code dual check). |
| `mcp-server` (Phase 0-1, ✅ done) | `src/frontend/mcp-server/` | `cmd/main.go` — MCP Server exposing 10 tools (6 read-only + 4 write: `scheduler_submit`/`cancel`/`hold`/`release`); pure Go, `CGO_ENABLED=0`, uses `schedulerclient` as API client, authenticates via `XCE_API_KEY` env var. Uses `github.com/mark3labs/mcp-go@v0.57.0` for MCP stdio transport. `internal/templates.go` — job template registry + `general_serial` (通用串行脚本); `scheduler_submit` has `dry_run` review mode (returns resolved JobSpec before submitting). Read-only tools annotated ReadOnlyHint=true; write tools DestructiveHint=true. `go build`/`go vet`/`go test` clean, binary ~12MB. |
## Build & Test
```bash