lei.chen/myxce
1
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

apiserver: persist API keys across restarts via state file

Browse source 
API keys were a pure in-memory map (store/apikey.go) — every apiserver
restart silently revoked every service-account key (advisor, MCP
server) until re-issued, with only an inexplicable 401 to show for it.

Keys now persist to <top>/state/apikeys.json: hash → metadata only
(never the raw key, preserving the existing "a dump cannot leak usable
credentials" property), written atomically (temp file + fsync + rename,
0600) on each issuance, loaded at InitStores. The load also restores
the user→role mapping, without which a reloaded key would authenticate
but be denied every permission check. A missing file is a normal fresh
install; a corrupt one fails fast at startup instead of quietly
starting with zero keys. A failed save fails the issuance request —
the raw key is shown exactly once, so we must not hand out a key we
could not persist.

The memory-only path (no SetAPIKeyStatePath) is kept for unit tests.

Cluster-verified on t1-t3: issue key → restart apiserver → key still
authenticates (hosts OK, advisor:report POST published 1) — previously
impossible. Verify job submitted, ran, exited; all daemons healthy.

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
lei.chen 2026-10-20 03:12:13 +00:00
parent e7fbedd047
commit 9dbb75adb3
8 changed files with 312 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

4
src/docs/advisor-demo-cases.md
View file

@ -24,7 +24,7 @@ curl -s -H "Authorization: Bearer $KEY" http://172.31.1.30:8080/xce/v1/hosts \
# 4. LLM 路由可用（用例 2 需要；不通则 advisor 自动降级 echo，演示仍可做但亮点变少）
curl -s -m 10 http://127.0.0.1:6789/v1/models -H "Authorization: Bearer sk-litellm-router" | head -c 120
# 5. API key 是否有效（apiserver 重启过就失效，见"坑"一节）
# 5. API key 是否有效（2026-10-20 起重启不再失效；若 401 见"坑"一节）
curl -s -H "Authorization: Bearer $KEY" "http://172.31.1.30:8080/xce/v1/jobs?limit=1" | head -c 80
```
@ -196,7 +196,7 @@ EOF
| 坑 | 影响 | 预案 |
|---|---|---|
| **apiserver 重启使所有 API key 失效**（key 是纯内存 map，无持久化） | 所有 curl/xce 401 | 重新发：logon 拿 admin token → `POST /xce/v1/apikeys`（role_name=advisor_agent，username=advisor-svc）→ 存回 `/tmp/advisor_apikey_v2` → **重启 advisor** |
| ~~apiserver 重启使所有 API key 失效~~（2026-10-20 起已修：key 持久化到 `/opt/xce/state/apikeys.json`，存 hash+metadata，重启自动恢复） | — | 仅当 state 文件丢失/损坏时才需重发：logon 拿 admin token → `POST /xce/v1/apikeys`（role_name=advisor_agent，username=advisor-svc）→ 存回 `/tmp/advisor_apikey_v2` → **重启 advisor** |
| glm 路由超时（偶发） | 用例 2 降级为 echo（degraded=true） | 顺势讲兜底设计；或演示前 `curl /v1/models` 预检；**直接换 deepseek-v4-flash**（快且稳，见用例 2） |
| SSE 监听用 `curl -m 180` 会在 5min 阈值前断 | 以为推送没到 | 监听窗口用 `-m 600` 以上，或用 MCP 工具事后查 |
| stall 阈值 5 分钟 | 用例 1 有 5-6 分钟干等 | 先讲架构图/MCP 工具填这段时间；或把演示作业提前 5 分钟提交 |

3
src/docs/advisor-design.md
View file

@ -119,7 +119,8 @@ tokens / findings / raw / state(+digest) / degraded`。JSONL、按大小轮转
| 阶段 | 内容 |
|---|---|
| P1 ✅ | propose 档：findings 推送（SSE 广播 + MCP 工具，已实现；CLI 直出未做）；评估 echo vs LLM 的对照数据 |
| P1 ✅ | propose 档：findings 推送（SSE 广播 + MCP 工具，已实现；CLI 直出未做）；评估 echo vs LLM 的对照数据（`advisor compare` 子命令，d4bd504f） |
| — ✅ | 依赖项修复：apiserver API key 持久化（advisor/MCP 的服务账号 key 原先纯内存，apiserver 重启即全部失效；现落盘 `<top>/state/apikeys.json`，hash+metadata，签发即原子写、启动即加载，重启 key 存活） |
| P2 | act 档（仅当引入带校准置信度的 provider——见附录 A）：allowlist + 爆炸半径上限 + 审计 |
| — | 自洽性伪置信度（k 次采样一致率）作为 act 档的非 Jev 替代——5 倍成本，只在关键判断用 |