Ask before switching while sessions are live, and recover the ones already broken
Acts on docs/incident-mode-switch-corrupts-live-sessions.md, which is added here as the record of why. The report identifies a consequence that was not modelled. A failed call is recoverable; a *successful* one may not be. If a running session takes even one completion from the provider being switched to - which happens when that mode matches the base URL it already had cached - that provider's message-id format lands in its transcript. OpenRouter issues `gen-<epoch>-<rand>` where Anthropic issues `msg_...`, and native Anthropic then refuses to resume the session at all, with a 400 naming previous_message_id. The only way back is to truncate the transcript, losing every turn after the cut. That happened here, and was fixed by hand. Two changes follow. Sessions are now settled before the write, not reported after it. A switch with anything running stops, names the sessions, explains what is about to happen to them, and offers restart (the only answer that ends with everything on the mode the bar now claims), close, proceed anyway, or abort - defaulting to abort. Non-interactively it refuses outright unless given --yes. The old after-the-fact reporter is deleted rather than left as a second, contradictory account. `claude-mode repair-session` replaces the hand surgery: it finds a project's transcripts, reports which are resumable, and on --apply backs the file up and truncates to the last Anthropic-issued message. Verified against the real corrupted transcript from the incident - it reproduces the manual cut exactly, 1921 lines to 1813, dropping the two `gen-` completions and the error placeholders after them, leaving a transcript that ends on a genuine msg_ id. It refuses a transcript written to in the last 90 seconds, since that one belongs to a session still running. The panel passes --yes, having already asked in its own card, and that card now names the transcript risk rather than only the inconvenient one. Requirement 4 of the report - documenting the mechanism - landed in 9c301e1; the README now carries the unrecoverable half as well.
This commit is contained in:
smoido
+205
-22
@@ -278,7 +278,9 @@ claude-mode - switch Claude Code between Anthropic, OpenRouter, Z.AI, LM Studio
|
||||
claude-mode preflight <mode> [preset] check a mode can actually serve, without switching
|
||||
claude-mode sessions [--stop|--restart]
|
||||
running sessions; close or reopen them
|
||||
claude-mode repair-session [id] make a session resumable again after a bad switch
|
||||
claude-mode <mode> --force switch even if preflight says no
|
||||
claude-mode <mode> --yes switch without asking about running sessions
|
||||
EOF
|
||||
}
|
||||
|
||||
@@ -818,26 +820,6 @@ EOF_ROWS
|
||||
return 0
|
||||
}
|
||||
|
||||
# Named after the switch, not before it: the switch has already happened, and
|
||||
# these are the sessions it did not reach.
|
||||
cm_report_live_sessions() {
|
||||
local rows n busy
|
||||
rows="$(cm_session_rows)"
|
||||
n="$(printf '%s' "$rows" | grep -c . || true)"
|
||||
[ "${n:-0}" -gt 0 ] || return 0
|
||||
|
||||
busy="$(printf '%s' "$rows" | cut -f4 | grep -c '^yes$' || true)"
|
||||
printf '\n'
|
||||
warn "$n Claude Code session(s) are running and will start failing their calls"
|
||||
say 'their key is re-fetched on a timer and now resolves to the new mode, which'
|
||||
say 'the endpoint they are still pointed at will not accept. Restart them.'
|
||||
if [ "${busy:-0}" -gt 0 ]; then
|
||||
warn "$busy of them is mid-request and will break wherever it happens to be"
|
||||
fi
|
||||
printf ' %sclaude-mode sessions what is running%s\n' "$C_DIM" "$C_RESET"
|
||||
printf ' %sclaude-mode sessions --restart close and reopen them on the new provider%s\n' "$C_DIM" "$C_RESET"
|
||||
}
|
||||
|
||||
cm_terminal_cmd() {
|
||||
local t
|
||||
for t in "${TERMINAL:-}" foot alacritty ghostty kitty; do
|
||||
@@ -898,6 +880,8 @@ set_mode() {
|
||||
[ -f "$preset_file" ] || { err "preset '$preset_name' not found"; return 1; }
|
||||
fi
|
||||
|
||||
cm_confirm_sessions "$mode" || return 1
|
||||
|
||||
backup="$(backup_settings)"
|
||||
|
||||
if ! "$PY" "$JSON" apply "$CM_SETTINGS" "$CM_STATE" "$mode" "$preset_file" "$CM_HELPER" >/dev/null; then
|
||||
@@ -945,7 +929,7 @@ set_mode() {
|
||||
check_stale_models "$mode"
|
||||
write_health "$mode" "$preset_name"
|
||||
|
||||
cm_report_live_sessions
|
||||
cm_apply_session_action
|
||||
|
||||
printf '\n %srestart claude (and reload the VS Code window) to pick this up%s\n' "$C_DIM" "$C_RESET"
|
||||
}
|
||||
@@ -2034,6 +2018,203 @@ cmd_setup() {
|
||||
return 0
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Live sessions: asked before the write, not reported after it
|
||||
#
|
||||
# The damage a switch does to a running session is not limited to it failing
|
||||
# calls. If the session takes even one completion from the new provider before
|
||||
# anything notices, that provider's message-id format lands in its transcript -
|
||||
# OpenRouter issues `gen-<epoch>-<rand>` where Anthropic issues `msg_...` - and
|
||||
# native Anthropic then refuses to resume the session at all:
|
||||
#
|
||||
# API Error: 400 diagnostics.previous_message_id: must be the `id` from a
|
||||
# prior /v1/messages response (starts with `msg_`)
|
||||
#
|
||||
# There is no supported way back from that. The only fix is to truncate the
|
||||
# transcript to the last message Anthropic issued, losing everything after it
|
||||
# (see `claude-mode repair-session`). A confirmation that costs one keystroke is
|
||||
# cheap against a failure that costs an afternoon of conversation.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
CM_ASSUME_YES=0
|
||||
CM_SESSION_ACTION=none
|
||||
CM_SESSION_ROWS=''
|
||||
|
||||
cm_confirm_sessions() {
|
||||
local mode="$1" rows n busy reply
|
||||
CM_SESSION_ACTION=none
|
||||
CM_SESSION_ROWS=''
|
||||
|
||||
rows="$(cm_session_rows)"
|
||||
n="$(printf '%s' "$rows" | grep -c . || true)"
|
||||
[ "${n:-0}" -gt 0 ] || return 0
|
||||
CM_SESSION_ROWS="$rows"
|
||||
|
||||
busy="$(printf '%s' "$rows" | cut -f4 | grep -c '^yes$' || true)"
|
||||
|
||||
printf '\n'
|
||||
warn "$n Claude Code session(s) are running right now"
|
||||
local pid ppid tty b cwd isself pcmd tag
|
||||
while IFS=$'\t' read -r pid ppid tty b cwd isself pcmd; do
|
||||
[ -n "$pid" ] || continue
|
||||
tag=''
|
||||
[ "$b" = yes ] && tag=" ${C_YELLOW}working${C_RESET}"
|
||||
[ "$isself" = yes ] && tag="$tag ${C_DIM}(this one)${C_RESET}"
|
||||
printf ' %-8s %-8s %s%s\n' "$pid" "$tty" "$cwd" "$tag"
|
||||
done <<EOF_ROWS
|
||||
$rows
|
||||
EOF_ROWS
|
||||
|
||||
printf '\n'
|
||||
say 'Their key is re-fetched on a timer and will resolve to the new mode,'
|
||||
say 'which the endpoint they are still pointed at will not accept. If one'
|
||||
say 'of them takes a reply from the new provider first, that provider'"'"'s'
|
||||
say 'message-id format goes into its transcript and Anthropic will then'
|
||||
say 'refuse to resume that session at all - recoverable only by truncating'
|
||||
say 'it (claude-mode repair-session), which loses the turns after the cut.'
|
||||
[ "${busy:-0}" -gt 0 ] && warn "$busy of them is mid-request and is the most likely to be caught"
|
||||
|
||||
if [ "$CM_ASSUME_YES" -eq 1 ]; then
|
||||
say 'proceeding (--yes)'
|
||||
return 0
|
||||
fi
|
||||
|
||||
if ! ui_interactive; then
|
||||
printf '\n'
|
||||
err 'refusing to switch while sessions are running'
|
||||
say 'restart or close them first, or pass --yes to switch anyway'
|
||||
return 1
|
||||
fi
|
||||
|
||||
printf '\n'
|
||||
say 'r switch, then close and reopen them on the new mode (safest)'
|
||||
say 'c switch, then close them'
|
||||
say 's switch and leave them running (risks the above)'
|
||||
say 'a abort'
|
||||
printf '\n [r/c/s/A] '
|
||||
IFS= read -r reply
|
||||
case "$reply" in
|
||||
r|R) CM_SESSION_ACTION=restart; return 0 ;;
|
||||
c|C) CM_SESSION_ACTION=stop; return 0 ;;
|
||||
s|S) CM_SESSION_ACTION=none; return 0 ;;
|
||||
*) say 'aborted; nothing was changed'; return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Run after the write, never before: a session reopened first would come back up
|
||||
# on the mode being left behind.
|
||||
cm_apply_session_action() {
|
||||
[ "$CM_SESSION_ACTION" = "none" ] && return 0
|
||||
[ -n "$CM_SESSION_ROWS" ] || return 0
|
||||
printf '\n'
|
||||
cm_session_act "$CM_SESSION_ACTION" "$CM_SESSION_ROWS" 0
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Transcript repair
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# ~/.claude/projects/<cwd with every slash turned into a dash>
|
||||
cm_project_dir() {
|
||||
printf '%s/projects/%s' "$CM_SETTINGS_DIR" "$(printf '%s' "${1:-$PWD}" | sed 's|/|-|g')"
|
||||
}
|
||||
|
||||
cmd_repair_session() {
|
||||
local target='' apply=0 dir='' a file verdict age
|
||||
for a in "$@"; do
|
||||
case "$a" in
|
||||
--apply) apply=1 ;;
|
||||
--dry-run) apply=0 ;;
|
||||
--list) target='--list' ;;
|
||||
-*) err "unknown option '$a'"; return 1 ;;
|
||||
*) target="$a" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Claude Code keys transcripts by the directory the session was started in,
|
||||
# which is rarely the one you are standing in when you come to fix it. Walk
|
||||
# up first, and for a named session fall back to looking through every
|
||||
# project - the id is unique, so there is nothing ambiguous to resolve.
|
||||
local probe="$PWD"
|
||||
while [ -n "$probe" ]; do
|
||||
[ -d "$(cm_project_dir "$probe")" ] && { dir="$(cm_project_dir "$probe")"; break; }
|
||||
[ "$probe" = "/" ] && break
|
||||
probe="$(dirname "$probe")"
|
||||
done
|
||||
|
||||
if [ -n "$target" ] && [ "$target" != "--list" ]; then
|
||||
if [ -z "$dir" ] || [ ! -f "$dir/${target%.jsonl}.jsonl" ]; then
|
||||
local hit
|
||||
hit="$(ls -1 "$CM_SETTINGS_DIR"/projects/*/"${target%.jsonl}".jsonl 2>/dev/null | head -n1)"
|
||||
[ -n "$hit" ] && dir="$(dirname "$hit")"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "$dir" ] || [ ! -d "$dir" ]; then
|
||||
err 'no session transcripts found for this directory'
|
||||
say 'run it from the project the session belongs to, or name the session id'
|
||||
return 1
|
||||
fi
|
||||
|
||||
if [ -z "$target" ] || [ "$target" = "--list" ]; then
|
||||
head_ 'session transcripts here'
|
||||
local f v
|
||||
for f in $(ls -1t "$dir"/*.jsonl 2>/dev/null); do
|
||||
v="$("$PY" "$JSON" repair-session "$f" 2>/dev/null)" || continue
|
||||
printf '%s' "$v" | "$PY" -c "
|
||||
import json,sys,os
|
||||
d=json.load(sys.stdin)
|
||||
state = 'ok' if d['healthy'] else ('repairable, would drop %d line(s)' % d['dropLines'] if d['repairable'] else 'no anthropic message found')
|
||||
print(' %-40s %s' % (os.path.basename(d['path'])[:-6], state))
|
||||
"
|
||||
done
|
||||
printf '\n %sclaude-mode repair-session <session-id> --apply%s\n' "$C_DIM" "$C_RESET"
|
||||
return 0
|
||||
fi
|
||||
|
||||
file="$dir/${target%.jsonl}.jsonl"
|
||||
[ -f "$file" ] || { err "no transcript $file"; return 1; }
|
||||
|
||||
# A transcript that is still being appended to belongs to a session that is
|
||||
# still alive; truncating it underneath a running process helps nobody.
|
||||
age=$(( $(date +%s) - $(stat -c %Y "$file" 2>/dev/null || echo 0) ))
|
||||
if [ "$age" -lt 90 ] && [ "$apply" -eq 1 ]; then
|
||||
err "that transcript was written to ${age}s ago - it looks live"
|
||||
say 'close the session that owns it first'
|
||||
return 1
|
||||
fi
|
||||
|
||||
verdict="$("$PY" "$JSON" repair-session "$file" $([ "$apply" -eq 1 ] && printf -- '--apply'))" || {
|
||||
err 'could not read that transcript'; return 1; }
|
||||
|
||||
printf '%s' "$verdict" | "$PY" -c "
|
||||
import json,sys
|
||||
d=json.load(sys.stdin)
|
||||
G,Y,R,D,X = '\033[32m','\033[33m','\033[91m','\033[90m','\033[0m'
|
||||
print()
|
||||
if d['healthy']:
|
||||
print(' %sok %s last message is Anthropic-issued; nothing to repair' % (G,X))
|
||||
raise SystemExit(0)
|
||||
if not d['repairable']:
|
||||
print(' %sFAIL%s no Anthropic-issued message anywhere in this transcript' % (R,X))
|
||||
raise SystemExit(1)
|
||||
for f in d['foreignIds']:
|
||||
print(' %swarn%s line %d carries a %s id from %s' % (Y,X,f['line'],f['id'].split('-')[0]+'-',f['model'] or 'another provider'))
|
||||
n = sum(1 for s in d['syntheticIds'] if s['apiError'] and s['line'] > d['lastGoodLine'])
|
||||
if n:
|
||||
print(' %swarn%s %d client-side error placeholder(s) after the last good message' % (Y,X,n))
|
||||
if d['applied']:
|
||||
print(' %sok %s truncated to line %d, dropping %d' % (G,X,d['lastGoodLine'],d['dropLines']))
|
||||
print(' %sok %s original saved as %s' % (G,X,d['backup']))
|
||||
print()
|
||||
print(' %sthat session should resume again; the dropped turns are gone%s' % (D,X))
|
||||
else:
|
||||
print(' %swarn%s would truncate to line %d, dropping %d line(s)' % (Y,X,d['lastGoodLine'],d['dropLines']))
|
||||
print()
|
||||
print(' %sre-run with --apply to do it (the original is backed up first)%s' % (D,X))
|
||||
"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Dispatch
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -2046,7 +2227,8 @@ init_root
|
||||
_args=()
|
||||
for _a in "$@"; do
|
||||
case "$_a" in
|
||||
--force) CM_FORCE=1 ;;
|
||||
--force) CM_FORCE=1 ;;
|
||||
-y|--yes) CM_ASSUME_YES=1 ;;
|
||||
*) _args+=("$_a") ;;
|
||||
esac
|
||||
done
|
||||
@@ -2109,6 +2291,7 @@ case "$cmd" in
|
||||
cmd_setup "$_mode" "$_rest"
|
||||
fi ;;
|
||||
sessions) cmd_sessions "$@" ;;
|
||||
repair-session) cmd_repair_session "$@" ;;
|
||||
repair)
|
||||
scope=''
|
||||
for a in "$@"; do [ "$a" = "--all" ] && scope=all; done
|
||||
|
||||
@@ -603,10 +603,107 @@ def cmd_set_all(argv):
|
||||
save(path, p)
|
||||
print(model)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Session transcript repair
|
||||
#
|
||||
# Native Anthropic requires previous_message_id to be an id it issued itself -
|
||||
# one starting `msg_`. A session that took even one completion from a gateway
|
||||
# while the mode was switched under it has that provider's id format in its
|
||||
# transcript instead (OpenRouter issues `gen-<epoch>-<rand>`), and every attempt
|
||||
# to resume it afterwards fails with a 400 naming previous_message_id. Client-
|
||||
# side error placeholders, written as model `<synthetic>` with a UUID for an id,
|
||||
# do the same thing when one is last.
|
||||
#
|
||||
# The transcript is newline-delimited JSON, one independent object per line, so
|
||||
# rolling back to the last message Anthropic actually issued is a truncation.
|
||||
# Everything after it is lost - which is the cost, and why nothing here runs
|
||||
# without being asked twice.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _msg_id(obj):
|
||||
m = obj.get("message")
|
||||
if isinstance(m, dict) and m.get("id"):
|
||||
return str(m["id"])
|
||||
return None
|
||||
|
||||
|
||||
def cmd_repair_session(argv):
|
||||
"""repair-session <transcript.jsonl> [--apply]
|
||||
|
||||
Prints a JSON verdict. With --apply, backs the file up and truncates it to
|
||||
the last Anthropic-issued message.
|
||||
"""
|
||||
path = argv[0]
|
||||
apply_it = "--apply" in argv[1:]
|
||||
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
lines = fh.read().splitlines()
|
||||
|
||||
last_good = -1 # index of the last line carrying a msg_ id
|
||||
foreign, synthetic = [], []
|
||||
for i, line in enumerate(lines):
|
||||
if not line.strip():
|
||||
continue
|
||||
try:
|
||||
obj = json.loads(line)
|
||||
except Exception:
|
||||
continue
|
||||
mid = _msg_id(obj)
|
||||
if mid is None:
|
||||
continue
|
||||
if mid.startswith("msg_"):
|
||||
last_good = i
|
||||
elif mid.startswith("gen-"):
|
||||
foreign.append({"line": i + 1, "id": mid,
|
||||
"model": str((obj.get("message") or {}).get("model", ""))})
|
||||
else:
|
||||
synthetic.append({"line": i + 1, "id": mid,
|
||||
"apiError": bool(obj.get("isApiErrorMessage"))})
|
||||
|
||||
# Only the *last* id matters for resuming: an error placeholder in the
|
||||
# middle of a long-finished turn is history, not a blocker.
|
||||
tail_ids = [i for i in range(len(lines) - 1, last_good, -1)
|
||||
if lines[i].strip() and _msg_id(_safe(lines[i])) is not None]
|
||||
healthy = (last_good >= 0 and not tail_ids)
|
||||
|
||||
out = {
|
||||
"path": path,
|
||||
"lines": len(lines),
|
||||
"lastGoodLine": last_good + 1 if last_good >= 0 else 0,
|
||||
"dropLines": 0 if healthy or last_good < 0 else len(lines) - (last_good + 1),
|
||||
"foreignIds": foreign,
|
||||
"syntheticIds": synthetic,
|
||||
"healthy": healthy,
|
||||
"repairable": (not healthy) and last_good >= 0,
|
||||
"applied": False,
|
||||
"backup": "",
|
||||
}
|
||||
|
||||
if apply_it and out["repairable"]:
|
||||
stamp = __import__("datetime").datetime.now().strftime("%Y%m%d-%H%M%S")
|
||||
backup = "%s.pre-repair-backup-%s" % (path, stamp)
|
||||
with open(backup, "w", encoding="utf-8") as fh:
|
||||
fh.write("\n".join(lines) + ("\n" if lines else ""))
|
||||
with open(path, "w", encoding="utf-8") as fh:
|
||||
fh.write("\n".join(lines[:last_good + 1]) + "\n")
|
||||
out["applied"] = True
|
||||
out["backup"] = backup
|
||||
|
||||
print(json.dumps(out))
|
||||
|
||||
|
||||
def _safe(line):
|
||||
try:
|
||||
return json.loads(line)
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
COMMANDS = {
|
||||
"health": cmd_health,
|
||||
"preflight-json": cmd_preflight_json,
|
||||
"sessions-json": cmd_sessions_json,
|
||||
"repair-session": cmd_repair_session,
|
||||
"stale-models": cmd_stale_models,
|
||||
"strip-tags": cmd_strip_tags,
|
||||
"or-models": cmd_or_models,
|
||||
|
||||
Reference in New Issue
Block a user