-
Notifications
You must be signed in to change notification settings - Fork 593
Description
Problem
In lib/response_analyzer.sh, the heuristic exit condition triggers at a very low confidence score:
```bash
line ~641
if [[ $confidence_score -ge 40 || "$has_completion_signal" == "true" ]]; then
exit_signal=true
fi
```
A score of 40 can be reached from just 4 keyword matches (+10 each for "done", "complete", etc.). This means:
- Writing a CHANGELOG entry ("implementation complete")
- Updating a README ("setup is done")
- Any documentation mentioning completion words
...can all trigger premature exit mid-task.
Impact
Ralph exits before the task is actually finished whenever Claude writes documentation or comments containing common completion words. This is a silent failure — the loop ends, no error is shown, but work is incomplete.
Suggested Fix
Raise the threshold significantly and require explicit EXIT_SIGNAL when in JSON mode (which is the default):
```bash
For JSON output mode (default), require explicit EXIT_SIGNAL
if [[ "$CLAUDE_OUTPUT_FORMAT" == "json" ]]; then
# JSON mode: only exit on explicit EXIT_SIGNAL from RALPH_STATUS block
# Never use heuristic confidence scoring — Claude controls exit explicitly
: # exit_signal already set from JSON parse above
else
# Text mode fallback: raise threshold to 70 (was 40)
if [[ $confidence_score -ge 70 && "$has_completion_signal" == "true" ]]; then
exit_signal=true
fi
fi
```
This respects the existing JSON mode design intent (Claude sends explicit EXIT_SIGNAL) and makes text mode heuristics less fragile.