…GRESS) (#326)

* add propose + plan for index output rework (gating spike passed)

Propose (propose/active/INDEX-OUTPUT-REWORK-PROPOSE.md), plan
(plans/active/PLAN-INDEX-OUTPUT-REWORK.md), and per-PR agent prompts
(plans/active/AGENT-PROMPTS-INDEX-OUTPUT-REWORK.md) for unifying the
index-build output across init/increment/install/reprocess/update and
adding a real progress bar (rich + JCIRAG_PROGRESS protocol).

Gating spike passed 2026-06-14: a print(file=sys.stderr) inside a
CocoIndex flow function reaches the parent's captured stderr (130/130
lines relayed; pre-walk divergence 0 on the fixture). PR-1 unblocked.

Planning only; no production code changed.

Co-Authored-By: Claude <[email protected]>

* add rich + progress protocol/renderer skeleton

rich>=14,<15 (cocoindex[lancedb] requires rich>=14). New
java_codebase_rag/progress.py: ProgressEvent, parse_progress_line,
IndexProgressRenderer (rich Live + non-TTY fallback), ProgressRelay
(parse-first, single-writer). 13 light unit tests. No production
caller yet; PR-2+ wire it in.

Co-Authored-By: Claude <[email protected]>

* correct rich version pin to >=14,<15

cocoindex[lancedb]>=1.0.0a43 transitively requires rich>=14, so the
originally-suggested rich>=13.7,<14 cap is unsatisfiable (pip install -e .
fails). Corrected in propose Open Q5 + risk row, plan Resolved decisions +
PR-1 step + DoD, and the PR-1 agent prompt. Verified compatible on rich
14.3.4.

Co-Authored-By: Claude <[email protected]>

* address PR-1 review: port _suppress_next, two-way clamp, monotonic done, carry-forward totals

- ProgressRelay now mirrors _LineFilter's noise-continuation suppression
  (_suppress_next) + new test_progress_relay_suppresses_noise_continuation
- remove dead _is_partial param from _route_line
- clamp completed→total on status=done both directions (over-count safe)
- monotonic completed (max with current) so a stray smaller done can't rewind
- non-TTY done concise line carries forward last-seen total/elapsed_s

Minor #6 (apply/stop atomicity) deferred to PR-2 (drain-thread wiring).

Co-Authored-By: Claude <[email protected]>

* note PR-2 follow-up: IndexProgressRenderer apply/stop atomicity

PR-1 code-quality review Minor #6: apply() issues several locked rich
calls not atomic vs. a concurrent stop() from the main thread. Only
matters once PR-2 wires the drain thread. Added as cross-PR risk #10
with the PR-2 mitigation (gate apply on _started, or hold the Live
lock across the sequence).

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>

#327)

* wire graph-phase index progress (count-first pass1 + pass steps)

build_ast_graph emits JCIRAG_PROGRESS kind=graph under --verbose
(count-first exact total in pass 1; pass 2-6 step lines). The sync
and async subprocess drains route progress events to an on_progress
callback via ProgressRelay (parse-first, single-writer). init/increment/
reprocess render a determinate graph-phase bar in default TTY mode
(running only after the builder spawns); --quiet silent; --verbose raw.
Renderer.apply() is a no-op after stop() (drain-thread safety).

Co-Authored-By: Claude <[email protected]>

* harden PR-2: survive renderer exceptions, exact incremental total, relay factory

- ProgressRelay guards renderer.apply() so a render-chain exception
  can't silently kill the drain thread (try/except + stderr note)
- incremental pass-1 total excludes removed files (done no longer
  undercounts then clamps)
- make_relay() factory centralizes the sync+async drain wiring
- drop unreachable console fallback in ProgressRelay._route_line

erase unused-renderer (review Minor #3) and failing-builder tests
(Minor #6) deferred.

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>

…/3/4) (#330)

* add propose + plan for index output rework (gating spike passed)

Propose (propose/active/INDEX-OUTPUT-REWORK-PROPOSE.md), plan
(plans/active/PLAN-INDEX-OUTPUT-REWORK.md), and per-PR agent prompts
(plans/active/AGENT-PROMPTS-INDEX-OUTPUT-REWORK.md) for unifying the
index-build output across init/increment/install/reprocess/update and
adding a real progress bar (rich + JCIRAG_PROGRESS protocol).

Gating spike passed 2026-06-14: a print(file=sys.stderr) inside a
CocoIndex flow function reaches the parent's captured stderr (130/130
lines relayed; pre-walk divergence 0 on the fixture). PR-1 unblocked.

Planning only; no production code changed.

Co-Authored-By: Claude <[email protected]>

* add rich + progress protocol/renderer skeleton

rich>=14,<15 (cocoindex[lancedb] requires rich>=14). New
java_codebase_rag/progress.py: ProgressEvent, parse_progress_line,
IndexProgressRenderer (rich Live + non-TTY fallback), ProgressRelay
(parse-first, single-writer). 13 light unit tests. No production
caller yet; PR-2+ wire it in.

Co-Authored-By: Claude <[email protected]>

* correct rich version pin to >=14,<15

cocoindex[lancedb]>=1.0.0a43 transitively requires rich>=14, so the
originally-suggested rich>=13.7,<14 cap is unsatisfiable (pip install -e .
fails). Corrected in propose Open Q5 + risk row, plan Resolved decisions +
PR-1 step + DoD, and the PR-1 agent prompt. Verified compatible on rich
14.3.4.

Co-Authored-By: Claude <[email protected]>

* address PR-1 review: port _suppress_next, two-way clamp, monotonic done, carry-forward totals

- ProgressRelay now mirrors _LineFilter's noise-continuation suppression
  (_suppress_next) + new test_progress_relay_suppresses_noise_continuation
- remove dead _is_partial param from _route_line
- clamp completed→total on status=done both directions (over-count safe)
- monotonic completed (max with current) so a stray smaller done can't rewind
- non-TTY done concise line carries forward last-seen total/elapsed_s

Minor #6 (apply/stop atomicity) deferred to PR-2 (drain-thread wiring).

Co-Authored-By: Claude <[email protected]>

* note PR-2 follow-up: IndexProgressRenderer apply/stop atomicity

PR-1 code-quality review Minor #6: apply() issues several locked rich
calls not atomic vs. a concurrent stop() from the main thread. Only
matters once PR-2 wires the drain thread. Added as cross-PR risk #10
with the PR-2 mitigation (gate apply on _started, or hold the Live
lock across the sequence).

Co-Authored-By: Claude <[email protected]>

* wire graph-phase index progress (count-first pass1 + pass steps)

build_ast_graph emits JCIRAG_PROGRESS kind=graph under --verbose
(count-first exact total in pass 1; pass 2-6 step lines). The sync
and async subprocess drains route progress events to an on_progress
callback via ProgressRelay (parse-first, single-writer). init/increment/
reprocess render a determinate graph-phase bar in default TTY mode
(running only after the builder spawns); --quiet silent; --verbose raw.
Renderer.apply() is a no-op after stop() (drain-thread safety).

Co-Authored-By: Claude <[email protected]>

* harden PR-2: survive renderer exceptions, exact incremental total, relay factory

- ProgressRelay guards renderer.apply() so a render-chain exception
  can't silently kill the drain thread (try/except + stderr note)
- incremental pass-1 total excludes removed files (done no longer
  undercounts then clamps)
- make_relay() factory centralizes the sync+async drain wiring
- drop unreachable console fallback in ProgressRelay._route_line

erase unused-renderer (review Minor #3) and failing-builder tests
(Minor #6) deferred.

Co-Authored-By: Claude <[email protected]>

* wire vectors-phase index progress; retire Spinner/emit_vectors; optimize phase

process_*_file emit JCIRAG_PROGRESS kind=vectors (approximate total from
app_main pre-walk + throttled per-file ticks); the parent emits the
terminal vectors event on cocoindex exit (drives clamp-on-completion).
optimize_lance_tables emits kind=optimize in-process (covers both call
sites). server.run_refresh_pipeline wires the async drain + renderer.
Retire Spinner + emit_vectors_start/_finish. Operator commands render
Vectors -> Optimize -> Graph.

Co-Authored-By: Claude <[email protected]>

* fix PR-3: count application YAML in pre-walk; tighten divergence test

_approximate_vectors_total's YAML predicate ("/application" in fn)
was always False for a bare filename, so the pre-walk under-counted
application YAML and the divergence went negative (gap=-4). Use
fn.startswith("application"). Move the vectors tick emission under
the lock to match its comment. Drop a dead type-ignore. Tighten the
divergence test to assert gap==0 on the fixture.

Co-Authored-By: Claude <[email protected]>

* align install/update index progress; wire --quiet/--verbose

install/update render the unified Vectors->Optimize->Graph progress on
stderr during their indexing sub-step (no longer stdout chatter / silent
quiet=True); wizard conversational stdout is preserved. _cmd_update
forwards --quiet/--verbose (both ignored today); _cmd_install forwards
--verbose. CLI docs + README updated.

Co-Authored-By: Claude <[email protected]>

* fix PR-4: best-effort graph catch-up in update; truthful failure footer

run_update's graph catch-up is best-effort again (graph failure -> exit 0
+ Warning, not exit 1) — matches the original intent and the output/UX-
only scope. Both installer brackets catch BaseException to set the failed
footer marker before re-raising. Footer uses perf_counter like the
operator footer.

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>