Windowed auto-ranging experiment #14330

bryevdv · 2025-02-15T02:58:43Z

bryevdv
Feb 15, 2025
Maintainer

It occurred to me that now that most (all?) glyphs use the spatial index that perhaps we could just leverage that for a windowed auto-range. Maybe it would not be perfect but possibly it could be good enough. So my first experiment was just to try and pass a Rect with x coords matching the current range, and unbounded y coords:

diff --git a/bokehjs/src/lib/core/util/bbox.ts b/bokehjs/src/lib/core/util/bbox.ts
index d63cb52c4d..190a3b6d96 100644
--- a/bokehjs/src/lib/core/util/bbox.ts
+++ b/bokehjs/src/lib/core/util/bbox.ts
@@ -36,6 +36,15 @@ export function positive_y(): Rect {
   }
 }

+export function for_x_range(x0: number, x1: number): Rect {
+  return {
+    x0,
+    y0: -Infinity,
+    x1,
+    y1: Infinity,
+  }
+}
+
 function _min(a: number, b: number): number {
   if (isNaN(a)) {
     return b
diff --git a/bokehjs/src/lib/models/glyphs/glyph.ts b/bokehjs/src/lib/models/glyphs/glyph.ts
index 49297385d7..f8c484e107 100644
--- a/bokehjs/src/lib/models/glyphs/glyph.ts
+++ b/bokehjs/src/lib/models/glyphs/glyph.ts
@@ -149,7 +149,14 @@ export abstract class GlyphView extends DOMComponentView {
   }

   bounds(): Rect {
-    return this._bounds(this.index.bbox)
+    const {x_range_name} = this.renderer.model
+    const x_range = this.renderer.plot_view.frame.x_ranges.get(x_range_name)!
+    if (isNaN(x_range.start) || isNaN(x_range.end)) {
+      return this._bounds(this.index.bbox)
+    }
+    const {x0, x1} = this.index.bbox
+    const {y0, y1} = this.index.bounds(bbox.for_x_range(x_range.start, x_range.end))
+    return this._bounds({x0, y0, x1, y1})
   }

   log_bounds(): Rect {

This actually sort of seems to work for a little while then clearly stops working after a short order (see example code below). @mattpap is there anything obviously wrong with this approach? It seems like it ought to keep things in the ballpark at least, but even that is not the case after ~10 seconds.

bryevdv · 2025-02-15T02:59:19Z

bryevdv
Feb 15, 2025
Maintainer Author

Simplified OHLC I was testing with

import numpy as np

from bokeh.driving import count
from bokeh.plotting import curdoc, figure, ColumnDataSource

np.random.seed(1)

source = ColumnDataSource(dict(time=[], average=[]))

p = figure(height=500, tools="", y_axis_location="right")
p.x_range.follow = "end"
p.x_range.follow_interval = 100
p.x_range.range_padding = 0

r = p.line(x='time', y='average', alpha=0.2, line_width=3, color='navy', source=source)
p.y_range.renderers = [r]

def _average(t):
    last_average = 100 if t==0 else source.data['average'][-1]
    returns = np.asarray(np.random.lognormal(0.001, 0.01, 1))
    average =  last_average * np.cumprod(returns)
    return average[0]

@count()
def update(t):
    source.stream(dict(time=[t], average=[_average(t)]), 2000)

curdoc().add_root(p)
curdoc().add_periodic_callback(update, 100)

6 replies

bryevdv Feb 17, 2025
Maintainer Author

Hrm I think this logic in SpatialIndex.bounds may be the issue:

      if (x0 >= rect.x0 && x0 < bounds.x0) {
        bounds.x0 = x0
      }
      if (x1 <= rect.x1 && x1 > bounds.x1) {
        bounds.x1 = x1
      }
      if (y0 >= rect.y0 && y0 < bounds.y0) {
        bounds.y0 = y0
      }
      if (y1 <= rect.y1 && y1 > bounds.y1) {
        bounds.y1 = y1
      }

It seem to me that e.g. the y bounds can get set based on the current index box, even if the index box is entirely outside the rect on the x-dimension.

bryevdv Feb 17, 2025
Maintainer Author

Adding this diff gets tantalizingly close but there are still very noticeable excursions after a while (the bounds now seem too small sometimes)

diff --git a/bokehjs/src/lib/core/util/spatial.ts b/bokehjs/src/lib/core/util/spatial.ts
index a6ed334e81..e7340c907c 100644
--- a/bokehjs/src/lib/core/util/spatial.ts
+++ b/bokehjs/src/lib/core/util/spatial.ts
@@ -139,6 +139,9 @@ export class SpatialIndex {
       const y0 = boxes[4*i + 1]
       const x1 = boxes[4*i + 2]
       const y1 = boxes[4*i + 3]
+      if (x0 > rect.x1 || x1 < rect.x0 || y0 > rect.y1 || y1 < rect.y0) {
+        continue
+      }
       if (x0 >= rect.x0 && x0 < bounds.x0) {
         bounds.x0 = x0
       }

bryevdv Feb 17, 2025
Maintainer Author

That it seems like this code in search_indices is also already supposed to guard against that (but is apparently not doing so) so maybe there is a deeper issue in search_indices

        if (maxX < nodeMinX || maxY < nodeMinY || minX > nodeMaxX || minY > nodeMaxY) {
          continue
        }

bryevdv Feb 17, 2025
Maintainer Author

@mattpap bounds is using the indices returned by search_indices to iterate over boxes, but I am pretty sure that is not correct. The indices returned by search_indices are analogous to Flatbush's search which are "indices of items intersecting or touching the given bounding box" i.e. the index provided when add is called. But the indices for boxes are nodeIndex which can be bigger than the number items AFAIK.

bryevdv Feb 17, 2025
Maintainer Author

This was definitely the bug, windowed auto-ranging actually works really well when this bug is fixed #14339

bryevdv · 2025-02-17T19:12:19Z

bryevdv
Feb 17, 2025
Maintainer Author

Just recording my thoughts here as they come. Things are on a path to working (hopefully) with the above example because streaming data triggers data-range updates. But another (maybe more common) case is where say a plot is panned in the x-direction, or a range-tool updates an x-range, without new data added. This currently would not trigger a range update anywhere, so a full solution here will need a way to make a data range for update in additional circumstances.

2 replies

bryevdv Feb 17, 2025
Maintainer Author

For example. here is an updated range_tool.py that ought to work with the top plot y-range updating as the range tool is used.

import numpy as np

from bokeh.layouts import column
from bokeh.models import ColumnDataSource, RangeTool
from bokeh.plotting import figure, show
from bokeh.sampledata.stocks import AAPL

dates = np.array(AAPL['date'], dtype=np.datetime64)
source = ColumnDataSource(data=dict(date=dates, close=AAPL['adj_close']))

p = figure(height=300, width=800, tools="xpan", toolbar_location=None,
           x_axis_type="datetime", x_axis_location="above",
           background_fill_color="#efefef", x_range=(dates[1500], dates[2500]))

p.line('date', 'close', source=source)
p.yaxis.axis_label = 'Price'

select = figure(title="Drag the middle and edges of the selection box to change the range above",
                height=130, width=800,
                x_axis_type="datetime", y_axis_type=None,
                tools="", toolbar_location=None, background_fill_color="#efefef")

range_tool = RangeTool(x_range=p.x_range, start_gesture="pan")
range_tool.overlay.fill_color = "navy"
range_tool.overlay.fill_alpha = 0.2

select.line('date', 'close', source=source)
select.ygrid.grid_line_color = None
select.add_tools(range_tool)

show(column(p, select))

bryevdv Feb 17, 2025
Maintainer Author

e.g. adding a something like this gets things kinda working, modulo the spatial index issues

p.x_range.js_on_change("start", CustomJS(args=dict(source=source), code="""
    source.change.emit()
"""))

Will need to figure out a good API and mechanism to handle that internally rather than this hack.

Uh oh!

Windowed auto-ranging experiment #14330

Uh oh!

bryevdv Feb 15, 2025 Maintainer

Replies: 2 comments · 8 replies

Uh oh!

bryevdv Feb 15, 2025 Maintainer Author

Uh oh!

Uh oh!

bryevdv Feb 17, 2025 Maintainer Author

Uh oh!

bryevdv Feb 17, 2025 Maintainer Author

Uh oh!

bryevdv Feb 17, 2025 Maintainer Author

Uh oh!

bryevdv Feb 17, 2025 Maintainer Author

Uh oh!

Uh oh!

bryevdv Feb 17, 2025 Maintainer Author

Uh oh!

Uh oh!

bryevdv Feb 17, 2025 Maintainer Author

Uh oh!

bryevdv Feb 17, 2025 Maintainer Author

Uh oh!

Uh oh!

bryevdv Feb 17, 2025 Maintainer Author

bryevdv
Feb 15, 2025
Maintainer

Replies: 2 comments 8 replies

bryevdv
Feb 15, 2025
Maintainer Author

bryevdv Feb 17, 2025
Maintainer Author

bryevdv Feb 17, 2025
Maintainer Author

bryevdv Feb 17, 2025
Maintainer Author

bryevdv Feb 17, 2025
Maintainer Author

bryevdv Feb 17, 2025
Maintainer Author

bryevdv
Feb 17, 2025
Maintainer Author

bryevdv Feb 17, 2025
Maintainer Author

bryevdv Feb 17, 2025
Maintainer Author