-
Notifications
You must be signed in to change notification settings - Fork 169
experimental/API added roi parameter to pixel classification pipeline #2396
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
| self._predict_op.LabelsCount.setValue(classifer.label_count) | ||
|
|
||
| def predict(self, data): | ||
| def predict(self, data, xarray_roi=dict()): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
def predict(self, data, xarray_roi=None):
if xarray_roi is None:
xarray_roi = {}There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is the objection against the default value?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because default values for function arguments are evaluated and assigned on function definition, not evaluation. This doesn't play well with mutable objects: https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
crap it does...gee, what a mistake...
|
|
||
|
|
||
| def xarray_roi_to_slicing(xarray_roi, vigra_axistags): | ||
| if any(k not in vigra_axistags for k in xarray_roi.keys()): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| if any(k not in vigra_axistags for k in xarray_roi.keys()): | |
| if set(xarray_roi).difference(vigra_axistags): |
| out_slicing = [] | ||
| for axis in vigra_axistags: | ||
| if axis in xarray_roi: | ||
| out_slicing.append(xarray_roi[axis]) | ||
| else: | ||
| out_slicing.append(slice(None, None, None)) | ||
| return tuple(out_slicing) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| out_slicing = [] | |
| for axis in vigra_axistags: | |
| if axis in xarray_roi: | |
| out_slicing.append(xarray_roi[axis]) | |
| else: | |
| out_slicing.append(slice(None, None, None)) | |
| return tuple(out_slicing) | |
| return tuple(xarray_roi.get(axis, slice(None)) for axis in vigra_axistags) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cheers, I was actually hoping to get this from xarrray and didn't look into their docs yet... :) But I'll use your suggestion if there is nothing to be found.
| def xarray_roi_to_slicing(xarray_roi, vigra_axistags): | ||
| if any(k not in vigra_axistags for k in xarray_roi.keys()): | ||
| raise ValueError(f"xarray_roi contains keys ({xarray_roi.keys()} not in axistags ({vigra_axistags})") | ||
| if not xarray_roi: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need this special case? That is, does a[...] == a[:, :, :, :, :] always hold in volumina?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it depends on the data, unfortunately. I wanted to send a clear message for what happens, when no roi is provided.
|
Some objections against dictionaries are:
|
|
Another idea to return intermediate objects when we call predict and only materialize it on direct query. block_prediction = pipeline.predict(ds).isel(x=slice(0,10), y=slice(0, 17)).result()
# this kind of block interaction is nicer to read imo
predictor = pipeline.Predictor(ds)
computed = []
for block in tile(ds.shape, 256):
tile_prediction = predictor.isel(**block).result()
computed.append(tile_prediction)it would also play nicer if we have multi-value returns |
I agree, there are some nice pro's tho:
What I liked about the first iteration of the api is that was a single call to get a prediction. That is staying with the original |
this is only half-true, though. def __init__(self, x: Tuple[int, int], y: Tuple[int, int], z: Tuple[int, int], c: Tuple[int, int], t: Tuple[int, int])which comes with the added boni that the second version can have explicit default values and does not silently accept bad keys
True that, but basic python functionality will still work. This produces no errors at the call site: predict(roi={"X": (100, 200), "y": (300, 400)})While this: predict(roi=CustomRoi(X=(100, 200), y=(300,400)))immediately throws a "TypeError: Got an unexpected keyword argument: X" , right at the call site.
I guess that could also work; the syntax is indeed more ergonomic at the cost of runtime validation. But if ever we need to add some validation in the constructor or add some methods to the rois (enlarge, split, get_tiles, etc), we're back to a custom Roi class |
|
How about we hide that from users and offer mypy compatibility: RoiDict = TypedDict('RoiDict', {"x": ..., "y", ....}, total=False)
class Classifier:
...
def predict(roi: RoiDict):
croi = CustomRoi(**roi)Wouldn't this be way nicer for users? |
In order to be more useful for larger data, this PR adds a
roiparameter to thepredictmethod of the pixel classification api.Right now I went for
xarraydict-like specification, which looks very sensible.Assumption: Our pipelines will always handle halo's internally (given they have access to the raw data).
Example:
cc @m-novikov @emilmelnikov @Tomaz-Vieira