Skip to content

Streaming API refinements #223

@goodboy

Description

@goodboy

As a follow up to the core work done for #53 in #219 (originally #209) and the sidekick debugger improvements in #220 this is the list of outstandings and would like to haves that deserve at least further discussion and possibly implementation for next release.


API and internals related:

  • possibly a better internal design for tracking bidir vs unidir context usage to avoid hacky if not self._ctx._portal checking inside ReceiveMsgStream.aclose()

  • move to a split SendMsgStream / ReceiveMsgStream type set and staple them together using a channel/messaging equivalent of trio.StapledStream?

    • one approach would be to make all streams a MsgStream and just don't allow send on receive only (which should be minority use case i'd imagine eventually).
    • currently this would allow making the stream arg to @stream funcs a MsgStream instead of a Context and we shouldn't have to do anything on the caller side other then prevent .send() from existing or going with the split types approacah.
    • turns out anyio has something similar: StapledObjectStream (obvs we won't use *Object* since we're msgpack type contrained). Actual code is here.
  • from Ems to bidir streaming pikers/piker#190

    would be nice if in tractor we can require either a ctx arg, or a named arg with ctx in it and a type annotation of tractor.Context instead of strictly requiring a ctx arg.


docs and tutorials

we obviously need extensive docs and a full tutorial on both receive only and 2 way streaming.
This is something to work out in detail as part of #175. Content basis will likely be from our test set.


theory / design-y questions:

The main question was moreso about cancellation race conditions that can arise where the local channel is killed after it's sent the stop and whether or not we should wait / shield the mem chan until the msg is processed later (also presumably this is all before the sub-actor is killed).

  • this really has to do with whether or not we want a channel teardown transaction eventually. I personally think right now it's not a requirement (and makes facing 2-general's more immediate). the more sane thing to address first is the naive discovery issues we have as per Multi-root discovery: pragmatic, simple consensus. #216.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions