One question, why setting it in response context instead of request, which seems a more proper place to store this information. The response context here is mainly for modifying the properties of the response to be returned.

@frostming thats a good point and when I saw that the response context is mainly for modifying the properties of the response to be returned I also questioned that.

My initial thought was that when I call the submit_task endpoint, then I get the task_id in the response. And from the documentation I understood that the context allows you to access information about the incoming request and it's response. And since the task_id came with the response to my request, I naturally first looked into the response context in my inference function, but didn't find it there and therefore thought about adding it there.

I can edit it to add the task_id to request.state which based on the starlette documentation is there to add additional information, would that be better?

Also _run_task is called after the response is sent, so as background task, so I wonder how you can access this property.

Another point I didn't mention is the whole task implementation is local only, it works when you serve the bento without your own infras. And for bento cloud we have another implementation for tasks.

Also _run_task is called after the response is sent, so as background task, so I wonder how you can access this property.

Well I guess then the actual api function which gets the context is called after the response was sent.
Within _run_task the function api_endpoint_wrapper is called which calls the function api_endpoint. Within those two functions the other context variables are set and I guess the actual api function is called. So by accessing it in _run_task I am accessing it even earlier than the main code as far as I understood it.

Another point I didn't mention is the whole task implementation is local only, it works when you serve the bento without your own infras. And for bento cloud we have another implementation for tasks.

I guess it would still be beneficial to cover this aspect for local implementations. At least I don't see any drawbacks in it.