Skip to content

feat: AutoscaledPool implementation#55

Merged
janbuchar merged 24 commits intomasterfrom
autoscaled-pool
Feb 27, 2024
Merged

feat: AutoscaledPool implementation#55
janbuchar merged 24 commits intomasterfrom
autoscaled-pool

Conversation

@janbuchar
Copy link
Copy Markdown
Collaborator

@janbuchar janbuchar commented Feb 20, 2024

The current concurrency model is "launch desired_concurrency worker tasks that execute the task function in a loop". Maybe we could achieve simpler and slightly more efficient code if each execution of the task function had its own asyncio task, similarly to the original implementation. Follow-up tasks would be started in the done_callback, or in a recurring task (both of these would be needed).

@github-actions github-actions bot added this to the 83rd sprint - Tooling team milestone Feb 20, 2024
@github-actions github-actions bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Feb 20, 2024
@janbuchar janbuchar marked this pull request as draft February 20, 2024 14:08
@github-actions github-actions bot added the tested Temporary label used only programatically for some analytics. label Feb 21, 2024
@janbuchar janbuchar requested a review from vdusek February 22, 2024 14:54
@janbuchar janbuchar marked this pull request as ready for review February 22, 2024 14:54
Copy link
Copy Markdown
Collaborator

@vdusek vdusek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just comments regarding the measure time util.

Copy link
Copy Markdown
Collaborator

@vdusek vdusek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we will have a different opinion on how much we should document the code 🙂. However, I'd be glad if you could add at least some comments explaining what's going on in main functions like run(), autoscale, and worker. So that it's easier to understand it when we come back to this module. In a similar way as it is in the original TS implementation. Also, I'd like the class docstring to be extended to a similar length as it is in the TS implementation. Again, I believe you can copy the doc from there and just adjust it.

@vdusek
Copy link
Copy Markdown
Collaborator

vdusek commented Feb 24, 2024

At first, good job 👏. I went through all of it, some parts were harder to understand, so I'd be glad if we could have more comments, as I wrote earlier, otherwise, it's great.

To your question, let's discuss it on Slack...

@janbuchar janbuchar requested a review from vdusek February 27, 2024 09:35
@janbuchar
Copy link
Copy Markdown
Collaborator Author

Maybe we will have a different opinion on how much we should document the code 🙂. However, I'd be glad if you could add at least some comments explaining what's going on in main functions like run(), autoscale, and worker.

I filled in what seemed non-obvious. Please do point out if something is still unclear.

Also, I'd like the class docstring to be extended to a similar length as it is in the TS implementation. Again, I believe you can copy the doc from there and just adjust it.

Honestly, the TS implementation seems overdocumented to me (can't believe that I said that 😄). It seems like a very internal thing and as such it should not need things like usage examples. Also, copying documentation between projects just feels wrong.

Copy link
Copy Markdown
Collaborator

@vdusek vdusek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! basically just formatting things, otherwise it's great

Copy link
Copy Markdown
Collaborator

@vdusek vdusek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@janbuchar janbuchar merged commit 621ada2 into master Feb 27, 2024
@janbuchar janbuchar deleted the autoscaled-pool branch February 27, 2024 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement AutoscaledPool

2 participants