Skip to content

autoscaled_pool.desired_concurrency_ratio is useless #759

@Pijukatel

Description

@Pijukatel

It seems to me that autoscaled_pool.desired_concurrency_ratio is currently completely useless.

It is in init here:
https://github.com/apify/crawlee-python/blob/master/src/crawlee/_autoscaling/autoscaled_pool.py#L57

It is checked for bounds 0> desired_concurrency_ratio >=1
https://github.com/apify/crawlee-python/blob/master/src/crawlee/_autoscaling/autoscaled_pool.py#L97

It is used in only one place:
https://github.com/apify/crawlee-python/blob/master/src/crawlee/_autoscaling/autoscaled_pool.py#L198

And from there in condition here:
https://github.com/apify/crawlee-python/blob/master/src/crawlee/_autoscaling/autoscaled_pool.py#L202

That condition will always be true for currently runtime enforced values 0> desired_concurrency_ratio >=1:

self.current_concurrency >= math.floor(self._desired_concurrency_ratio * self.current_concurrency)

Did I miss something?

Metadata

Metadata

Assignees

Labels

bugSomething isn't working.t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions