Skip to content

feat: blocking detection for playwright crawler#328

Merged
vdusek merged 2 commits intomasterfrom
playwright-blocking
Jul 19, 2024
Merged

feat: blocking detection for playwright crawler#328
vdusek merged 2 commits intomasterfrom
playwright-blocking

Conversation

@vdusek
Copy link
Copy Markdown
Collaborator

@vdusek vdusek commented Jul 19, 2024

Description

  • Add blocking detection for PlaywrightCrawler.
  • Improve documentation of the PlaywrightCrawler and related components.

Issues

Testing

Checklist

  • Changes are described in the CHANGELOG.md
  • CI passed

@vdusek vdusek requested a review from janbuchar July 19, 2024 11:19
@vdusek vdusek self-assigned this Jul 19, 2024
@github-actions github-actions bot added this to the 94th sprint - Tooling team milestone Jul 19, 2024
@github-actions github-actions bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Jul 19, 2024
Comment on lines +16 to +19
Args:
page: The Playwright `Page` object.
response: The Playwright `Response` object.
enqueue_links: The `PlaywrightCrawler` implementation of the `EnqueueLinksFunction` function.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this correctly rendered in the documentation?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, args in API docs currently do not work - #324.

@vdusek vdusek merged commit 49ff6e2 into master Jul 19, 2024
@vdusek vdusek deleted the playwright-blocking branch July 19, 2024 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check for selectors that indicate blocking in PlaywrightCrawler

2 participants