You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would like to be able to toggle on/off crawler behaviors when doing a crawl so that I can modify what gets crawled.
Context
I was doing a crawl of a website and it pulled in a lot of youtube content that put me over my storage quota. It was great that the youtube crawling worked, but I wasn't able to get a good snapshot of the entire website. It would have been useful if there were check boxes for other behaviors similar to the auto-scroll behavior. This would have allowed be to turn off auto-play which hopefully would have given me a complete snapshot of the website (minus the youtube content).
The text was updated successfully, but these errors were encountered:
What change would you like to see?
I would like to be able to toggle on/off crawler behaviors when doing a crawl so that I can modify what gets crawled.
Context
I was doing a crawl of a website and it pulled in a lot of youtube content that put me over my storage quota. It was great that the youtube crawling worked, but I wasn't able to get a good snapshot of the entire website. It would have been useful if there were check boxes for other behaviors similar to the auto-scroll behavior. This would have allowed be to turn off
auto-play
which hopefully would have given me a complete snapshot of the website (minus the youtube content).The text was updated successfully, but these errors were encountered: