Skip to content

[Feature]: Add support for custom crawl headers #2108

Open
@tw4l

Description

@tw4l

What change would you like to see?

Requested on IIPC Slack:

"We need the option to set a request header name and value in the configuration. It could be e.g. cookie: Zxcv1234 and it should then only be set in the request headers if domain name or URL is specified else set on all URLs request.
Header Name = cookie
Header Value = Zxcv1234"

Context

From IIPC Slack: "What we are missing is the option to manually add multiple custom headers and cookies to a crawl configuration, it could be the same place as user-agent is now. Some hosts offer that as the only access to their content behind a paywall."

This would require adding support for custom headers to the crawler as well as the Browsertrix workflow editor.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementRequests a change to a feature

    Type

    No type

    Projects

    Status

    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions