Skip to content

feat: Replace HeaderGenerator implementation by browserforge implementation#960

Merged
vdusek merged 41 commits intomasterfrom
remove-header-generator
Feb 25, 2025
Merged

feat: Replace HeaderGenerator implementation by browserforge implementation#960
vdusek merged 41 commits intomasterfrom
remove-header-generator

Conversation

@Pijukatel
Copy link
Copy Markdown
Collaborator

Description

Replaces internal HeaderGenerator implementation by BrowserForge implementation.

Issues

Pijukatel and others added 30 commits December 16, 2024 14:50
TODO: Tests, JS page function for injecting fingerprint json.
Todo: Make it work with page.add_init_script
Added dev test
TODO:
Add proper tests
Sync header generation with fingerprint generation (use gen from browserforge)
Added more tests.
Use browser_pool_options from crawler to pass fingerprint related stuff to be similar to JS
Draft adapter and basic test.
TODO: Solve circular imports
Remove types unused in this change.
Format, lint, type check.
(Hint about browserforge being just implementation detail and not core functionality.)
Update docstrings.
Add example code + doc page.
(To avoid some import race conditions downloads when running pytest with multiple processes)
@github-actions github-actions bot added this to the 107th sprint - Tooling team milestone Feb 6, 2025
@github-actions github-actions bot added t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics. labels Feb 6, 2025
@Pijukatel Pijukatel changed the title Replace internal header generator by browserforge implementation feat: Replace HeaderGenerator implementation by browserforge implementation Feb 6, 2025
@Pijukatel Pijukatel marked this pull request as ready for review February 6, 2025 09:06
@Pijukatel Pijukatel requested review from janbuchar and vdusek February 6, 2025 09:06
@Pijukatel Pijukatel added the enhancement New feature or request. label Feb 6, 2025
@vdusek vdusek requested a review from Mantisus February 19, 2025 07:54
Copy link
Copy Markdown
Collaborator

@Mantisus Mantisus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Pijukatel Pijukatel requested a review from vdusek February 25, 2025 12:11
Copy link
Copy Markdown
Collaborator

@vdusek vdusek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vdusek vdusek merged commit c2f8c93 into master Feb 25, 2025
23 checks passed
@vdusek vdusek deleted the remove-header-generator branch February 25, 2025 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace HeaderGenerator by browserforge implementation

4 participants