Last active
January 24, 2025 22:34
-
-
Save 0xdevalias/b34feb567bd50b37161293694066dd53 to your computer and use it in GitHub Desktop.
Revisions
-
0xdevalias revised this gist
May 3, 2024 . 1 changed file with 10 additions and 6 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -39,9 +39,13 @@ Some notes/resources for bypassing anti-bot/scraping features on Cloudflare, Aka ### My Other Related Deepdive Gist's and Projects - https://github.com/0xdevalias/chatgpt-source-watch : Analyzing the evolution of ChatGPT's codebase through time with curated archives and scripts. - [Reverse engineering ChatGPT's frontend web app + deep dive explorations of the code (0xdevalias' gist)](https://gist.github.com/0xdevalias/4ac297ee3f794c17d0997b4673a2f160#reverse-engineering-chatgpts-frontend-web-app--deep-dive-explorations-of-the-code) - [Deobfuscating / Unminifying Obfuscated Web App Code (0xdevalias' gist)](https://gist.github.com/0xdevalias/d8b743efb82c0e9406fc69da0d6c6581#deobfuscating--unminifying-obfuscated-web-app-code) - [Reverse Engineering Webpack Apps (0xdevalias' gist)](https://gist.github.com/0xdevalias/8c621c5d09d780b1d321bfdb86d67cdd#reverse-engineering-webpack-apps) - [Reverse Engineered Webpack Tailwind-Styled-Component (0xdevalias' gist)](https://gist.github.com/0xdevalias/916e4ababd3cb5e3470b07a024cf3125#reverse-engineered-webpack-tailwind-styled-component) - [React Server Components, Next.js v13+, and Webpack: Notes on Streaming Wire Format (`__next_f`, etc) (0xdevalias' gist))](https://gist.github.com/0xdevalias/ac465fb2f7e6fded183c2a4273d21e61#react-server-components-nextjs-v13-and-webpack-notes-on-streaming-wire-format-__next_f-etc) - [Fingerprinting Minified JavaScript Libraries / AST Fingerprinting / Source Code Similarity / Etc (0xdevalias' gist)](https://gist.github.com/0xdevalias/31c6574891db3e36f15069b859065267#fingerprinting-minified-javascript-libraries--ast-fingerprinting--source-code-similarity--etc) - [Debugging Electron Apps (and related memory issues) (0xdevalias' gist)](https://gist.github.com/0xdevalias/428e56a146e3c09ec129ee58584583ba#debugging-electron-apps-and-related-memory-issues) - [devalias' Beeper CSS Hacks (0xdevalias' gist)](https://gist.github.com/0xdevalias/3d2f5a861335cc1277b21a29d1285cfe#beeper-custom-theme-styles) - [Reverse Engineering Golang (0xdevalias' gist)](https://gist.github.com/0xdevalias/4e430914124c3fd2c51cb7ac2801acba#reverse-engineering-golang) - [Reverse Engineering on macOS (0xdevalias' gist)](https://gist.github.com/0xdevalias/256a8018473839695e8684e37da92c25#reverse-engineering-on-macos) -
0xdevalias revised this gist
Apr 30, 2024 . 1 changed file with 10 additions and 0 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -2,6 +2,16 @@ Some notes/resources for bypassing anti-bot/scraping features on Cloudflare, Akamai, etc. ## Table of Contents <!-- TOC start (generated with https://derlin.github.io/bitdowntoc/) --> - [Notes](#notes) - [See Also](#see-also) - [My Other Related Deepdive Gist's and Projects](#my-other-related-deepdive-gists-and-projects) <!-- TOC end --> ## Notes - https://www.zenrows.com/blog/bypass-cloudflare - https://www.zenrows.com/blog/bypass-cloudflare#waiting-room-reverse-engineer - Lots of neat content in this section about the high level aspects of how the Cloudflare JavaScript check works, some of it's obfuscation techniques, and the general flow of things. -
0xdevalias revised this gist
Apr 30, 2024 . 1 changed file with 1 addition and 0 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -30,6 +30,7 @@ Some notes/resources for bypassing anti-bot/scraping features on Cloudflare, Aka - https://github.com/0xdevalias/chatgpt-source-watch : Analyzing the evolution of ChatGPT's codebase through time with curated archives and scripts. - [Deobfuscating / Unminifying Obfuscated Web App Code (0xdevalias gist)](https://gist.github.com/0xdevalias/d8b743efb82c0e9406fc69da0d6c6581#deobfuscating--unminifying-obfuscated-web-app-code) - [Fingerprinting Minified JavaScript Libraries / AST Fingerprinting / Source Code Similarity / Etc (0xdevalias gist)](https://gist.github.com/0xdevalias/31c6574891db3e36f15069b859065267#fingerprinting-minified-javascript-libraries--ast-fingerprinting--source-code-similarity--etc) - [Reverse Engineering Webpack Apps (0xdevalias gist)](https://gist.github.com/0xdevalias/8c621c5d09d780b1d321bfdb86d67cdd#reverse-engineering-webpack-apps) - [Reverse engineering ChatGPT's frontend web app + deep dive explorations of the code (0xdevalias gist)](https://gist.github.com/0xdevalias/4ac297ee3f794c17d0997b4673a2f160#reverse-engineering-chatgpts-frontend-web-app--deep-dive-explorations-of-the-code) - [Debugging Electron Apps (and related memory issues) (0xdevalias gist)](https://gist.github.com/0xdevalias/428e56a146e3c09ec129ee58584583ba#debugging-electron-apps-and-related-memory-issues) -
0xdevalias created this gist
Jun 30, 2023 .There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,36 @@ # Bypassing Cloudflare, Akamai, etc Some notes/resources for bypassing anti-bot/scraping features on Cloudflare, Akamai, etc. - https://www.zenrows.com/blog/bypass-cloudflare - https://www.zenrows.com/blog/bypass-cloudflare#waiting-room-reverse-engineer - Lots of neat content in this section about the high level aspects of how the Cloudflare JavaScript check works, some of it's obfuscation techniques, and the general flow of things. - https://www.zenrows.com/blog/bypass-akamai - https://www.zenrows.com/blog/bypass-akamai#akamais-javascript-challenge-explained - Less detail than the Cloudflare post had, but still some useful detail - https://github.com/FlareSolverr/FlareSolverr - > Proxy server to bypass Cloudflare protection - > FlareSolverr starts a proxy server, and it waits for user requests in an idle state using few resources. When some request arrives, it uses Selenium with the undetected-chromedriver to create a web browser (Chrome). It opens the URL with user parameters and waits until the Cloudflare challenge is solved (or timeout). The HTML code and the cookies are sent back to the user, and those cookies can be used to bypass Cloudflare using other HTTP clients. - https://github.com/ultrafunkamsterdam/undetected-chromedriver - > Optimized Selenium Chromedriver patch which does not trigger anti-bot services like Distill Network / Imperva / DataDome / Botprotect.io Automatically downloads the driver binary and patches it. - https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth - > A plugin for puppeteer-extra and playwright-extra to prevent detection. - > puppeteer-extra with stealth passes all public bot tests. > > Please note: I consider this a friendly competition in a rather interesting cat and mouse game. If the other team (👋) wants to detect headless chromium there are still ways to do that (at least I noticed a few, which I'll tackle in future updates). > > It's probably impossible to prevent all ways to detect headless chromium, but it should be possible to make it so difficult that it becomes cost-prohibitive or triggers too many false-positives to be feasible. > > If something new comes up or you experience a problem, please do your homework and create a PR in a respectful way (this is Github, not reddit) or I might not be motivated to help. :) - https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth/evasions ## See Also ### My Other Related Deepdive Gist's and Projects - https://github.com/0xdevalias/chatgpt-source-watch : Analyzing the evolution of ChatGPT's codebase through time with curated archives and scripts. - [Deobfuscating / Unminifying Obfuscated Web App Code (0xdevalias gist)](https://gist.github.com/0xdevalias/d8b743efb82c0e9406fc69da0d6c6581#deobfuscating--unminifying-obfuscated-web-app-code) - [Reverse Engineering Webpack Apps (0xdevalias gist)](https://gist.github.com/0xdevalias/8c621c5d09d780b1d321bfdb86d67cdd#reverse-engineering-webpack-apps) - [Reverse engineering ChatGPT's frontend web app + deep dive explorations of the code (0xdevalias gist)](https://gist.github.com/0xdevalias/4ac297ee3f794c17d0997b4673a2f160#reverse-engineering-chatgpts-frontend-web-app--deep-dive-explorations-of-the-code) - [Debugging Electron Apps (and related memory issues) (0xdevalias gist)](https://gist.github.com/0xdevalias/428e56a146e3c09ec129ee58584583ba#debugging-electron-apps-and-related-memory-issues) - [devalias' Beeper CSS Hacks (0xdevalias gist)](https://gist.github.com/0xdevalias/3d2f5a861335cc1277b21a29d1285cfe#beeper-custom-theme-styles)