Skip to content

Add searxng for search endpoint#1193

Merged
mogery merged 6 commits intofirecrawl:mainfrom
loorisr:add_searxng
Feb 20, 2025
Merged

Add searxng for search endpoint#1193
mogery merged 6 commits intofirecrawl:mainfrom
loorisr:add_searxng

Conversation

@loorisr
Copy link
Contributor

@loorisr loorisr commented Feb 17, 2025

Fixes #1145

SearXNG has to be configured with json format

It adds 3 new var env:

@loorisr loorisr mentioned this pull request Feb 17, 2025
@loorisr loorisr closed this Feb 18, 2025
@loorisr loorisr deleted the add_searxng branch February 18, 2025 13:42
@loorisr loorisr restored the add_searxng branch February 18, 2025 13:42
@mogery
Copy link
Member

mogery commented Feb 20, 2025

Hey! We'd love to merge this. Any particular reason you've closed it?

@loorisr
Copy link
Contributor Author

loorisr commented Feb 20, 2025

I've deleted by mistake this branch on my repo and it automatically closed it :(

@loorisr loorisr reopened this Feb 20, 2025
@mogery mogery force-pushed the add_searxng branch 2 times, most recently from 5188448 to 7bac95e Compare February 20, 2025 09:52
@mogery mogery force-pushed the add_searxng branch 2 times, most recently from ad416d1 to 59ab50d Compare February 20, 2025 10:21
@mogery
Copy link
Member

mogery commented Feb 20, 2025

@loorisr do you know of any public SearXNG instances with JSON enabled that we can point to for testing?

@loorisr
Copy link
Contributor Author

loorisr commented Feb 20, 2025

you can use https://search.url4irl.com

@mogery
Copy link
Member

mogery commented Feb 20, 2025

you can use https://search.url4irl.com

It's getting a 429 Too Many Requests

@loorisr
Copy link
Contributor Author

loorisr commented Feb 20, 2025

it is very rate limited. you may have to setup your own instance.

@mogery mogery force-pushed the add_searxng branch 4 times, most recently from 5c6455f to cbe35b8 Compare February 20, 2025 11:11
@mogery
Copy link
Member

mogery commented Feb 20, 2025

Ended up bootstrapping a whole SearXNG instance in the worker. Dodgy, but works! Merging if this passes

@mogery mogery merged commit 100168d into firecrawl:main Feb 20, 2025
8 of 9 checks passed
@mogery
Copy link
Member

mogery commented Feb 20, 2025

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feat] SearXNG Support

2 participants