Skip to content

[Bug] headers don't pass to the api in crawl url #909

Open
@TChengZ

Description

@TChengZ

Describe the Bug
Provide a clear and concise description of what the bug is.
i crawl the url 'https://www.xchuxing.com/car?bid=203', in this page, it will send a request like
image
to get some information to show on the web,but the crawl result don't have the above message,
i figure out there must have some cookie to simulate the log in state,and i use postman to check that once i add the cookie below for the request(/all-brand?bid=203), it can return the information
image

and i past the cookie data to the pageOptions,but still can't crawl the full message from the website
To Reproduce
Steps to reproduce the issue:

  1. Configure the environment or settings with '...'
  2. Run the command '...'
  3. Observe the error or unexpected output at '...'
  4. Log output/error message

Expected Behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots or copies of the command line output to help explain the issue.

Environment (please complete the following information):

  • OS: [e.g. macOS, Linux, Windows]
  • Firecrawl Version: [e.g. 1.2.3]
  • Node.js Version: [e.g. 14.x]

Logs
If applicable, include detailed logs to help understand the problem.

Additional Context
Add any other context about the problem here, such as configuration specifics, network conditions, data volumes, etc.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingquestionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions