Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Explicitly set limits for prompt length #31

Open
QUANGLEA opened this issue Jul 12, 2023 · 4 comments
Open

Explicitly set limits for prompt length #31

QUANGLEA opened this issue Jul 12, 2023 · 4 comments
Labels
good first issue Good for newcomers help wanted Extra attention is needed

Comments

@QUANGLEA
Copy link

QUANGLEA commented Jul 12, 2023 •
edited
Loading

Hi, I'm getting this error 502 Server Error: Bad Gateway for url: https://alpha.rebuff.ai/api/detect when I use detect_injection() with long user inputs. A couple of sentences is fine, but a longer DAN prompt doesn't work. Appreciate any help I could get with this, thanks!
@woop
Copy link
Contributor

woop commented Jul 22, 2023

Hey @QUANGLEA, thanks for raising!

@cis-muzahid
Copy link

I am also getting same issue, is there any way something like calculate the tokens or string size and pass into chunks like tiktoken or something?
Thanks.

@woop
Copy link
Contributor

woop commented Aug 4, 2023

@cis-muzahid what would the idea API look like for you if we were to break up the prompt into chunks? That seems like something the caller should do, not the PI library. What do you think?

@seanpmorgan seanpmorgan changed the title 502 Server Error Explicitly set limits for prompt length Oct 4, 2023
@seanpmorgan seanpmorgan added help wanted Extra attention is needed good first issue Good for newcomers labels Oct 4, 2023
@seanpmorgan
Copy link
Member

Hi @QUANGLEA and @cis-muzahid. Thanks for reporting! I've edited the issue title so that Rebuff will explicitly set what the maximum prompt length is for this endpoint. After we've achieved that we can discuss if we should support more optimal handling or if that's on the caller to do.

Do you have a reproducible example we can use to debug?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@woop @seanpmorgan @cis-muzahid @QUANGLEA