System Info

Python 3.11.5
Langchain (pip show) 0.0.327
Windows OS
Visual Studio Code

Who can help?

@eyurtsev

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

I searched and was surprised this has not come up.

I am using LangChain for a RAG workflow - and when I send a document, if that document contains { } - it throws a missing key error - it is treating the content of the document, as it would a normal prompt where you might have "question {question}" and expect an input key of 'question' it then reports back that all of the { } are in fact different missing keys.

For example, my data contains this:

"...1 2 ------------------------------------ {w14 w15 w16se w16cid w16 w16cex w16sdtdh wp14}{DP}{AD}{S::}"

It will say that we are missing numerous keys:

ValueError: Missing some input keys: {'AD', 'w14 w15 w16se w16cid w16 w16cex w16sdtdh wp14', ...}

Now, I can clean the data prior to sending, but I was wondering whether it should behave like this given that this document is already within { } as content?

I use the "FewShotPromptTemplate" to create a prompt which includes a "Suffix" and my suffix is:

def get_suffix():
    return """
    Document: {content}
    Question: {question}
    """

Here content is the content of the document that contains the { } set out above.

I build the prompt like this:

prompt_template = FewShotPromptTemplate(
            examples = examples,
            example_prompt = get_prompt_template(example_template, example_variables),
            prefix = prefix,
            suffix = suffix,
            input_variables = input_variables
        )
        prompt = prompt_template.format(question=question, context=context)
        return prompt

I also did a test using another piece of code:

document_context = text_response + "{AD}"
prompt = ChatPromptTemplate.from_template("my_specific_prompt": {document}.\n{format_instructions}")
formated_prompt = prompt.format(**{"document": document_context, "format_instructions":output_parser.get_format_instructions()})

Introducing a random {AD} into the text response. It did not fail. It messed up the results, but it didn't actually cause any missing input key errors.

So this may be limited to the FewShotPromptTemplate?

Expected behavior

I would have thought that anything passed within a curly bracket set would be considered as plain text, not parsed for further keys that might be embedded in that curly bracket set and throw an error when it cannot find them?

Maybe I am wrong, but that is what I would have expected and is what appears to happen when using the ChatPromptTemplate.from_template?