Skip to content

How to specific customized force_token_ids in whisper #34107

@MonolithFoundation

Description

@MonolithFoundation
ValueError: A custom logits processor of type <class 'transformers.generation.logits_process.ForceTokensLogitsProcessor'> with values <transformers.generation.logits_process.ForceTokensLogitsProcessor object at 0x7f4230cfac50> has been passed to `.generate()`, but it has already been created with the values <transformers.generation.logits_process.ForceTokensLogitsProcessor object at 0x7f422829c510>. <transformers.generation.logits_process.ForceTokensLogitsProcessor object at 0x7f422829c510> has been created by passing the corresponding arguments to generate or by the model's config default values. If you just want to change the default values of logits processor consider passing them as arguments to `.generate()` instead of using a custom logits processor

this way don't work:

inputs = inputs.to(self.model.dtype)
        with torch.no_grad():
            if forced_decoder_ids is not None:
                generated_ids = self.model.generate(
                    inputs, forced_decoder_ids=forced_decoder_ids
                )
            else:
                generated_ids = self.model.generate(inputs)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions