To achieve features like function calling or structured output we need a way to constrain the output with a grammar. This process needs to run along the inference on cpu while the gpu is generating tokens. For every new token the grammar needs to reevaluate and filter the logic biases.