Skip to content

Exception - Structured Output - Vision Models (Llama 3.2) #5599

Open

Description

Description

Using structured output with vision models like gpt-4o-mini works. I'd like to do the same for Llama-3.2-11B-Vision-Instruct from GitHub models. Currently it throws an exception.

Reproduction Steps

  1. Clone this repo: https://github.com/lqdev/AIBookmarks/
  2. Set modelName in Program.cs to Llama-3.2-11B-Vision-Instruct
  3. Run the application

Working Azure AI Inference SDK Code

Replacing the inference loop with this code works

foreach(var path in filePaths)
{
    var file = await File.ReadAllBytesAsync(path);
    var messages = new List<ChatRequestMessage>
    {
        new ChatRequestSystemMessage(systemPrompt),
        new ChatRequestUserMessage(new ChatMessageImageContentItem(BinaryData.FromBytes(file), "image/jpeg")),
        new ChatRequestUserMessage("Extract the marked passages from the image")
    };

    var response = await aoaiClient.CompleteAsync(new ChatCompletionsOptions{
        Messages = messages,
        Model = modelName
    });

    Console.WriteLine(response.Value.Content);
}

Expected Output

The marked passages are:

* If we try ten experiments and none of them work, we have a choice. We can take it personally, and think of ourselves as a failure and question our ability to solve the problems that come our way.
* We are required to believe in something that doesn't exist in order to allow it to come into being.
* Faith is rewarded, perhaps even more than talent or ability.

These passages are found on pages 278 and 279.

Similarly, using MEAI without structured output

foreach(var path in filePaths)
{
    var file = await File.ReadAllBytesAsync(path);
    var messages = new List<ChatMessage>
    {
        new ChatMessage(ChatRole.System, systemPrompt),
        new ChatMessage(ChatRole.User, new AIContent[] {
            new ImageContent(file, "image/jpeg"),
            new TextContent("Extract the marked passages from the image"),
        })
    };

    var response = await meaiClient.CompleteAsync(messages, options: new ChatOptions {Temperature = 0.1f});

    Console.WriteLine(response.Message);
}

Expected behavior

Response returns LLM response as specified type parameter, even if it's not supported by the model.

See generic completion API for reference.

Actual behavior

Exception is thrown.

Unhandled exception. Azure.RequestFailedException: An error occurred during the request processing: All connection attempts failed | Type: ConnectError | Traceback (most recent call last):||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_transports/default.py", line 66, in map_httpcore_exceptions||    yield||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_transports/default.py", line 366, in handle_async_request||    resp = await self._pool.handle_async_request(req)||           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request||    raise exc from None||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request||    response = await connection.handle_async_request(||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request||    raise exc||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request||    stream = await self._connect(request)||             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_async/connection.py", line 122, in _connect||    stream = await self._network_backend.connect_tcp(**kwargs)||             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_backends/auto.py", line 30, in connect_tcp||    return await self._backend.connect_tcp(||           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 115, in connect_tcp||    with map_exceptions(exc_map):||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/contextlib.py", line 158, in __exit__||    self.gen.throw(typ, value, traceback)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions||    raise to_exc(exc) from exc||httpcore.ConnectError: All connection attempts failed||||The above exception was the direct cause of the following exception:||||Traceback (most recent call last):||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__||    await self.app(scope, receive, _send)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/billingproxy/main.py", line 411, in __call__||    return await handler_task||           ^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__||    with collapse_excgroups():||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/contextlib.py", line 158, in __exit__||    self.gen.throw(typ, value, traceback)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups||    raise exc||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__||    response = await self.dispatch_func(request, call_next)||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/billingproxy/main.py", line 377, in dispatch||    response = await call_next(request)||               ^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next||    raise app_exc||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro||    await self.app(scope, receive_or_disconnect, send_no_error)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__||    with collapse_excgroups():||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/contextlib.py", line 158, in __exit__||    self.gen.throw(typ, value, traceback)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups||    raise exc||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__||    response = await self.dispatch_func(request, call_next)||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/billingproxy/main.py", line 353, in dispatch||    response = await call_next(request)||               ^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next||    raise app_exc||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro||    await self.app(scope, receive_or_disconnect, send_no_error)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__||    await self.app(scope, receive, send)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__||    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app||    raise exc||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app||    await app(scope, receive, sender)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__||    await self.middleware_stack(scope, receive, send)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/routing.py", line 776, in app||    await route.handle(scope, receive, send)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle||    await self.app(scope, receive, send)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/routing.py", line 77, in app||    await wrap_app_handling_exceptions(app, request)(scope, receive, send)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app||    raise exc||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app||    await app(scope, receive, sender)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/routing.py", line 72, in app||    response = await func(request)||               ^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app||    raw_response = await run_endpoint_function(||                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function||    return await dependant.call(**values)||           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/billingproxy/apis/vllm/chat.py", line 322, in chat||    response = await downstream_client_getter().send(downstream_request)||          
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_client.py", line 1617, in send||    response = await self._send_handling_auth(||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_client.py", line 1645, in _send_handling_auth||    response = await self._send_handling_redirects(||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects||    response = await self._send_single_request(request)||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_client.py", line 1719, in _send_single_request||    response = await transport.handle_async_request(request)||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_transports/default.py", line 365, in handle_async_request||    with map_httpcore_exceptions():||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/contextlib.py", line 158, in __exit__||    self.gen.throw(typ, value, traceback)||  File "/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_transports/default.py", line 83, in map_httpcore_exceptions||    raise mapped_exc(message) from exc||httpx.ConnectError: All connection attempts failed||
Status: 500 (Internal Server Error)
ErrorCode: Internal Server Error

Content:
{"error":{"code":"Internal Server Error","message":"An error occurred during the request processing: All connection attempts failed | Type: ConnectError | Traceback (most recent call last):||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_transports/default.py\", line 66, in map_httpcore_exceptions||    yield||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_transports/default.py\", line 366, in handle_async_request||    resp = await self._pool.handle_async_request(req)||           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_async/connection_pool.py\", line 216, in handle_async_request||    raise exc from None||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_async/connection_pool.py\", line 196, in handle_async_request||    response = await connection.handle_async_request(||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_async/connection.py\", line 99, in handle_async_request||    raise exc||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_async/connection.py\", line 76, in handle_async_request||    stream = await self._connect(request)||             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_async/connection.py\", line 122, in _connect||    stream = await self._network_backend.connect_tcp(**kwargs)||             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_backends/auto.py\", line 30, in connect_tcp||    return await self._backend.connect_tcp(||           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_backends/anyio.py\", line 115, in connect_tcp||    with map_exceptions(exc_map):||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/contextlib.py\", line 158, in __exit__||    self.gen.throw(typ, value, traceback)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpcore/_exceptions.py\", line 14, in map_exceptions||    raise to_exc(exc) from exc||httpcore.ConnectError: All connection attempts failed||||The above exception was the direct cause of the following exception:||||Traceback (most recent call last):||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/errors.py\", line 164, in __call__||    await self.app(scope, receive, _send)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/billingproxy/main.py\", line 411, in __call__||    return await handler_task||           ^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py\", line 189, in __call__||    with collapse_excgroups():||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/contextlib.py\", line 158, in __exit__||    self.gen.throw(typ, value, traceback)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_utils.py\", line 93, in collapse_excgroups||    raise exc||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py\", line 191, in __call__||    response = await self.dispatch_func(request, call_next)||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/billingproxy/main.py\", line 377, in dispatch||    response = await call_next(request)||               ^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py\", line 165, in call_next||    raise app_exc||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py\", line 151, in coro||    await self.app(scope, receive_or_disconnect, send_no_error)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py\", line 189, in __call__||    with collapse_excgroups():||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/contextlib.py\", line 158, in __exit__||    self.gen.throw(typ, value, traceback)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_utils.py\", line 93, in collapse_excgroups||    raise exc||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py\", line 191, in __call__||    response = await self.dispatch_func(request, call_next)||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/billingproxy/main.py\", line 353, in dispatch||    response = await call_next(request)||               ^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py\", line 165, in call_next||    raise app_exc||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/base.py\", line 151, in coro||    await self.app(scope, receive_or_disconnect, send_no_error)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/cors.py\", line 85, in __call__||    await self.app(scope, receive, send)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/middleware/exceptions.py\", line 65, in __call__||    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_exception_handler.py\", line 64, in wrapped_app||    raise exc||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_exception_handler.py\", line 53, in wrapped_app||    await app(scope, receive, sender)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/routing.py\", line 756, in __call__||    await self.middleware_stack(scope, receive, send)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/routing.py\", line 776, in app||    await route.handle(scope, receive, send)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/routing.py\", line 297, in handle||    await self.app(scope, receive, send)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/routing.py\", line 77, in app||    await wrap_app_handling_exceptions(app, request)(scope, receive, send)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_exception_handler.py\", line 64, in wrapped_app||    raise exc||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/_exception_handler.py\", line 53, in wrapped_app||    await app(scope, receive, sender)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/starlette/routing.py\", line 72, in app||    response = await func(request)||               ^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/fastapi/routing.py\", line 278, in app||    raw_response = await run_endpoint_function(||                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/fastapi/routing.py\", line 191, in run_endpoint_function||    return await dependant.call(**values)||           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/billingproxy/apis/vllm/chat.py\", line 322, in chat||    response = await downstream_client_getter().send(downstream_request)||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_client.py\", line 1617, in send||    response = await self._send_handling_auth(||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_client.py\", line 1645, in _send_handling_auth||    response = await self._send_handling_redirects(||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_client.py\", line 1682, in _send_handling_redirects||    response = await self._send_single_request(request)||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_client.py\", line 1719, in _send_single_request||    response = await transport.handle_async_request(request)||               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_transports/default.py\", line 365, in handle_async_request||    with map_httpcore_exceptions():||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/contextlib.py\", line 158, in __exit__||    self.gen.throw(typ, value, traceback)||  File \"/opt/microsoft-maas/miniconda/envs/proxy/lib/python3.11/site-packages/httpx/_transports/default.py\", line 83, in map_httpcore_exceptions||    raise mapped_exc(message) from exc||httpx.ConnectError: All connection attempts failed||","status":500}}

Headers:
Date: Mon, 04 Nov 2024 23:54:21 GMT
Transfer-Encoding: chunked
Connection: keep-alive
azureml-model-group: REDACTED
request-context: REDACTED
x-ms-rai-invoked: REDACTED
X-Request-ID: REDACTED
ms-azureml-model-error-reason: REDACTED
ms-azureml-model-error-statuscode: REDACTED
x-ms-client-request-id: 09647581-2868-4eef-b19d-be48a4c54033
azureml-model-deployment: REDACTED
azureml-model-session: REDACTED
Strict-Transport-Security: REDACTED
X-Content-Type-Options: REDACTED
x-aml-cluster: REDACTED
x-request-time: REDACTED
Content-Type: application/json

   at Azure.Core.HttpPipelineExtensions.ProcessMessageAsync(HttpPipeline pipeline, HttpMessage message, RequestContext requestContext, CancellationToken cancellationToken)
   at Azure.AI.Inference.ChatCompletionsClient.CompleteAsync(RequestContent content, String extraParams, RequestContext context)
   at Azure.AI.Inference.ChatCompletionsClient.CompleteAsync(ChatCompletionsOptions chatCompletionsOptions, CancellationToken cancellationToken)
   at Microsoft.Extensions.AI.AzureAIInferenceChatClient.CompleteAsync(IList`1 chatMessages, ChatOptions options, CancellationToken cancellationToken)
   at Microsoft.Extensions.AI.ChatClientStructuredOutputExtensions.CompleteAsync[T](IChatClient chatClient, IList`1 chatMessages, JsonSerializerOptions serializerOptions, ChatOptions options, Nullable`1 useNativeJsonSchema, CancellationToken cancellationToken)
   at Program.<Main>$(String[] args) in C:\Dev\AIBookmarks\Program.cs:line 76
   at Program.<Main>(String[] args)

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Labels

area-AIbugThis issue describes a behavior which is not expected - a bug.untriaged

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions