Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature detection for HDR canvas #4828

Open
kainino0x opened this issue Aug 27, 2024 · 39 comments
Open

Feature detection for HDR canvas #4828

kainino0x opened this issue Aug 27, 2024 · 39 comments
Labels
api resolved Resolved - waiting for a change to the API specification api WebGPU API
Milestone

Comments

@kainino0x
Copy link
Contributor

Right now it seems we don't have a way for webpages to feature-detect whether WebGPU can do HDR canvas. Only whether the device is HDR (by checking the media query).

See discussion here webgpu/webgpu-samples#432 (comment)
cc @beaufortfrancois @ccameron-chromium

@kainino0x kainino0x added the api WebGPU API label Aug 27, 2024
@kainino0x kainino0x added this to the Milestone 1 milestone Aug 27, 2024
@ccameron-chromium
Copy link
Contributor

It appears that several of the properties set in the GPUCanvasConfiguration are not query-able (GPUCanvasAlphaMode and the PredefinedColorSpace). If we were to be able to query "what is the current GPU canvas configuration", we could check to see what the tone mapping property (and all of those other ones) were set to. And if there is no tone mapping property, then we know that it isn't supported.

@beaufortfrancois
Copy link
Contributor

Following the similar CanvasRenderingContext2D: getContextAttributes() method, I'd propose something like GPUCanvasContext: getContextAttributes() method:

dictionary GPUCanvasContextAttributes {
  PredefinedColorSpace colorSpace = "srgb";
  GPUCanvasToneMapping toneMapping = {};
  GPUCanvasAlphaMode alphaMode = "opaque";
};

partial interface GPUCanvasContext {
    GPUCanvasContextAttributes getContextAttributes();
};

And the spec text change would be as simple as something like: https://html.spec.whatwg.org/multipage/canvas.html#dom-context-2d-canvas-getcontextattributes

@mdrejhon
Copy link

mdrejhon commented Sep 4, 2024

FYI, as of September 1st, I have now publicly launched a HDR CANVAs testing website also good for browser engine devs: https://beta.testufo.com and its HDR tester at https://beta.testufo.com/hdr

It would be nice at the javascript level if I could feature-detect whether HDR support is GPU-accelerated or not, since some HDR operations are ginormously slow compared to SDR operations. The prevailing recommendation is instead internally benchmark HDR performance though, and disable HDR when it's not performant enough. But that slows startup if I have to briefly benchmark first.

On MacBook M1, it runs really fast, but on some really old Intel internal GPUs from years ago (when HDR first introduced), enabling HDR suddenly causes some things to really slow down in mysterious ways -- highly suggestive of some HDR operations not GPU accelerated.

@mwyrzykowski
Copy link

Would feature detection of HDR canvas be beneficial? Even assuming a browser supports HDR canvas and the display does as well, the brightness of the display combined with absolute headroom may result in available headroom not much above 1 at maximal brightness.

E.g., on my Studio Display with 600 nits brightness, a difference in HDR particles is observable at lower brightness display levels but unobservable at maximal brightness due to reduced headroom.

@kainino0x
Copy link
Contributor Author

kainino0x commented Sep 11, 2024

I was actually thinking mainly about detecting browser support. The media query provides hardware support info (and any headroom info should be communicated through some media query too).
The issue is a browser can have HDR support via the media query without having it via WebGPU, and apps need to be able to feature-detect the specific path they're going to use.

Of course the detection could be rolled together in some way, but I think they are best separate. This depends on what browsers that support WebGPU HDR do on devices that don't support it. IIRC it is allowed and the browser just does some tonemapping to SDR?

@ccameron-chromium
Copy link
Contributor

While the browser detection is a "nice to have, but hopefully will be forgotten in the mists of time once all browsers support this", the suggested API in this comment has benefits that are worth considering on their own.

In particular, if I am a library and I am handed a canvas, I really really do want to know what the GPUCanvasAlphaMode of the canvas is. And so being able to query all of these things is extremely useful on its own.

@mdrejhon
Copy link

Would feature detection of HDR canvas be beneficial? Even assuming a browser supports HDR canvas

Yes. At beta.testufo.com you can turn on/off HDR canvas via the gear icon. I hand multiple offscreen canvas to central functions for processing and I've needed to track separately what color space these canvas are.

Would be nice to get straight from the horse's mouth instead of an accompanying variable or property that tracks this.

@kainino0x kainino0x added the api resolved Resolved - waiting for a change to the API specification label Sep 18, 2024
@Kangz
Copy link
Contributor

Kangz commented Sep 24, 2024

GPU Web WG 2024-09-18
  • KN: came up on WebGPU Samples - have an HDR sample now. ccameron suggests reflecting what the canvas configuration is, so you know whether your tone mapping request was honored. (Paid attention to by the browser, and obeyed.) How does feature detection work for hardware support? Know if the browser's looking at the option, or not.
  • MW: thumbs up
  • KG: there's a way to know if the browser consumed the request. Add a get observer to the dictionary, find out whether the browser actually accessed it. You still don't know whether the browser used it. Was recommended by HTML spec authors for WebGL-related dictionaries. Could do something similar here.
  • KN: messy but can be done. Would you prefer to rely on that rather than reflecting the configuration?
  • KG: I like reflection.
  • KN: sounds like everyone's happy with that.
  • KG/KN: testufo also asked for a query about “whether HDR support is GPU-accelerated or not”, not totally clear what this would mean
  • Resolved - reflect the information.

@beaufortfrancois
Copy link
Contributor

What are the next steps?

@Kangz
Copy link
Contributor

Kangz commented Sep 25, 2024

Someone makes a proposal for how information is reflected.

@beaufortfrancois
Copy link
Contributor

I've already proposed #4828 (comment). What do you think?

@Kangz
Copy link
Contributor

Kangz commented Sep 25, 2024

I don't have an opinion at this point, maybe editors or other folks would?

@beaufortfrancois
Copy link
Contributor

@Kangz Gotcha!

@kainino0x @jimblandy @toji Shall I send a spec PR matching #4828 (comment) or do you already have something else in mind?

@toji
Copy link
Member

toji commented Sep 25, 2024

I think something like that proposal is fine and does align with WebGL. I don't recommend getContextAttributes() because that implies values passed in at context creation, as with WebGL. Perhaps getConfiguration(), since it's reflecting values passed into configure(). Also note that it would need to be able to return null in the event that the canvas hasn't been configured yet or was explicitly unconfigured.

Finally, I don't know that it's necessary to have a new dictionary? Why not just return the full GPUCanvasConfiguration?

@beaufortfrancois
Copy link
Contributor

I like getConfiguration() as well.

Re: the new dictionary, I'm afraid we may add stuff into GPUCanvasConfiguration in the future that may be relevant only when setting the configuration and may not make sense when retrieving the final configuration.
Maybe WebGL folks can chime in and explain why it was done this way with getContextAttributes()

@kainino0x
Copy link
Contributor Author

+1 to getConfiguration()

WebGL uses the same type for both input and output, it's just a bit muddied by the fact that getContext is generic, not WebGL-specific - it takes the attributes as any and then WebGL converts that to WebGLContextAttributes.

I think it's fine to use the same type for configure() and getConfiguration(). The only member that I'm not sure should be reflected is device - it forces the configured canvas to hold a strong ref to the GPUDevice, but this is probably fine (we have unconfigure() to break the ref if necessary).

In any case, it would be a non-breaking change if we wanted to fork the type into two versions in the future, when adding an input we don't want as an output.

@beaufortfrancois
Copy link
Contributor

Thank you! I've started #4899

@beaufortfrancois
Copy link
Contributor

I believe we can close this issue as #4899 has been merged.

@greggman
Copy link
Contributor

I feel like I'm missing something. That value, GPUCanvasToneMapping is provided by the user. How does that tell them whether the device supports HDR?

Is it you do this?

context.configure({
  ...
  toneMapping: { mode: 'extended' },
});

const isReallyHDR = context.getConfiguration().toneMapping?.mode === 'extended';

???

The spec does not seem to suggest toneMapping will ever be anything than what the user set it to.

@beaufortfrancois
Copy link
Contributor

You'd need to test whether the display is HDR as well with the a CSS media query to detect whether or not the browser has canvas HDR support for WebGPU.

const hdrMediaQuery = window.matchMedia("(dynamic-range: high)");
const configuration = context.getConfiguration();
const isReallyHDR =
  configuration?.toneMapping?.mode === "extended" && hdrMediaQuery.matches;

@greggman
Copy link
Contributor

You lost me. In your example it appears toneMapping.mode is set automatically but it's not, it's set by the user. So, given the API design. I'd expect this to work

let isReallyHDR = false;
const hdrMediaQuery = window.matchMedia("(dynamic-range: high)");
if (hdrMediaQuery.matches) {
  context.configure({
    ...
    toneMapping: { mode: 'extended' },
  });
  isReallyHDR = true;
}

Correct or incorrect?

@beaufortfrancois
Copy link
Contributor

Sorry I should have added the following code as well in my response to make it clear.

context.configure({
  ...
  toneMapping: { mode: 'extended' },
});

Your code is correct for browsers that actually implement HDR support for WebGPU canvas. Implementations that lack it could still be feature-detected with isReallyHDR = configuration?.toneMapping?.mode === "extended" in your example.

@beaufortfrancois
Copy link
Contributor

See how WebKit does this for instance: WebKit/WebKit@60fafaa

@kainino0x
Copy link
Contributor Author

Closing, but re-open if it seems like something is still missing.

@greggman
Copy link
Contributor

greggman commented Oct 1, 2024

I'm sorry if I'm being pedantic but, it sounds like what you're saying is this configuration setting, unlike all of the others, doesn't respect the user's request. Instead, the user sets it, and then sees if the setting stuck. If it didn't stick then the device doesn't implement HDR support for WebGPU canvas.

// ask for HDR (no guarantee this works)
context.configure({
  ...
  toneMapping: { mode: 'extended' },
});

// Check if I actually got HDR
const isHDR = context.getConfiguration().toneMapping.mode === 'extended';

No ?s are required here. No WebGPU implementation that supports getConfiguration would be missing toneMapping since it's require by the spec. But, the implementation is allowed to ignore that the user specified extended mode?

If that's what you're saying I don't see where in the spec it says the user's configuration for toneMapping: { mode: 'extended' } can be ignored. Is that yet to be added?

@mwyrzykowski
Copy link

I do share some concerns to what @greggman is saying.

Basically it seems like the configure attempts to configure the canvas, while getConfiguration only returns the current state. For a device which does not support HDR content will always report toneMappingMode == standard.

Or worded differently, a UA is free to ignore a request for .toneMappingMode = extended if the hardware does not support it.

In some sense, this is beneficial as it allows the website to determine if the display supports HDR without the media query.

But maybe some additional spec text is needed?

@beaufortfrancois
Copy link
Contributor

The note in https://gpuweb.github.io/gpuweb/#dom-gpucanvascontext-getconfiguration was supposed to imply that the user's configuration for toneMapping: { mode: 'extended' } could be ignored as we say "In scenarios where getConfiguration() shows that toneMapping is implemented and...". How you like to update it to address your concerns?

In some sense, this is beneficial as it allows the website to determine if the display supports HDR without the media query.

I believe it's not exactly true. The display could be non-HDR but getConfiguration().toneMapping could still be "extended" if HDR is supported by the WebGPU implementation. Web developers still need to check for the media query to know if the display is HDR.

@beaufortfrancois
Copy link
Contributor

FYI I've started webgpu/webgpu-samples#451 to show how web developers can use this handy method to check whether HDR canvas is implemented.

@greggman
Copy link
Contributor

greggman commented Oct 2, 2024

I think it would be good to be explicit in the spec for configure.

It should say toneMapping.mode is a request that may be ignored. It should be spelled out when it is ignored and what value will it will be set it when it is ignored.

As it is, it says toneMapping: defaults to {} even though it says GPUToneMapping defaults to { mode: 'standard' }. That's seems to suggest toneMapping.mode can be undefined even after calling configure

context.configure({ device, format }}; 
console.log(context.getConfiguration().toneMapping.mode); // prints undefined

context.configure({ device, format, toneMapping: { mode: 'standard' }};
console.log(context.getConfiguration().toneMapping.mode); // prints 'standard'

context.configure({ device, format, toneMapping: { mode: 'extended' }};
console.log(context.getConfiguration().toneMapping.mode); // prints ???

According to the spec currently it seems like in this last case toneMapping.mode will be "extended", always. If it's supposed to be ignorable then the configure spec seems like it should spell out when, how and what it becomes instead.

@beaufortfrancois
Copy link
Contributor

@greggman Here's a PR that hopefully addresses your valid concerns: #4909
Thank you!

@kainino0x kainino0x reopened this Oct 2, 2024
@kainino0x
Copy link
Contributor Author

That proposal is probably fine with me (with wording tweaks), but I don't think it's necessary.

No WebGPU implementation that supports getConfiguration would be missing toneMapping since it's require by the spec. But, the implementation is allowed to ignore that the user specified extended mode?

I don't want HDR canvas to be optional, I want our spec to require it but also provide a way for applications to feature-detect whether the browser actually implements that part of the spec or not. It's similar to checking whether, like, GPUCommandEncoder.prototype.clearBuffer exists, even though the spec requires it, because it's newer. My expectation was:

  • Browsers with actual HDR support would avoid adding toneMapping to their dictionary definition until they've actually implemented it, because clamping to SDR on an HDR display would be out-of-spec. They could still implement other new parts of the spec while staying behind on this.
  • Browsers that don't have HDR support (or that want to prevent some webpages from accessing HDR) would pretend all displays are SDR, so they would be able to add toneMapping because "extended" and "standard" behave the same on SDR displays per spec.

We could include a full explanation of that in the spec, non-normatively.

Or we could make it all explicit/normative and make getConfiguration() able to return toneMapping: undefined / not return toneMapping, or we could do something like #4909.

Or worded differently, a UA is free to ignore a request for .toneMappingMode = extended if the hardware does not support it.

Even if we do #4909 it should only be allowed when the browser doesn't support HDR WebGPU canvas at all. It shouldn't be allowed based on hardware because it's a per-display ability, and can change if for example an HDR display is plugged into a laptop that has an SDR internal display. Technically the request could be ignored if the GPU is somehow incapable of HDR, but we should not allow sites to observe the difference between that, and the displays being incapable.

Also the value of getConfiguration().toneMapping.mode shouldn't change when the window is dragged between displays; if it could, we would want to expose some kind of event for that, but media query change events already provide what sites need.

@beaufortfrancois
Copy link
Contributor

That proposal is probably fine with me (with wording tweaks), but I don't think it's necessary.

What do you suggest as next steps? We could make #4909 non normative so that it's clear that the spec requires HDR canvas BUT we explain how web apps can feature-detect when the browser does not support yet HDR canvas. OR we can close #4909 and decide this does not belong to the spec world and should/will be explained in developer documentation (which I'll do).

Even if we do #4909 it should only be allowed when the browser doesn't support HDR WebGPU canvas at all.

That was the plan indeed. The value of getConfiguration().toneMapping.mode shouldn't change until configure(newConfig) or unconfigure() is called.

@greggman
Copy link
Contributor

greggman commented Oct 7, 2024

Is it per display thing?

I think I'm confused about the different features

  1. can the display show HDR.

    I thought this was done with media queries

  2. can WebGPU provide values > 1.0

    I thought this was the toneMapping thing

So, show a toneMapping: { mode: 'extended' } on a display that doesn't support HDR and it gets clipped or remapped (or something). ?

As always, this color management stuff has too many parts for me to follow 😅

@beaufortfrancois
Copy link
Contributor

So, show a toneMapping: { mode: 'extended' } on a display that doesn't support HDR and it gets clipped or remapped (or something). ?

"extended" unlocks the full HDR range of the screen. This mode matches "standard" in the [0, 1] range of the screen. Clamping or projection is done to the extended dynamic range of the screen but not [0, 1]. See https://developer.chrome.com/blog/new-in-webgpu-129#hdr_support_with_canvas_tone_mapping_mode

@ccameron-chromium
Copy link
Contributor

There should be no way for a rendering context to reject a request for any value of toneMapping.

I view this primarily as a way to query a context for the current value of toneMapping (and the GPUCanvasAlphaMode, and everything else).

In the short term this can be used to check "has this particular browser implemented toneMapping yet", but in the long term the answer will always only be "yes".

@beaufortfrancois
Copy link
Contributor

@kainino0x (gentle ping)

@kainino0x
Copy link
Contributor Author

In the short term this can be used to check "has this particular browser implemented toneMapping yet", but in the long term the answer will always only be "yes".

This matches what I'm suggesting. In order to make it clearer we can update the non-normative parts in the spec to explain what you're supposed to do with it, and what implementations that don't support HDR WebGPU are supposed to do (i.e. exactly how they should violate the spec to indicate their nonconformance in a detectable way).

@beaufortfrancois
Copy link
Contributor

@kainino0x I've started #4922. Let me know what you think.

@Saleemfarheen

This comment was marked as spam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api resolved Resolved - waiting for a change to the API specification api WebGPU API
Projects
None yet
Development

No branches or pull requests

9 participants