Skip to content

Conversation

@yoavweiss
Copy link
Contributor

As discussed in #716 and #775, this PR adds a recommended Sec- prefix to Client Hints request header values.

Open questions:

  • Should it be Sec-CH?
  • Should it be a MUST?

/cc @igrigorik @reschke @mnot @mikewest @arturjanc

@arturjanc
Copy link

My opinion doesn't carry a lot of weight here, but I like both MUST and Sec-CH (or potentially some other namespace-like thing, e.g. Sec-Hint) -- this mirrors what @mikewest did for Sec-Fetch-*.

Ilya's suggestion also LGTM.

@yoavweiss
Copy link
Contributor Author

Apologies for my slowness on this. Adopted Ilya's suggestion.
I know that @reschke was wary of adding more namespace like things though.
@reschke - can you repeat your concerns around this?

@reschke
Copy link
Contributor

reschke commented Jul 18, 2019

Well, the discussion is in #716 - maybe this is a good topic for the WG meetings in Montreal?

@yoavweiss
Copy link
Contributor Author

OK, I'll revive the discussion there

@yoavweiss yoavweiss force-pushed the client_hints_sec_prefix branch from 66c12d7 to 50248e9 Compare July 25, 2019 14:49
@yoavweiss
Copy link
Contributor Author

Fixes #767 and #716

@yoavweiss
Copy link
Contributor Author

@igrigorik - integrated your suggestions and then suggestions from @sleevi. PTAL?

Copy link
Member

@igrigorik igrigorik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, this looks great!

Per Ryan's earlier comment.. I don't have a strong opinion on whether CH- bit is necessary. I'll defer to @mnot and @martinthomson's guidance here.

@yoavweiss yoavweiss force-pushed the client_hints_sec_prefix branch from 9c1b603 to a2f6bab Compare August 13, 2019 10:56
@yoavweiss
Copy link
Contributor Author

mnot@ - Friendly ping :) Do you think the "CH-" bit is required?

## Deployment and Security Risks
Deployment of new request headers requires several considerations:

- Potential conflicts due to existing use of field name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's managed by registering the header field, and is well-documented in HTTP. Why is this important to call out here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example I had in mind were the conflicts @mikewest ran into when deploying the initial version of upgrade-insecure-requests as https. I believe that resulted in some server side conflicts with some servers that already used that name internally, and read all incoming request headers to be global variables. (@mikewest - please correct me if I remember that wrong)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some versions of PHP do strange things with certain headers. https was one of them. I'm not sure that has much weight in either direction, except as an anecdote that gave me a very bad week a few years ago. :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I just don't know why we're doing this specifically here, rather than relying on the generic HTTP text (DRY and all that). Not a huge deal, just struck me as odd.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So Yoav explained that the main reason here is that the reason for the Sec- prefix was to avoid problems like the one where header fields are turned into environment variables in a non-injective fashion by some frameworks. So you get the value of the "Foo" header field mixed in with the "foo" query parameter. That motivates a prefix, but it doesn't motivate the "Sec-" one.

I hadn't heard that reason properly before. It took me a while to internalize it.

And then it didn't make sense. The more I think about this, the more that I think that the "CH-" prefix is useful, but the "Sec-" one isn't. On the contrary, I see value in giving script the ability to query for different content using these header fields. For one, it might allow an app that is preparing for going offline in the browser the ability to request content based on anticipated rather than current needs.

For example, assuming for a moment that I thought Save-Data was useful (it's not), then the content could request the lightweight content that this targets, to avoid expending resources on the fetch or storage. At that point, I don't have to have an opinion on Save-Data from the browser perspective, it might as well be yet another proprietary header field.

Copy link
Contributor Author

@yoavweiss yoavweiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing! :)

## Deployment and Security Risks
Deployment of new request headers requires several considerations:

- Potential conflicts due to existing use of field name
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example I had in mind were the conflicts @mikewest ran into when deploying the initial version of upgrade-insecure-requests as https. I believe that resulted in some server side conflicts with some servers that already used that name internally, and read all incoming request headers to be global variables. (@mikewest - please correct me if I remember that wrong)

@yoavweiss
Copy link
Contributor Author

Friendly ping! :)

Copy link
Member

@igrigorik igrigorik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good on this end 👍

@mnot I'll defer to you for final once over and approve.

Copy link
Contributor

@martinthomson martinthomson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally, I was sure I didn't understand this. Then I thought that I might have understood. Now I'm closer to being sure that I think this is not the right answer.

- Potential conflicts due to existing use of field name
- Properties of the data communicated in field value

Specifications and features relying on Client Hints are advised to carfully consider prefixing request header names with a "Sec-" prefix. User agents, such as web browsers, ensure such prefixed headers can only be emitted by the browser, and application content, such as scripts, are forbidden from emitting such headers {{FETCH}}. This provides assurance that, when transmitted by user agents, the type, format and value of data communicated in such request headers are enforced by the user agent.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

carfully?

Also, the number, of, commas, here, is, starting to, get tiresome. Maybe there are two sentences in that second one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's much more clear to say something like:

Authors of new Client Hints are advised to carefully consider whether they should be able to be added by client-side content (e.g., scripts), or whether they should be exclusively set by the user agent. In the latter case, the Sec- prefix on the header field name has the effect of preventing scripts and other application content from setting them in many user agents. See {{FETCH}} for more information.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is definitely better phrasing. Assuming that you still want to do this based on my earlier comments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops regarding "carfully"...

Otherwise, I like @mnot's proposed text, assuming we are fine with user-added Client-Hints. I need to give them some more thought.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, Mark's text seems fine to me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the "Sec-" prefix signals to servers that the user agent - and not application content - generated the values.

## Deployment and Security Risks
Deployment of new request headers requires several considerations:

- Potential conflicts due to existing use of field name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So Yoav explained that the main reason here is that the reason for the Sec- prefix was to avoid problems like the one where header fields are turned into environment variables in a non-injective fashion by some frameworks. So you get the value of the "Foo" header field mixed in with the "foo" query parameter. That motivates a prefix, but it doesn't motivate the "Sec-" one.

I hadn't heard that reason properly before. It took me a while to internalize it.

And then it didn't make sense. The more I think about this, the more that I think that the "CH-" prefix is useful, but the "Sec-" one isn't. On the contrary, I see value in giving script the ability to query for different content using these header fields. For one, it might allow an app that is preparing for going offline in the browser the ability to request content based on anticipated rather than current needs.

For example, assuming for a moment that I thought Save-Data was useful (it's not), then the content could request the lightweight content that this targets, to avoid expending resources on the fetch or storage. At that point, I don't have to have an opinion on Save-Data from the browser perspective, it might as well be yet another proprietary header field.

@arturjanc
Copy link

The Sec- prefix guarantees that application developers are not currently sending requests with that header to their server (i.e. they're not adding it in XHR/Fetch because it's a forbidden header name.) If a browser starts sending a new header without the prefix, it can result in unexpected behavior if the header name clashes with a custom header already in use by an application. In some cases this can create a security issue, e.g. if an application uses the presence of such a header to identify trusted requests -- this approach is a relatively common CSRF protection because the server can be sure that requests with a custom header were sent same-origin, or the server had opted into them in response to a CORS preflight.

I don't feel very strongly about this particular case, but in general I'm worried about browsers sending new unprefixed headers by default for this reason.

@arturjanc
Copy link

I believe @annevk had some comments about similar situations in the past.

@mnot
Copy link
Member

mnot commented Nov 6, 2019

@arturjanc that basically means we can't define any new request HTTP headers, which is pretty demonstrably untrue. What we shouldn't do is choose especially generic names, like HTTPS. Don't generalise from one bad experience.

@martinthomson
Copy link
Contributor

Also, the establishment of the convention to use a "CH-" prefix would seem to avoid that problem amply. But @mnot's point is the stronger one. I realize that having a standard bulldoze your established use of "My-Header" is an experience people are reluctant to be exposed to the possibility of, but they could always register header field names.

@annevk
Copy link
Contributor

annevk commented Nov 7, 2019

Each new request header that a browser is to transmit cross-origin and is not prefixed with Sec- (or subject to a CORS preflight) would be another enshrined exception to the same-origin policy and a possible security issue until servers are updated to expect it.

Furthermore, some new request headers are not supposed to be spoofable same-origin and for those the prefix is important too. We have not had to update https://fetch.spec.whatwg.org/#forbidden-header-name for about a decade now and it would be nice to keep it that way.

(There's also a problem if an attacker can control these fields somehow. We've had to impose length restrictions on Referer and the CORS-safelisted request-headers. I suspect that's not applicable to client hints.)

@martinthomson
Copy link
Contributor

(OK, disclaimer: I'm not 100% clear on the model for cross-origin use of client hints then, but this is my operating model. If this model is incorrect somehow, I think that the fetch changes here are what needs adjusting...)

Given that these aren't sent unless the server specifically requests them, I think that the cross-origin concern isn't that significant. Rather than insisting on a one-by-one exception, a systemic exception for client hints would be preferable.

The way that this draft models CH header fields is that the site collects the information and injects the header field in every request as it leaves for the server (as if by service worker fetch interception). The browser is just helping out. If we are able to adhere to that model, then allowing insertion is different than header fields like User-Agent or Origin or other security-critical fields.

If this model is correct, then it doesn't make sense for these to be forbidden or for them to require special processing for exemption. Note that this changes how the information needs to be generated (it can't access privileged information; that's already in this doc), and it means that these are treated like any other header field.

As for concerns about interaction with ambient authority that a server might associate with the request, I think that I will point back at the bit where the server explicitly asked for the hint to be provided. I concede that cross-resource concerns might be a problem (resource A asks, resource B is unprepared), but the set of fields will be small, carefully groomed by browser makers, mostly prefixed with CH-, and carrying information that is not obviously useful in security-critical decisions.

@annevk
Copy link
Contributor

annevk commented Nov 7, 2019

Agreed that if the server asks for the browser to set these headers on subsequent requests the cross-origin concern does not apply. However, that does assume these headers are not attacker-controlled and depending on when they are set (and the ask has been that they are set early to expose them to service workers as normal headers) they would also have to bypass CORS and we'd have to ensure they cannot be manipulated, etc.

And writing that down I realize there's a security problem there as now A can figure out that B requested client hints (and see what hints B will get).

Also, does the spoofing concern not apply?

It's also not entirely clear if server opt-in is tenable as we've seen requests for client hint like data to be available on the first request.

@yoavweiss
Copy link
Contributor Author

Given that these aren't sent unless the server specifically requests

That is true for the same origin case, but not for the cross-origin one. For privacy purposes, we've limited the Client Hints opt-in to top-level documents, so a cross-origin resource cannot opt-in to client hints. And with Feature Policy, the top-level document can delegate hints to certain origins.

That means that those origins will get the hints without opting into them, which increases the risk.

At the same time, we want to avoid Client Hints request headers from triggering preflights, while being sure that it's indeed safe to do so.

@martinthomson
Copy link
Contributor

A can figure out that B requested client hints (and see what hints B will get).

A seeing the values isn't an issue as it is A that put the hints there (and A sees the same values).

I don't see A having influence over values (viewport width maybe...).

B being unprepared for the hints does seem to be a problem. I know that avoiding preflight is appealing from a performance perspective, but that's how we've decided to operate for these things. Because A is essentially forcing the inclusion of header fields into cross-origin requests, I think that we have a few options:

  1. perform some sort of analysis for each client hint to verify that no server conditions behaviour on the newly introduced header field in a way that would expose security-critical information

  2. preflight always

  3. find a better alternative to preflights (origin policy, the TLS extension, etc...)

  4. pretend that Sec--prefixed header fields aren't subject to these same concerns

  5. require the use of CH- prefixes and perform generic analysis

FWIW, for the first reason, we can't remember that B was OK with hints when it was top-level or framed in C because that lets A observe that B has been visited elsewhere.

@arturjanc
Copy link

  • pretend that Sec--prefixed header fields aren't subject to these same concerns

I'm not sure we need to do a lot of pretending here ;-) We know that applications can't get Sec-prefixed headers with ambient authority, so developers don't use them in their applications (unlike other headers which may be set same-origin or allowed via a response to a preflight). I believe delivering hints with a Sec- prefix would indeed avoid the class of problems discussed above; is there a specific issue that you're still worried about in this case?

@yoavweiss
Copy link
Contributor Author

@martinthomson - does @arturjanc's response answer concerns you had around option (4)?
As I'm not sure how to go about either (1) or (5), (2) would be a significant performance regression and (3) seems somewhat orthogonal, (4) seems like the best path forward.

@yoavweiss
Copy link
Contributor Author

At IETF 106, we agreed that it might be better to modify this so that the infrastructure draft indicates that features need to make sure that adding new request headers would be safe for servers. Individual features would then be able to decide if they enforce Sec-, live with a preflight and its performance regressions, or otherwise add their request headers to the CORS safe list. An alternative PR outlining that will follow.

@yoavweiss yoavweiss force-pushed the client_hints_sec_prefix branch from d2882b8 to 30afe42 Compare December 4, 2019 11:09
@yoavweiss
Copy link
Contributor Author

At IETF 106, we agreed that it might be better to modify this so that the infrastructure draft indicates that features need to make sure that adding new request headers would be safe for servers. Individual features would then be able to decide if they enforce Sec-, live with a preflight and its performance regressions, or otherwise add their request headers to the CORS safe list. An alternative PR outlining that will follow.

@martinthomson - looking at the current language, it seems to well-align with that decision without much modification. Can you take a look and let me know if you agree?

@yoavweiss
Copy link
Contributor Author

@martinthomson - friendly ping! :)

@yoavweiss
Copy link
Contributor Author

Post holidays ping! :)

Copy link
Contributor

@martinthomson martinthomson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm seeing pretty strong encouragement to use "Sec-" here.

- Potential conflicts due to existing use of field name
- Properties of the data communicated in field value

Specifications and features relying on Client Hints are advised to carfully consider prefixing request header names with a "Sec-" prefix. User agents, such as web browsers, ensure such prefixed headers can only be emitted by the browser, and application content, such as scripts, are forbidden from emitting such headers {{FETCH}}. This provides assurance that, when transmitted by user agents, the type, format and value of data communicated in such request headers are enforced by the user agent.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the "Sec-" prefix signals to servers that the user agent - and not application content - generated the values.

Authors of new Client Hints are advised to carefully consider whether they should be able to be added by client-side content (e.g., scripts), or whether they should be exclusively set by the user agent. In the latter case, the Sec- prefix on the header field name has the effect of preventing scripts and other application content from setting them in user agents.
Using the "Sec-" prefix signals to servers that the user agent - and not application content - generated the values. See {{FETCH}} for more information.

By convention, request headers that are client hints are encouraged to use a CH- prefix, to make them easier to identify as using this framework; for example, Sec-CH-Foo. Doing so makes them easier to identify programmatically (e.g., for stripping unrecognised hints from requests by privacy filters).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By convention, request headers that are client hints are encouraged to use a CH- prefix, to make them easier to identify as using this framework; for example, CH-Foo or, with a "Sec-" prefix, Sec-CH-Foo. Doing so makes them easier to identify programmatically (e.g., for stripping unrecognised hints from requests by privacy filters).

@yoavweiss yoavweiss merged commit dd24850 into httpwg:master Jan 20, 2020
@yoavweiss yoavweiss deleted the client_hints_sec_prefix branch January 20, 2020 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

9 participants