Authoritative Metadata

2.1 Scenario 1: Silent recovery from error

Stuart runs his own Web server at "http://www.example.org/". He creates an HTML page and means to serve it as "text/html", but misconfigures the Web server so that the content is served via HTTP/1.1 [RFC2616] as "text/plain". Tim's browser looks inside the page, detects some markup that suggests that this is an HTML document (e.g., a <!DOCTYPE declaration or <title> element), and, without informing Tim, proceeds as though the content were "text/html", rendering it according to the HTML and CSS specifications. Janet's browser displays the content as plain text.

Which party has neglected a principle of Web architecture: Stuart for the server misconfiguration, Tim's browser for silently overriding the HTTP headers from the server, or Janet's browser for not detecting that the content looked like HTML?

Answer: By silently overriding metadata from the representation provider in the HTTP headers, Tim's browser did not respect Web architecture principles that promote shared understanding and security.

2.2 Scenario 2: Server misconfiguration

Norm publishes an XHTML document that includes this link:

<link href="cool-style" type="text/css" rel="stylesheet"/>

Although the link refers to an XSLT style sheet, Norm has set the type attribute to "text/css". Stuart has configured the Web server so that the style sheet is served via HTTP/1.1 as "application/xslt+xml". With a user agent that understands XSLT but not CSS, Janet requests the content that includes this link. As it interprets the representation data, Janet's user agent reads the type hint and does not fetch the style sheet."

Which party is responsible for the fact that Janet did not receive content she should have: Stuart for the server configuration, Norm for stating that the style sheet is served as "text/css" when in fact it's served with a different media type, or Janet's user agent for not double-checking the media type with the server?

Answer: Though not a violation of principles of Web architecture, Norm's mislabeling of content deprived Janet of content she should have received.

2.3 Scenario 3: Metadata hints in specifications

The MyFormat specification specifies a type attribute and that, when type is present, a receiving agent must use its value and ignore conflicting metadata provided by the sender. The MyFormat specification designers explain that such a definition of the type attribute allows content authors to work around misconfigured servers. They contend that this is necessary because in many environments content authors do not have sufficient access to server managers to affect server configuration.

Should the MyFormat specification designers ignore a principle of Web architecture or define type this way to remedy this social problem?

Answer: The TAG does not believe that author-specified overrides in representation data offer the proper solution to social problems such as interactions with server managers. An agent that silently overrides server-provided metadata can create security risks and prevent errors from being detected and corrected.

3.1 The Role of Metadata

The sequence of numbers "324033" might be a license plate number in the state of Arkansas or an old-style telephone number in Italy. In general, one cannot determine the nature of data in the absence of context. One way to provide a context for interpretation is through metadata. On the Web, examples of important metadata include the Internet media type, which explains what data format specification can be used to interpret the data, and the character encoding, which explains how octets map to characters.

In The Web architecture, data and metadata are distinguished during an exchange between agents (in the protocol used to carry out the exchange). Agents exchange resource state information through a "representation," which consists of two parts:

1. Representation data, expressed in one or more formats (e.g., XHTML, SVG, PNG) used separately or in combination.
2. Representation metadata, expressed in the protocol. This metadata includes the Internet media type (e.g., "text/html" or "image/jpeg") of the representation data.

Not only does this separation promote shared understanding, it enables more efficient processing. It is far easier to dispatch behavior on the basis of inspecting metadata (typically short strings) than it is to invoke a generic document parser and try to divine the purpose of data by inspecting the data itself (with no guarantee of success). Separating data from metadata also increases the Web's flexibility as data formats rise, evolve, and fall.

3.2 Sources of Metadata

One can imagine different (competing) sources of metadata:

• From the agent providing the representation data.
• From the agent receiving the representation data.
• From a third party referring to a resource.

To enable the greatest number of independent agents to interpret representation data in a consistent manner (i.e., according to a common set of specifications), the Web architecture adopts the first choice: representation metadata, when provided by the sender of a representation, is authoritative in defining the nature of the representation being sent.

Thus, if the sender asserts through a protocol that "the following representation data has the Internet media type text/html", that assertion is authoritative. The IANA media type registry maps these short strings such as "text/html" or "image/png" to data format specifications (e.g., XHTML, CSS, PNG, XLink, RDF/XML, etc.) via intermediate media type registrations. For instance, in the IANA registry, the content type "text/html" is associated with [RFC2854], which in turn states that:

The Internet media type asserts "this is X", not "process this as follows." Representation metadata does not constrain the receiving agent to process the representation data in one particular way. It does allow the designer of the receiving agent to create applications that correctly interpret the sender's intentions, while taking into account the desires of the party employing the agent (e.g., expressed through configuration and user choices).

3.3 Risks Associated with Overriding Authoritative Metadata

A user agent represents the user for protocol-level interactions with representation providers. A user agent that does not respect protocol specifications can violate user privacy, produce security holes, and otherwise create confusion. For example, a user agent can create a security problem by ignoring a "Content-Type" header with value "text/plain", guessing that representation data is a shell script, and executing the script on the user's machine without the user's knowledge. Because of such risks, it is an error for an agent to ignore or override authoritative metadata without the consent of the party employing the agent. For this reason, the HTTP/1.1 specification states, "If and only if the media type is not given by a "Content-Type" field, the recipient MAY attempt to guess the media type via inspection of its content and/or the name extension(s) of the URI used to identify the resource."

In scenario 1, in terms of Web architecture, Stuart is innocent; misconfiguration of the server is not an architectural error, it's just a human error. Instead, Tim's browser is the culprit since it misrepresents the resource provider by ignoring the authoritative metadata, without Tim's consent. Janet's browser respected the "Content-Type" header field, and by doing so, helps Janet and Stuart detect a server misconfiguration.

Note the difference between an agent taking authoritative metadata into account and an agent ignoring the metadata without the consent of the user. The first scenario below is an error, the second is not:

• The agent silently ignores the authoritative Internet media type "text/plain" and renders (without user consent) the representation data as though the Internet media type were "text/html".
• The agent recognizes the authoritative Internet media type of "text/plain", and because the user has so requested, tries to detect any URIs in the content and then dereferences them to determine whether representations are available for the identified resources.

4.1 Recipient Handling of Inconsistency

Consent does not necessarily imply that the receiving agent must interrupt the user and require selection of one option or another. User consent may be achieved in the form of pre-selected configuration options, modes, or selectable user interface toggles, with appropriate reporting to the user when the agent detects an error. For example, a small "bug" icon in a graphical browser's user interface can indicate that the user agent has overridden sender metadata and can also act as a button through which a curious user might inspect the error or reverse the agent's choice. Other agent behavior when faced with inconsistencies includes prompting the user for interactive guidance and advancing according to an ordered list of user preferences. Naturally, appropriate behavior and interfaces are unique to each type of receiving agent and application context. It is beyond the scope of this finding to anticipate the range of possible errors and ways in which interface designers might obtain user feedback to address them.

In Scenario 2, Norm is responsible for Janet not having access to representation data she was meant to receive. The HTML 4.01 Recommendation states that "Authors who use [the type] attribute take responsibility to manage the risk that it may become inconsistent with the content available at the link target address." Janet's client could have done more than merely read the type hint and decide to skip the style sheet" Users benefit from clients that allow different configurations for handling hints, including:

• Query the server, and when there is an inconsistency, choose the authoritative metadata, or
• Query the server, and when there is an inconsistency, prompt the user for instructions on how to proceed.

4.2 Self-describing data and Risk of Inconsistency

Data is "self-describing" if it includes enough information to allow two parties to establish a consistent interpretation without additional clues. If the representation provider intends for the data to be interpreted in a manner other than what is self-described (e.g., "treat this XML content as plain text"), then clarifying metadata is required (e.g., in protocol headers). As illustrated above, providing redundant metadata for data that is self-describing can lead to inconsistencies.

Representation providers SHOULD NOT in general specify the character encoding for XML data in protocol headers since the data is self-describing.

4.3 Reducing the Risk of Inconsistency

Representation providers can help reduce the risk of inconsistency through careful assignment of representation metadata (especially that which applies across representations). In particular:

• Server software designers SHOULD NOT specify a default Internet media type in the default configuration shipped with the server.
• Server managers SHOULD be wary of specifying a default Internet media type.
• Server managers SHOULD NOT specify an arbitrary Internet media type (e.g., "text/plain" or "application/octet-stream") when the Internet media type is unknown.
• Content authors SHOULD inform server managers of metadata misconfigurations.
• Server managers SHOULD provide authors with a means to override a metadata configuration when it is inappropriate for a specific representation. This does not mean that the representation data should override the representation metadata, only that the author should have a way to supply correct metadata.