JSONRequest

Douglas Crockford
[email protected]
2006-04-17 (Updated 2012-08-10)

Abstract

XMLHttpRequest has a security model that is inadequate for supporting the next generation of web applications. JSONRequest is proposed as a new browser service that allows for two-way data exchange with any JSON data server without exposing users or organization to harm. It exchanges data between scripts on pages with JSON servers in the web. It is hoped that browser makers will build this feature into their products in order to enable the next advance in web application development.

Motivation

The next generation of web applications will be much more data intensive. They will want to go to a server, any server, and exchange data. The XMLHttpRequest interface suggests, but does not achieve, this. It is severely limited by a defective security model.

XMLHttpRequest is constrained by the Same Origin Policy. This constrains the interface to only connect with the server that delivered the base page. This rule deals with some common, long-standing security flaws in web architecture.

If the Same Origin Policy were not in place, then the user could be harmed by XSS (cross site scripting) attacks. In the following examples, we will have a page that was created by miscreants at pirate.net. That page will attempt to compromise the user's relationship with penzance.org.

If the Same Origin Policy were not in effect, a pirate.net page could send a request via XMLHttpRequest to penzance.org. That request would carry the penzance.org cookies. If penzance.org were using cookies for authentication, then it would be tricked into acting on the request as though it had been initiated by the user. Any request to a site will carry the cookies associated with the site. This would allow pirate.net the right to use penzance.org cookies.

If penzance.org sits behind a firewall, and if the internal servers assume that the firewall makes explicit authorization unnecessary, then the pirate.net page could be used as a proxy, accessing the contents of penzance.org for transmission back to pirate.net. This would be possible because XMLHttpRequest can obtain XML-like data (such as HTML documents) as well as non-XML text.

The Same Origin Policy frustrates these attacks, but it also frustrates a larger class of legitimate uses. It should be possible for a script in a page to access data from other servers without compromising the security of the user or his organization.

Surprisingly, the Same Origin Policy does not apply to scripts. So some developers have begun to dynamically generate <script> tags to connect to any server. The server sends back a script which delivers some data. Unfortunately, the script runs with the same authority as a scripts from the originating page, allowing the script to steal cookies or directly access the originating server. This is unsafe. If a penzance.org page loaded a script from pirate.net, the script could damage the relationship between penzance.org and its users by stealing cookies and making requests of the penzance.org server.

This document proposes a safe, reliable data service which will allow a script on any page to connect to any server and exchange data. It would make it possible for a page from pirate.net to access data from any server without compromising penzance.org, and for penzance.org to access pirate.net data on its pages without compromising its own users.

JSON

JSON is a data interchange format which is based on a safe subset of JavaScript. JSON can represent simple or complex structured data. JSON cannot represent functions or expressions. It is strictly data. It has very specific rules of syntax, so it is very straightforward to determine that a JSON text is syntactically well formed. A JSON text can easily be converted into a JavaScript value, which makes it a very convenient format for use with JavaScript. There is support for use of JSON with many other languages, including C#, Java, Perl, PHP, Python, and Ruby. More information on JSON can be found at www.JSON.org.

JSON does not look like XML, so HTML text fed to a JSON parser will produce an error.

JSONRequest

JSONRequest is a global JavaScript object. It provides three methods: post, get, and cancel.

`JSONRequest.post`

JSONRequest.post does an HTTP POST of the serialization of a JavaScript object or array, gets the response, and parses the response into a JavaScript value. If the parse is successful, it returns the value to the requesting script. In making the request, no HTTP authentication or cookies are sent. Any cookies returned by the server cause the request to fail. The JSONRequest service can only be used to send and receive JSON-encoded values. JSONRequest cannot be used to retrieve other text formats.

JSONRequest.post takes four parameters:

parameter	type	description
`url`	string	The URL to `POST` to. The URL does not need to be related to the page's URL.
`send`	object	The JavaScript object or array to send as the `POST` data. It will be serialized as JSON text. Cyclical structures will fail.
`done`	function (requestNumber, value, exception)	The function to be called when the request is completed. If the request was successful, the function will receive the request number and the returned value. If it is not successful, it will receive the request number and an exception object. The `done` function will not be called until after the call to `JSONRequest` returns a serial number.
`timeout`	number	The number of milliseconds to wait for the response. This parameter is optional. The default is `10000` (10 seconds).

JSONRequest.post returns a serial number if the request parameters are acceptable. It throws a JSONRequestError exception if the request is rejected. The request will be rejected if

The url string is not a properly formatted URL.
The send value cannot be serialized. It will be rejected if it is not an object or array or if it is cyclical. (Functions and host objects will not be included in the serialization.)
The done value is not a function.
The timeout value is not a positive number.

The request number may be used by a script to match requests with responses. It is provided as a convenience for programmers who are not comfortable with the use of function values and closures, and for request cancellation.

Example:

requestNumber = JSONRequest.post(
    "https://json.penzance.org/request",
    {
        user: "[email protected]",
        t: "vlIj",
        zip: 94089,
        forecast: 7
    },
    function (requestNumber, value, exception) {
        if (value) {
            processResponse(value);
        } else {
            processError(exception);
        }
    }
);

After JSONRequest.post has verified the parameters, it will queue the request and return the request number. The done function value will be invoked later when the outcome of the request is known.

No cookies or implicit authentication information are sent with the POST operation. Any authentication information must be placed in the send data or in the url. The JSON text that was serialized from the send data is used as the body of the request. The character encoding is UTF-8. An implementation may choose to gzip the JSON text.

The request may use either http or https. This choice is independent of the security of the page.

POST /request HTTP/1.1
Accept: application/jsonrequest
Content-Encoding: identity
Content-Length: 72
Content-Type: application/jsonrequest
Host: json.penzance.org

{"user":"[email protected]","forecast":7,"t":"vlIj","zip":94089}

After the server has acknowledged the request, it has until the time limit expires to produce a response. If the time limit is exceeded, or if the connection is closed before a complete response is sent, then the request fails. If the HTTP status code is not 200 OK, then the request fails.

HTTP/1.1 200 OK
Content-Type: application/jsonrequest
Content-Length: xxxx

The body of the response is a JSON text, encoded in UTF-8. If the text contains any JSON encoding errors, then the request fails.

If the request succeeds, then the done function is called with the request number and the value obtained from the parsing of the JSON text. If the request fails, then the done function is called with the request number and an exception object, indicating the communication failure. If a server wishes to communicate an application-level error, then it should return it as a JSON text with an HTTP status code of 200 OK.

The browser must be able to keep open at least two requests per host per page. Excess requests may be queued. The browser should attempt to keep alive connections. These connections are counted separately from the connections that the browser uses to fetch HTML and related resources.

`JSONRequest.get`

JSONRequest.get does an HTTP GET request, gets the response, and parses the response into a JavaScript value. If the parse is successful, it returns the value to the requesting script. In making the request, no HTTP authentication or cookies are sent. Any cookies returned by the server cause the request to fail. The JSONRequest.get service can only be used to obtain JSON-encoded values. JSONRequest.get cannot be used to retrieve other text formats.

JSONRequest.get takes three parameters:

parameter	type	description
`url`	string	The URL to `GET` from. The URL does not need to be related to the page's URL.
`done`	function (requestNumber, value, exception)	The function to be called when the request is completed. If the request was successful, the function will receive the request number and the returned value. If it is not successful, it will receive the request number and an exception object. The `done` function will not be called until after the call to `JSONRequest` returns a serial number.
`timeout`	number	The number of milliseconds to wait for the response. This parameter is optional. The default is `10000` (10 seconds).

JSONRequest.get returns a serial number if the request parameters are acceptable. It throws a JSONRequestError exception if the request is rejected. The request will be rejected if

The url string is not a properly formatted URL.
The done value is not a function.

The request number can be used by a script to match requests with responses.

Example:

requestNumber = JSONRequest.get(
    "https://json.penzance.org/request",
    function (requestNumber, value, exception) {
        if (value) {
            processResponse(value);
        } else {
            processError(exception);
        }
    }
);

After JSONRequest.get has verified the parameters, it will queue the request and return the request number. The done function value will be invoked later when the outcome of the request is known.

No cookies or implicit authentication information is sent with the GET operation. Any authentication information must be placed in the url.

The request may use either http or https. This choice is independent of the security of the page.

GET /request HTTP/1.1
Accept: application/jsonrequest
Host: json.penzance.org

After the server has acknowledged the request, it has until the timeout interval expires to produce a response. If the time limit is exceeded, or if the connection is closed before a complete response is sent, then the request fails.

HTTP/1.1 200 OK
Content-Type: application/jsonrequest
Content-Length: xxxx

The body of the response is a JSON text, encoded in UTF-8. If the text contains any JSON encoding errors, then the request fails.

An implementation may cache the results of GET requests.

`JSONRequest.clear`

A document can be removed from the GET cache by calling JSONRequest.clear with its url. Nothing is returned. It is not possible to determine with this function if the document had ever been in the cache.

JSONRequest.clear(url);

`JSONRequest.cancel`

A request can be canceled by calling JSONRequest.cancel with the request number as the only parameter. Nothing is returned. There is no guarantee that the request will not be sent to the server since it is possible that it had been transmitted before the cancel request was made.

JSONRequest.cancel(requestNumber);

If the request is still in the outgoing message queue, it will be deleted from the queue.

If the request is in progress, an attempt will be made to abort it.

If the request cannot be found, then the cancellation will be ignored.

When a message is successfully canceled, the done callback function of the request is called with an exception message of "canceled".

It is possible that from the server's point of view, the transaction was completed normally, but in the client's point of view was canceled.

HTTP Header Fields

`Accept`

The only accept type used with JSONRequest is application/jsonrequest. The use of this unique type prevents JSONRequest from interacting with legacy systems that assumed that a firewall was sufficient to protect them from unintended web access.

`Content-Type`

The only content type used with JSONRequest is application/jsonrequest.

`Content-Encoding`

The content encoding can be identity (the default) or gzip.

Exceptions

Exceptions can be produced either when the JSONRequest function is called, or when the done callback function is invoked. An exception object contains a name member whose value will always be the string "JSONRequestError", and a message member, which contains a string that explains the error.

{name: "JSONRequestError", message: "error message"}

These are the messages that can be produced.

`message`	meaning
`"bad URL"`	The URL was not formatted correctly and could not be used to make a request.
`"bad data"`	The send data was not an object or array, or was cyclical, or was too big.
`"bad function"`	The callback function was not a function with an arity of 3.
`"bad timeout"`	The timeout parameter is not a positive integer.
`"not ok"`	The server supplied a response that was not `200` `OK`.
`"no response"`	The server did not respond, or a timeout occurred.
`"bad response"`	The response was not a valid JSON text, or had unexpected material in the HTTP response header.
`"canceled"`	The response was canceled.

Delay

When a request fails (delivering an exception object with the message "not ok", "no response", or "bad response" to a done callback function), then a delay will be added to the dispatching of all subsequent requests. This is intended to frustrate denial of service attacks, timing analysis attacks, and exhaustive search attacks.

Each failure increases the current delay value by 500 milliseconds plus a random number of milliseconds (between 0 and 511). Each successful request reduces the delay by 10 milliseconds until the delay goes to zero. Each page keeps its own delay value.

Canceled requests increase the delay by 20 milliseconds.

Security

The JSONRequest has some features that allow it to be exempted from the Same Origin Policy.

JSONRequest does not send or receive cookies or passwords in HTTP headers. This avoids false authorization situations. Knowing the name of a site does not grant the ability to use its browser credentials.
JSONRequest works only with JSON text. The JSONRequest cannot be used to access legacy data or documents or scripts. This avoids attacks on internal websites which assume that access is sufficient authorization. A request will fail if the response is not perfectly UTF-8 encoded. Suboptimal aliases and surrogates will fail. A request will fail if the response is not strictly in JSON format. A request will fail if the server does not respond to POST with a JSON payload.
Reponses will be rejected unless they contain a JSONRequest content type. This makes it impossible to use JSONRequest to obtain data from insecure legacy servers.
JSONRequest reveals very little error information. In some cases, the goal of a miscreant is to access the information that can be obtained from an error message. JSONRequest does not return this information to the requesting script. It may provide the information to the user through a log or other mechanism, but not in a form that the script can ordinarily access.
JSONRequest accumulates random delays before acting on new requests when previous requests have failed. This is to frustrate timing analysis attacks and denial of service attacks.

The JSONRequest does only one thing: It exchanges data between scripts on pages with JSON servers in the web. It provides this highly valuable service while introducing no new security vulnerabilities.

A browser within a filewall may have the capability to interact with a server (penzance.org). Computers on the outside do not have that capability. Can a computer on the outside (pirate.net) cause a browser to act as its agent in interacting with an internal server?

Current, XMLHttpRequest does not allow a script from a page from pirate.net to connect to penzance.org because of the Same Origin Policy.

JSONRequest does allow the connection, but with some limitations:

The Content-Type in both directions is application/jsonrequest.
The POST body data will be in JSON format.
The response data will be in JSON format.
The character encoding in both directions will be UTF-8, strictly enforced.

But what of legacy applications that accept POST. Could JSONRequest be used to improperly POST to these applications, thereby corrupting databases? JSONRequest mitigates this danger because Cookies and HTTP authentication are not sent.

Contrast this to form.submit, which can send a POST body and cookies and HTTP authentication. JSONRequest is more secure than the form.submit feature which is currently implemented everywhere. By switching to a policy of responding only to well-formatted JSONRequest, applications can be made more secure.

Duplex

JSONRequest is designed to support duplex connections. This permits applications in which the server can asynchronously initiate transmissions. This is done by using two simultaneous requests: one to send and the other to receive. By using the timeout parameter, a POST request can be left pending until the server determines that it has timely data to send.

Duplex connections can be used in realtime notification applications such as process management and finance. It can also be used in collaborative applications such as instant messaging, instant email, chat, games, presentation, and shared applications.