WebRTC 1.0: Real-time Communication Between Browsers

Table of Contents

• 1. Introduction
• 2. Conformance
• 3. Terminology
• 4. Network Stream API
  • 4.1 Introduction
  • 4.2 Interface definitions
    • 4.2.1 MediaStream
      • 4.2.1.1 label
      • 4.2.1.2 Events on MediaStream
    • 4.2.2 MediaStreamTrack
  • 4.3 AudioMediaStreamTrack
    • 4.3.1 Attributes
    • 4.3.2 Methods
• 5. Peer-to-peer connections
  • 5.1 RTCPeerConnection
    • 5.1.1 RTCSdpType
    • 5.1.2 RTCSessionDescription Class
      • 5.1.2.1 Attributes
      • 5.1.2.2 Methods
      • 5.1.2.3 Dictionary RTCSessionDescriptionInit Members
    • 5.1.3 RTCSessionDescriptionCallback
      • 5.1.3.1 Callback RTCSessionDescriptionCallback Parameters
    • 5.1.4 RTCVoidCallback
    • 5.1.5 RTCPeerConnectionErrorCallback
      • 5.1.5.1 Callback RTCPeerConnectionErrorCallback Parameters
    • 5.1.6 RTCPeerState Enum
    • 5.1.7 RTCIceState Enum
    • 5.1.8 RTCIceCandidate Type
      • 5.1.8.1 Attributes
      • 5.1.8.2 Methods
      • 5.1.8.3 Dictionary RTCIceCandidateInit Members
    • 5.1.9 RTCIceServer Type
      • 5.1.9.1 Dictionary RTCIceServer Members
    • 5.1.10 RTCConfiguration Type
      • 5.1.10.1 Dictionary RTCConfiguration Members
    • 5.1.11 RTCPeerConnection Interface
• 6. IANA Registrations
  • 6.1 Constraints
• 7. Simple Example
• 8. Advanced Example
• 9. Call Flow Browser to Browser
• 10. Call Flow Browser to MCU
• 11. Peer-to-peer Data API
  • 11.1 DataChannel
    • 11.1.1 Attributes
    • 11.1.2 Methods
    • 11.1.3 Dictionary DataChannelInit Members
  • 11.2 Examples
• 12. Garbage collection
• 13. Event definitions
  • 13.1 RTCPeerConnectionIceEvent
    • 13.1.1 Attributes
    • 13.1.2 Dictionary RTCPeerConnectionIceEventInit Members
  • 13.2 MediaStreamEvent
    • 13.2.1 Attributes
    • 13.2.2 Dictionary MediaStreamEventInit Members
  • 13.3 DataChannelEvent
    • 13.3.1 Attributes
    • 13.3.2 Dictionary DataChannelEventInit Members
• 14. Event summary
• 15. Security Considerations
• 16. Change Log
• A. Acknowledgements
• B. References
  • B.1 Normative references
  • B.2 Informative references

2. Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The key words must, must not, required, should, should not, recommended, may, and optional in this specification are to be interpreted as described in [RFC2119].

This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.

Implementations that use ECMAScript to implement the APIs defined in this specification must implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [WEBIDL], as this specification uses that specification and terminology.

3. Terminology

The EventHandler interface represents a callback used for event handlers as defined in [HTML5].

The concepts queue a task and fires a simple event are defined in [HTML5].

The terms event handlers and event handler event types are defined in [HTML5].

4. Network Stream API

interface AudioMediaStreamTrack : MediaStreamTrack {
    readonly attribute boolean canInsertDTMF;
    void insertDTMF (DOMString tones, optional long duration);
};

5. Peer-to-peer connections

A RTCPeerConnection allows two users to communicate directly, browser to browser. Communications are coordinated via a signaling channel which is provided by unspecified means, but generally by a script in the page via the server, e.g. using XMLHttpRequest.

Calling new RTCPeerConnection(configuration ) creates a RTCPeerConnection object.

The configuration has the information to find and access the [STUN] and [TURN] servers. There may be multiple servers of each type and any TURN server also acts as a STUN server.

A RTCPeerConnection object has an associated ICE agent, RTCPeerConnection readiness state, and ICE state. These are initialized when the object is created.

When the RTCPeerConnection() constructor is invoked, the user agent must run the following steps. This algorithm has a synchronous section (which is triggered as part of the event loop algorithm).

1. Create an ICE Agent and let connection’s RTCPeerConnection ICE Agent be that ICE Agent and provide it the STUN and TURN servers from the configuration array. The [ICE] will proceed with gathering as soon as the IceTransports constraint is not set to "none". At this point the ICE Agent does not know how many ICE components it needs (and hence the number of candidates to gather) but it can make a reasonable assumption and as the RTCPeerConnection object gets more information, it can adjust the number of components.

2. Set connection’s RTCPeerConnection readiness state to new .

3. Set connection’s RTCPeerConnection ice state to new .

4. Let connection’s localStreams attribute be an empty read-only MediaStream array.

5. Let connection’s remoteStreams attribute be an empty read-only MediaStream array.

6. Return connection, but continue these steps asynchronously.

7. Await a stable state. The synchronous section consists of the remaining steps of this algorithm.

During the lifetime of the RTCPeerConnection object, the following procedures are followed:

1. If iceState is "new" and the IceTransports constraint is not set to "none", it must queue a task to start gathering ICE address and set the iceState to "gathering".

2. If the ICE Agent has found one or more candidate pairs for any MediaStreamTrack that forms a valid connection, the ICE state is changed to "connected".

3. When the ICE Agent finishes checking all candidate pairs, if at least one connection has been found for some MediaStreamTrack, the iceState is changed to "completed" and if no connection has been found for any MediaStreamTrack, the iceState is changed to "failed".

   Issue 4

   ISSUE: Note that this means that if I was able to negotiate audio but not video via ICE, then iceState == "completed". Is this really what is desired?

4. If the iceState is "connected" or "completed" and both the local and remote session descriptions are set, the RTCPeerConnection state is set to "active".

5. If the iceState is "failed", a task is queued to calls the close method.

   Issue 5

   ISSUE:: CJ - this seems wrong to me.

User agents negotiate the codec resolution, bitrate, and other media parameters. User agents are recommended to initially negotiate for the maximum resolution of a video stream. For streams that are then rendered (using a video element), user agents are recommended to renegotiate for a resolution that matches the rendered display size.

The word "components" in this context refers to an RTP media flow and does not have anything to do with how [ICE] uses the term "component".

When a user agent has reached the point where a MediaStream can be created to represent incoming components, the user agent must run the following steps:

1. Let connection be the RTCPeerConnection expecting this media.

2. Create a MediaStream object to represent the media stream.

3. Run the following steps for each component in the media stream.

   1. Create a MediaStreamTrack object track to represent the component. [[EDITORIAL: Can we just reference 3.2.1.2 here?]]

   2. If track's kind attribute equals "audio", add it to the MediaStream object's audioTracks MediaStreamTrackList object.

   3. If track's kind attribute equals "video", add it to the MediaStream object's videoTracks MediaStreamTrackList object.

   Note

   The creation of new incoming MediaStreams may be triggered either by SDP negotiation or by the receipt of media on a given flow.

   Note

   The internal order in the MediaStreamTrackList objects on the receiving side should reflect the order on the sending side. One way to enforce this is to specify the order in the SDP.

4. Queue a task to run the following substeps:

   1. If the connection’s RTCPeerConnection readiness state is CLOSED (3), abort these steps.

   2. Add the newly created MediaStream object to the end of connection’s remoteStreams array.

   3. Fire a stream event named addstream with the newly created MediaStream object at the connection object.

When a user agent has negotiated media for a component that belongs to a media stream that is already represented by an existing MediaStream object, the user agent must associate the component with that MediaStream object.

When a RTCPeerConnection finds that a stream from the remote peer has been removed , the user agent must follow these steps:

1. Let connection be the RTCPeerConnection associated with the stream being removed.

2. Let stream be the MediaStream object that represents the media stream being removed, if any. If there isn't one, then abort these steps.

3. By definition, stream is now finished.

   Note

   A task is thus queued to update stream and fire an event.

4. Queue a task to run the following substeps:

   1. If the connection’s RTCPeerConnection readiness state is closed (3), abort these steps.

   2. Remove stream from connection’s remoteStreams array.

   3. Fire a stream event named removestream with stream at the connection object.

The task source for the tasks listed in this section is the networking task source.

If something in the browser changes that causes the RTCPeerConnection object to need to initiate a new session description negotiation, an negotiationneeded event is fired at the RTCPeerConnection object.

In particular, if a RTCPeerConnection object is consuming a MediaStream and a track is added to one of the stream's MediaStreamTrackList objects, by, e.g., the add() method being invoked, the RTCPeerConnection object must fire the "negotiationneeded" event. Removal of media components must also trigger "negotiationneeded".

To prevent network sniffing from allowing a fourth party to establish a connection to a peer using the information sent out-of-band to the other peer and thus spoofing the client, the configuration information should always be transmitted using an encrypted connection.

enum RTCSdpType {
    "offer",
    "pranswer",
    "answer"
};

[Constructor (optional RTCSessionDescriptionInit descriptionInitDict)]
interface RTCSessionDescription {
             attribute RTCSdpType? type;
             attribute DOMString?  sdp;
    stringifier DOMString ();
};
dictionary RTCSessionDescriptionInit {
    RTCSdpType type;
    DOMString  sdp;
};

callback RTCSessionDescriptionCallback = void (RTCSessionDescription sdp);

callback RTCVoidCallback = void ();

callback RTCPeerConnectionErrorCallback = void (DOMString errorInformation);

enum RTCPeerState {
    "new",
    "opening",
    "active",
    "closing",
    "closed"
};

enum RTCIceState {
    "new",
    "gathering",
    "waiting",
    "checking",
    "connected",
    "completed",
    "failed",
    "closed"
};

[Constructor (optional RTCIceCandidateInit candidateInitDict)]
interface RTCIceCandidate {
             attribute DOMString?      candidate;
             attribute DOMString?      sdpMid;
             attribute unsigned short? sdpMLineIndex;
    stringifier DOMString ();
};
dictionary RTCIceCandidateInit {
    DOMString      candidate;
    DOMString      sdpMid;
    unsigned short sdpMLineIndex;
};

dictionary RTCIceServer {
    DOMString          url;
    nullable DOMString credential;
};

dictionary RTCConfiguration {
    RTCIceServer[] iceServers;
};

typedef MediaStream[] MediaStreamArray;

[Constructor (RTCConfiguration configuration, optional MediaConstraints
        constraints)]
interface RTCPeerConnection : EventTarget  {
    void        createOffer (RTCSessionDescriptionCallback successCallback, optional RTCPeerConnectionErrorCallback failureCallback, optional MediaConstraints constraints);
    void        createAnswer (RTCSessionDescription offer, RTCSessionDescriptionCallback successCallback, optional RTCPeerConnectionErrorCallback? failureCallback = null, optional optional MediaConstraints constraints = null, optional optional boolean createProvisionalAnswer = false);
    void        setLocalDescription (RTCSessionDescription description, optional RTCVoidCallback successCallback, optional RTCPeerConnectionErrorCallback failureCallback);
    readonly attribute RTCSessionDescription localDescription;
    void        setRemoteDescription (RTCSessionDescription description, optional RTCVoidCallback successCallback, optional RTCPeerConnectionErrorCallback failureCallback);
    readonly attribute RTCSessionDescription remoteDescription;
    readonly attribute RTCPeerState          readyState;
    void        updateIce (optional RTCConfiguration? configuration = null, optional optional MediaConstraints? constraints = null, optional boolean restart = false);
    void        addIceCandidate (RTCIceCandidate candidate);
    readonly attribute RTCIceState           iceState;
    readonly attribute MediaStreamArray      localStreams;
    readonly attribute MediaStreamArray      remoteStreams;
    DataChannel createDataChannel ([TreatNullAs=EmptyString] DOMString label, optional DataChannelInit dataChannelDict);
             attribute EventHandler          ondatachannel;
    void        addStream (MediaStream stream, optional MediaConstraints constraints);
    void        removeStream (MediaStream stream);
    void        close ();
             attribute EventHandler          onnegotationneeded;
             attribute EventHandler          onicecandidate;
             attribute EventHandler          onopen;
             attribute EventHandler          onstatechange;
             attribute EventHandler          onaddstream;
             attribute EventHandler          onremovestream;
             attribute EventHandler          onicechange;
};

6. IANA Registrations

IANA is requested to register the constraints defined in Constraints Section as specified in [RTCWEB-CONSTRAINTS].

7. Simple Example

var signalingChannel = createSignalingChannel();
var pc;
var configuration = ...;

// run start(true) to initiate a call
function start(isCaller) {
    pc = new RTCPeerConnection(configuration);

    // send any ice candidates to the other peer
    pc.onicecandidate = function (evt) {
        signalingChannel.send(JSON.stringify({ "candidate": evt.candidate }));
    };

    // once remote stream arrives, show it in the remote video element
    pc.onaddstream = function (evt) {
        remoteView.src = URL.createObjectURL(evt.stream);
    };

    // get the local stream, show it in the local video element and send it
    navigator.getUserMedia({ "audio": true, "video": true }, function (stream) {
        selfView.src = URL.createObjectURL(stream);
        pc.addStream(stream);

        if (isCaller)
            pc.createOffer(gotDescription);
        else
            pc.createAnswer(pc.remoteDescription, gotDescription);

        function gotDescription(desc) {
            pc.setLocalDescription(desc);
            signalingChannel.send(JSON.stringify({ "sdp": desc }));
        }
    });
}

signalingChannel.onmessage = function (evt) {
    if (!pc)
        start(false);

    var signal = JSON.parse(evt.data);
    if (signal.sdp)
        pc.setRemoteDescription(new RTCSessionDescription(signal.sdp));
    else
        pc.addIceCandidate(new RTCIceCandidate(signal.candidate));
};

8. Advanced Example

TODO

9. Call Flow Browser to Browser

This shows an example of one possible call flow between two browsers. This does not show every callback that gets fired but instead tries to reduce it down to only show the key events and messages.

The following flow show a more complete set of the callbacks and events that happen.

10. Call Flow Browser to MCU

This shows an example of one possible call flow between a centralized conferencing server and a browser. This does not show every callback that gets fired but instead tries to reduce it down to only show the key events and messages.

11. Peer-to-peer Data API

The Peer-to-peer Data API lets a web application send and receive generic application data peer-to-peer.

interface DataChannel {
    readonly attribute DOMString        label;
    readonly attribute boolean          reliable;
    readonly attribute DataChannelState readyState;
    readonly attribute unsigned long    bufferedAmount;
             attribute EventHandler     onopen;
             attribute EventHandler     onerror;
             attribute EventHandler     onclose;
    void close ();
             attribute EventHandler     onmessage;
             attribute DOMString        binaryType;
    void send (DOMString data);
    void send (ArrayBuffer data);
    void send (Blob data);
};

dictionary DataChannelInit {
    boolean reliable;
};

enum DataChannelState {
    "connecting",
    "open",
    "closing",
    "closed"
};

var chan = peerConn.createDataChannel("mylabel");
chan.onmessage = function (evt) {
    // use evt.data };
    chan.send("hello");
}

peerConn.ondatachannel = function (evt) {
   var chan = evt.channel;
   chan.onmessage = function (evt) {
       // use evt.data
   };
   chan.onclose = function () {
          // remote side closed the data channel
   };
};

12. Garbage collection

A Window object has a strong reference to any RTCPeerConnection objects created from the constructor whose global object is that Window object.

13. Event definitions

[Constructor(DOMString type, RTCPeerConnectionIceEventInit eventInitDict)]
interface RTCPeerConnectionIceEvent : Event {
    readonly attribute RTCIceCandidate candidate;
};
dictionary RTCPeerConnectionIceEventInit : EventInit {
    RTCIceCandidate candidate;
};

[Constructor(DOMString type, MediaStreamEventInit eventInitDict)]
interface MediaStreamEvent : Event {
    readonly attribute MediaStream? stream;
};
dictionary MediaStreamEventInit : EventInit {
    MediaStream stream;
};

[Constructor(DOMString type, DataChannelEventInit eventInitDict)]
interface DataChannelEvent : Event {
    readonly attribute DataChannel channel;
};
dictionary DataChannelEventInit : EventInit {
    DataChannel channel;
};

15. Security Considerations

TBD.

16. Change Log

This section will be removed before publication.

Changes since Aug 13, 2012

1. Made the RTCSessionDescription and RTCIceCandidate constructors take dictionaries instead of a strings. Also added detailed stringifier algorithm.
2. Went through the list of issues (issue numbers are only valid with HEAD at fcda53c460). Closed (fixed/wontfix): 1, 8, 10, 13, 14, 16, 18, 19, 22, 23, 24. Converted to notes: 4, 12. Updated: 9.
3. Incorporate changes proposed by Li Li.
4. Use an enum for DataChannelState and fix IDLs where using an optional argument also requires all previous optional arguments to have a default value.

Changes since Jul 20, 2012

1. Added RTC Prefix to names (including the notes below).
2. Moved to new defintion of configuration and ice servers object.
3. Added correlating lines to candidate structure.
4. Converted setLocalDescription and setRemoteDescription to be asynchronous.
5. Added call flows.

Changes since Jul 13, 2012

1. Removed peer attribute from RTCPeerConnectionIceEvent (duplicates functionality of Event.target attribute).
2. Removed RTCIceCandidateCallback (no longer used).
3. Removed RTCPeerConnectionEvent (we use a simple event instead).
4. Removed RTCSdpType argument from setLocalDescription() and setRemoteDescription(). Updated simple example to match.

Changes since May 28, 2012

1. Changed names to use RTC Prefix.
2. Changed the data structure used to pass in STUN and TURN servers in configuration.
3. Updated simple RTCPeerConnection example (RTCPeerConnection constructor arguments; use icecandidate event).
4. Initial import of new Data API.
5. Removed some left-overs from the old Data Stream API.
6. Renamed "underlying data channel" to "underlying data transport". Fixed closing procedures. Fixed some typos.

Changes since April 27, 2012

1. Major rewrite of RTCPeerConnection section to line up with IETF JSEP draft.
2. Added simple RTCPeerConnection example. Initial update of RTCSessionDescription and RTCIceCandidate to support serialization and construction.

Changes since 21 April 2012

1. Moved MediaStream and related definitions to getUserMedia.
2. Removed section "Obtaining local multimedia content".
3. Updated getUserMedia() calls in examples (changes in Media Capture TF spec).
4. Introduced MediaStreamTrackList interface with support for adding and removing tracks.
5. Updated the algorithm that is run when RTCPeerConnection receives a stream (create new stream when negotiated instead of when data arrives).

Changes since 12 January 2012

1. Clarified the relation of Stream, Track, and Channel.

Changes since 17 October 2011

1. Tweak the introduction text and add a reference to the IETF RTCWEB group.
2. Changed the first argument to getUserMedia to be an object.
3. Added a MediaStreamHints object as a second argument to RTCPeerConnection.addStream.
4. Added AudioMediaStreamTrack class and DTMF interface.

Changes since 23 August 2011

1. Separated the SDP and ICE Agent into separate agents and added explicit state attributes for each.
2. Removed the send method from PeerConenction and associated callback function.
3. Modified MediaStream() constructor to take a list of MediaStreamTrack objects instead of a MediaStream. Removed text about MediaStream parent and child relationship.
4. Added abstract.
5. Moved a few paragraphs from the MediaStreamTrack.label section to the MediaStream.label section (where they belong).
6. Split MediaStream.tracks into MediaStream.audioTracks and MediaStream.videoTracks.
7. Removed a sentence that implied that track access is limited to LocalMediaStream.
8. Updated a few getUserMedia()-examples to use MediaStreamOptions.
9. Replaced calls to URL.getObjectURL() with URL.createObjectURL() in example code.
10. Fixed some broken getUserMedia() links.
11. Introduced state handling on MediaStreamTrack (removed state handling from MediaStream).
12. Reintroduced onended on MediaStream to simplify checking if all tracks are ended.
13. Aligned the MediaStreamTrack ended event dispatching behavior with that of MediaStream.
14. Updated the LocalMediaStream.stop() algorithm to implicitly use the end track algorithm.
15. Replaced an occurrence the term finished track with ended track (to align with rest of spec).
16. Moved (and extended) the explanation about track references and media sources from LocalMediaStream to MediaStreamTrack.