Stage 1 Draft / July 26, 2016

Template Literal Revision

Introduction#

Template literals should allow the embedding of languages (DSLs etc.). But restrictions on escape sequences make this problematic.

For example, consider making a latex processor with templates:

function latex(strings) {
  // ...
}

let document = latex`
\newcommand{\fun}{\textbf{Fun!}}  // works just fine
\newcommand{\unicode}{\textbf{Unicode!}} // Illegal token!
\newcommand{\xerxes}{\textbf{King!}} // Illegal token!

Breve over the h goes \u{h}ere // Illegal token!
`

The problem here is that \u is the start of a unicode escape, but ES grammar forces it to be of the form \u00FF or \u{42} and considers the token \unicode illegal. Similarly \x is the start of a hex escape like \xFF but \xerxes is illegal. Octal literal escapes have the same problem; \0100 is illegal.

Proposal Overview

Remove the restriction on escape sequences.

Lifting the restriction raises the question of how to handle cooked template values that contain illegal escape sequences. Currently, cooked template values are supposed to replace escape sequences with the "Unicode code point represented by the escape sequence" but this can't happen if the escape sequence is not valid.

The proposed solution is to set the cooked value to undefined for template values that contain illegal escape sequences. The raw value is still accessible via .raw so embedded DSLs that might contain undefined cooked values can just use the raw string:

function tag(strs) {
  strs[0] === undefined
  strs.raw[0] === "\\unicode and \\u{55}";
}
tag`\unicode and \u{55}`

This loosening of the escape sequence restriction only applies to tagged template literals; untagged templates still throw an early error for invalid escape sequences:

let bad = `bad escape sequence: \unicode`; // throws early error

1(11.8.6) Template Literal Lexical Components#

Syntax

Template::NoSubstitutionTemplate TemplateHead NoSubstitutionTemplate::`TemplateCharactersopt` TemplateHead::`TemplateCharactersopt${ TemplateSubstitutionTail::TemplateMiddle TemplateTail TemplateMiddle::}TemplateCharactersopt${ TemplateTail::}TemplateCharactersopt` TemplateCharacters::TemplateCharacterTemplateCharactersopt TemplateCharacter::$[lookahead ≠ {] \EscapeSequence LineContinuation LineTerminatorSequence SourceCharacterbut not one of ` or \ or $ or LineTerminator TemplateCharacter::$[lookahead ≠ {] \EscapeSequence \NotEscapeSequence LineContinuation LineTerminatorSequence SourceCharacterbut not one of ` or \ or $ or LineTerminator NotEscapeSequence::0DecimalDigit DecimalDigitbut not 0 x[lookahead ∉ HexDigit] xHexDigit[lookahead ∉ HexDigit] u[lookahead ∉ HexDigit][lookahead ≠ {] uHexDigit[lookahead ∉ HexDigit] uHexDigitHexDigit[lookahead ∉ HexDigit] uHexDigitHexDigitHexDigit[lookahead ∉ HexDigit] u{[lookahead ∉ HexDigit] u{NotCodePoint u{CodePoint[lookahead ≠ }] NotCodePoint::HexDigitsbut not if MV of HexDigits ≤ 0x10FFFF CodePoint::HexDigitsbut not if MV of HexDigits > 0x10FFFF

A conforming implementation must not use the extended definition of EscapeSequence described in B.1.2 when parsing a TemplateCharacter.

Note

TemplateSubstitutionTail is used by the InputElementTemplateTail alternative lexical goal.

2(11.8.6.1) Static Semantics: TV and TRV#

A template literal component is interpreted as a sequence of Unicode code points. The Template Value (TV) of a literal component is described in terms of code unit values (SV, 11.8.4) contributed by the various parts of the template literal component. As part of this process, some Unicode code points within the template component are interpreted as having a mathematical value (MV, 11.8.3). In determining a TV, escape sequences are replaced by the UTF-16 code unit(s) of the Unicode code point represented by the escape sequence. The Template Raw Value (TRV) is similar to a Template Value with the difference that in TRVs escape sequences are interpreted literally.

Note

TV excludes the code units of LineContinuation while TRV includes them. <CR><LF> and <CR> LineTerminatorSequences are normalized to <LF> for both TV and TRV. An explicit EscapeSequence is needed to include a <CR> or <CR><LF> sequence.

3(12.2.9) Template Literals#

Syntax

TemplateLiteral[Yield]:NoSubstitutionTemplate SubstitutionTemplate[?Yield] SubstitutionTemplate[Yield]:TemplateHeadExpression[In, ?Yield]TemplateSpans[?Yield] TemplateSpans[Yield]:TemplateTail TemplateMiddleList[?Yield]TemplateTail TemplateMiddleList[Yield]:TemplateMiddleExpression[In, ?Yield] TemplateMiddleList[?Yield]TemplateMiddleExpression[In, ?Yield] TemplateLiteral[Yield, Tagged]:NoSubstitutionTemplate SubstitutionTemplate[?Yield, ?Tagged] SubstitutionTemplate[Yield, Tagged]:TemplateHeadExpression[In, ?Yield]TemplateSpans[?Yield, ?Tagged] TemplateSpans[Yield, Tagged]:TemplateTail TemplateMiddleList[?Yield, ?Tagged]TemplateTail TemplateMiddleList[Yield, Tagged]:TemplateMiddleExpression[In, ?Yield] TemplateMiddleList[?Yield, ?Tagged]TemplateMiddleExpression[In, ?Yield]

4Static Semantics: Early Errors#

TemplateLiteral[Yield, Tagged]:NoSubstitutionTemplate SubstitutionTemplate[Yield, Tagged]:TemplateHeadExpression[In, ?Yield]TemplateSpans[?Yield, ?Tagged] TemplateSpans[Yield, Tagged]:TemplateTail TemplateMiddleList[Yield, Tagged]:TemplateMiddleExpression[In, ?Yield] TemplateMiddleList[?Yield, ?Tagged]TemplateMiddleExpression[In, ?Yield]

5(12.3) Left-Hand-Side Expressions#

Syntax

MemberExpression[Yield]:PrimaryExpression[?Yield] MemberExpression[?Yield][Expression[In, ?Yield]] MemberExpression[?Yield].IdentifierName MemberExpression[?Yield]TemplateLiteral[?Yield] MemberExpression[?Yield]TemplateLiteral[?Yield, Tagged] SuperProperty[?Yield] MetaProperty newMemberExpression[?Yield]Arguments[?Yield] CallExpression[Yield]:MemberExpression[?Yield]Arguments[?Yield] SuperCall[?Yield] CallExpression[?Yield]Arguments[?Yield] CallExpression[?Yield][Expression[?Yield]] CallExpression[?Yield].IdentifierName CallExpression[?Yield]TemplateLiteral[?Yield] CallExpression[?Yield]TemplateLiteral[?Yield, Tagged]

ACopyright & Software License#

Copyright Notice

© 2016 Tim Disney

Software License

All Software contained in this document ("Software") is protected by copyright and is being made available under the "BSD License", included below. This Software may be subject to third party rights (rights from parties other than Ecma International), including patent rights, and no licenses under such third party rights are granted under this license even if the third party concerned is a member of Ecma International. SEE THE ECMA CODE OF CONDUCT IN PATENT MATTERS AVAILABLE AT http://www.ecma-international.org/memento/codeofconduct.htm FOR INFORMATION REGARDING THE LICENSING OF PATENT CLAIMS THAT ARE REQUIRED TO IMPLEMENT ECMA INTERNATIONAL STANDARDS.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
  3. Neither the name of the authors nor Ecma International may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE ECMA INTERNATIONAL "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL ECMA INTERNATIONAL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.