XProc V2.0 Requirements

2.1 Simplify parameters

Experience with parameters in XProc 1.0 reveals that they are too complicated. They often cause user confusion and introduce syntactic complexity not justified by their function. XProc v2.0 must dramatically simplify parameters, perhaps simply removing parameter ports altogether without replacing them with a new mechanism of equivalent power (and complexity).

2.2 Integrate non-XML documents into pipelines

Experience has shown that real-world pipelines often involve non-XML documents. Several workarounds have been invented for special cases. The limitation that V1.0 can only pass XML between steps makes some pipelines difficult, if not impossible, to write.

Providing the ability to allow non-XML documents to flow between steps opens up the possibility of writing simple pipelines to work with images, JSON, Turtle, EPUB, etc.

2.3 Align with XPath 3.0 technologies

Alignment with [XQuery 3.0]/[XSLT 3.0] will keep features of XProc consistent with modern XML technologies: error handling, serialization options, [XDM] features, etc. In addition, support for XPath 1.0 no longer seems relevant; it adds complexity to the specification and is unlikely to be implemented today. XPath 1.0 support will be removed from XProc.

2.4 Add explicit flow handling

There are many pipelines for which the flow analysis does not provide a convenient or predictable ordering of steps. Because some steps have side effects not manifest in the pipeline, it may be necessary to ensure a particular order. This facility is not supported by XProc 1.0, but is available in implementation-defined extensions. XProc 2.0 will standardize this facility.

2.5 Allow arbitrary XDM values in variables

XProc 1.0 restricts the values of variables, options, and parameters to be only strings. This has proven to be an inconvenient limitation. XProc 2.0 will allow variables, options, and parameters to have any [XDM] value insofar as possible. XProc 2.0 will also allow the required types of variables, options, and parameters to be specified.

2.6 Allow attribute value templates

The syntactic sugar that allows step options to be expressed concisely as attribute values on a step is foiled whenever the value of the option must be computed by the pipeline. Allowing those options to contain XSLT-style attribute value templates (AVTs) would simplify many pipelines. Additionally, allowing AVTs in other places, such as the href attribute on p:document, will be considered.

[XSLT 3.0] introduces a feature which allows expressions in curly braces to be evaluated in element content. This feature is similar to the facility provided by the p:template step. Extending XProc to support curly braces in a manner consistent with [XSLT 3.0] will be considered.

2.7 Support a variety of syntactic simplifications

XProc 1.0 offers relatively few default behaviors, requiring instead that pipelines specify every construct fully. User experience has demonstrated that this leads to very verbose pipelines and has been a constant source of complaint. XProc 2.0 will introduce a variety of syntactic simplifications as an aid to readability and usability, including but not limited to:

2.8 Document backwards-incompatibilities

Backwards incompatiblity is painful for users and will be avoided wherever possible. However, XProc 2.0 will introduce language features that are not backwards compatible with 1.0. The specification must document these incompatibilities.

3.1 Editorial improvements

Implementation experience has demonstrated that there are areas of the specification that didn't get the balance right between precision for implementors and clarity for users, for example “non-step wrappers”. The XProc 2.0 specification should attempt to resolve these problems without introducing inordinate complexity.

The 1.0 specification also defines the p:pipeline element as a syntactic shortcut for a particular form of p:declare-step. While convenient in some circumstances, it has proven to be a source of some confusion especially among new users. XProc 2.0 may remove the p:pipeline element.

3.2 Associate arbitrary metadata with documents

Adding metadata to documents is a natural thing for pipelines to do, either for subsequent use by the pipeline or for eventual output. For example, the serialization options provided in an XSLT stylesheet could be carried forward to the eventual serialization of the result document by the pipeline. In XProc 1.0, there's no way to maintain that association. XProc 2.0 should support the ability to associate processor and user-defined metadata with documents.

3.3 Support steps with a dynamic number of ports

While most steps have a predetermined and static number of inputs and outputs, this is not universally the case. In XProc 1.0, a putative p:eval step which could run a dynamically constructed pipeline, for example, suffers from the limitation that the signature of the p:eval step usually differs from the signature of the evaluated pipeline.

XProc 2.0 should provide a facility for supporting steps with a variable number of inputs and outputs.

3.4 Provide improved status information

XProc 1.0 provides scant support for reporting the status of a pipeline and providing aid to users attempt to debug pipelines. Implementation-defined extensions have demonstrated that some additional facilities, such as a p:message step, would be an aid to users.

XProc 2.0 will add some mechanism for reporting status messages and will consider adding additional steps and/or language features to aid in analysing the behavior of a running pipeline.

3.5 Provide a mechanism for importing user-defined functions

Experience with user-defined functions in XQuery and XSLT reveals that they can be a powerful addition to the language. Providing some feature that allowed users to extend the vocabulary of functions available in, for example, the test expressions on p:when elements would greatly simplify some pipelines.

Such a mechanism might take the form of the ability to load extension functions defined in, for example, XQuery, or it might include adding the ability to define functions in XProc.

3.6 Enhance try/catch

Support for catching errors in XProc 1.0 is limited to a simple p:try/p:catch pair, which catches and handles all errors uniformly. To align XProc with modern languages, the try/catch mechanism will be extended to support the ability to catch specific errors and possibly with the addition of a “finally” construct.

3.7 Write a primer

A new user introduction to XProc would aid adoption.

3.8 Consider using XDM everywhere

In addition to supporting [XDM] values in variables, options, and parameters, XProc 2.0 might allow [XDM] values in more places, such as allowing p:for-each to iterate over a sequence of strings or integers.

3.9 Consider dividing the specification

XProc 1.0 is a specification that consists of both the language definition and the inventory of required and optional steps. Release management might be simplified by separating the language core from the vocabulary of steps and providing some sort of versioning strategy that allowed the vocabulary of steps to be revised more frequently. XProc 2.0 may be defined in more than one Rec-track specification document.

3.10 Consider additional steps and enhancements

The vocabulary of steps available in XProc is extensible. Users and implementors have developed additional steps. For example, to support pipelines that produce EPUB documents or manipulate files on disk. It is worth considering which, if any, new steps should be elevated to the XProc namespace. The candidates include, but are not limited to:

3.11 Simplify p:viewport and allow it to have multiple outputs

The use of an optional, single p:output binding in p:viewport creates confusion for users. The binding is used both to connect the inner workings of the viewport and as the name of the output port as seen from the outside.

In addition, the fact that viewport can produce only a single result means that for some tasks, multiple passes are required, using a combination of p:viewport and p:for-each. Consider the task of changing image references in an XHTML document from .svg to .png and generating the sequence of .png images. In XProc 1.0, this requires a p:viewport and a p:for-each.

Adding an explicit p:viewport-result allows us to remove the confusion between the input and the name of the output. Allowing multiple outputs allows us to collapse the p:viewport and p:for-each logic into a single step.

<p:viewport
  name? = NCName
  match = XSLTMatchPattern>
    ((p:viewport-source? &
      p:viewport-result? &
      p:output* &
      p:log?),
     subpipeline)
</p:viewport>

The viewport-result connects the transformation inside the viewport back into the source document over which viewport is operating. The transformed document always appears on a port named 'result'. Any other outputs are simply sequences analagous to p:for-each. It's a static error to name one of those outputs 'result'.

3.12 Provide a way to specify the base URI of a document

The base URI of a document created by the p:inline element is the base URI of the p:inline element. Specifying an xml:base attribute on the root element of the document does not help as that only applies to that element and its decendants.

Additionally, in some pipelines, it is desirable to be able to change the base URI of documents produced by other steps. No convenient mechanism exists in XProc V1.0 to satisfy these requirements.