The Piqi Project

Table of Contents

The "Short-term goals" section contains the list of features that are planned for the next release.

The "Mid-term goals" section lists things I consider working on in some not-so-distant future. If you would like to see something happening sooner rather than later, send me an email and I’ll adjust the priorities based on your request. Contributions are very welcome as well!

The remaining sections form an incomplete list of high-level ideas for the Piqi project with no particular priorities or time estimates in mind.

1. Short-term goals

1.1. Refactor command-line API

Piqi command-line API is due for refactoring. It has been evolving for over 3 years now and accumulated a fair number of inconsistencies.

1.2. Refactor piqi server interfaces

Following the command-line API refactoring, piqi-server interfaces need to go through an overhaul. The goal is to make them stable yet flexible enough for the next generation of bindings for various new languages.

1.3. Even better Protocol Buffers compatibility for Erlang and OCaml mappings

• Handle unknown enum and variant field values like in Protocol Buffers: treat them as if the field value was missing.

1.4. Packaging

in progress

Pre-built Piqi binaries are provided for all major platforms.

It would be useful to have them packaged using native package managers, such as Debian packages for Debian Linux or Homebrew for Mac OS X.

(Debian and RPM packages are ready).

2. Mid-term goals

2.1. Dynamic Protocol Buffers style extensions

At the moment, Piqi supports only static syntax-based extensions. Although static extensions can be used as a substitute for dynamic record extensions, sometimes, it makes things logistically harder than necessary than if Piqi supported dynamic execution-time extensions.

The need for dynamic extensions usually arises in layered applications which protocols span several independent abstraction layers.

2.2. Lexical support for multi-line string and binary literals in Piq

Currently, Piq doesn’t support multi-line string and binary literals. Wrapped strings don’t look pretty and they are not very comfortable to deal with when viewing or editing.

There should also be a way to pretty-print string literals so that they fit into 80 (or whatever number) of columns in order to prevent wrapping.

2.3. Support for message definitions in the Piqi language

Piqi supports definition of functions which is a model for two-way synchronous request-reply communication. This is a very useful abstraction. For instance, Piqi-RPC heavily relies on it.

However, support for communication patterns is incomplete without the ability to define a simple asynchronous one-way communication often regarded as messages.

3. Future ideas

3.1. Piqi support for other programming languages

Starting from 0.6.0 release, portable Piqi self-specification became stable enough to be used by various applications that want to work with .piqi modules.

One possible application is adding Piqi support for other programming languages. For instance, it will be straightforward to implement Piqi compiler backends in practically any language. This way, Piqi-Erlang compiler can be written in Erlang, Piqi-Haskell compiler — in Haskell, Piqi-C compiler in Python, etc.

Several potential candidates include: Haskell, Clojure, F#, C, JavaScipt and Go.

3.2. Piqdoc — Piq document markup language

Piq/Piqi is ready to support semi-structured data and there is a compelling application for it — documenting Piqi modules.

Initially Piqdoc will support a minimum set of features equivalent to Markdown and implement only HTML output. But unlike Markdown, Restructured text, Ascii-doc and other similar systems, it will not rely on ASCII-based abbreviations and decorations. It will be extensible and somewhat similar to Scribble

3.3. Piq language extensions

Piq syntax and data model are specially designed to allow easy addition of new language features.

3.3.1. Protoquery — data query language for Piq

Piqi was originally started as a different project named Protoquery. The idea was to make a query language/engine for Google Protocol Buffers but it hasn’t been finished as the project’s direction had changed.

In the meantime, the original idea is becoming even more interesting, because Piqi offers more general and elegant data model compared to Protocol Buffers. It would be quite exciting to develop a query language around Piqi data model.

The first step would be to implement the ability to locate and match Piq objects and sub-objects using pattern matching and path expressions. There is a mailing list conversation that contains some more information about it.

3.3.2. Macro-system and limited execution model

Once we have a foreign function interface for Protoquery, it would be interesting to implement Lisp-like macros and some limited execution model for the Piq language.

3.4. The Piq programming language

There is a chance that Piq evolves to a fully-featured (functional) programming language. At least I would love to have a programming language that natively supports Piqi data model.

Last modified: April 27, 2014