Page MenuHomePhabricator

Investigate alternatives to polestar in WDQS UI
Open, Needs TriagePublic

Description

Acceptance criteria:

  • a) Look into vega-lite (see T291928#10323465) and decide on it as a solution based on resources (effort, time, people) required
    • note that in a github comment one of the maintainers mentions that it could be done "easily" in a month or so
  • If point a) doesn't seem viable, make a list of alternatives that would be better served to solve the issue
  • Most important features required would be - the ability to build a graph with multiple variables, exporting the visualization and ideally being able to drag & drop build the visualization

Timebox: 2 days max

Event Timeline

As of 22nd May 2017, Polestar is no longer maintained (455f0f81). The suggested alternative, Voyager, is also no longer actively maintained according to this GitHub issue. A comment on that GitHub issue claims that "You could probably build your own version of Voyager without too much effort (i.e. a month of work or so) if you only need a specific feature set". I therefore didn't spend any more time looking into Voyager.

There are a lot of JavaScript graphing libraries out there such as D3.js and Plotly, however none of them have any "graph builder" features and would require a similar amount of effort as reimplementing Voyager with Vega-Lite.

The best alternative to Polestar/Voyager that I have found is RAWGraphs (Website, GitHub). The React web app (app.rawgraphs.io) allows you to load data, choose a chart type, chose mappings between data columns and chart dimensions via a drag and drop interface, customize the chart, and export in a variety of formats. The most interesting thing about the RAWGraphs web app is that you can load data via a SPARQL query, and https://query.wikidata.org/sparql is the default SPARQL endpoint.

Since RAWGraphs exists, I question whether there is much benefit in WMDE also maintaining a graph builder as part of the query service. Especially given that the current Polestar implementation was added without product input and the Query Service UI supports several other graph types. The Unix philosophy of do one thing and do it well comes to mind.

It might be useful to have a chat with the RAWGraph developers (there's an email address on their website) - they might have some useful insights and metrics, such as how often RAWGraph is used with different query services.

The workflow docs give a good overview of how the process of creating a visualization in the rawgraphs-app, and how the rawgraphs-core library and the rawgraphs-charts repository of chart implementations are used. I also created a jsfiddle that uses rawgraphs-core and rawgraphs-charts to display a simple barchart to help demonstrate what is provided and what we would have to implement.

  • 1a) data loading - this is already implemented in the Query Service UI, all that would need to be done is replace what is in the data variable with the result of the user's query.
  • 1b) defining data types - rawgraphs-core provides a method for inferring the data types of the data, but we would also need to implement a way for the user to change data types.
  • 2) visual model selection - we would need to implement a way for the user to select what chart (from the ones available in rawgraphs-charts) they wanted to build.
  • 3) mapping - we would need to implement a way for the user to select which data columns map to chart dimensions.
  • 4) customization - the charts in rawgraphs-charts defines what customization options are available for each chart, we would need to implement an interface to display these customization options to the user and update the chart if an option changes.
  • 5) export - the workflow docs claim this is handled by rawgraphs-core, however from what I can tell this is done via the web app with little help from the core library. This doesn't look too difficult, though.

I estimate that the team could have a proof of concept implemented during one prototype sprint.

Other things to consider:

  • RAWGraphs doesn't seem to have been worked on very actively in the past few years - an argument could be made for spending more time reimplementing the graph builder using a more popular graphing library that is likely to continue to receive support, even if that is extra work.
  • From my limited experiments, Polestar only seems to be able to create bar and line charts.
  • I notice that T370415: [EPIC] [QS-GUI] Modernize the Query Service Client into an updated tech stack exists. If it is decided to reimplement the graph builder functionality in the Query Service UI, it would make sense to do it after this epic is done to avoid duplicate work.
  • If we use RAWGraphs to reimplement the graph builder, would we want to also use it for the other graph types the Query Service UI supports as well?

Thanks @Ollie.Shotton_WMDE! Before I tag the other PMs, here's what I understand and think . The two options are:

  1. Remove the graph capability and somehow refer people to go to RAWGraphs - this is the "easiest" and we won't completely leave people who do use it without an alternative
  2. Put too much effort (imho) into making it fit into the QS with no guarantee that it will be maintained long term (or putting even more effort into finding other such graphing libraries which in turn might also stop being maintained and so on).

We're also blocked here by the ticket you linked above and we have no clue when that will be worked on

Yeah, I agree with your two points, with one small caveat. There is always a risk that a library you depend on stops being maintained. We just have to be aware of that and make informed decisions. The likelihood that a well established graphing library, like D3.js or similar, stops being maintained any time soon is low. The likelihood that RAWGraph stops being maintained is higher, but the benefit is that it would take us less time to implement a graph builder. We would have to weigh up that risk with the (potential) extra effort required to either reimplement or maintain should it be abandoned.

I'll also add a couple of extra points, which have since come to mind:

  • Removing the graph builder might provide us with useful contact and input from affected users. Even better might be to add a banner above the existing graph builder saying something along the lines of "this feature will be removed on XYZ date, if you require the features provided by the graph builder, please let us know via <insert Phab ticket, Wiki page, etc>. Otherwise, RAWGraph is a good open source alternative.".
  • We don't have to reinvent the wheel - we could take on the maintenance of one of the existing tools, e.g. Voyager or RAWGraph (if it turns out it is no longer maintained), if the codebase is in a good state and that would be less effort than reimplementing.

Tagging @Lydia_Pintscher and @Anton.Kokh since you both are stakeholders. Please let me know if you'd prefer one of the two options and we can chat about it in Jan when y'all are back.