Waiting on plotly/plotly.py#4926

No new release forthcoming so far :(

xref: plotly/Kaleido#242

Hi thread! @stefanv

There are new pre-releases slated on github of kaleido: https://github.com/plotly/Kaleido/tree/latest-tag

Once Kaleido >= v1.0.0rc1 is integrated into plotly.py (for figure.write_figure(PATH) shortcut API), it will be released as v1.0.0 official.

But the base Kaleido API is stable and can also be used like this:

import kaleido
kaleido.write_fig_sync(some_figure, path="./output./")

Here are some more examples:
https://github.com/plotly/Kaleido/blob/master/src/py/docs/examples.md
The new API enables multi-processor and more memory efficient processing, which may be of interest for this project.

The new kaleido release requires a chromium-ish browser (chromium, chrome, edge) to be installed and accesible.
This is also possible from a CLI that comes with kaleido:

kaleido_get_chrome --help

That command also uses tested chromiums in case new versions introduce bugs (they do sometimes).

Feel free to @ me for any help.

Thansk @ayjayt! Are you planning to make a new RC release on PyPI so we can try it?

If I convince the rest of the team the documentation that I wrote is sufficient for a rc I think we can push the next rc to pypi, I can bring it up tomorrow.

The rc will get bumped to a full release once plotly.py finishes integration and the technical writer approves, I imagine.

There is also a branch on github called latest-tag which follows the latest release (pre or full), so any eager users can install with:

pip install git+https://github.com/plotly/kaleido@latest-tag#subdirectory=src/py

Testing directly from gh is a bit finicky with our CI, but the last two commits here should take care of it.

@ayjayt Doesn't look like the plotly backend is picking up on kaleido rc6? It still talks about installing orca:

ImportError: orca and psutil are required to use the `sphinx-gallery-orca` renderer. See https://plotly.com/python/static-image-export/ for instructions on how to install orca. Alternatively, you can use the `sphinx-gallery` renderer (note that png thumbnails can only be generated with the `sphinx-gallery-orca` renderer).

See the failed "Test docs" CI entry above.

Yes exactly, that's what I've failed to articulate.

The kaleido api is stable in kaleido>=v1.0.0rc1 as such:

import kaleido
kaleido.write_fig_sync(fig, path)

But until the plotly integration is done, this won't work:

import plotly
fig = ...
fig.write_image(path)

Once the plotly.py integration is finished and this shortcut function is back, v1.0.0 will be released.

@stefanv, the kaleido api (not the plotly.Figure.write_image() shortcut) will be much faster and more memory efficient for batch processing and I imagine it will be a better choice for this project, but I'm assuming there are quite a few images here.

My personal goal in this exercise was to reasonably enable making movies from plotly figures.

Plotly animations work for simple graphs, but the new kaleido allows to output 100's of frames (without OOM killer, and in a reasonable timeframe) and string them into an MP4, create data movies:

That's awesome :)

My goal is just to get the skimage docs to build again, without pinning an ancient version of kaleido, which is not currently possible :(

@stefanv I can probably fix the PR, give me a day, I should be able to do it in <15 minutes with a grep

Well, I think we don't want to use unintuitive or non-standard API in our examples, so we may need to wait for the plotly integration, or figure out a backend hack.

I'll keep you posted

Thanks!

@ayjayt Any update on the kaleido / plotly integration and release?

Hi @stefanv, there is plotly/plotly.py#5062 which is slated for 6.1.0 AFAIK (this month)

I will be pushing forward on some of their tasks today, let me know if there is anything I can do for you in the meantime.

No go with latest plotly.

@ayjayt Are you still interested in this problem? The current failure is:

        AttributeError: module 'kaleido' has no attribute 'calc_fig_sync'

Yes this is still in progress! Some newer releases are out and official releases are just making their way through the pipeline. It's mainly fixing up documentation and integrating with Plotly! I'll give a more thorough review of issues this evening.

I wouldn't install with the package manager, I would install with our CLI tool since its the last known good version.

If I have access I can debug this tomorrow, I'm working on this stuff at a pycon sprint.

Alright, I'm in this now.

I have to:

• Modify workflow files for consistency (slim builds can be tough)
• Enable verbose logging in GHA
• Capture un-serializable object and either add encoder or switch serializer

Here's where my work will be: #7803

(some of this stuff is platform dependent, so I need to dump stuff in the CI, some of this stuff is not, so I can work on it locally if I can reproduce it)

Thanks, @ayjayt, appreciated. Currently, the error messages in CI hide much of the specifics of what's going wrong. It'd be good to expose at least some of that by default.

im going to add an environmental variable or check for one that circle ci/gha set. Chromium is very noisy!

We use pytest, which captures all logging output and dumps it if theres is an error. It would be nice if all CI tools did that.

choreographer.browsers._errors.BrowserFailedError: ('The browser seemed to close immediately after starting.', 'You can set the logging.Logger level lower to see more output.', 'You may try installing a known working copy of Chrome by running ', '$ choreo_get_chrome. It may be your copy auto-updated.')

I don't really like that it seems to be so picky about the Chrome version required.

choreographer.browsers._errors.BrowserFailedError: ('The browser seemed to close immediately after starting.', 'You can set the logging.Logger level lower to see more output.', 'You may try installing a known working copy of Chrome by running ', '$ choreo_get_chrome. It may be your copy auto-updated.')

I don't really like that it seems to be so picky about the Chrome version required.

Neither am I.

Plotly used to build and ship a chrome engine inside kaleido but that become infeasible when the binary got too big to host on package distributors.

I am building testing and a possible long term solution for this but it's sort of intractable, given browsers are our #1 interactive rendering engines and nobody ships a browser architected to use it in parts.

That said, I don't think linux-arm has a significant user base for the package. I'm not sure what you're seeing, but we don't get ANY issues for it outside of this one. And I would assume Google feels the same way, or else they would be offering a version of it for testing.

I could be wrong though, and I am still eyeing ways to solve it.

Maybe could try and debug the issue on a Raspberry Pi? I'd recommend adding an M.2 NVME hat, it really helps with disk access.