REANA 0.9.3 is a minor update that upgrades Snakemake workflow engine to version 7, improves job submission performance for massively-parallel workflows, improves the clean-up processing for stopped and failed workflows, and brings other minor improvements and bug fixes.

Please see the REANA 0.9.3 release blog post for more information.

• [Build] helm: add support for Kubernetes 1.29 (#775) (ae90500)
• [Features] helm: add value to customise env vars of job controller (#781) (634691f)
• [Features] helm: add value to customise PostgreSQL docker image (#774) (07a191f), closes #773
• [Features] helm: add value to customise URL of privacy notice (#778) (650ddbd)
• [Features] helm: add values to customise env vars of workflow engines (#781) (35ee032)
• [Features] helm: use PostgreSQL 14.10 in local dev deployment (#774) (43ead8a), closes #744
• [Features] reana-dev: add git-aggregate-changelog (#789) (6210b11)
• [Features] reana-dev: add date to commits bumping dependencies (#787) (a4cb84c)
• [Bug fixes] reana-dev: add PR number in commits bumping shared modules (#783) (57c6755)
• [Bug fixes] reana-dev: create commits that conform to conventional style (#777) (86d0133)
• [Bug fixes] reana-dev: delete extra files with git-submodule --update (#764) (e5680ce)
• [Bug fixes] reana-dev: detect number of already open PR in commit suffix (#783) (c533c34)
• [Bug fixes] reana-dev: update container image labels when releasing (#765) (fe6bc2c)
• [Code refactoring] docs: move from reST to Markdown (#776) (79aedb9)
• [Code style] black: format with black v24 (#772) (311e157)
• [Continuous integration] commitlint: addition of commit message linter (#767) (be77666)
• [Continuous integration] commitlint: allow release commit style (#785) (a6f95ac)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#771) (2c34634)
• [Continuous integration] release-please: initial configuration (#767) (bb45539)
• [Continuous integration] release-please: update version in Helm Chart (#770) (09c9210)
• [Continuous integration] shellcheck: fix exit code propagation (#771) (035d51c)
• [Documentation] authors: complete list of contributors (#779) (123eae8)
• [Documentation] chat: remove Gitter (#782) (aba8ac2)

• [Build] appimage: upgrade to Python 3.8.18 (#704) (783c17a)
• [Bug fixes] status: display correct duration of stopped workflows (#701) (b53def8), closes #699
• [Code refactoring] docs: move from reST to Markdown (#703) (c9c4d53)
• [Code style] black: format with black v24 (#702) (02dc830)
• [Test suite] snakemake: allow running Snakemake 7 tests on Python 3.11+ (#700) (8ad7ff1), closes #655
• [Continuous integration] commitlint: addition of commit message linter (#695) (2de7d61)
• [Continuous integration] commitlint: allow release commit style (#708) (f552752)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#698) (fa5b7c7)
• [Continuous integration] pytest: install tests package variant instead of all (#703) (fe0b00a)
• [Continuous integration] release-please: initial configuration (#695) (5b278f1)
• [Continuous integration] shellcheck: fix exit code propagation (#698) (fe696ea)
• [Documentation] authors: complete list of contributors (#705) (875997c)

• [Build] python: change extra names to comply with PEP 685 (#446) (9dad6da)
• [Build] python: require smart-open<7 for Python 3.6 (#446) (17fd581)
• [Build] python: restore snakemake reports extra (#446) (904178f)
• [Continuous integration] commitlint: allow release commit style (#447) (1208ccf)

• [Build] snakemake: require pulp<2.8.0 (#444) (5daa109)
• [Documentation] authors: complete list of contributors (#442) (4a74c10)

• [Features] config: allow customisation of runtime group name (#440) (5cec305)
• [Features] snakemake: upgrade to Snakemake 7.32.4 (#435) (20ae9ce)
• [Bug fixes] cache: handle deleted files when calculating access times (#437) (698900f)
• [Code refactoring] docs: move from reST to Markdown (#441) (36ce4e0)
• [Continuous integration] commitlint: addition of commit message linter (#432) (a67906f)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#438) (d3035dc)
• [Continuous integration] release-please: initial configuration (#432) (687f2f4)
• [Continuous integration] shellcheck: check all shell scripts recursively (#436) (709a685)
• [Continuous integration] shellcheck: fix exit code propagation (#438) (85d9a2a)

• Fixes installation by pinning bravado-core to versions lower than 6.1.1.

• [Code refactoring] docs: move from reST to Markdown (#225) (b48eb55)
• [Code style] black: format with black v24 (#224) (cc60522)
• [Continuous integration] commitlint: addition of commit message linter (#218) (ee0f7e5)
• [Continuous integration] commitlint: allow release commit style (#229) (adf15d7)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#223) (3d513f6)
• [Continuous integration] pytest: move to PostgreSQL 14.10 (#226) (4dac889)
• [Continuous integration] release-please: initial configuration (#218) (7c616d6)
• [Continuous integration] shellcheck: fix exit code propagation (#223) (b62ee1e)
• [Documentation] authors: complete list of contributors (#227) (3fbcf65)

• [Build] certificates: update expired CERN Grid CA certificate (#440) (8d6539a), closes #439
• [Build] docker: non-editable submodules in "latest" mode (#416) (3bdda63)
• [Build] python: bump all required packages as of 2024-03-04 (#442) (de119eb)
• [Build] python: bump shared REANA packages as of 2024-03-04 (#442) (fc77628)
• [Features] shutdown: stop all running jobs before stopping workflow (#423) (866675b)
• [Bug fixes] database: limit the number of open database connections (#437) (980f749)
• [Performance improvements] cache: avoid caching jobs when the cache is disabled (#435) (553468f), closes #422
• [Code refactoring] db: set job status also in the main database (#423) (9d6fc99)
• [Code refactoring] docs: move from reST to Markdown (#428) (4732884)
• [Code refactoring] monitor: centralise logs and status updates (#423) (3685b01)
• [Code refactoring] monitor: move fetching of logs to job-manager (#423) (1fc117e)
• [Code style] black: format with black v24 (#426) (8a2757e)
• [Continuous integration] commitlint: addition of commit message linter (#417) (f547d3b)
• [Continuous integration] commitlint: allow release commit style (#443) (0fc9794)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#425) (35bc1c5)
• [Continuous integration] pytest: move to PostgreSQL 14.10 (#429) (42622fa)
• [Continuous integration] release-please: initial configuration (#417) (fca6f74)
• [Continuous integration] release-please: update version in Dockerfile/OpenAPI specs (#421) (e6742f2)
• [Continuous integration] shellcheck: fix exit code propagation (#425) (8e74a85)
• [Documentation] authors: complete list of contributors (#434) (b9f8364)

• [Bug fixes] startup: handle signals for graceful shutdown (#59) (abb8969)
• [Code refactoring] docs: move from reST to Markdown (#65) (e5bd869)
• [Continuous integration] commitlint: addition of commit message linter (#60) (a9ee4bb)
• [Continuous integration] commitlint: allow release commit style (#67) (600aa01)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#64) (8283064)
• [Continuous integration] release-please: initial configuration (#60) (02b3595)
• [Continuous integration] release-please: update version in Dockerfile (#63) (548f3e1)
• [Continuous integration] shellcheck: fix exit code propagation (#64) (8a2a7fc)
• [Documentation] authors: complete list of contributors (#66) (56fbc8c)

• [Build] deps: pin invenio-userprofiles to 1.2.4 (#665) (d6cb168)
• [Build] docker: non-editable submodules in "latest" mode (#656) (d16fefb)
• [Build] python: bump all required packages as of 2024-03-04 (#674) (f40b82f)
• [Build] python: bump shared modules (#676) (47ad3ca)
• [Build] python: bump shared REANA packages as of 2024-03-04 (#674) (aa18394)
• [Code refactoring] docs: move from reST to Markdown (#671) (b6d1799)
• [Code style] black: format with black v24 (#670) (6d2b898)
• [Continuous integration] commitlint: addition of commit message linter (#665) (2b43ecc)
• [Continuous integration] commitlint: allow release commit style (#675) (e0299ef)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#669) (87c6145)
• [Continuous integration] pytest: move to PostgreSQL 14.10 (#672) (e888ddd)
• [Continuous integration] release-please: initial configuration (#665) (1d5e7c5)
• [Continuous integration] release-please: update version in Dockerfile/OpenAPI specs (#668) (3b3dc41)
• [Continuous integration] shellcheck: fix exit code propagation (#669) (d7eac6b)
• [Documentation] authors: complete list of contributors (#673) (71b3f38)

• [Build] package: require jsroot<7.6.0 (#399) (d53b290)
• [Build] package: update yarn.lock (#399) (10e41b1)
• [Features] footer: link privacy notice to configured URL (#393) (f0edde6), closes #392
• [Bug fixes] launcher: remove dollar sign in generated Markdown (#389) (8ad4afd)
• [Bug fixes] progress: update failed workflows duration using finish time (#387) (809fdc5), closes #386
• [Bug fixes] router: show 404 page for invalid URLs (#382) (c18e81d), closes #379
• [Code refactoring] docs: move from reST to Markdown (#391) (8d58277)
• [Continuous integration] commitlint: addition of commit message linter (#380) (1c9ec74)
• [Continuous integration] commitlint: allow release commit style (#400) (426a2b0)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#390) (e938f60)
• [Continuous integration] release-please: initial configuration (#380) (db2e82b)
• [Continuous integration] release-please: switch to simple release strategy (#383) (2c64085)
• [Continuous integration] release-please: update version in package.json and Dockerfile (#385) (5d232af)
• [Continuous integration] shellcheck: exclude node_modules from the analyzed paths (#387) (8913e4d)
• [Continuous integration] shellcheck: fix exit code propagation (#390) (7b5f29e)
• [Documentation] authors: complete list of contributors (#396) (814d68e)

• [Build] docker: non-editable submodules in "latest" mode (#551) (af74d0b)
• [Build] python: bump all required packages as of 2024-03-04 (#574) (1373f4c)
• [Build] python: bump shared REANA packages as of 2024-03-04 (#574) (e31d903)
• [Features] manager: call shutdown endpoint before workflow stop (#559) (719fa37)
• [Features] manager: increase termination period of run-batch pods (#572) (f05096a)
• [Features] manager: pass custom env variables to job controller (#571) (646f071)
• [Features] manager: pass custom env variables to workflow engines (#571) (cb9369b)
• [Bug fixes] manager: graceful shutdown of job-controller (#559) (817b019)
• [Bug fixes] manager: use valid group name when calling groupadd (#566) (73a9929), closes #561
• [Bug fixes] stop: store engine logs of stopped workflow (#563) (199c163), closes #560
• [Code refactoring] consumer: do not update status of jobs (#559) (5992034)
• [Code refactoring] docs: move from reST to Markdown (#567) (4fbdb74)
• [Code style] black: format with black v24 (#564) (2329437)
• [Continuous integration] commitlint: addition of commit message linter (#555) (b9df20a)
• [Continuous integration] commitlint: allow release commit style (#575) (b013d49)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#562) (4b8f539)
• [Continuous integration] pytest: move to PostgreSQL 14.10 (#568) (9b6bfa0)
• [Continuous integration] release-please: initial configuration (#555) (672083d)
• [Continuous integration] release-please: update version in Dockerfile/OpenAPI specs (#558) (4be8086)
• [Continuous integration] shellcheck: fix exit code propagation (#562) (c5d4982)
• [Documentation] authors: complete list of contributors (#570) (08ab9a3)

• [Build] docker: install correct extras of reana-commons submodule (#261) (21957fe)
• [Build] docker: non-editable submodules in "latest" mode (#255) (a6acc88)
• [Build] python: bump all required packages as of 2024-03-04 (#267) (ed6a846)
• [Build] python: bump shared REANA packages as of 2024-03-04 (#267) (47155ef)
• [Bug fixes] progress: handle stopped jobs (#260) (bc36cb7)
• [Code refactoring] docs: move from reST to Markdown (#263) (3cf272f)
• [Continuous integration] commitlint: addition of commit message linter (#256) (021854e)
• [Continuous integration] commitlint: allow release commit style (#268) (ed7ad11)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#262) (9a45817)
• [Continuous integration] release-please: initial configuration (#256) (bcd87d1)
• [Continuous integration] release-please: update version in Dockerfile (#259) (0961257)
• [Continuous integration] shellcheck: fix exit code propagation (#262) (6568b9b)
• [Documentation] authors: complete list of contributors (#266) (2960cd9)
• [Documentation] conformance-tests: update CWL conformance test badges (#264) (45afa2e)

• [Build] docker: install correct extras of reana-commons submodule (#196) (b23f4df)
• [Build] docker: non-editable submodules in "latest" mode (#190) (03a15cf)
• [Build] python: bump all required packages as of 2024-03-04 (#200) (ffc8aec)
• [Build] python: bump shared REANA packages as of 2024-03-04 (#200) (47c26cc)
• [Bug fixes] progress: handle stopped jobs (#195) (a232a76)
• [Code refactoring] docs: move from reST to Markdown (#198) (7507d12)
• [Continuous integration] commitlint: addition of commit message linter (#191) (b7a6ef1)
• [Continuous integration] commitlint: allow release commit style (#201) (b50b6d0)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#197) (1813ac3)
• [Continuous integration] release-please: initial configuration (#191) (d40a675)
• [Continuous integration] release-please: update version in Dockerfile (#194) (52c34ec)
• [Continuous integration] shellcheck: fix exit code propagation (#197) (5565b29)
• [Documentation] authors: complete list of contributors (#199) (e9b25b6)

• [Build] docker: install correct extras of reana-commons submodule (#79) (fd9b88a)
• [Build] docker: non-editable submodules in "latest" mode (#73) (c3595c2)
• [Build] python: bump all required packages as of 2024-03-04 (#85) (66e81e2)
• [Build] python: bump shared REANA packages as of 2024-03-04 (#85) (d07f91f)
• [Features] config: get max number of parallel jobs from env vars (#84) (69cfad4)
• [Features] executor: upgrade to Snakemake v7.32.4 (#81) (4a3f359)
• [Bug fixes] progress: handle stopped jobs (#78) (4829d80)
• [Code refactoring] docs: move from reST to Markdown (#82) (31de94f)
• [Continuous integration] commitlint: addition of commit message linter (#74) (145b7e7)
• [Continuous integration] commitlint: allow release commit style (#86) (fd032db)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#80) (b677913)
• [Continuous integration] release-please: initial configuration (#74) (9b16bd0)
• [Continuous integration] release-please: update version in Dockerfile (#77) (3c35a67)
• [Continuous integration] shellcheck: fix exit code propagation (#80) (ad15c0d)
• [Documentation] authors: complete list of contributors (#83) (4782678)

• [Build] docker: install correct extras of reana-commons submodule (#256) (8b4caf0)
• [Build] docker: non-editable submodules in "latest" mode (#249) (a57716a)
• [Build] python: bump all required packages as of 2024-03-04 (#261) (2a02e19)
• [Build] python: bump shared REANA packages as of 2024-03-04 (#261) (493aee1)
• [Bug fixes] progress: correctly handle running and stopped jobs (#258) (56ef6a4)
• [Code refactoring] docs: move from reST to Markdown (#259) (37f05e6)
• [Continuous integration] commitlint: addition of commit message linter (#251) (f180e21)
• [Continuous integration] commitlint: allow release commit style (#262) (1b8b6b8)
• [Continuous integration] commitlint: check for the presence of concrete PR number (#257) (9ddb488)
• [Continuous integration] release-please: initial configuration (#251) (dc4fa7a)
• [Continuous integration] release-please: update version in Dockerfile (#254) (8f18751)
• [Continuous integration] shellcheck: fix exit code propagation (#257) (8831d9e)
• [Documentation] authors: complete list of contributors (#260) (68f97a0)

• Users:

  • Adds web interface form allowing to generate launcher URL for any user-provided public analysis, as well as the Markdown snippet for the corresponding Launch-on-REANA badge.
  • Adds web interface option to delete all the runs of a workflow.
  • Changes the Launch-on-REANA web interface page to improve how workflow parameters are shown by displaying them inside a table.
  • Changes CVMFS support to allow users to automatically mount any available repository.
  • Changes the REANA specification schema to use the draft-07 version of the JSON Schema specification.
  • Changes reana-client validate command to show detailed errors when the specification file is not a valid YAML file.
  • Changes validation of REANA specification to make the environment property mandatory for the steps of serial workflows.
  • Changes validation of REANA specification to raise a warning for unexpected properties for the steps of serial workflows.
  • Changes validation of REANA specification to report improved validation warnings which also indicate where unexpected properties are located in the file.
  • Changes workflow restarts to allow for more than nine restarts of the same workflows.
  • Changes workflow scheduler logging behaviour to also report the main reason behind scheduling errors to the users.
  • Fixes reana-client list command to accept case-insensitive column names when sorting the returned workflow runs via the --sort option.
  • Fixes reana-client run wrapper command for workflows that do not contain inputs clause in their specification.
  • Fixes reana-client's create_workflow_from_json API function to always load and send the workflow specification to the server.
  • Fixes creation of image thumbnails for output files in Snakemake HTML execution reports.

• Administrators:

  • Changes several database index definitions in order to improve performance of most common database queries.
  • Changes the names of database table, column, index and key constraints in order to follow the SQLAlchemy upstream naming conventions everywhere.
  • Changes the Workflow table to replace the run_number column with two new columns run_number_major and run_number_minor in order to allow for more than nine restarts of user workflows.
  • Changes CVMFS support to allow users to automatically mount any available repository, thanks to CVMFS CSI v2.
  • Fixes the mounting of CVMFS volumes for the REANA deployments that use non-default Kubernetes namespace.
  • Fixes container image building of cluster components for the arm64 architecture.
  • Fixes job monitoring in cases when job creation fails, for example when it is not possible to successfully mount volumes.
  • Fixes job status consumer exception while attempting to fetch workflow engine logs for workflows that could not have been successfully scheduled.
  • Fixes the creation of Kubernetes jobs by retrying in case of error and by correctly handling the error after reaching the retry limit.

• Developers:

  • Adds automated multi-platform container image building of cluster components for amd64 and arm64 architectures.
  • Adds new --image-name option to the reana-dev docker-push command to customise the name of the docker image to publish.
  • Adds new --platform option to the reana-dev docker-build and reana-dev release-docker commands to build and publish multi-platform images.
  • Adds new --registry option to the reana-dev docker-push and reana-dev release-docker commands to specify the registry where the built docker images should be pushed to.
  • Adds new --tags-only option to the reana-dev release-docker command to only print the final docker image names, without pushing the images to the registry.
  • Adds new reana-dev git-create-release-branch command to create a new Git branch for a new release.
  • Adds new reana-dev git-upgrade-requirements command to upgrade the requirements.txt file before a new release.
  • Changes reana-dev git-fork and reana-dev git-create-pr to use the gh CLI client instead of hub.
  • Changes reana-dev python-run-tests command to allow excluding certain Python components.
  • Changes reana-dev python-run-tests command to allow execution of selected pytests only by passing over PYTEST_ADDOPTS environment variable.
  • Changes validation of REANA specification to expose functions for loading workflow input parameters and workflow specifications.
  • Changes version of reana-ui Node.js Docker image from 16 to 18.
  • Changes the workflow deletion endpoint to return a different and more appropriate message when deleting all the runs of a workflow.
  • Changes the workflow list endpoint on how pagination is performed in order to avoid counting twice the total number of records.
  • Fixes reana-dev python-run-tests command to create Python-3.8 based virtual environments to use the same version as container images.

• Users:

  • Adds support for Python 3.12.
  • Adds support for previewing PDF files present in a workflow's workspace.
  • Adds support for previewing ROOT files present in a workflow's workspace.
  • Adds support for signing-in with a custom third-party Keycloak instance.
  • Adds a new menu item to the workflow actions popup to allow stopping running workflows.
  • Adds prune command to delete all intermediate files of a given workflow. Use with care.
  • Changes the deletion of a workflow to automatically delete an open interactive session attached to its workspace.
  • Changes the workflow deletion message to clarify that attached interactive sessions are also closed when a workflow is deleted.
  • Changes the workflow progress bar to always display it as completed for finished workflows.
  • Changes the interactive session notification to also report that the session will be closed after a specified number of days of inactivity.
  • Changes the workflow-details page to make it possible to scroll through the list of workflow steps in the job logs section.
  • Changes the workflow-details page to not automatically refresh the selected job when viewing the related logs, but keeping the user-selected one active.
  • Changes the page titles to conform to the same sentence case style.
  • Changes the launcher page to show warnings when validating the REANA specification file of the workflow to be launched.
  • Changes open command to inform user about the auto-closure of interactive sessions after a certain inactivity timeout.
  • Changes validate command to display non-critical validation warnings when checking the REANA specification file.
  • Changes Rucio authentication helper to allow users to override the Rucio server and authentication hosts independently of VO name.
  • Fixes job status inconsistency when stopping a workflow by setting the job statuses to stopped for any running jobs.
  • Fixes calculation of workflow runtime durations for stopped workflows.
  • Fixes list command to correctly list workflows when sorting them by their run number or by the size of their workspace.
  • Fixes du command help message typo.
  • Fixes validation --environments command to correctly handle fully-qualified image names.
  • Fixes deletion of failed jobs not being performed when Kerberos is enabled.
  • Fixes job monitoring to consider OOM-killed jobs as failed.
  • Fixes detection of default Rucio server and authentication host for ATLAS VO.
  • Fixes the reported total number of jobs for restarted workflows by excluding cached jobs that were simply reused from previous runs in the workspace and not really executed by Snakemake.
  • Fixes an issue where Snakemake workflows could get stuck waiting for already-finished jobs.

• Administrators:

  • Adds support for Kubernetes clusters 1.26, 1.27, 1.28.
  • Adds new configuration option components.reana_ui.launcher_examples to customise the demo examples that are shown in the launch page in the REANA UI.
  • Adds new configuration option interactive_sessions.maximum_inactivity_period to set a limit in days for the maximum inactivity period of interactive sessions after which they will be closed.
  • Adds new configuration option interactive_sessions.cronjob_schedule to set how often interactive session cleanup should be performed.
  • Adds new configuration option ingress.extra to define extra Ingress resources, in order to support redirecting HTTP requests to HTTPS with traefik v2 version.
  • Adds new configuration option ingress.tls.hosts to define hosts that are present in the TLS certificate, in order to support cert-manager's automatic creation of certificates.
  • Adds new configuration option notifications.email_config.smtp_ssl to use SSL when connecting to the SMTP email server.
  • Adds new configuration option notifications.email_config.smtp_starttls to use the STARTTLS command to enable encryption after connecting to the SMTP email server.
  • Adds new configuration option components.reana_ui.file_preview_size_limit to set the maximum file size that can be previewed in the web interface.
  • Adds new configuration options login and secrets.login for configuring Keycloak SSO login with third-party authentication services.
  • Adds new interactive-session-cleanup command that can be used by REANA administrators to close interactive sessions that are inactive for more than the specified number of days.
  • Adds progress meter to the logs of the periodic quota updater.
  • Adds the content of the secrets.gitlab.REANA_GITLAB_HOST configuration option to the list of GitLab instances from which it is possible to launch a workflow.
  • Changes uWSGI configuration to increase buffer size, add vacuum option, etc.
  • Changes CPU and disk quota calculations to improve the performance of periodic quota updater.
  • Changes the system status report to simplify and clarify the disk usage summary.
  • Changes check-workflows command to also check the presence of workspaces on the shared volume.
  • Changes check-workflows command to not show in-sync runs by default. If needed, they can be shown using the new --show-all option.
  • Changes reana-admin command options to require the passing of --admin-access-token argument more globally.
  • Changes the k8s specification for interactive session pods to include labels for improved subset selection of objects.
  • Changes the k8s specification for interactive session ingress resource to include annotations.
  • Changes nginx configuration to save bandwidth by serving gzip-compressed static files.
  • Changes HTCondor to version 9.0.17 (LTS).
  • Fixes uWSGI memory consumption on systems with very high allowed number of open files.
  • Fixes cronjob failures due to database connection issues when REANA is deployed with non-default namespace or prefix.
  • Fixes ingress.enabled option to correctly enable or disable the creation of Ingresses.
  • Fixes graceful shutdown for reana-server and reana-workflow-controller.
  • Fixes the workflow priority calculation to avoid workflows stuck in the queued status when the number of allowed concurrent workflow is set to zero.
  • Fixes GitLab integration to automatically redirect the user to the correct URL when the access request is accepted.
  • Fixes authentication flow to correctly deny access to past revoked tokens in case the same user has also other new active tokens.
  • Fixes email templates to show the correct kubectl commands when REANA is deployed inside a non-default namespace or with a custom component name prefix.
  • Fixes email sender for system emails to notifications.email_config.sender Helm value.
  • Fixes email receiver for token request emails to use notifications.email_config.receiver Helm value.
  • Fixes quota-set-default-limits command to propagate default quota limits to all users without custom quota limit values.
  • Fixes job status consumer to correctly rollback the database transaction when an error occurs.
  • Fixes intermittent Slurm connection issues by DNS-resolving the Slurm head node IPv4 address before establishing connections.
  • Fixes Slurm command generation issues when using fully-qualified image names.
  • Fixes high memory usage of RabbitMQ by limiting the maximum number of open file descriptors.
  • Removes support for Kubernetes version prior to 1.21.

• Developers:

  • Adds new prune_workspace endpoint to allow users to delete all the files of a workflow, specifying whether to also delete the inputs and/or the outputs.
  • Adds the timestamp of when the workflow was stopped (run_stopped_at) to the workflow list and the workflow status endpoints.
  • Adds unique error messages to Kubernetes job monitor to more easily identify source of problems.
  • Adds new --parallel option to docker-build, cluster-build and run-ci to build multiple docker images in parallel.
  • Changes launch endpoint to also include the warnings of the validation of the workflow specification.
  • Changes OpenAPI specification of the info endpoint to return the maximum inactivity time before automatic closure of interactive sessions.
  • Changes apispec dependency version in order to be compatible with PyYAML v6.
  • Changes Paramiko to version 3.0.0.
  • Changes remote storage file support for Snakemake workflows to use XRootD 5.6.0.
  • Fixes cluster-deploy, cluster-undeploy and client-setup-environment commands when using custom instance name or kubernetes namespace.
  • Fixes the git-tag command to display the component name.
  • Fixes container image names to be Podman-compatible.
  • Fixes location of HTCondor build dependencies.

• Users:

  • Adds support for Rucio authentication for workflow jobs.
  • Adds support for Kerberos authentication for workflow orchestration.
  • Adds support for specifying slurm_partition and slurm_time for Slurm compute backend jobs.
  • Adds support for XRootD remote file locations in Snakemake workflow specification definitions.
  • Adds support for Python 3.11.
  • Adds Launch on REANA page allowing the submission of workflows via badge-clicking.
  • Adds notifications to inform users when critical levels of quota usage is reached.
  • Adds 404 Not Found error page.
  • Adds tab titles to all the pages.
  • Adds the REANA_WORKSPACE environment variable to jupyter notebooks and terminals.
  • Adds option to sort workflows by most disk and cpu quota usage to the workflow list endpoint.
  • Adds support for specifying and listing workspace file retention rules.
  • Adds support for .gitignore and .reanaignore to specify files that should not be uploaded to REANA.
  • Adds retention-rules-list command to list the retention rules of a workflow.
  • Changes REANA specification to allow specifying retention_days for the workflow.
  • Changes default Slurm partition to inf-short.
  • Changes GitLab integration to also retrieve user's projects that are in groups and subgroups.
  • Changes the workflow-details page to show the logs of the workflow engine.
  • Changes the workflow-details page to show file sizes in a human-readable format.
  • Changes the workflow-details page to show the workspace's retention rules.
  • Changes the workflow-details page to show the duration of the workflow's jobs.
  • Changes the workflow-details page to display a label of the workflow launcher URL remote origin.
  • Changes the workflow-details page to periodically refresh the content of the page.
  • Changes the workflow-details page to refresh after the deletion of a workflow.
  • Changes the workflow-list page to add a way to hide deleted workflows.
  • Changes the workflow-list page to add new workflows sorting options by most used disk and cpu quota.
  • Changes the deletion of a workflow to always clean up the workspace and to update the user disk quota usage.
  • Changes the CWD of jupyter's terminals to the directory of the workflow's workspace.
  • Changes percentage ranges used to calculate the health status of user resource quota usage.
  • Changes create and restart commands to always upload the REANA specification file.
  • Changes delete command to always delete the workflow's workspace.
  • Changes delete_workflow Python API function to always delete the workflow's workspace.
  • Changes download command to add the possibility to write files to the standard output via -o - option.
  • Changes list command to hide deleted workflows by default.
  • Changes list command to allow displaying deleted workflows via --all and --show-deleted-runs options.
  • Changes list and status commands to allow displaying the duration of workflows with the --include-duration option.
  • Changes mv command to allow moving files while a workflow is running.
  • Changes upload command to prevent uploading symlinks.
  • Changes validation --environment command to use Docker registry v2 APIs to check that a Docker image exists in DockerHub.
  • Fixes list command to highlight the workflow specified in REANA_WORKON correctly.
  • Fixes secrets-delete command error message when deleting non existing secrets.
  • Fixes start command to report failed workflows as errors.
  • Fixes start and run commands to correctly follow the execution of the workflow until termination.
  • Fixes status command to respect output format provided by the --format option.
  • Fixes upload command to report when input directories are listed under the files section in the REANA specification file and vice versa.
  • Fixes validate --environment command to detect illegal whitespace characters in Docker image names.
  • Fixes Kerberos authentication for long-running workflows by renewing the Kerberos ticket periodically.
  • Fixes status reporting for failed CWL and Snakemake jobs that were incorrectly considered successful.
  • Fixes redirection chain for non-signed-in CERN SSO users to access the desired target page after sign-in.
  • Fixes the ordering by size of the files showed in the Workspace tab of the workflow-details page.
  • Fixes CERN OIDC authentication to possibly allow eduGAIN and social login users.
  • Fixes wrong numbering of restarted workflows by limiting the number of times a workflow can be restarted to nine.

• Administrators:

  • Adds new configuration environment variable reana_server.environment.REANA_SCHEDULER_REQUEUE_COUNT to set workflow requeue count in case of scheduling errors or busy cluster situations.
  • Adds "infinity" option to REANA_SCHEDULER_REQUEUE_COUNT to disable requeue count.
  • Adds support for Kubernetes clusters 1.22, 1.23, 1.24, 1.25.
  • Adds new configuration option workspaces.retention_rules.maximum_period to set a default period for workspace retention rules.
  • Adds new configuration option workspaces.retention_rules.cronjob_schedule to set how often pending retention rules should be applied.
  • Adds configuration environment variable reana_server.environment.REANA_RATELIMIT_SLOW to limit API requests to some protected endpoints e.g launch workflow.
  • Adds configuration environment variable reana_server.environment.REANA_WORKFLOW_SCHEDULING_READINESS_CHECK_LEVEL to define checks that are performed to assess whether the cluster is ready to start new workflows.
  • Adds new configuration option ingress.tls.self_signed_cert to enable the generation of a self-signed TLS certificate.
  • Adds new configuration option ingress.tls.secret_name to specify the name of the Kubernetes secret containing the TLS certificate to be used.
  • Adds support for configuring an additional volume to be used by the database and the message broker.
  • Adds new configuration option maintenance.enabled to scale down the cluster for maintenance.
  • Adds support for Unicode characters inside email body.
  • Adds queue-consume command that can be used by REANA administrators to remove specific messages from the queue.
  • Adds retention-rules-apply command that can be used by REANA administrators to apply pending retention rules.
  • Adds retention-rules-extend command that can be used by REANA administrators to extend the duration of active retentions rules.
  • Adds check-workflows command that can be used by REANA administrators to check for out-of-sync workflows and interactive sessions.
  • Changes configuration option quota.workflow_termination_update_policy to deactivate workflow termination accounting by default.
  • Changes Helm template to use PostgreSQL 12.13 version.
  • Changes the base image for most of the components to Ubuntu 20.04 LTS and reduces final Docker image size by removing build-time dependencies.
  • Changes reana-auth-vomsproxy sidecar to the latest stable version to support client-side proxy file generation technique and ESCAPE VOMS.
  • Changes OAuth configuration to enable the new CERN SSO.
  • Changes job status consumer to improve logging for not-alive workflows.
  • Changes the deployment of interactive sessions to improve security by not automounting the Kubernetes service account token.
  • Changes the deployment of job-controller to avoid unnecessarily mounting the database's directory.
  • Changes the announcements to support limited HTML markup.
  • Changes REANA specification loading functionality to allow specifying different working directories.
  • Changes global setting of maximum number of parallel jobs to 300 for Snakemake workflow engine.
  • Fixes job status consumer by discarding invalid job IDs.
  • Fixes GitLab integration error reporting in case user exceeds CPU or Disk quota usage limits.
  • Fixes issue when irregular number formats are passed to REANA_SCHEDULER_REQUEUE_COUNT configuration environment variable.
  • Fixes quota updater to reduce memory usage.
  • Fixes conversion of possibly-negative resource usage values to human-readable formats.
  • Fixes disk quota updater to prevent setting negative disk quota usage values.
  • Removes support for Kubernetes version prior to 1.19.

• Developers:

  • Adds OpenAPI specification support for launch endpoint that allows running workflows from remote sources.
  • Adds OpenAPI specification support for get_workflow_retention_rules endpoint that allows to retrieve the workspace file retention rules of a workflow.
  • Adds the remote origin of workflows submitted via Launch-on-REANA (launcher_url) to the workflow list endpoint.
  • Adds common utility functions for managing workspace files to reana-commons.
  • Changes default consumer prefetch count to handle 10 messages instead of 200 in order to reduce the probability of 406 PRECONDITION errors on message acknowledgement.
  • Changes git-upgrade-shared-modules to generate the correct upper-bound in setup.py.
  • Changes REANA specification loading and validation functionalities by porting some of the logic to reana-commons.
  • Changes OpenAPI specification to include missing response schema elements.
  • Changes the Kubernetes Python client to use the networking/v1 API.
  • Changes the deployment of interactive sessions to use networking/v1 Kubernetes API.
  • Changes to Flask v2.
  • Changes /api/info endpoint to also include the kubernetes maximum memory limit, the kubernetes default memory limit and the maximum workspace retention period.
  • Changes start_workflow endpoint to validate the REANA specification of the workflow.
  • Changes create_workflow endpoint to populate workspace retention rules for the workflow.
  • Changes start_workflow endpoint to disallow restarting a workflow when retention rules are pending.
  • Changes API rate limiter error messages to be more verbose.
  • Changes workflow scheduler to allow defining the checks needed to assess whether the cluster can start new workflows.
  • Changes workflow list endpoint to add the possibility to filter by workflow ID.
  • Changes the move_files endpoint to allow moving files while a workflow is running.
  • Changes the k8s specification of interactive sessions' pods to remove the environment variables used for service discovery.
  • Changes GitLab integration to use reana as pipeline name instead of default when setting status of a commit.
  • Changes the loading of Snakemake specifications to preserve the current working directory.
  • Changes the Invenio dependencies to the latest versions.
  • Fixes the submission of jobs by stripping potential leading and trailing whitespaces in Docker image names.
  • Fixes fetchWorkflow action to fetch a specific workflow instead of the entire user workflow list. (reana-ui)
  • Fixes the download of files by changing the default MIME type to application/octet-stream.
  • Fixes the workflow list endpoint to correctly parse the boolean parameters include_progress, include_workspace_size and include_retention_rules.

• Users:

  • Adds support for specifying kubernetes_job_timeout for Kubernetes compute backend jobs.
  • Adds Kubernetes job memory limits validation before accepting workflows for execution.
  • Adds support for HTML preview of workspace files in the web user interface.
  • Adds an option to search for concrete file names in the workflow's workspace web user interface page.
  • Changes the Cluster Health web interface page to display the cluster status information based on resource availability rather than only usage.
  • Changes info command to include the list of supported compute backends.
  • Fixes workflow stuck in pending status due to early Yadage failures.
  • Fixes formatting of error messages and sets appropriate exit status codes.

• Administrators:

  • Adds new configuration option to set default job timeout value for the Kubernetes compute backend jobs (kubernetes_jobs_timeout_limit).
  • Adds new configuration option to set maximum job timeout that users can assign to their jobs for the Kubernetes compute backend (kubernetes_jobs_max_user_timeout_limit).
  • Adds new configuration option compute_backends to specify the supported list of compute backends for validation purposes.
  • Adds new configuration option reana_server.uwsgi.log_all to toggle the logging of all the HTTP requests.
  • Adds new configuration options reana_server.uwsgi.log_4xx and reana_server.uwsgi.log_5xx to only log HTTP error requests, i.e. HTTP requests with status code 4XX and 5XX. To make this configuration effective reana_server.uwsgi.log_all must be false.
  • Adds new configuration options node_label_infrastructuremq and node_label_infrastructuredb to have the possibility to run the Message Broker and the Database pods in specific nodes.
  • Changes uWSGI configuration to log all HTTP requests in REANA-Server by default.
  • Changes quota.disk_update to quota.periodic_update_policy to also update the CPU quota. Keeps quota.disk_update for backward compatibility.
  • Changes the name of configuration option quota.termination_update_policy to quota.workflow_termination_update_policy. Keeps quota.termination_update_policy for backward compatibility.

• Developers:

  • Adds workflow name validation to the create_workflow endpoint, restricting special characters like dots.
  • Changes /api/info endpoint to return a list of supported compute backends.
  • Changes /api/status endpoint to calculate the cluster health status based on the availability instead of the usage.
  • Changes the way of determining Snakemake job statuses, polling the Job Controller API instead of checking local files.

• Users:

  • Adds support for running and validating Snakemake workflows.
  • Adds support for outputs.directories in reana.yaml allowing to easily download output directories.
  • Adds new command quota-show to retrieve information about total CPU and Disk usage and quota limits.
  • Adds new command info that retrieves general information about the cluster, such as available workspace path settings.
  • Changes validate command to add the possibility to check the workflow against server capabilities such as desired workspace path via --server-capabilities option.
  • Changes list command to add the possibility to filter by workflow status and search by workflow name via --filter option.
  • Changes list command to add the possibility to filter and display all the runs of a given workflow via -w option.
  • Changes list command to stop including workflow progress and workspace size by default. Please use new options --include-progress and --include-workspace-size to show this information.
  • Changes list --sessions command to display the status of interactive sessions.
  • Changes logs command to display also the start and finish times of individual jobs.
  • Changes ls command to add the possibility to filter by file name, size and last-modified values via --filter option.
  • Changes du command to add the possibility filter by file name and size via --filter option.
  • Changes delete command to prevent hard-deletion of workflows.
  • Changes Yadage workflow specification loading to be done in reana-commons.
  • Changes CWL workflow engine to cwltool version 3.1.20210628163208.
  • Removes support for Python 2.7. Please use Python 3.6 or higher from now on.

• Administrators:

  • Adds new configuration options node_label_runtimebatch, node_label_runtimejobs, node_label_runtimesessions allowing to set cluster node labels for splitting runtime workload into dedicated workflow batch nodes, workflow job nodes and interactive session nodes.
  • Adds new configuration option workspaces.paths allowing to set a dictionary of available workspace paths to pairs of cluster_node_path:cluster_pod_mountpath for mounting directories from cluster nodes.
  • Adds new configuration option quota.enabled to enable or disable CPU and Disk quota accounting for users.
  • Adds new configuration option quota.termination_update_policy to select the quota resources such as CPU and Disk for which the quota usage will be calculated immediately at the workflow termination time.
  • Adds new periodic cron job to update Disk quotas nightly. Useful if the quota.termination_update_policy does not include Disk quota resource.
  • Adds configuration environment variable reana_server.environment.REANA_WORKFLOW_SCHEDULING_POLICY allowing to set workflow scheduling policy (first-in first-out, user-balanced and workflow-complexity balanced).
  • Adds configuration environment variables reana_server.environment.REANA_RATELIMIT_GUEST_USER, reana_server.environment.REANA_RATELIMIT_AUTHENTICATED_USER allowing to set REST API rate limit values.
  • Adds configuration environment variable reana_server.environment.REANA_SCHEDULER_REQUEUE_SLEEP to set a time to wait between processing queued workflows.
  • Adds configuration environment variable reana_workflow_controller.environment.REANA_JOB_STATUS_CONSUMER_PREFETCH_COUNT allowing to set a prefetch count for the job status consumer.
  • Adds support for Kubernetes 1.21 version clusters.
  • Adds default kubernetes_memory_limit value (4 GiB) that will be used for all user jobs unless they specify otherwise.
  • Changes Helm template to use PostgreSQL 12.8 version.
  • Changes Helm template for reana-db component to allow 300 maximum number of database connections by default.
  • Fixes email validation procedure during create-admin-user command to recognize more permissive email address formats.

• Developers:

  • Changes git-* commands to add the possibility of excluding certain components via the --exclude-components option.
  • Changes git-create-release-commit command to bump all version files in a component.
  • Changes git-log command to show diff patch or to pass any wanted argument.
  • Changes helm-upgrade-components command to also upgrade the image tags in prefetch-images.sh script.

• Users:

  • Adds support for file listing wildcard matching patterns to ls command.
  • Adds support for directory download and wildcard matching patterns to download command.
  • Adds support for specifying kubernetes_memory_limit for Kubernetes compute backend jobs for CWL, Serial and Yadage workflows.
  • Changes list command to include deleted workflows by default.
  • Changes validate command to warn about incorrectly used workflow parameters for each step.
  • Changes validate command to display more granular workflow validation output.
  • Fixes workflow step job command formatting bug for CWL workflows on HTCondor compute backend.
  • Fixes validate command output for verifying environment image UID values.
  • Fixes upload_to_server() Python API function to silently skip uploading in case of none-like inputs.
  • Fixes validate command for environment image validation to not test repetitively the same image found in different steps.

• Administrators:

  • Adds support for Kubernetes 1.21.
  • Adds configuration environment variable to set default job memory limits for the Kubernetes compute backend (REANA_KUBERNETES_JOBS_MEMORY_LIMIT).
  • Adds configuration environment variable to set maximum custom memory limits that users can assign to their jobs for the Kubernetes compute backend (REANA_KUBERNETES_JOBS_MAX_USER_MEMORY_LIMIT).
  • Changes HTCondor compute backend to 8.9.11 and myschedd package and configuration to latest versions.
  • Fixes Kubernetes job log capture to include information about failures caused by external factors such as out-of-memory situations (OOMKilled).

• Developers:

  • Adds new functions to serialise/deserialise job commands between REANA components.
  • Changes client dependencies to unpin six so that client may be installed in more contexts.
  • Changes cluster dependencies to remove click and pins several dependencies.
  • Changes reana_ready() function location to REANA-Server.

• Users:

  • Adds reana-client validate options to detect possible issues with workflow input parameters and environment images.
  • Fixes problem with failed jobs being reported as still running in case of network problems.
  • Fixes job command encoding issues when dispatching jobs to HTCondor and Slurm backends.

• Administrators:

  • Adds new configuration to toggle Kubernetes user jobs clean up. : (REANA_RUNTIME_KUBERNETES_KEEP_ALIVE_JOBS_WITH_STATUSES in components.reana_workflow_controller.environment)
  • Improves platform resilience.

• Developers:

  • Adds new command-line options to reana-dev run-example command allowing full parallel asynchronous execution of demo examples.
  • Adds default configuration for developer deployment mode to keep failed workflow and job pods for easier debugging.
  • Changes job status consumer communications to improve overall platform resilience.

• Administrators:

  • Adds support for deployments on Kubernetes 1.20 clusters.
  • Adds deployment option to disable user email confirmation step after sign-up. (REANA_USER_EMAIL_CONFIRMATION in components.reana_server.environment)
  • Adds deployment option to disable user sign-up feature completely. (components.reana_ui.hide_signup)
  • Adds deployment option to display CERN Privacy Notice for CERN deployments. (components.reana_ui.cern_ropo)

• Developers:

  • Adds support for Python 3.9.
  • Fixes minor code warnings.
  • Changes CI system to include Python flake8 and Dockerfile hadolint checkers.

• Users:

  • Adds support for specifying htcondor_max_runtime and htcondor_accounting_group for HTCondor compute backend jobs.
  • Fixes restarting of Yadage and CWL workflows.
  • Fixes REANA <-> GitLab synchronisation for projects having additional external webhooks.
  • Changes ping command output to include REANA client and server version information.

• Developers:

  • Fixes conflicting kombu installation requirements by requiring Celery version 4.
  • Changes /api/you endpoint to include REANA server version information.
  • Changes continuous integration platform from Travis CI to GitHub Actions.

• Users:

  • Adds new restart command to restart previously run or failed workflows.
  • Adds option to logs command to filter job logs according to compute backend, docker image, job status and step name.
  • Adds option to specify operational options in the reana.yaml of the workflow.
  • Adds option to specify unpacked Docker images as workflow step requirement.
  • Adds option to specify Kubernetes UID for jobs.
  • Adds support for VOMS proxy as a new authentication method.
  • Adds support for pulling private Docker images.
  • Adds pagination on the workflow list and workflow detailed web interface pages.
  • Adds user profile page to the web interface.
  • Adds page refresh button to workflow detailed page.
  • Adds local user web forms for sign-in and sign-up functionalities for local deployments.
  • Fixes user experience by preventing dots as part of the workflow name to avoid confusion with restart runs.
  • Fixes workflow specification display to show runtime parameters.
  • Fixes file preview functionality experience to allow/disallow certain file formats.
  • Changes Yadage workflow engine to version 0.20.1.
  • Changes CERN HTCondor compute backend to use the new myschedd connection library.
  • Changes CERN Slurm compute backend to improve job status detection.
  • Changes documentation to move large parts to docs.reana.io.
  • Changes du command output format.
  • Changes logs command to enhance formatting using marks and colours.
  • Changes ping command to perform user access token validation.
  • Changes diff command to improve output formatting.
  • Changes defaults to accept both reana.yaml and reana.yml filenames.
  • Changes from Bravado to requests to improve download performance.
  • Changes file loading to optimise CLI performance.

• Administrators:

  • Adds Helm chart and switches to Helm-based deployment technique instead of using now-deprecated reana-cluster.
  • Adds email notification service to inform administrators about system health.
  • Adds announcement configuration option to display any desired text on the web UI.
  • Adds pinning of all Python dependencies allowing to easily rebuild component images at later times.
  • Adds support for local user management and web forms for sign-in and sign-up functionalities.
  • Adds support for database upgrades using Alembic.
  • Changes installation procedures to move database initialisation and admin creation after Helm installation.
  • Changes service exposure to stop exposing unused Invenio-Accounts views.
  • Changes runtime job instantiation into the configured runtime namespace.
  • Changes CVMFS to be read-only mount.

• Developers:

  • Adds several new reana-dev commands to help with merging, releasing, unit testing.
  • Changes base image to use Python 3.8 for all REANA cluster components.
  • Changes pre-requisites to node version 12 and latest npm dependencies.
  • Changes back-end code formatting to respect black coding style.
  • Changes front-end code formatting to respect updated prettier version coding style.
  • Changes test strategy to start PostgreSQL DB container to run tests locally.
  • Changes auto-generated component documentation to single-page layout.

• Administrators:

  • Fixes installation troubles for REANA 0.6.x release series by pinning several dependencies.
  • Upgrades REANA-Commons package to latest Kubernetes Python client version.
  • Amends documentation for minikube start to include VirtualBox hypervisor explicitly.

• Users:

  • Adds support for HTCondor compute backend for all workflow engines (CWL, Serial, Yadage).
  • Adds support for Slurm compute backend for all workflow engines (CWL, Serial, Yadage).
  • Allows to run hybrid analysis pipelines where different parts of the workflow can run on different compute backends (HTCondor, Kubernetes, Slurm).
  • Adds support for Kerberos authentication mechanism for user workflows.
  • Introduces user secrets management commands secrets-add, secrets-list and secrets-delete.
  • Fixes upload command behaviour for uploading very large files.
  • Upgrades CWL workflow engine to 1.0.20191022103248.
  • Upgrades Yadage workflow engine to 0.20.0 with Packtivity 0.14.21.
  • Adds support for Python 3.8.
  • See additional changes in reana-client 0.6.0 release notes.

• Administrators:

  • Upgrades to Kubernetes 1.16 and moves Traefik installation to Helm 3.0.0.
  • Creates a new Kubernetes service account for REANA with appropriate permissions.
  • Makes database connection details configurable so that REANA can connect to databases external to the cluster.
  • Autogenerates deployment secrets if not provided by administrator at cluster creation time.
  • Adds an interactive mode on cluster initialisation to allow providing deployment secrets.
  • Adds CERN specific Kerberos configuration files and CERN EOS storage support.
  • See additional changes in reana-cluster 0.6.0 release notes.

• Developers:

  • Modifies the batch workflow runtime pod creation including an instance of job controller running alongside workflow engine using the sidecar pattern.
  • Adds generic job manager class and provides example classes for CERN HTCondor and CERN Slurm clusters.
  • Provides user secrets to the job container runtime tasks.
  • Adds sidecar container to the Kubernetes job pod if Kerberos authentication is required.
  • Refactors job monitoring using the singleton pattern.
  • Enriches make behaviour for developer-oriented installations with live code reload changes and debugging.
  • Enriches git-status component status reporting for developers.
  • See additional changes in individual REANA 0.6.0 platform components.

• Users:

  • Allows to explore workflow results by running interactive Jupyter notebook sessions on the workspace files.
  • Allows to declare computing resources needed for workflow runs, such as access to CVMFS repositories.
  • Improves reana-client command-line client with new options to stop workflows, diff workflows, move and remove files.
  • Upgrades CWL engine to 1.0.20181118133959.
  • See additional changes in reana-client 0.5.0 release notes.

• Administrators:

  • Upgrades to Kubernetes 1.14, Helm 2.13 and Minikube 1.0.
  • Separates cluster infrastructure pods from runtime workflow engine pods that will be created by workflow controller.
  • Introduces configurable CVMFS and CephFS shared volume mounts.
  • Adds support for optional HTTPS protocol termination.
  • Introduces incoming workflow queue for additional safety in case of user storms.
  • Makes infrastructure pods container image slimmer to reduce the memory footprint.
  • See additional changes in reana-cluster 0.5.0 release notes.

• Developers:

  • Enhances development process by using git-submodule-like behaviour for shared components.
  • Introduces simple Makefile for (fast) local testing and (slow) nightly building purposes.
  • Centralises logging level and common Celery tasks.
  • Adds helpers for test suite fixtures and improves code coverage.
  • See additional changes in individual REANA 0.5.0 platform components.

• Uses common OpenAPI client in communications between workflow engines and job controller.
• Improves AMQP re-connection handling.
• Enhances test suite and increases code coverage.
• Changes license to MIT.

• Introduces new Serial workflow engine for simple sequential workflow needs.
• Enhances progress reporting for CWL, Serial and Yadage workflow engines.
• Simplifies reana-client command set and usage scenarios.
• Introduces multi-user capabilities with mandatory access tokens.
• Adds support for multi-node clusters using shared CephFS volumes.
• Adds support for Kubernetes 1.11, Minikube 0.28.2.
• Upgrades CWL workflow engine to use latest cwltool version.
• Fixes several bugs such as binary file download with Python 3.

• Adds support for Common Workflow Language workflows.
• Adds support for persistent user-selected workflow names.
• Enables file and directory input uploading using absolute paths.
• Enriches reana-client and reana-cluster command set.
• Reduces verbosity level for commands and improves error messages.

• Initial public release.