Introduction to Zero Downtime Migration
Zero Downtime Migration provides a simple and reliable way for you to migrate applications from any CQL-based cluster (Apache Cassandra®, DataStax Enterprise (DSE), Astra DB, or any type of CQL-based database) to any other CQL-based cluster, without any interruption of service to the client applications and data.
-
You can move your application to Astra DB, DSE, or Cassandra with no downtime and with minimal configuration changes.
-
Your clusters are kept in sync at all times by a dual-write logic configuration.
-
You can roll back at any point, for complete peace of mind.
|
This suite of tools allows for zero downtime migration only if your database meets the minimum requirements. If your database does not meet these requirements, you can complete the migration from Origin to Target, but downtime might be necessary to finish the migration. |
|
The Zero Downtime Migration process requires you to be able to perform rolling restarts of your client applications during the migration. This is standard practice for client applications that are deployed over multiple instances and is a widely used approach to roll out releases and configuration changes. |
Supported releases
Overall, you can use ZDM Proxy to migrate:
-
From: Any Cassandra 2.1.6 or higher release, or from any DSE 4.7.1 or higher release.
-
To: Any equivalent or higher release of Cassandra, or to any equivalent or higher release of DSE, or to Astra DB.
Migration scenarios
There are many reasons why you may decide to migrate your data and client applications from one cluster to another, for example:
-
Moving to a different type of CQL database, for example an on-demand cloud-based proposition such as Astra DB.
-
Upgrading a cluster to a newer version, or newer infrastructure, in as little as one step while leaving your existing cluster untouched throughout the process.
-
Moving one or more client applications out of a shared cluster and onto a dedicated one, in order to manage and configure each cluster independently.
-
Consolidating client applications, which may be currently running on separate clusters, onto a shared one in order to reduce overall database footprint and maintenance overhead.
Here are just a few examples of migration scenarios that are supported when moving from one type of CQL-based database to another:
-
From an existing self-managed Cassandra or DSE cluster to cloud-native Astra DB. For example:
-
Cassandra 2.1.6+, 3.11.x, 4.0.x, or 4.1.x to Astra DB.
-
DSE 4.7.1+, 4.8.x, 5.1.x, 6.7.x or 6.8.x to Astra DB.
-
-
From an existing Cassandra or DSE cluster to another Cassandra or DSE cluster. For example:
-
Cassandra 2.1.6+ or 3.11.x to Cassandra 4.0.x or 4.1.x.
-
DSE 4.7.1+, 4.8.x, 5.1.x or 6.7.x to DSE 6.8.x.
-
Cassandra 2.1.6+, 3.11.x, 4.0.x, or 4.1.x to DSE 6.8.x.
-
DSE 4.7.1+ or 4.8.x to Cassandra 4.0.x or 4.1.x.
-
-
From Astra DB Classic to Astra DB Serverless.
-
From any CQL-based database type/version to the equivalent CQL-based database type/version.
|
An important migration prerequisite is that you already have the matching schema on Target. A CQL statement that your client application sends to ZDM Proxy must be able to succeed on both Origin and Target clusters. This means that any keyspace that your client application uses must exist on both Origin and Target with the same name. Table names must also match. For more, see Schema/keyspace compatibility. |
Migration phases
First, a couple of key terms used throughout the ZDM documentation and software components:
-
Origin: This cluster is your existing Cassandra-based environment, whether it’s open-source Apache Cassandra, DSE, or Astra DB Classic.
-
Target: This cluster is the new environment to which you want to migrate client applications and data.
Migration diagram
Discover the migration concepts, software components, and sequence of operations.
Your migration project occurs through a sequence of phases, which matches the structure of the Zero Downtime Migration documentation.
The highlighted components in each phase emphasize how your client applications perform read and write operations on your Origin and Target clusters.
Pre-migration client application operations
Here’s a look at a pre-migration from a high-level view. At this point, your client applications are performing read/write operations with an existing CQL-compatible database such as Apache Cassandra, DSE, or Astra DB.
Phase 1: Deploy ZDM Proxy and connect client applications
In this first phase, deploy the ZDM Proxy instances and connect client applications to the proxies. This phase activates the dual-write logic. Writes are bifurcated (sent to both Origin and Target), while reads are executed on Origin only.
Phase 2: Migrate data
In this phase, migrate existing data using Cassandra Data Migrator and/or DSBulk Loader. Validate that the migrated data is correct, while continuing to perform dual writes.
Phase 3: Enable asynchronous dual reads
In this phase, you can optionally enable asynchronous dual reads. The idea is to test performance and verify that Target can handle your application’s live request load before cutting over from Origin to Target.
Phase 4: Route reads to Target
In this phase, read routing on the ZDM Proxy is switched to Target so that all reads are executed on it, while writes are still sent to both clusters. In other words, Target becomes the primary cluster.
Phase 5: Connect directly to Target
In this phase, move your client applications off the ZDM Proxy and connect the apps directly to Target. Once that happens, the migration is complete.
A fun way to learn: Zero Downtime Migration Interactive Lab
Now that you’ve seen a conceptual overview of the process, let’s put what you learned into practice.
We’ve built a complementary learning resource that is a companion to this comprehensive ZDM documentation. It’s the Zero Downtime Migration Interactive Lab.
-
All you need is a browser and a GitHub account.
-
There’s nothing to install for the lab, which opens in a pre-configured GitPod environment.
-
You’ll learn about a full migration without leaving your browser!
|
To run the lab, all major browsers are supported, except Safari. For more, see the lab’s start page. |
We encourage you to explore this free hands-on interactive lab from DataStax Academy. It’s an excellent, detailed view of the migration process. The lab describes and demonstrates all the steps and automation performed to prepare for, and complete, a migration from any Cassandra/DSE/Astra DB database to another Cassandra/DSE/Astra DB database across clusters.
The interactive lab spans the pre-migration prerequisites and each of the five key migration phases illustrated above.
