@Arshil-Akkala My pleasure. It's recommended to enable the enable_experimental_write_cache too, it's good for performance, please read the performance tips doc:
https://docs.greptime.com/user-guide/administration/performance-tuning-tips#using-cache-for-object-stores

I had another question, what if the pod fails and get reallocated to another node.. what will happen in that case?

@Arshil-Akkala

If you use our official helm-charts to deploy your DB, we use the following default configuration:

• Use K8s StatefulSets to manage Datanode or GreptimeDB Standalone instance;
• Use default storageclass to allocate the PVC, which is used for wal and cache data;

Suppose your default storageclass is distributed block storage, for example, AWS EBS(public cloud) or Ceph(self-host). In that case, the PVC remains even if your pod is terminated or relocated to another node. Whenever your pod recovers, it will attach to the original PVC and use the original data.

The storage will be bound to the specific node if your underly storageclass is local-storage(for example, local pv). Although your DB may work, you should still be careful with the binding. If you don't set the pod affinity(the pod must be scheduled to the specific node), your pod may not work well when it fails or is rescheduled(it probably will fail to be scheduled). We don't recommend using this deployment because it's hard to operate.

I meant in reference to caching, so is there something like distributed caching?

I meant in reference to caching, so is there something like distributed caching?

No, we don't design the distributed caching in the system right now, but based on the PVC and statefulsets, we can archive almost the same mechanism just like it. We plan to design the in-house distributed caching.