Open
Description
Describe the problem you're observing
In the case of producer-consumer workloads where a subset of producers produce and then another subset of processes on a different set of nodes consume, currently the reading of consumers is bottlenecked by the unifyfs server which holds the data (generally the set of nodes where data was produced).
We can add a migration step between the unifyfs servers to migrate this data closer to the consumers so that they can access data node-locally instead of remote access.
Activity