We need the same bahavior as with the JS version:
- crawlee implements the base storage classes
- every async operation checks if it was the first call, and purges automatically unless opted-out via
CRAWLEE_PURGE_ON_START env var (with a falsy value like 0 or false)
- we have this method that it called on many places in the storage methods like
open or getInput https://crawlee.dev/api/core/function/purgeDefaultStorages
- since the SDK uses those storage classes, it has the same behavior out of box
- internally this works by calling
purge method on the storage client, so this also means both memory storage and apify client need to implement this purge method
Related: apify/apify-cli#545
We need the same bahavior as with the JS version:
CRAWLEE_PURGE_ON_STARTenv var (with a falsy value like0orfalse)openorgetInputhttps://crawlee.dev/api/core/function/purgeDefaultStoragespurgemethod on the storage client, so this also means both memory storage and apify client need to implement thispurgemethodRelated: apify/apify-cli#545