StorageCache documents byAlias as keyed by a "run-scoped alias", but neither backend purges an aliased storage on start, so alias-keyed storages outlive the run like named ones do.
FileSystemStorageBackend.purge opens exactly three stores, all { alias: '__default__' }, and purges those — it never enumerates the cache. MemoryStorageBackend.purge filters on name === 'default' || cacheKey === 'default', and an alias resolves to its own cache key with name === undefined, so it never matches.- Visible without opting into anything:
openOwnedRequestQueue gives every crawler instance after the first an { alias: '__default_<id>__' } queue, so a second crawler in the same process silently resumes the previous run's requests even with purgeOnStart enabled.
- crawlee-python purges any storage whose
name is None, which covers aliases — _purge_if_needed there is explicit that named storages are the ones meant to survive.
Reproduced cross-process on the file-system backend with purgeOnStart: true: a request written to { alias: 'throttled-example.com' } is still pending after purgeDefaultStorages() in a fresh process, while the default queue's leftover is correctly cleared.
Found while reviewing #3741, which relies on the current behaviour to leave rate-limited requests queued for a later run — worth deciding the intended semantics before fixing, since that PR would need a different mechanism.
StorageCachedocumentsbyAliasas keyed by a "run-scoped alias", but neither backend purges an aliased storage on start, so alias-keyed storages outlive the run like named ones do.FileSystemStorageBackend.purgeopens exactly three stores, all{ alias: '__default__' }, and purges those — it never enumerates the cache.MemoryStorageBackend.purgefilters onname === 'default' || cacheKey === 'default', and an alias resolves to its own cache key withname === undefined, so it never matches.openOwnedRequestQueuegives every crawler instance after the first an{ alias: '__default_<id>__' }queue, so a second crawler in the same process silently resumes the previous run's requests even withpurgeOnStartenabled.nameisNone, which covers aliases —_purge_if_neededthere is explicit that named storages are the ones meant to survive.Reproduced cross-process on the file-system backend with
purgeOnStart: true: a request written to{ alias: 'throttled-example.com' }is still pending afterpurgeDefaultStorages()in a fresh process, while the default queue's leftover is correctly cleared.Found while reviewing #3741, which relies on the current behaviour to leave rate-limited requests queued for a later run — worth deciding the intended semantics before fixing, since that PR would need a different mechanism.