The tarball crawl bundles Bamboo currently use make it expensive to retrieve the logs from for QA reporting and analysis. UKWA use zip64 crawl log/config bundles, might nice to use the same structure.
@anjackson pointed me at the relevant part of their code: https://github.com/ukwa/python-shepherd/blob/2016-pulse/crawl/job/output.py#L153-L175
The tarball crawl bundles Bamboo currently use make it expensive to retrieve the logs from for QA reporting and analysis. UKWA use zip64 crawl log/config bundles, might nice to use the same structure.
@anjackson pointed me at the relevant part of their code: https://github.com/ukwa/python-shepherd/blob/2016-pulse/crawl/job/output.py#L153-L175