This repository was archived by the owner on Jun 2, 2024. It is now read-only.

Pipelined parallel extract - #407

Closed

cosmicexplorer wants to merge 11 commits into

zip-rs:masterfrom

cosmicexplorer:pipelined-extract

Commits on Sep 18, 2023

and we got a benchmark
cosmicexplorer
committed
47d2c39
significantly faster pipelined extraction!!!
cosmicexplorer
committed
009c6dd

Commits on Sep 20, 2023

using a secondary rayon threadpool is SLOWER than manual threading!!
cosmicexplorer
committed
5d939b8
convert Vec<u8> to Bytes/BytesMut
cosmicexplorer
committed
ddd7cf5
add once_cell
cosmicexplorer
committed
e0afdd4
conver to rayon::join
cosmicexplorer
committed
ebb9148
reuse the file handle to set permissions
cosmicexplorer
committed
321ffca
clarify randomness in bench (means incompressibility)
cosmicexplorer
committed
24e76c7
move dir creation out of the hot path
cosmicexplorer
committed
1922204
avoid the need to reverse by hand
cosmicexplorer
committed
e4dbf6e

Commits on Sep 21, 2023

ok the pipelining works perfectly and lucidrously fast now
cosmicexplorer
committed
f4c11f1