LZ4 compression
- Anmelden oder Registrieren um Kommentare zu schreiben
I wanted to make a backup of several large software packages officially released in bzip2 or xz format. The extraction of official packages took a minute or more. Compression could take even longer.
I typed the compression command (tar + lz4) in the terminal. When I lifted my finger from the Enter key, the compression completed. 2 or 3 seconds at most. The resulting .tar.lz4 file is only ~10% larger than the original .tar.bz2 or .tar.xz package.
Given the fact that I need to backup TBs of experimental data in my lab, I feel LZ4 suitable for my application. (Zstd is not fast enough.)
Yes, lz4 is very fast.
https://en.wikipedia.org/wiki/LZ4_(compression_algorithm)
According to the official benchmarks, the decompression speed of latest LZ4 approaches 5 GB/s (based on high-performance CPU), even faster than a PCIe x4 NVMe SSD.
On my most powerful mobile workstation (Xeon processor, 64 GiB RAM, two NVMe SSDs plus one SATA SSD), it does instantly decompress almost anything.
- Anmelden oder Registrieren um Kommentare zu schreiben