Comparison of gzip, bzip2 and xz compression tools.
When storing data for archiving or transferring big files over the network the use of compression is often considered. Then we need to…
When storing data for archiving or transferring big files over the network the use of compression is often considered. Then we need to decide which tool to be use. In this article will test three of the most common compression tools to compress a 100MB ASCII file using all available compression levels VS compression speed, decompression speed, memory consumption, compression ratio.
Tests preparation / environment used
- The file (compressed) that we will use for our tests can be find here http://mattmahoney.net/dc/enwik8.zip
- gzip version 1.10
- bzip2 version 1.0.8
- xz version 5.2.4 (liblzma 5.2.4)
- OS: Windows 11 / WSL 2 Ubuntu 20.04
- CPU: AMD Ryzen 7 Mobile 3700U, 4 Cores 8 Threads
- RAM: 16GB DDR4
- SSD: WDC PC SN530
Notes on the environment: Most likely if the same test is going to be performed on another environment the measurements will differ, and this might or might not affect the comparison of the tools in terms which is better, so if you want to test it on your environment you can find the script that performs the tests in my GitHub repository
kpatronas/compression_test: Do compression tests with gzip,bzip2 and xz (github.com)
Testing gzip
- Using level 1 compression , compression percentage was 57% in 1.7 sec
- Using level 4 compression, compression percentage was 62% in 2.5 sec
- levels 5 to 9 had not any significant compression change from level 4 and compression speed was much slower
- max memory consumption for compression and decompression was low in any level
- decompression time had not great variations despite different compression levels, average decompression speed was 0.8 sec
It seems that gzip best use case is to be used with level 4 compression if you want good compression to compression speed ratio, level 1 compression was 57.7% and level 9 was 63.55%
Testing bzip2
- Using level 1 compression, compression percentage was 66.74% in 8.1 sec.
- Using level 5 compression, compression percentage was 70% in 8.0 sec.
- Using level 9 compression, compression percentage was 70.99% in 8.7 sec.
- Max memory usage increases for both compression and decompression levels increase, but the needed resources are not high.
- Decompression time for level 1 compressed file is 3.6 sec and 5.2 sec for a level 9 file (44% increase of decompression time).
It seems that bzip2 compression performance is slow for any given level but does not increase despite the level is used, level 1 compression offers a 66.74% compression and level 9 a 70.99%, but there is a decompression slowdown up to 44% if the same file is compressed with a level 9 compression.
Testing xz
- Using level 1 compression, compression percentage was 66.72% in 13.0 sec.
- Using level 5 compression, compression percentage was 73.48% in 1m22sec.
- Using level 9 compression, compression percentage was 75.13% in 1m59sec.
- Max memory usage is very high if we consider that we try to compress/decompress a 100MB file, its about 11MB for level 1 compression and 690MB for level 9
- Compression is very slow, 13 sec for level 1 and 1m:59sec for level 9
- Decompression is very fast! ~ 1.5 sec for any level
XZ offers the best compression in its levels 4 and up but is very slow and memory consuming, but it offers very fast decompression
Conclusion
Its tool has its pros and cons, and depending on the needs and the limitations an organization can choose the right tool, for the average user that needs a tool to do casual compression of files and does not care too much about the compression time bzip2 with a level 5 is the best choice.
One more thing
There is a cool web site named loopcv , its super cool! you just create an account, upload your CV and automatically apply for jobs that match your interests.
LoopCV — The first job search automation platform
