Zstandard – Super fast data compression tool for Linux

This is known topic and we were using in our day to day activity to compress and decompress files & folders. You might already know zip,tar,7-zip, etc and you would have used all these application for your requirement.

Even today also we are going to discuss about similar kind of topic, the tool name is Zstandard. It is super fast data compression tool and compression ratio is very very low.

Zstd is lossless data compression algorithm developed by Yann Collet at Facebook.

Due to high data compression ratio many of the popular companies and databases are using this tool.

What is Zstandard

Zstandard called zstd as short version, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios.

It’s backed by a very fast entropy stage, provided by Huff0 and FSE library. Zstd can also offer stronger compression ratios at the cost of compression speed. Speed vs Compression trade-off is configurable by small increments.

Decompression speed is preserved and remains roughly the same at all settings, a property shared by most LZ compression algorithms, such as zlib or lzma. It also offers a special mode for small data, called dictionary compression, and can create dictionaries from any sample set.

These compression gains are achieved while simultaneously providing faster compression and decompression speeds. This utility producing and decoding .zst, .gz, .xz and .lz4 files.

How to install Zstandard

Zstandard is available in most of the distribution official repository that can be installed through package manager.

For Debian/Ubuntu, use APT-GET Command or APT Command to install Zstandard.

$ sudo apt install zstd

For RHEL/CentOS, use YUM Command to install Zstandard.

$ sudo yum install zstd

For Fedora, use DNF Command to install Zstandard.

$ sudo dnf install zstd

For Arch Linux, use Pacman Command to install Zstandard.

$ sudo pacman -S zstd

For openSUSE, use Zypper Command to install Zstandard.

$ sudo zypper in zstd

How to use Zstandard

It uses standard Linux format so, just use zstd command followed by the filename to compress the file.

$ zstd filename

The above command, create a compressed file with appending .zst extension to the original filename.

Add -o option followed by the file name to create a compressed file with new name.

$ zstd filename -o newfile.zst

Run below commands to uncompress any file compressed with zstandard.

$ zstd -d filename.zst
$ unzstd filename.zst

Zstandard Comparison Table

As discussed in the beginning of article. Zstandard offer super speed and high compression ratio. Hence, we are going to comparing this with other compression tools called zip & tar.

I have a .sql file with 102MB for this testing and the results are follow.

$ ls -lh daygeekc_magi.sql 
-rw-rw-r-- 1 daygeek daygeek 102M Jun 21 16:15 daygeekc_magi.sql

For Zstd: It taken only 1 seconds and created 5.8MB of compressed file.

$ date; zstd daygeekc_magi.sql; date
Thu Jun 21 16:16:26 IST 2018
daygeekc_magi.sql    :  5.62%   (106770640 => 6001513 bytes, daygeekc_magi.sql.zst) 
Thu Jun 21 16:16:27 IST 2018

$ ls -lh daygeekc_magi.sql.zst 
-rw-rw-r-- 1 daygeek daygeek 5.8M Jun 21 16:15 daygeekc_magi.sql.zst

For zip: It taken only 2 seconds and created 9.7MB of compressed file.

$ date; zip daygeekc_magi.sql.zip daygeekc_magi.sql; date
Thu Jun 21 16:17:02 IST 2018
  adding: daygeekc_magi.sql (deflated 90%)
Thu Jun 21 16:17:04 IST 2018

$ ls -lh daygeekc_magi.sql.zip 
-rw-r--r-- 1 daygeek daygeek 9.7M Jun 21 16:17 daygeekc_magi.sql.zip

For tgz: It taken only 2 seconds and created 9.7MB of compressed file.

$ date; tar czf daygeekc_magi.sql.tgz daygeekc_magi.sql; date
Thu Jun 21 16:18:37 IST 2018
Thu Jun 21 16:18:39 IST 2018

$ ls -lh daygeekc_magi.sql.tgz 
-rw-r--r-- 1 daygeek daygeek 9.7M Jun 21 16:18 daygeekc_magi.sql.tgz

About Prakash Subramanian

Prakash Subramanian is a Linux lover and has 3.5+ years of experience in linux server administration with major Linux distribution such as (RHEL, CentOS, Ubuntu). He is currently working as a Senior L2 Linux Server administrator.

View all posts by Prakash Subramanian

Leave a Reply

Your email address will not be published. Required fields are marked *