ZSTD Bindings for Python
.. |releaseW| image:: https://github.com/sergey-dryabzhinsky/python-zstd/actions/workflows/build-wheels.yml/badge.svg?branch=v1.5.5.1 :target: https://github.com/sergey-dryabzhinsky/python-zstd/actions/workflows/build-wheels.yml
.. |masterW| image:: https://github.com/sergey-dryabzhinsky/python-zstd/actions/workflows/build-wheels.yml/badge.svg :target: https://github.com/sergey-dryabzhinsky/python-zstd/actions/workflows/build-wheels.yml
+---------+------------+ | branch | status | +=========+============+ | Release | |releaseW| | +---------+------------+ | Master | |masterW| | +---------+------------+
Simple python bindings to Yann Collet ZSTD compression library.
Zstd, short for Zstandard, is a new lossless compression algorithm, which provides both good compression ratio and speed for your standard compression needs. "Standard" translates into everyday situations which neither look for highest possible ratio (which LZMA and ZPAQ cover) nor extreme speeds (which LZ4 covers).
It is provided as a BSD-license package, hosted on GitHub_.
.. _GitHub: https://github.com/facebook/zstd
If you setup 1.0.0.99.1 version - remove it manualy to able to update. PIP matching version strings not tuple of numbers.
Result generated by versions prior to 1.0.0.99.1 is not compatible with orignial Zstd by any means. It generates custom header and can be read only by zstd python module.
As of 1.0.0.99.1 version it uses standard Zstd output, not modified.
To prevent data loss there is two functions now: compress_old and decompress_old.
They are works just like in old versions prior to 1.0.0.99.1.
As of 1.1.4 version module build without them by default.
As of 1.3.4 version these functions are deprecated and will be removed in future releases.
As of 1.5.0 version these functions are removed.
DISCLAIMER
These python bindings are kept simple and blunt.
Support of dictionaries and streaming is not planned.
$ git clone https://github.com/sergey-dryabzhinsky/python-zstd $ git submodule update --init $ apt-get install python-dev python3-dev python-setuptools python3-setuptools $ python setup.py build_ext clean $ python3 setup.py build_ext clean
Note: Zstd legacy format support disabled by default.
To build with Zstd legacy versions support - pass --legacy option to setup.py script:
$ python setup.py build_ext --legacy clean
When using a PEP 517 builder you can use ZSTD_LEGACY environment variable instead:
$ ZSTD_LEGACY=1 python -m build -w
Note: Python-Zstd legacy format support removed since 1.5.0.
If you need to convert old data - checkout 1.4.9.1 module version. Support of it disabled by default.
To build with python-zstd legacy format support (pre 1.1.2) - pass --pyzstd-legacy option to setup.py script:
$ python setup.py build_ext --pyzstd-legacy clean
If you want to build with existing distribution of libzstd just add --external option.
But beware! Legacy formats support state is unknown in this case.
And if your version not equal with python-zstd - tests may not pass.
$ python setup.py build_ext --external clean
When using a PEP 517 builder you can use ZSTD_EXTERNAL environment variable instead:
$ ZSTD_EXTERNAL=1 python -m build -w
If paths to header file zstd.h and libraries is uncommon - use common build params:
--libraries --include-dirs --library-dirs.
$ python setup.py build_ext --external --include-dirs /opt/zstd/usr/include --libraries zstd --library-dirs /opt/zstd/lib clean
for Python 2.7+
$ pip install zstd
or for Python 3.4+
$ pip3 install zstd
API
Error Standard python Exception for zstd module
ZSTD_compress (data[, level, threads]): string|bytes Function, compress input data block via mutliple threads, return compressed block, or raises Error.
Params:
Aliases: compress(...), dumps(...)
Exception if:
Max number of threads:
Since: 0.1
ZSTD_uncompress (data): string|bytes Function, decompress input compressed data block, return decompressed block, or raises Error.
Support compressed data with multiple/concatenated frames (blocks) (since 1.5.5.1).
Params:
Aliases: decompress(...), uncompress(...), loads(...)
Since: 0.1
version (): string|bytes Returns this module doted version string.
The first three digits are folow libzstd version. Fourth digit - module release number for that version.
Since: 1.3.4.3
ZSTD_version (): string|bytes Returns ZSTD library doted version string.
Since: 1.3.4.3
ZSTD_version_number (): int Returns ZSTD library version in format: MAJOR100100 + MINOR*100 + RELEASE.
Since: 1.3.4.3
ZSTD_threads_count (): int Returns ZSTD determined CPU cores count.
Since: 1.5.4.1
ZSTD_max_threads_count (): int Returns ZSTD library determined maximum working threads count.
Since: 1.5.4.1
ZSTD_external (): int Returns 0 of 1 if ZSTD library build as external.
Since: 1.5.0.2
Removed
ZSTD_compress_old (data[, level]): string|bytes Function, compress input data block, return compressed block, or raises Error.
DEPRECATED: Returns not compatible with ZSTD block header
REMOVED: since 1.5.0
Params:
Since: 1.0.0.99.1
ZSTD_uncompress_old (data): string|bytes Function, decompress input compressed data block, return decompressed block, or raises Error.
DEPRECATED: Accepts data with not compatible with ZSTD block header
REMOVED: since 1.5.0
Params:
Since: 1.0.0.99.1
Use
Module has simple API:
import zstd dir(zstd) ['Error', 'ZSTD_compress', 'ZSTD_external', 'ZSTD_uncompress', 'ZSTD_version', 'ZSTD_version_number', 'doc', 'file', 'loader', 'name', 'package', 'spec', 'compress', 'decompress', 'dumps', 'loads', 'uncompress', 'version'] zstd.version() '1.5.1.0' zstd.ZSTD_version() '1.5.1' zstd.ZSTD_version_number() 10501 zstd.ZSTD_external() 0
In python2
data = "123456qwert"
In python3 use bytes
data = b"123456qwert"
cdata = zstd.compress(data, 1) data == zstd.decompress(cdata) True cdata_mt = zstd.compress(data, 1, 4) cdata == cdata_mt True data == zstd.decompress(cdata_mt) True