compress
import compress
The compress library provides in-memory and streaming compression and
decompression across several algorithms, plus archive-format support for
TAR and ZIP.
Compression methods
| Method | Submodule | Encode | Decode | Streaming | Notes |
|---|---|---|---|---|---|
| Deflate | compress.deflate | yes | yes | yes | RFC 1951 raw compress stream. |
| Zlib | compress.zlib | yes | yes | yes | RFC 1950 wrapper around Deflate; also re-exports Deflate’s GzipDecoder/GzipEncoder as DeflateDecoder/DeflateEncoder. |
| Gzip | compress.gzip | yes | yes | yes | RFC 1952 wrapper around Deflate. GzFile treats a .gz file like a regular file. |
| Zstandard | compress.zstd | yes | yes | yes | Pure-Rust implementation (zrip). |
| LZ4 | compress.lz4 | yes | yes | yes | Frame format, via lz4_flex. |
| Bzip2 | compress.bzip2 | yes | yes | yes | Pure-Rust implementation (libbz2-rs-sys). BzFile treats a .bz2 file like a regular file. Only decodes the first stream in a bzip2 “multistream” (see the submodule’s own docs). |
| Brotli | compress.brotli | yes | yes | decode only | Pure-Rust implementation. The streaming encoder buffers all input and compresses once in finish() rather than incrementally: see BrotliEncoder’s own docs. |
Every streaming submodule follows the same shape: a compress()/
decompress() one-shot pair for when the whole buffer is already in
memory, plus an Encoder/Decoder class pair
(write()/flush()/finish()/reset() and
read()/read_exact()/read_all()/read_as_string() respectively)
for when the input arrives incrementally or the output needs to be
consumed as it’s produced. total_in()/
total_out()/available()/finished() are available on both sides of
every streaming class for introspection.
compress.checksum provides CRC-32 and Adler-32, independent of any
particular compression method: used internally by zip to validate
extracted entries, but also usable standalone.
Archive formats
| Format | Submodule | Read | Write | Compression methods |
|---|---|---|---|---|
| TAR | compress.tar | yes | yes | none, gzip, bzip2 (COMPRESS_NONE/COMPRESS_GZIP/COMPRESS_BZIP) |
| ZIP | compress.zip | yes | yes | stored, deflate, bzip2 (ZIP_STORED/ZIP_DEFLATE/ZIP_BZIP2); zip64 supported for archives/files exceeding the classic 4 GiB limit |
Both tar and zip expose a low-level class (tar.Tar,
zip.ZipArchive) for building an archive entry by entry or inspecting
one in detail, and a pair of one-shot free functions
(compress()/extract()) for the common case of “archive this whole
file or directory” / “extract this whole archive”. zip.extract()
refuses to write outside the destination directory even if the archive
contains ../-style path-traversal entry names (a “Zip Slip” attack); a
ZIP entry that fails to decode cleanly (an unsupported or encrypted
entry, or one whose CRC-32 doesn’t match what the archive claims) is
reported per-entry via ZipItem.error rather than silently skipped or
allowed to abort the whole read.
TAR does not currently support the LZMA/xz compression method some
.tar.xz archives use, nor Brotli (no established TAR convention exists
for it). ZIP does not currently support LZMA (method 14), PPMd (method
98), or any of the ZIP encryption schemes (traditional PKWARE or AES):
an encrypted entry is reported via ZipItem.error rather than
attempted.
The compress API
Every public name in compress, wherever it is declared. Each links to
the page that documents it.
| Name | Kind | Summary |
|---|---|---|
compress.BEST_COMPRESSION | constant | Best compression level. |
compress.BEST_SPEED | constant | Best speed compression. |
compress.DEFAULT_COMPRESSION | constant | Default compression level. |
compress.DEFAULT_MEMORY_LEVEL | constant | Default memory level |
compress.DEFAULT_STRATEGY | constant | Default compression strategy. |
compress.FILTERED | constant | Filtered compression strategy. |
compress.FIXED | constant | Fixed compression strategy. |
compress.HUFFMAN_ONLY | constant | huffman only compression strategy |
compress.MAX_WBITS | constant | Maximum windows bit. |
compress.NO_COMPRESSION | constant | No compression level. |
compress.RLE | constant | Rle compression strategy. |
compress.ZlibDecoder | class | Streaming zlib decompressor implemention. |
compress.ZlibEncoder | class | A streaming Deflate encoder. |
compress.brotli.BrotliDecoder | class | Streaming Brotli decompressor implementation. |
compress.brotli.BrotliEncoder | class | A streaming Brotli encoder. |
compress.brotli.compress | function | Compress data using Brotli. |
compress.brotli.decompress | function | Decompress Brotli-compressed data. |
compress.bzip2.BzFile | class | The BzFile class implements a Bzip2 based I/O system that allows you to treat Bzip2 streams (bytes) as if… |
compress.bzip2.Bzip2Decoder | class | Streaming bzip2 decompressor implementation. |
compress.bzip2.Bzip2Encoder | class | A streaming Bzip2 encoder. |
compress.bzip2.compress | function | Compress data using the default options for Bzip2. |
compress.bzip2.decompress | function | Decompress Bzip2-compressed data. |
compress.checksum.adler32 | function | Updates a running Adler-32 checksum with the bytes buf[0,len-1] and return the updated checksum. |
compress.checksum.crc32 | function | Update a running CRC-32 checksum with the bytes buf[0,len-1] and return the updated CRC-32 checksum. |
compress.compress | function | Compress compresses as much data as possible, and stops when the input buffer becomes empty or the output… |
compress.decompress | function | Decompress decompresses as much data as possible, and stops when the input buffer becomes empty or the output… |
compress.deflate.DeflateDecoder | class | Streaming deflate decompressor implemention. |
compress.deflate.DeflateEncoder | class | A streaming Deflate encoder. |
compress.deflate.compress | function | Compress data using the default options for Deflate. |
compress.deflate.decompress | function | Decompress a deflated data using default options. |
compress.gzip.GzFile | class | The GzFile class implements a GZip based I/O system that allows you use treat Gzip streams (bytes) as if they… |
compress.gzip.GzipDecoder | class | Streaming gzip decompressor implemention. |
compress.gzip.GzipEncoder | class | A streaming GZip encoder. |
compress.gzip.compress | function | Compress data using the default options for GZip. |
compress.gzip.decompress | function | Decompress a GZipped data using default options. |
compress.lz4.Lz4Decoder | class | Streaming reader for decompressing the LZ4 frame format. |
compress.lz4.Lz4Encoder | class | Streaming lz4 compressor implemention. |
compress.lz4.compress | function | Compress data using the Lz4 block format. |
compress.lz4.decompress | function | Decompress a Lz4 compressed data. |
compress.tar.COMPRESS_AUTO | constant | Automatically select and detect compression type (Default). |
compress.tar.COMPRESS_BZIP | constant | Create and read archives with the BZip2 compression method. |
compress.tar.COMPRESS_GZIP | constant | Create and read archives with the GZip compression method. |
compress.tar.COMPRESS_NONE | constant | Create and read archives without any compression. |
compress.tar.Tar | class | |
compress.tar.TarCorruptedError | class | Error thrown when a TAR archive is corrupted. |
compress.tar.TarIOError | class | Error thrown when an I/O error occurs. |
compress.tar.TarIllegalCompressionError | class | Error thrown when an illegal compression type is used. |
compress.tar.compress | function | Create a new TAR ball from the file or directory in the given path and saves it to the destination path or… |
compress.tar.extract | function | Extracts a TAR file to the given destination or to the same directory as the source file with the same name… |
compress.zip.ZIP_BZIP2 | constant | Compression method that indicates Bzip2 compression |
compress.zip.ZIP_DEFLATE | constant | Compression method that indicates zlib Deflate compression |
compress.zip.ZIP_EXT | constant | The default zip file extension |
compress.zip.ZIP_FILE_COUNT_LIMIT | constant | The maximum number of files in a zip archive when zip64 is not used |
compress.zip.ZIP_FILE_MAX | constant | |
compress.zip.ZIP_MAX | constant | The maximum size of a zip archive when zip64 is not used |
compress.zip.ZIP_STORED | constant | Compression method that indicates no compression |
compress.zip.ZipArchive | class | ZipArchive provides a class for zip archive creation, manipulation and extraction. |
compress.zip.ZipFile | class | ZipFile represents an instance of zip file. |
compress.zip.ZipItem | class | ZipItem represents a single file or directory in a zip archive. |
compress.zip.compress | function | Compresses the given path (file or directory) into the destination zip archive. |
compress.zip.extract | function | Extracts the zip archive at the file path to the given destination directory. |
compress.zstd.ZstdDecoder | class | Streaming zstd decompressor implemention. |
compress.zstd.ZstdEncoder | class | Streaming zstd compressor implemention. |
compress.zstd.compress | function | Compress data using the default options for Zstd. |
compress.zstd.decompress | function | Decompress a Zstd compressed data. |
Submodules
| Module | Reached as | Summary |
|---|---|---|
compress.brotli | compress.brotli.* | The Brotli submodule for the compress module. |
compress.bzip2 | compress.bzip2.* | The Bzip2 submodule for the compress module. |
compress.checksum | compress.checksum.* | This is the checksum submodule for the compress module. |
compress.deflate | compress.deflate.* | This is the Deflate submodule for the compress module. |
compress.gzip | compress.gzip.* | The is the GZip submodule for the compress module. |
compress.lz4 | compress.lz4.* | This is the Lz4 submodule for the compress module. |
compress.tar | compress.tar.* | This module adds support for creating and extracting TAR archives. |
compress.zip | compress.zip.* | The zip module contains classes and functions to make working with zip archives easy. |
compress.zlib | compress.* | This is the Zlib submodule for the compress module. |
compress.zstd | compress.zstd.* | This is the Zstd submodule for the compress module. |
1995-2017 Jean-loup Gailly and Mark Adler