zpaq is a journaling, incremental, deduplicating archiver for Windows and Linux. "Journaling" means that when you update a file or directory, both the old and new versions are saved. You can extract from the archive as it existed at any time in the past. "Incremental", means that when you back up your entire hard drive, for example:
zpaq -add e:backup.zpaq c:\*that only those files whose last-modified date has changed since the previous backup are added. For 100 GB of files, this typically takes 1-2 minutes, vs. a few hours to create the first version. "Deduplicating" means that identical files or fragments are stored only once to save time and space.
Files are compressed in the open standard ZPAQ format, which is supported by a specification, reference decoder, and a test case (which should decode to the Calgary corpus), and a public domain API in C++ that provides compression and decompression services. The format is not known to be encumbered by any patents. ZPAQ stores the decompression algorithm descripion in the archive, which means that when the archive is updated to a better compression algorithm, older extractors will still be able to read the new archives.
zpaq v6.27 (zip) includes the following files:
You might need unzip.exe v6.00 to extract zip archives from the Windows command line.
Github fork of zpaq v6.16 maintained by Yonggang Luo.
WinZPAQ GUI interface by Sportman.
The following test shows what happens when the 32 bit MinGW g++ compiler version 4.4.0 is backed up, followed by version 4.5.0, which has many files (40% of total size) in common, and then both directories are extracted.
Archiver Create mingw44 Time Add mingw45 Time Extract (CPU) Free Open Spec
------- -------------- ----- ----------- ---- ------- ----- ---- ---- ----
uncompressed 116,655,760 276,348,369
tar -r (Win32) 117,841,920 1.7 278,814,720 2.9 17.7 7.7 Yes Yes Yes
(Linux) 117,903,360 0.13 278,927,360 0.20 0.30
exdupe (Linux) 60,453,170 4.0 159,094,874 4.2 2.1 0.85 No Yes No
zip (Win32) 54,704,677 14.8 140,666,060 19.3 20.0 11.1 Yes Yes Yes
zip -9 54,479,404 23.4 140,178,119 29.3 19.0 11.4
zip (Linux) 54,587,804 5.0 140,418,752 6.7 2.6 2.0
zip -9 54,362,531 9.5 139,930,811 12.1 2.4 1.9
rar (Win32) 49,766,504 30.3 129,667,053 43.9 37.9 9.3 No UnRAR No
rar -s 41,892,363 28.8 112,591,881 42.9 32.8 9.3
rar -s -m5 41,796,989 35.6 112,379,699 51.8 31.8 8.6
rar -ma5 49,622,311 29.6 128,072,934 44.9 38.9 11.0
rar -s -ma5 39,353,664 33.3 80,580,793 51.8 31.8 8.6
rar -s -m5 -ma5 39,247,216 44.6 80,323,216 58.2 33.1 9.0
7zip (Win32) 38,517,699 54.5 105,112,527 74.1 40.8 12.1 Yes Yes No
7zip -mx 37,372,117 60.1 76,420,114 83.3 31.1 11.8
7zip (Linux) 38,688,771 32.5 105,709,753 44.1 8.5 8.0
7zip -mx 37,796,121 39.7 77,478,904 54.9 6.5 6.0
arc -m1 (Win32) 47,532,588 3.1 93,410,320 7.0 21.5 10.8 Yes Yes No
arc -m2 41,709,395 5.4 85,154,022 14.3 28.1 15.7
arc -m3 40,758,956 8.8 82,910,637 25.0 29.3 16.6
arc -m4 39,959,885 18.0 81,110,925 43.1 30.4 18.9
arc -m5 37,276,959 56.4 73,346,814 118.2 37.8 21.0
arc -m7 37,005,360 57.9 72,251,977 126.5 39.2 23.3
arc -m9 37,004,608 60.7 71,272,360 154.0 38.2 22.6
arc -m9x 37,406,338 69.9 72,041,614 182.2 33.6 17.4
nz -cf (Win32) 53,923,985 1.5 132,731,778 3.9 22.1 7.0 Yes No No
nz -cd 42,954,791 3.9 82,506,506 7.8 21.3 6.9
nz -co 37,653,519 66.0 73,259,856 115.6 39.5
nz -cc 35,089,232 332.5 68,373,452 815.3 777.7
nz -cf (Linux) 132,734,286 1.4 2.5
nz -m3.5g -cf 132,734,714 1.3 1.4
nz -cF 90,646,897 1.9 2.0
nz -m3.5g -cF 90,646,570 1.9 1.9
nz -cd 82,505,268 3.9 1.1
nz -m3.5g -cd 82,505,524 3.8 1.0
nz -cdp 79,212,425 8.4 1.2
nz -m3.5g -cdp 79,212,436 8.4 1.8
nz -cdP 78,470,838 9.4 1.2
nz -m3.5g -cdP 78,470,838 9.5 1.2
nz -cD 79,093,418 8.8 1.3
nz -m3.5g -cD 79,093,249 8.8 1.5
nz -cDp 77,175,812 17.5 1.5
nz -m3.5g -cDp 77,175,791 17.8 1.5
nz -cDP 76,279,837 16.9 1.5
nz -m3.5g -cDP 76,279,837 16.8 1.5
nz -co 73,263,227 59.2 16.6
nz -m3.5g -co 72,565,375 76.9 21.0
nz -cO 69,391,570 162.6 51.8
nz -m3.5g -cO 69,284,756 170.3 50.7
nz -cc 68,377,052 433.8 421.6
nz -m3.5g -cc 68,027,938 486.4 472.2
zcm -r (Win32)45,004,528 276.8 119,374,717 313.3 298.4 Yes No No
zcm -r -m6 -t2 -s 38,021,034 94.2 78,026,436 118.2 175.3
zcm -r -m6 -t1 -s 36,174,231 68.5 73,968,725 93.2 161.0
zcm -r -m7 -t1 -s 36,115,632 86.0 73,846,883 101.6 176.9
zpaq -method 1 (win32)46,156,823 6.6 93,587,997 9.5 21.5 11.8 Yes Yes Yes
zpaq -method 2 42,800,065 15.1 86,537,822 22.5 32.0
zpaq -method 3 40,921,225 46.0 83,097,275 47.7 74.3
zpaq -method 4 37,728,233 145.7 76,693,484 152.6 275.1
zpaq -method 5 36,393,788 572.5 73,505,007 570.7 1126.1
zpaq -method 1 (Linux)46,156,912 3.0 93,588,793 4.1 2.8
zpaq -method 2 42,800,154 5.8 86,538,618 9.0 9.3
zpaq -method 3 40,921,314 20.3 83,098,071 21.1 38.4
zpaq -method 4 37,728,322 68.5 76,694,280 71.2 138.8
zpaq -method 5 36,393,877 268.5 73,505,803 281.8 512.7
zpaq -method 6 35,861,071 281.7 72,647,140 326.8 528.5
zpaq -method 7 35,829,691 467.2 72,339,570 493.0 570.7
The test data (88 MB, compressed with zpaq -method 2) consists of two directories:
tar and zip are single threaded. Other programs are multi-threaded and use all available cores.
Sample configuration files. zpaqd is a development tool allows you to create, test, debug, and optimize new compression algorithms that are compatible with all existing ZPAQ decompressors such as zpaq, zpipe, zpsfx, and the reference decoder. An algorithm is described in the ZPAQL language by a configuration (.cfg) file and sometimes an external preprocessor. Neither file is needed to decompress. See libzpaq.h for a description of the ZPAQL language.
fast.cfg, mid.cfg, and max.cfg describe the built in models for levels 1, 2, and 3 in libzpaq and zpipe. mid is equivalent to compression level -method 4 in zpaq. They are context mixing models without preprocessors using 2, 8, and 22 components respectively. With zpaq, you can pass an argument to increase memory usage to improve compression of large files.
bwt.1.zip contains 4 BWT based configurations with preprocessors (C++ source and Win32 executables) similar to -method 3. All are based on the Burrow-Wheeler transform, which sort the input by its following context, then compress with a low order model. The preprocessor uses Yuta Mori's libdivsufsort (open source MIT license).
bwt_slowmode1.zip is a low memory BWT compression algorithm by Jan Ondrus. It is based on on BBB slow mode with block sizes up to 1 GB using 1443 MB memory, instead of the usual 5 times block size. It compresses enwik9 to 163,565,006 bytes in a single block.
bmp_j4c.zip by Jan Ondrus uses a color transform preprocessor and a context model based on paq8px_v64 for compressing .bmp images. It achieves the best known result for the rafale.bmp benchmark, 521,992 bytes.
exe_j1.zip by Jan Ondrus contains a E8E9 preprocessor that improves compression of .exe and .dll files by translating x86 JMP and CALL addresses from relative to absolute. The context model is the same as max.cfg, but it typically compresses these file types 7-10% smaller at the same speed.
jpg_test2.zip by Jan Ondrus compresses JPEG images (which are already compressed) by an additional 15%. It uses a preprocessor that expands Huffman codes to whole bytes, followed by context modeling.
pi.cfg compresses pi.txt from the Canterbury miscellaneous corpus from 1 MB to 114 bytes. No other compressor is known to compress it to less than 415,241 bytes. It uses a postprocessor that ignores the decoded data and computes pi to 1 million digits. Warning: it may take several hours or days to run. It works only on this file. Discussion.
lz1.zip is a fast LZ77 based model similar to -method 1 and 2. It compresses enwik9 to 271,702,398 bytes in 391 seconds and decompresses in 227 sec. on a 2.0 GHz T3200 (Win32) in a single thread using 66 MB memory to compress and 18 MB to decompress. It include lz1.cfg and a preprocessor lzpre.cpp and lzpre.exe. The preprocessor can also compress by itself with less compression but better speed (327,501,489 bytes, 215 sec, 26 sec). The preprocessed format is a byte-aligned LZ77 using 2, 3, or 4 bytes to encode matches of up to 64 bytes with offsets up to 16 MB as explained in the source comments.
lazy v1.00 is a fast LZ77 compressor (source, Win32 .exe) with 5 compression levels, and equivelent config file (lazy.cfg). Level 3 compresses enwik9 to 325,609,617 in 125 sec (2 GHz T3200) and decompresses in 64 sec to disk or 21 sec. to nul:. It is the compressor used by zpaq -method 1 versions 6.00 through 6.14 (at level 3).
lazy v2.10 is like lazy, but with an E8E9 filter for better compression of .exe and .dll files. File size is limited to 1 GB (unlike lazy, which has no limit), and it needs about 1 GB memory. It includes an equivalent lazy2.cfg for decompression (but limited to 16 MB using 16 MB memory).
Miscellaneous Applications
zpipe (user's guide) is a simple application that compresses or decompresses from standard input to standard output. It illustrates the use of libzpaq. Contains GPL source and a Windows executable (updated to libzpaq v5.01 on Feb. 2, 2012).
zpsfx v1.01 (Apr. 4, 2012) is a self extracting archive stub for Windows. To use: compile, then append an archive created with -streaming or -solid to the executable. Be sure that the archive begins with a header locator tag (included by default). To extract, run the program. For example:
g++ -O2 -s zpsfx101.cpp libzpaq.cpp -o zpsfx.exe upx zpsfx.exe zpaq c archive files... copy/b zpsfx.exe+archive.zpaq archive.exeTo extract:
archive.exe
For either program, you need libzpaq to compile.
tiny_unzpaq.cpp v1.0, Mar. 21, 2012, is a small ZPAQ level 2 extractor derived from unzpaq200.cpp, with error checking, extra code, comments, and white space removed to make the source code compress as small as possible. It compiles by itself. It is written by Matt Mahoney and released to the public domain. To run: tiny_unzpaq archive.zpaq. It will create files as named in the archive. It only extracts files created with zpaqd (not in journaling format).
Technology
ZPAQ is based on the PAQ context mixing model, which was developed over the period 2000-2009. A set of context models independently predict the next bit of input. Then the predictions (probabilities) are combined and used to arithmetic code the bit. Using lots of predictors improves compression but takes more time. ZPAQ allows up to 255 components. The most important types are:
A configuration file describes how the components are connected. For example, mid.cfg describes an order 0 indirect context model whose output is adjusted by a chain of indirect secondary symbol estimators with increasing context order, a high order match model, and a final mixer. The context hashes are computed by a ZPAQL program that is called once for each uncompressed byte.
Preprocessing and postprocessing can be used to implement other algorithms such as LZ77, LZP, BWT, and various specialized transforms like E8E9 for x86, Huffman code expansion for JPEG, and color transforms for BMP. Postprocessing after decompression is written in ZPAQL. Preprocessing before compression is built in to zpaq (LZ77 for -method 1 and 2, BWT for -method 3) or done by an external program. zpaq will verify that postprocessing restores an externally preprocessed file prior to compression. zpaq also has a debugger that allows you to trace ZPAQL code or run it as a stand-alone program.
A ZPAQ archive in streaming format are designed to be read quickly in a single pass like a tar file. It consists of a sequence of blocks that can be decompressed independently in parallel. Each block consists of a sequence of segments that must be decompressed in order. Each segment has an optional filename, an optional comment, compressed data, and an optional SHA-1 checksum. A segment without a filename denotes a continuation of the same file from the previous segment. The end of the compressed data is marked by a sequence of 4 zero bytes, allowing it to be found quickly without having to decompress the data. The arithmetic coder is designed so that it never encodes such a sequence. A block header can be marked with a 13 byte tag that allows it to be found when embedded in other data such as a self extracting archive.
zpaq requires a different format to support journaling and deduplication. Files are fragmented along content-dependent boundaries to an average size of 64 KB and grouped into 16 MB data blocks to be compressed. There are separate blocks containing tables of fragment sizes and SHA-1 hashes, and lists of file additions and deletions. Additions include a list of pointers to fragments and the current date. Data blocks are preceded by a block containing the compressed size, allowing the data to be skipped to create a listing quickly. When a file is added, it is fragmented using a rolling hash that depends on the last 32 bytes not predicted by an order-1 context. The fragment SHA-1 hash is computed and compared with those stored in the archive. If there is a match, then the fragment is omitted and the file is coded using a pointer to the fragment already stored.
What is zpaq for?
zpaq is an archiver. It collects one or more files together and
compresses them so they take up less space. You can use it to make backups,
or to create a package of files to send to someone as an email attachment
or to download from a website. The recipient will need zpaq to
extract the files.
I clicked on the zpaq icon and nothing happened.
zpaq is a command-line program. You have to run it from
a command window in Windows or a Linux shell.
How do I do X?
Read the command line documentation in zpaq.cpp.
Why isn't there a graphical user interface (GUI)?
Because I prefer to work on the engine than the body.
peazip has a GUI and will
read (but not create or update) zpaq archives created in streaming
(but not journaling) formats.
Will it work in Linux or on a Macintosh?
Yes, but you will have to compile it.
How do I compile zpaq?
See the comments in zpaq.cpp.
zpaq crashes on my Windows 95 PC
zpaq (actually libzpaq) uses SSE2 instructions, which are
not available on Intel processors made before 2001 or AMD processors
before 2003. You can try recompiling with -DNOJIT to disable
the just-in-time (JIT) optimization feature, which uses SSE2. It will
run about twice as slow.
What is libzpaq?
It is an application programming interface (API) in C++ that allows developers
to easily compress and decompress data in the ZPAQ format, either in
memory or on disk.
Will libzpaq work for phone or tablet apps?
It has been tested on at least one ARM processor.
You will at least need to compile with -DNOJIT to disable the x86-32/64
just-in-time optimization feature.
How is zpaq licensed?
GPL v3. It is free to use. You can also modify it and distribute modified
versions, but any distribution must be free and under the same license and include
source code. zpaqd is public domain.
How is libzpaq licensed?
It is public domain. You can do anything you want with the code.
Is ZPAQ covered by patents?
As far as I know, no. I have not filed for any patents on any of the methods
used (including PAQ, from which it is derived)
and will not do so. I could not, even if I wanted to, because the
methods have already been disclosed for over a year.
Why haven't you patented it?
Because patents would kill any chance of ZPAQ ever becoming a standard. That is
what happened to CTW, the sort transform, and the arithmetic coding modes
of BZIP and JPEG. deflate succeeded precisely because it was not patented.
Why do you keep all of the old versions of the software?
To establish proof of disclosure of prior art so that nobody can claim any
patents on the technology.
ZPAQ development began on Feb. 15, 2009 with a series of mutually incompatible experimental programs (v0.01 through v0.09, now obsolete). Archives created with these versions could only be read by the same version. The level 1 standard became fixed on Mar. 12, 2009 with version 1.00. All versions from 1.00 onward are level 1 compliant and can extract each other's archives.
The SHA-1 code used in these older versions (prior to libzpaq 1.00) is derived from RFC 3174, which is copyright (C) 2001, The Internet Society. Please see this document for the full license.
zpaq v0.01
Open source (C++) and Win32 executables, Feb. 15, 2009.
zpaq v0.02 adds E8E9 transform. Fully
supports post-processing. Not compatible with v0.01. Feb. 19, 2009.
zpaq v0.03 modifies MIX, MIX2, IMIX to
fix poor compression on large files. Not compatible with v0.02.
Feb. 19, 2009.
zpaq v0.04 modifies train() and
squash() for improved compression. Not compatible with v0.03.
Feb. 21, 2009.
zpaq v0.05 modifies probability
representation and mixer weights to prevent mixer overflow and
to improve compression for highly redundant data.
Not compatible with v0.04. Feb. 26, 2009.
zpaq v0.06 adds SHA1 checksums,
replaces IMIX2 with ISSE. Not compatible with v0.05. Feb. 27, 2009.
zpaq v0.07 improves ISSE and bit-history
state table. Not compatible with v0.06. Feb. 28, 2009.
zpaq v0.08 adds LZP transform and
minor improvements. Not compatible with v0.07. Mar. 8, 2009.
zpaq v0.09 removes counters from
ISSE and ICM to improve speed. Not compatible with v0.09. Mar. 9, 2009.
zpaq v1.00 (first level 1 compliant version) includes
unzpaq1
candidate reference decoder. Simplified bit history tables.
Not compatible with earlier versions. Mar. 12, 2009.
fast.cfg written Apr. 26, 2010.
Now part of libzpaq distribution.
unzpaq 1.01 updates reference
decoder comments and help message
and fixes some VS2005 compiler issues. Compatible with 1.00.
Apr. 27, 2009.
unzpaq 1.02 and zpaq 1.02
closes extracted files immediately after decompression instead of when
program exits. Fixes g++ 4.4 warnings. Compatible with 1.00 and 1.01.
June 14, 2009.
unzpaq 1.03 and zpaq 1.03
has a default compression mode (mid.cfg),
supports compressing files in segments to separate blocks and extracting
them as suggested in part 7 of the spec. Does not store paths by default.
Does not extract to absolute paths by default. Some minor improvements.
Sept. 7, 2009 (added zpaq.exe Sept. 8, 2009).
zpaq103b adds zpaqsfx 1.03,
a stub for creating self extracting
archives. No changes to zpaq or unzpaq. Sept. 14, 2009.
zpaq104 can list and extract from self
extracting archives without running them. Added progress meter.
zpaqsfx.exe stub is slightly smaller. unzpaq unchanged. Sept. 18, 2009.
zpaq105 removes built in x and p preprocessors
and makes them separate programs called from config files with compile
time postprocessor testing. Adds if-else-endif
and do-while to ZPAQL. Many small changes. Sept. 28, 2009.
zpaq106 adds "ta" to append locater tags to
allow ZPAQ streams to be detected when embedded in arbitrary data.
zpaq1.pdf revision 1 adds this recommendation.
unzpaq106.cpp implements it.
Sept. 29, 2009.
zpaqsfx 1.06 self extracting archive stub is now
separate from the ZPAQ distribution.
Updated Sept. 29, 2009, posted Oct. 26, 2009. (Replaced by zpsfx in libzpaq 2.01)
zpipe v1.00, a simple streaming compressor,
Sept. 30, 2009. Linux patch added Jan. 18, 2010.
zpaq107 adds config file parameters and fixes
some bugs. From now on the specification and reference decoder are not
included unless they change. Oct. 2, 2009.
bwt_j2 is a config file (by Jan Ondrus) and
preprocessor for BWT compression. Posted Oct. 7, 2009.
bwt_j3 is a bug fix for bwt_j2 to accept
multiple files. Jan Ondrus, Oct 7, 2009.
exe_j1 is a config file and preprocessor
for .exe and .dll files. It extends the E8E9 transform in exe.cfg
to conditional jumps. Jan Ondrus, Oct. 7, 2009.
unzpaq108.cpp removes undefined behavior
of ZPAQL shifts larger than 31 bits on non x86 hardware.
Oct. 14, 2009.
zpaq108 generates optimized code that runs
about twice as fast on systems with a C++ compiler installed.
Oct. 14, 2009.
bmp_j4, configuration for .bmp files by Jan Ondrus,
Oct. 14, 2009.
bwt_slowmode1 BWT compression based
on BBB slow mode. Jan Ondrus, Oct. 15, 2009.
jpg_test2 JPEG config by Jan Ondrus, Oct. 20, 2009,
posted Oct. 26, 2009.
zpaq109 Linux port and some cosmetic bug fixes.
Oct. 21, 2009.
zpaq110 bug fix for Linux/g++ 4.4.1, Dec. 28, 2009.
zp v1.00 simple ZPAQ compatible archiver with 3 optimized
compression levels, Apr. 26, 2010.
Added a license file to zpaq 1.10, zpipe 1.00, zpaqsfx 1.06, and zp 1.00
distributions on May 23, 2010. No software changes.
libzpaq 0.01, Sept. 27, 2010.
libzpaq 0.02, Sept. 28, 2010.
zpipe 2.00 updated to use libzpaq 0.02, Sept. 28, 2010.
libzpaq 1.00, Sept. 29, 2010. Package includes
libzpaq, ZPAQ specification, reference decoder, zp, zpipe, and fast, mid, max
config files.
libzpaq 1.01, Oct. 14, 2010. Updates libzpaq
interface to use inheritance instead of templates, requiring changes
to zp and zpipe. Now compiles faster.
libzpaq 1.02, Oct. 20, 2010. Adds zpsfx self extracting
archive stub. Separates optimized models from libzpaq.cpp to libzpaqo.cpp.
libzpaq 2.00, Oct. 30, 2010. Ports zpaq to
libzpaq, replacing zp.
libzpaq 2.01, Nov. 5, 2010. Added optimized self
extracting archives. Simplified installation.
libzpaq 2.02, Nov. 13, 2010. zpaq shows
compression component statistics. Libzpaq support added.
zpaq 2.03, Dec. 23, 2010,
adds Linux support. The remaining
code is split into libzpaq 2.02,
zpipe 2.01,
zpsfx 1.00,
and configuration files
min, fast,
mid, and max.
zpaq 2.04, Dec.29, 2010,
adds support for Visual C++, Borland, and Mars compilers in addition to g++.
A Windows install script is added.
zpaq 2.05, Jan. 5, 2011. Fixed a bug in which
zpaq crashed when decompressing an unnamed file (as created with zpipe
or zpaq nc) without renaming. Separated zpaq.1.pod.
(Updated corrupted install.sh on Jan. 13, 2011).
libzpaq 2.02a, Jan. 6, 2011. Updates the documentation.
pzpaq 0.01 parallel file compressor,
Jan. 21, 2011.
pzpaq 0.02, Jan. 26, 2011,
adds large file support (over 2 GB) to Windows.
pzpaq 0.03, Feb. 2, 2011,
optimizes decompression for nonstandard compression levels by recompiling
itself with g++ (like "zpaq ox").
pzpaq 0.04, Feb. 4, 2011,
Windows version uses native threads and no longer requires pthreads-win32.
pzpaq 0.05, Feb. 10, 2011
removes -s option, puts temporary files in $TMPDIR or %TEMP%.
bwt v1, Mar. 16, 2011. 4 BWT based configurations.
unzp 1.00, May 10, 2011, a block level parallel decompresser
optimized for fast, mid, max, bwtrle1, bwt2 models with source level JIT
for other models.
zp 1.01, May 12, 2011, a block level parallel compressor
with 4 levels (bwtrle1, bwt2, mid, max). With unzp replaces pzpaq.
zp 1.02, May 16, 2011. Fixed -t option.
May 18, 2011. Undated zp.102.zip and unzp.100.zip with static x86-64 Linux binaries.
zp 1.03, May 26, 2011. Merges the compressor and decompresser
unzp into one program.
wbpe 1.00, June 12, 2011. Dictionary preprocessor
for text files.
wbpe 1.10, June 21, 2011.
zpaq 3.00, July 16, 2011. Combines features of
zpaq v2.05 and zp v1.03. zp support is discontinued. Windows only.
zpaq 3.01, July 21, 2011. Adds 64 bit Linux support.
Includes libzpaq 3.00.
bmp_j4a, July 21, 2011. Updated bmp_j4 .bmp
configuration for zpaq v3.01.
libzpaq 3.00, July 28, 2011, from zpaq v3.01
but as a separate download.
libzpaq 4.00, Nov. 13, 2011. libzpaq.cpp,
libzpaq.h, libzpaq.3.pod. Replaces
source-level JIT with internal JIT for x86-32 and x86-64.
zpaq 4.00, Nov. 13, 2011. zpaq.cpp, zpaq.1.pod
for use with libzpaq 4.00. Removes source generation,
b and e commands and -j option.
calgarytest.zpaq, Nov. 13, 2011.
Test case for ZPAQ compliance.
zpipe v2.01, Nov. 13, 2011. zpipe.exe linked
to libzpaq v4.00. Source unchanged.
zpaq v4.01, Nov. 26, 2011. Source code adds
incremental update and extraction.
zpaq v4.02, Nov. 28, 2011. Source code adds
commands c, x output/, list hcomp/pcomp. Updated pi.cfg
for this version.
libzpaq v4.01, Dec. 20, 2011. Fix for Mac OS
(MAP_ANONYMOUS -> MAP_ANON).
zpaq v4.03, Dec. 21, 2011. Adds -n, -r, and -f options.
Fixed bug in u (did not save filenames with no args).
lz1.zip, Dec. 29, 2011. LZ77 model.
ZPAQ level 2 standard,
unzpaq200.cpp reference decoder,
libzpaq 5.00 support, and
calgarytest2.zpaq test case, Feb. 1, 2012.
Level 2 allows the COMP section to be empty to store uncompressed (but possibly
preprocessed) data to support faster compression models.
libzpaq 5.01, Feb. 2, 2012. Removed debugging code
from libzpaq.cpp.
tiny_unzpaq.cpp v1.0, Mar. 21, 2012.
zpaq v4.04, Mar. 26, 2012. Fixed bug in r command that
truncated output file.
zpsfx v1.01, Apr. 4, 2012. Self extractor modified
by Klaus Post to create directories as needed.
zpaq v5.00, Aug. 27, 2012. Candidate release,
primarily a small development tool. Updates libzpaq to v6.00a
to include ZPAQL compiler.
zpaq v6.00, Sept.26, 2012. Candidate release. Adds
journaling, incremental update, and deduplication to support large backups.
Includes 4 compression levels (fast and slow LZ77, BWT, mid) plus all
v5.00 features.
zpaq201.pdf, Sept. 28, 2012. Updated specification
to describe streaming and journaling archive formats.
zpaq v6.01, Sept. 28, 2012. Adds -method 0, -list -force,
improves -list -detailed, and bug fixes.
zpaq v6.02, Sept. 29, 2012. Speed and compression
improvements. Adds -quiet option.
zpaq v6.03, Sept. 30, 2012. Saves and restores
file attributes. Cleans up -list.
zpaq v6.04, Oct. 1, 2012. Compression and speed
improvements by sorting by filename extension and storing uncompressible
data. Adds -list -quiet.
bmp_j4b.zip, Oct. 1, 2012. Updated bmp_j4 .bmp
model to work with zpaq v6.xx
zpaq v6.05, Oct. 2, 2012. Adds -list -history -summary,
Linux port, bug fixes, and improved docs.
zpaq v6.06, Oct. 4, 2012. Simplifies -list and
adds -compare.
zpaq v6.07, Oct. 7, 2012. Fixes porting issues
with Mac OS/X and Visual C++.
lazy v1.00, Oct. 10, 2012. A fast LZ77
compressor/preprocessor and and config.
lazy2 v1.00, Oct. 31, 2012. lazy with an E8E9
filter (and 1 GB file size limit).
zpaq v6.16, Nov. 5, 2012. Better compression
using lazy (-method 1) + e8e9 (all methods). Adds -test and -post.
zpaq v6.17, Dec. 13, 2012. Fixed display of
international characters. libzpaq v6.17 has slightly faster SHA1.
Has bugs. Do not use.
zpaq v6.18, Dec. 14, 2012. Bug fix.
zpaq v6.19, Jan. 23, 2013. Splits into zpaq (journaling
archiver) and zpaqd (development tool). Adds methods 5-9.
libzpaq v6.19 adds single pass compression checksums.
bmp_j4c, Jan. 24, 2013. Updated .bmp config file
to work with new zpaq/zpaqd syntax.
zpaq v6.20, Feb. 1, 2013. Improved compression
for methods 5 through 9. zpaq64.exe added Feb. 4, 2013.
zpaq v6.21, Feb. 6, 2013. Extract directories restores
timestamps and attributes. Adds -until date. Lists alphabetically. Fixed docs.
zpaq621-64.exe added Feb. 8, 2013.
zpaq v6.22, Feb. 13, 2013. -method supports custom
algorithms. zpaqd and libzpaq fixes for Win64. Command line accepts
international characters.
zpaq v6.23, Mar. 25, 2013. -method supports config
files without preprocessors. zpaqd 6.23 speed improvements for g++ 4.7.0
and "ds" command. libzpaq 6.23 faster initialization.
zpaq v6.24, Apr. 12, 2013. Adds d (delete) command.
Works with wildcards. zpaqd adds built-in configs 1..3.
zpaq v6.24a, Apr. 14, 2013. Recompile zpaq.exe, zpaqd.exe
to get around compiler bug in 64 bit version of MinGW causing 32 bit
zpaq to crash in WinXP.
zpaq v6.25, Apr. 19, 2013.
libzpaq optimizations (3-5% faster) and
bug fix for WinXP. No changes to zpaq or zpaqd except version number.
zpaq v6.26, May 9, 2013. Optimizations:
zpaq improves grouping of incompressible files into blocks, faster StringBuffer.
libzpaq JIT optimizes consecutive ZPAQL increments. zpaqd fixes compiler
warning.
zpaq v6.27, May 13, 2013. Adds -all and -test
options. Improved recovery of damaged archives. zpaqd updated to verify
checksums when listing journaling archives.
Discussion about ZPAQ updates.
ZPAQ is intended to replace PAQ and its variants (PAQ8, PAQ9A, LPAQ, LPQ1, etc) with similar or better compression in a portable, standard format. Current versions of PAQ break archive compatibility with each compression improvement. ZPAQ is intended to fix that. I no longer maintain the older PAQ code.
ZPAQ was written by Matt Mahoney, mattmahoneyfl (at) gmail (dot) com