Release notes

Unreleased

0.16.5

Fixes

• Restore prior optional dependency behavior for ‘crc32c’ group. By Davis Bennett, #802

0.16.4

Enhancements

• The Zstandard codec can now decode bytes containing multiple frames. By Mark Kittisopikul, #757

• Add Python 3.14 support to the test matrix. By Max Jones, #799

• Add Python 3.14 wheel builds. By Davis Bennett, #796

Fixes

• Fix PCodec.decode() to handle numpy arrays by calling ensure_bytes on input buffers. By Sam Levang, #756

Maintenance

• Replace crc32c dependency with google-crc32c. By Joe Hamman, #792

• Refactor numcodecs.zarr3 to re-export classes from zarr.codecs._numcodecs instead of defining them locally. By Davis Bennett, #780

• Add testing against zarr-python main branch. By Max Jones, #800

• Add pcodec and zfpy to 32-bit CI testing. By Mark Kittisopikul, #786

• Improve CRC32C codec test skipping to properly report skipped tests using pytest.skipif. By Agriya Khetarpal, #740

• Add nightly wheel upload to scientific-python channel. By Ian Hunt-Isaak, #787

• Fetch tags to version nightly wheels correctly. By Ian Hunt-Isaak, #790

• Bump pypa/cibuildwheel from 2.23.2 to 3.1.3. By dependabot, #771

• Bump actions/checkout from 4 to 5. By dependabot, #775

• Bump actions/download-artifact from 4 to 6. By dependabot, #794

• Bump prefix-dev/setup-pixi from 0.8.14 to 0.9.0. By dependabot, #783

• Bump jirutka/setup-alpine from 1.2.0 to 1.3.0. By dependabot, #789

• Bump conda-incubator/setup-miniconda from 3.1.1 to 3.2.0. By dependabot, #753

0.16.3

Fixes

• Fix Zstd decompression negative size issue on 32-bit platforms. By Mark Kittisopikul, #782

Maintenance

• Test 32-bit support via x86 Alpine Setup Action Linux. By Mark Kittisopikul, #784

• Bump to main branch of c-blosc to fix gcc15 compilation. By Thomas Caswell, #772

0.16.2

Enhancements

• Add streaming decompression for ZSTD_CONTENTSIZE_UNKNOWN case. By Mark Kittisopikul, #707

Maintenance

• Handle zarr 3.1.0 compatibility. By Davis Bennett, #766

0.16.1

Enhancements

• In vlen, define and use const HEADER_LENGTH. By John Kirkham, #723

• All codecs are now pickleable. By Tom Nicholas, #745

Fixes

• Fix const discard warnings in fletcher32. By John Kirkham, #728

• Update license* metadata and include third-party licenses. By John Kirkham, #729

• Add #ifndef guard around PyBytes_RESIZE. By John Kirkham, #732

• Remove typesize from Blosc.get_config output. By Ilan Gold, #739

0.16.0

Enhancements

• Add typesize argument to Blosc to allow for buffers that are passed to encode use that information. zarr v3 currently has its Blosc codec as bytes-to-bytes but does retain the size information so using it here allows for massive compression ratio gains. By Ilan Gold

• Add support for the Linux aarch64 architecture, By Agriya Khetarpal, #288.

• Preallocate output buffers and resize directly as needed. By John Kirkham, #656

• Checksum32 is now an abstract base clase, with checksum() as an abstract method. This helps ensure that any subclasses implement the checksum() method. #711

Removals

The following blosc funcitons are removed, with no replacement. This is because they were not intended to be public API.

• numcodecs.blosc.init

• numcodecs.blosc.destroy

• numcodecs.blosc.compname_to_compcode

• numcodecs.blosc.cbuffer_sizes

• numcodecs.blosc.cbuffer_metainfo

In addition, numcodecs.blosc.decompress_partial is removed as has always been experimental and there is no equivalent in the official blsoc Python package. By David Stansby, #712

Maintenance

• Replace internal Buffer usage with memoryviews. By John Kirkham, #656

• Bump the minimum macOS deployment target for x86_64 to 10.13. By Agriya Khetarpal, #288.

0.15.1

Improvements

• Add streaming decompression for ZSTD (#699) By Mark Kittisopikul.

• Raise a custom UnknownCodecError when trying to retrieve an unavailable codec. By Cas Wognum.

Fixes

• Remove redundant id from codec metadata serialization in Zarr3 codecs. By Norman Rzepka, #685

0.15.0

Breaking changes

• All arguments to the PCodec constructor except for level are now keyword only, to support the updated API. By Sam Levang, #623

Deprecations

The following blosc functions are deprecated, with no replacement. This is because they are not intended to be public API.

• numcodecs.blosc.init

• numcodecs.blosc.destroy

• numcodecs.blosc.compname_to_compcode

• numcodecs.blosc.cbuffer_sizes

• numcodecs.blosc.cbuffer_metainfo

In addition, numcodecs.blosc.decompress_partial is deprecated as has always been experimental and there is no equivalent in the official blsoc Python package. By David Stansby, #619

Fixes

• Fixes issue with Delta Zarr 3 codec not working with astype. By Norman Rzepka, #664

• Cleanup PCodec soft dependency. Previously importing numcodecs.pcodec would work if pcodec is not installed, but now it will fail to import. This mirrors the behaviour of other optional dependencies. By John Kirkham, #647

• Fixes issues with the upcoming zarr 3.0.0 release. By Norman Rzepka, #675

• Removed Version Check: The previous code included a check for the NumPy version and a warning if the version was incompatible with zfpy. This check has been removed because zfpy now supports the newer versions of NumPy. By Meher Gajula, #672

Improvements

• Add support for pcodec 0.3. This exposes the new delta_spec and paging_spec arguments, but maintains full backwards compatibility for data written with older package versions. By Sam Levang, #623

• If an import error is raised when trying to define a codec that is not an optional dependency, it is no longer silently caught. Instead it will be propagated to the user, as this indicates an issue with the installed package.

  Import errors caused by optional dependencies (ZFPY, MsgPack, CRC32C, and PCodec) are still silently caught. By David Stansby, #550.

0.14.1

Fixes

• Cleanups to the crc32c soft dependency. Whereas in numcodecs 0.14.0 a runtime error was raised if the CRC32C codec used without the crc32c installed, the CRC32C codec is no longer defined at import time if crc32c is not installed. This has been changed to match the behaviour of other optional dependencies/codecs. By John Kirkham, #637

Improvements

• Add noexcept to _utils C-equiv functions By John Kirkham, #641.

0.14.0

Enhancements

• Add Crc32c checksum codec. By Norman Rzepka, #613.

• Add codec wrappers for Zarr 3. By Norman Rzepka, #524

• Added mypy type checking to continuous integration. By David Stansby, #460.

Fixes

• Fix in-place mutation of input array in BitRound. By Sam Levang, #608

• Fix an issue where importing numcodecs would lock the state of multiprocessing and prevent user code to call multiprocessing.set_start_method(“spawn”) subsequently. By Clément Robert #522

Maintenance

• The minimum supported Python version is now Python 3.11. By David Stansby, #622

• The minimum supported numpy version is now 1.24. By David Stansby, #622

0.13.1

Breaking changes

• Zstd.default_level, Zstd.min_level, and Zstd.max_level are now class methods instead of properties. This means they must now be called like Zstd.default_level() instead of Zstd.default_level. This breaking change has been made because Python 3.13 removes support for class properties. By David Stansby, #576.

Enhancements

• Update bundled c-blosc to v1.26.1. This updates Zlib to v1.3.1 and Zstd to v1.5.6. By David Stansby, #560.

• Added support for Python 3.13 David Stansby, #576.

• Cleaned up the table of contents in the documentation to list codecs by category David Stansby, #458.

Maintenance

• Change format() and old string formatting to f-strings. By Dimitri Papadopoulos Orfanos, #439.

• Remove pin on Sphinx By Elliott Sales de Andrade, #552.

0.13.0

Enhancements

• Add checksum flag to zstd and sets the default level to 0. By Norman Rzepka, #519.

• Add PCodec By Ryan Abernathey, #501.

• Use PyData theme for docs By John Kirkham, #485.

• Improve the structure of docs By David Stansby, #458.

Fix

• Fix VLenUTF8 encoding for read-only buffers. By Isaac Virshup, #514.

• Fix skip of entry points backport tests By Elliott Sales de Andrade, #487.

• Fix Upgrade to Zstd 1.5.5 due to potential corruption. By Mark Kittisopikul, #429

• Add version constraint(<2.0) for numpy in zfpy. By Tom Liang, #540.

Maintenance

• The minimum supported Python version is now Python 3.10. By David Stansby, #531

• Add numpy 2 compatibility. By David Stansby, #535

• Update c-blosc to 1.21.0 to 1.21.5, zstd from 1.4.8 to 1.5.5, lz4 from 1.9.3 to 1.9.4, and zlib from 1.2.8 to to 1.2.13 By Mark Kittisopikul, #500

0.12.1

Fix

• Codec is now derived from abc.ABC By Mads R. B. Kristensen, #472.

• Fix handling of entry points on older Python versions where importlib_metadata compatibility is concerned By Vyas Ramasubramani, #478.

• Make shuffle pyx functions noexcept By Martin Durant, #477.

0.12.0

Enhancements

• Add fletcher32 checksum codec By Martin Durant, #410.

• Add jenkins_lookup3 checksum codec By Mark Kittisopkul, #445.

• Support Python 3.12. By John Kirkham, #471.

Fix

• Fixed docs/Makefile error message when sphinx is not present By Mark Kittisopikul, #451.

• Add missing RTD requirements By John Kirkham, #455.

Maintenance

• Cleanup imports in adhoc/blosc_memleak_check.py By John Kirkham, #408.

0.11.0

Fix

• Speed up isinstance checks of numcodecs.ndarray_like.NDArrayLike, numcodecs.ndarray_like.DType and numcodecs.ndarray_like.FlagsObj. By Andreas Poehlmann, #379.

Maintenance

• Remove unnecessary None argument to .get(), it is the default value. By Dimitri Papadopoulos Orfanos, #395.

• Apply refurb suggestions. By Dimitri Papadopoulos Orfanos, #372.

• Migrate codespell configuration to pyproject.toml and get rid of setup.cfg. By Dimitri Papadopoulos Orfanos #374.

• Unvendor cpuinfo. By Dimitri Papadopoulos Orfanos #373.

• Drop headers. By John Kirkham, #375.

• Remove Python 2 code. By Dimitri Papadopoulos Orfanos #368, #387.

• Support Python 3.11. By Dimitri Papadopoulos Orfanos, #369.

• Drop Python 3.7. By John Kirkham,, #405 #406.

• Test with zfpy 1.0.0. By John Kirkham, #385.

• Remove vendored C files and re-generate them on the fly using Cython. Add a pyproject.toml file to define Cython as a build dependency. By Dimitri Papadopoulos Orfanos, #369.

• Update cpuinfo to 9.0.0. By Dimitri Papadopoulos Orfanos #370.

• Add tests for all registry classes. By Josh Moore, #349.

• Add workflow to label PRs with “needs release notes”. By Saransh Chopra #396.

• Finish pyproject.toml migration. By John Kirkham #382.

• Drop tox. By John Kirkham, #384.

• Update ReadTheDocs. By John Kirkham, #383.

• Use conda-incubator/setup-miniconda@v2.2.0 (and use Conda on Linux). By John Kirkham #398.

• Bring coverage back up to 100%. By John Kirkham and Martin Durant, #392 and #393.

• Collect coverage on all OSes & enable Codecov. By John Kirkham, #386, #388, #390, #391.

0.10.2

Fix

• Add BitRound (0.10.0) to registry. By Josh Moore, #342.

0.10.1

Maintenance

• Add entrypoints to setup.py. By Josh Moore, #332.

• Fix spelling. By Dimitri Papadopoulos Orfanos, #336.

• Drop Python 3.6 from tests By Dimitri Papadopoulos Orfanos, #338, #339.

• Remove trailing spaces and empty lines. By Dimitri Papadopoulos Orfanos, #341.

• Add LGTM.com configuration file By Dimitri Papadopoulos Orfanos, #337.

0.10.0

Enhancements

• Add support of alternative array classes (other than NumPy arrays) By Mads R. B. Kristensen, #305.

• Add ability to find codecs via entrypoint numcodecs.codecs. By Martin Durant, #290.

• Add bitround codec By Ryan Abernathy and Martin Durant, #298.

• Introduce a flat option to ensure_contiguous_ndarray to switch off flatten for ZFPY codec By Haiying Xu, #307.

Bug fixes

• Fix a flatten array error for ZFPY, ZFPY codec is supported on Python 3.9 and 3.10 on Linux and MacOS, the docs about ZFPY is also available. By Haiying Xu, John Kirkham, Ryan Abernathey #303.

• Codex: make encode and decode @abstractmethods By Mads R. B. Kristensen, #306.

• Fix expected result test for Shuffle. By Elliott Sales de Andrade, #282.

Maintenance

• Multiple code linting fixes. By Dimitri Papadopoulos Orfanos, #295, #294, #293, and #292.

• Drop Python 3.6 By Josh Moore, #318.

• Fix macOS Python 3.10 By John Kirkham, #311.

• chore: bump cibuildwheel version, use action By Henry Schreiner, #309.

• Specify language as ‘en’ instead of None. By John Kirkham, #329.

• Move master to main. By John Kirkham, #322.

• Drop fastparquet benchmark. By John Kirkham, #321.

• Trim wheel builds. By John Kirkham, #320.

0.9.1

• Fix inaccurate docstrings for Blosc. By James Webber, #287.

0.9.0

• c-blosc upgrade 1.18.1 -> 1.21.0. Warning: this temporarily removes support for snappy compression! By kindjacket, #283.

• Fix an ImportError with Blosc on Android. By Daniel Jewell, #284.

0.8.1

• Fix an ImportError with Blosc on Android. By Daniel Jewell, #284.

0.8.0

• The numcodecs.zfpy.ZFPY codec is now supported on Python 3.8 if zfpy==0.5.5 is installed. By haiying xu, #229.

• Add support for byte Shuffle filter By Paul Branson and Martin Durant #273.

• Update Windows + Mac CI to run all tests. By Jackson Maxfield Brown, #276. Help from Oleg Höfling, #273.

• Update cpuinfo to 8.0.0. By Florian Jetter, #280.

• Drop out-of-date manual release docs. By John Kirkham, #272.

• Add support for Python 3.9 and Update GitHub Actions.

0.7.3

• Add support for Python 3.9 and Update GitHub Actions. By Jackson Maxfield Brown, #270.

• Remove support for Python 3.5 which is end of life. While the code base might still be compatible; the source dist and wheel are marked as Python 3.6+ and pip will not install them. Continuous integration on Python 3.5 has been disabled. By Matthias Bussonnier, #266 and #267.

0.7.2

• Disable avx2 for wheel. By Grzegorz Bokota, #253.

• Add Base64 fixtures. By John Kirkham, #251.

• Update docs regarding wheels. By Josh Moore, #250.

0.7.1

• Fix build of wheels. By Grzegorz Bokota, #244.

0.7.0

• Automatically release to PyPI. By Josh Moore, #241.

• Build wheels on github actions. By Grzegorz Bokota, #224.

• Add Base64 codec. By Trevor Manz, #176.

• Add partial decompression of Blosc compressed arrays. By Andrew Fulton, #235.

• Remove LegacyJSON codec. By James Bourbeau, #226.

• Remove LegacyMsgPack codec. By James Bourbeau, #218.

• Drop support for Python 2. By James Bourbeau, #220.

0.6.4

• Update Cython to 0.29.14. By John Kirkham, #168, #177, #204.

• The bundled c-blosc sources have been upgraded to version 1.17.0. This fixes compilation with newer versions of gcc. By Joe Jevnik, #194.

• Create .pep8speaks.yml. By Alistair Miles.

• Simplify datetime/timedelta check. By John Kirkham, #170, #171.

• Update URL metadata for PyPI. By Elliott Sales de Andrade, #178.

• Enable pytest rewriting in test helper functions. By Elliott Sales de Andrade, #185.

• Rewrites the ensure_text implementation. By John Kirkham, #201, #205, #206.

• Add macOS to CI. By Alistair Miles, #192.

• Fix test failures on big-endian systems. By Elliott Sales de Andrade, #186.

• Use unittest.mock on Python 3. By Elliott Sales de Andrade, #179.

• Don’t mask compile errors in setup.py. By Joe Jevnik, #197.

• Allow pickles when loading test fixture data. By Elliott Sales de Andrade, #193.

• Update cpuinfo.py. By John Kirkham, #202.

• Use ensure_text in JSON codecs. By John Kirkham, #207.

• Support Python 3.8. By John Kirkham, #208.

0.6.3

• Drop support for 32-bit Windows. By Alistair Miles, #97, #156.

• Raise a TypeError if an object array is passed to ensure_bytes. By John Kirkham, #162.

• Update Cython to 0.29.3. By John Kirkham, #165.

0.6.2

• Handle (new) buffer protocol conforming types in Pickle.decode. By John Kirkham, #143, #150.

• Use (new) buffer protocol in MsgPack codec decode() method. By John Kirkham, #148.

• Use (new) buffer protocol in JSON codec decode() method. By John Kirkham, #151.

• Avoid copying into data in GZip’s decode() method on Python 2. By John Kirkham, #152.

• Revert ndarray coercion of encode returned data. By John Kirkham, #155.

• The bundled c-blosc sources have been upgraded to version 1.15.0. By Alistair Miles and John Kirkham, #142, #145.

0.6.1

• Resolved minor issue in backwards-compatibility tests (by Alistair Miles, #138, #139).

0.6.0

• The encoding format used by the JSON and MsgPack codecs has been changed to resolve an issue with correctly encoding and decoding some object arrays. Now the encoded data includes the original shape of the array, which enables the correct shape to be restored on decoding. The previous encoding format is still supported, so that any data encoded using a previous version of numcodecs can still be read. Thus no changes to user code and applications should be required, other than upgrading numcodecs. By Jerome Kelleher; #74, #75.

• Updated the msgpack dependency (by Jerome Kelleher; #74, #75).

• Added support for ppc64le architecture by updating cpuinfo.py from upstream (by Anand S; #82).

• Allow numcodecs.blosc.Blosc compressor to run on systems where locks are not present (by Marcus Kinsella, #83; and Tom White, #93).

• Drop Python 3.4 (by John Kirkham; #89).

• Add Python 3.7 (by John Kirkham; #92).

• Add codec numcodecs.gzip.GZip to replace gzip alias for zlib, which was incorrect (by Jan Funke; #87; and John Kirkham, #134).

• Corrects handling of NaT in datetime64 and timedelta64 in various compressors (by John Kirkham; #127, #131).

• Improvements to the compatibility layer used for normalising inputs to encode and decode methods in most codecs. This removes unnecessary memory copies for some codecs, and also simplifies the implementation of some codecs, improving code readability and maintainability. By John Kirkham and Alistair Miles; #119, #121, #128.

• Return values from encode() and decode() methods are now returned as numpy arrays for consistency across codecs. By John Kirkham, #136.

• Improvements to handling of errors in the numcodecs.blosc.Blosc and numcodecs.lz4.LZ4 codecs when the maximum allowed size of an input buffer is exceeded. By Jerome Kelleher, #80, #81.

0.5.5

• The bundled c-blosc sources have been upgraded to version 1.14.3 (#72).

0.5.4

• The bundled c-blosc sources have been upgraded to version 1.14.0 (#71).

0.5.3

• The test suite has been migrated to use pytest instead of nosetests (#61, #62).

• The bundled c-blosc library has been updated to version 1.13.4 (#63, #64).

0.5.2

• Add support for encoding None values in VLen… codecs (#59).

0.5.1

• Fixed a compatibility issue with the Zlib codec to ensure it can handle bytearray objects under Python 2.7 (#57).

• Restricted the numcodecs.categorize.Categorize codec to object (‘O’) and unicode (‘U’) dtypes and disallowed bytes (‘S’) dtypes because these do not round-trip through JSON configuration.

0.5.0

• Added new codecs for encoding arrays with variable-length unicode strings (numcodecs.vlen.VLenUTF8), variable-length byte strings (numcodecs.vlen.VLenBytes) and variable-length numerical arrays ((numcodecs.vlen.VLenArray) (#56).

0.4.1

• Resolved an issue where providing an array with dtype object as the destination when decoding could cause segfaults with some codecs (#55).

0.4.0

• Added a new numcodecs.json.JSON codec as an alternative for encoding of object arrays (#54).

0.3.1

• Revert the default shuffle argument to SHUFFLE (byte shuffle) for the numcodecs.blosc.Blosc codec for compatibility and consistency with previous code.

0.3.0

• The numcodecs.blosc.Blosc codec has been made robust for usage in both multithreading and multiprocessing programs, regardless of whether Blosc has been configured to use multiple threads internally or not (#41, #42).

• The numcodecs.blosc.Blosc codec now supports an AUTOSHUFFLE argument when encoding (compressing) which activates bit- or byte-shuffle depending on the itemsize of the incoming buffer (#37, #42). This is also now the default.

• The numcodecs.blosc.Blosc codec now raises an exception when an invalid compressor name is provided under all circumstances (#40, #42).

• The bundled version of the c-blosc library has been upgraded to version 1.12.1 (#45, #42).

• An improvement has been made to the system detection capabilities during compilation of C extensions (by Prakhar Goel; #36, #38).

• Arrays with datetime64 or timedelta64 can now be passed directly to compressor codecs (#39, #46).

0.2.1

The bundled c-blosc library has been upgraded to version 1.11.3 (#34, #35).

0.2.0

New codecs:

• The numcodecs.quantize.Quantize codec, which provides support for reducing the precision of floating-point data, has been ported over from Zarr (#28, #31).

Other changes:

• The numcodecs.zlib.Zlib codec is now also registered under the alias ‘gzip’ (#29, #32).

Maintenance work:

• A data fixture has been added to the test suite to add some protection against changes to codecs that break backwards-compatibility with data encoded using a previous release of numcodecs (#30, #33).

0.1.1

This release includes a small modification to the setup.py script to provide greater control over how compiler options for different instruction sets are configured (#24, #27).

0.1.0

New codecs:

• Two new compressor codecs numcodecs.zstd.Zstd and numcodecs.lz4.LZ4 have been added (#3, #22). These provide direct support for compression/decompression using Zstandard and LZ4 respectively.

• A new numcodecs.msgpacks.MsgPack codec has been added which uses msgpack-python to perform encoding/decoding, including support for arrays of Python objects (Jeff Reback; #5, #6, #8, #21).

• A new numcodecs.pickles.Pickle codec has been added which uses the Python pickle protocol to perform encoding/decoding, including support for arrays of Python objects (Jeff Reback; #5, #6, #21).

• A new numcodecs.astype.AsType codec has been added which uses NumPy to perform type conversion (John Kirkham; #7, #12, #14).

Other new features:

• The numcodecs.lzma.LZMA codec is now supported on Python 2.7 if backports.lzma is installed (John Kirkham; #11, #13).

• The bundled c-blosc library has been upgraded to version 1.11.2 (#10, #18).

• An option has been added to the numcodecs.blosc.Blosc codec to allow the block size to be manually configured (#9, #19).

• The representation string for the numcodecs.blosc.Blosc codec has been tweaked to help with understanding the shuffle option (#4, #19).

• Options have been added to manually control how the C extensions are built regardless of the architecture of the system on which the build is run. To disable support for AVX2 set the environment variable “DISABLE_NUMCODECS_AVX2”. To disable support for SSE2 set the environment variable “DISABLE_NUMCODECS_SSE2”. To disable C extensions altogether set the environment variable “DISABLE_NUMCODECS_CEXT” (#24, #26).

Maintenance work:

• CI tests now run under Python 3.6 as well as 2.7, 3.4, 3.5 (#16, #17).

• Test coverage is now monitored via coveralls (#15, #20).

0.0.1

Fixed project description in setup.py.

0.0.0

First release. This version is a port of the codecs module from Zarr 2.1.0. The following changes have been made from the original Zarr module:

• Codec classes have been re-organized into separate modules, mostly one per codec class, for ease of maintenance.

• Two new codec classes have been added based on 32-bit checksums: numcodecs.checksum32.CRC32 and numcodecs.checksum32.Adler32.

• The Blosc extension has been refactored to remove code duplications related to handling of buffer compatibility.