Pickle#

Warning

Decoding pickled data can execute arbitrary code. Pickle.decode raises numcodecs.errors.UnsafeDecodeError unless the process opts in, either by setting numcodecs.pickles.ALLOW_UNSAFE_PICKLE_DECODE = True or by setting the environment variable NUMCODECS_ALLOW_UNSAFE_PICKLE_DECODE=1 (or "yes" / "true") before importing numcodecs. Only opt in if you trust the source of the data.

numcodecs.pickles.ALLOW_UNSAFE_PICKLE_DECODE = False#: Whether Pickle.decode() may unpickle data. Unpickling can execute arbitrary code, so this is False unless the NUMCODECS_ALLOW_UNSAFE_PICKLE_DECODE environment variable is set to 1, true or yes when numcodecs is imported. This is deliberately a process-wide setting rather than a codec parameter, because codec parameters are read from array metadata, which may be untrusted.

class numcodecs.pickles.Pickle(protocol=5)[source]#

Codec to encode data as as pickled bytes. Useful for encoding an array of Python string objects.

Warning

Decoding pickled data can execute arbitrary code. decode() refuses to run unless the process opts in by setting numcodecs.pickles.ALLOW_UNSAFE_PICKLE_DECODE = True (or the NUMCODECS_ALLOW_UNSAFE_PICKLE_DECODE=1 environment variable). Only opt in if you trust the source of the data. numcodecs.vlen.VLenUTF8, numcodecs.vlen.VLenBytes, numcodecs.json.JSON and numcodecs.msgpacks.MsgPack encode arrays of Python objects without this risk.

Parameters:

protocolint, defaults to pickle.HIGHEST_PROTOCOL: The protocol used to pickle data.

See also

numcodecs.msgpacks.MsgPack

Examples

>>> import numcodecs as codecs
>>> import numpy as np
>>> x = np.array(['foo', 'bar', 'baz'], dtype='object')
>>> f = codecs.Pickle()
>>> codecs.pickles.ALLOW_UNSAFE_PICKLE_DECODE = True  # only for trusted data
>>> f.decode(f.encode(x))
array(['foo', 'bar', 'baz'], dtype=object)
>>> codecs.pickles.ALLOW_UNSAFE_PICKLE_DECODE = False

codec_id: str | None = 'pickle'#: Codec identifier.

encode(buf)[source]#

Encode data in buf.

Parameters:

bufbuffer-like: Data to be encoded. May be any object supporting the new-style buffer protocol.

Returns:

encbuffer-like: Encoded data. May be any object supporting the new-style buffer protocol.

decode(buf, out=None)[source]#

Decode data in buf.

Parameters:

bufbuffer-like: Encoded data. May be any object supporting the new-style buffer protocol.
outbuffer-like, optional: Writable buffer to store decoded data. N.B. if provided, this buffer must be exactly the right size to store the decoded data.

Returns:

decbuffer-like: Decoded data. May be any object supporting the new-style buffer protocol.

get_config()[source]#: Return a dictionary holding configuration parameters for this codec. Must include an ‘id’ field with the codec identifier. All values must be compatible with JSON encoding.

classmethod from_config(config)#: Instantiate codec from a configuration object.