write_feather
Transforms the input event stream to Feather byte stream.
write_feather [compression_level=int, compression_type=str, min_space_savings=double]Description
Section titled “Description”Transforms the input event stream to Feather (a thin wrapper around Apache Arrow’s IPC wire format) byte stream.
compression_level = int (optional)
Section titled “compression_level = int (optional)”An optional compression level for the corresponding compression type. This option is ignored if no compression type is specified.
Defaults to the compression type’s default compression level.
compression_type = str (optional)
Section titled “compression_type = str (optional)”Supported options are zstd for Zstandard compression
and lz4 for LZ4 Frame compression.
:::tip Why would I use this over the compress operator?
The Feather format offers more efficient compression compared to the
compress operator. This is because it compresses
the data column-by-column, leaving metadata that needs to be accessed frequently
uncompressed.
:::
min_space_savings = double (optional)
Section titled “min_space_savings = double (optional)”Minimum space savings percentage required for compression to be applied. This option is ignored if no compression is specified. The provided value must be between 0 and 1 inclusive.
Space savings are calculated as 1.0 - compressed_size / uncompressed_size.
For example, with a minimum space savings rate of 0.1, a 100-byte body buffer will not
be compressed if its expected compressed size exceeds 90 bytes.
Defaults to 0, i.e., always applying compression.
Examples
Section titled “Examples”Convert a JSON stream into a Feather file
Section titled “Convert a JSON stream into a Feather file”load_file "input.json"read_jsonwrite_feathersave_file "output.feather"