Documentation/staging/xz.rst

Source file repositories/reference/linux-study-clean/Documentation/staging/xz.rst

File Facts

System
Linux kernel
Corpus path
Documentation/staging/xz.rst
Extension
.rst
Size
4128 bytes
Lines
99
Domain
Support Tooling And Documentation
Bucket
Documentation
Inferred role
Support Tooling And Documentation: documentation
Status
atlas-only

Why This File Exists

Repository support layer: documentation, build tooling, samples, user-space helper tools, generated initramfs support, licenses, and validation utilities.

Dependency Surface

Detected Declarations

Annotated Snippet

.. SPDX-License-Identifier: 0BSD

============================
XZ data compression in Linux
============================

Introduction
============

XZ is a general purpose data compression format with high compression
ratio. The XZ decompressor in Linux is called XZ Embedded. It supports
the LZMA2 filter and optionally also Branch/Call/Jump (BCJ) filters
for executable code. CRC32 is supported for integrity checking.

See the `XZ Embedded`_ home page for the latest version which includes
a few optional extra features that aren't required in the Linux kernel
and information about using the code outside the Linux kernel.

For userspace, `XZ Utils`_ provide a zlib-like compression library
and a gzip-like command line tool.

.. _XZ Embedded: https://tukaani.org/xz/embedded.html
.. _XZ Utils: https://tukaani.org/xz/

XZ related components in the kernel
===================================

The xz_dec module provides XZ decompressor with single-call (buffer
to buffer) and multi-call (stateful) APIs in include/linux/xz.h.

For decompressing the kernel image, initramfs, and initrd, there
is a wrapper function in lib/decompress_unxz.c. Its API is the
same as in other decompress_*.c files, which is defined in
include/linux/decompress/generic.h.

For kernel makefiles, three commands are provided for use with
``$(call if_changed)``. They require the xz tool from XZ Utils.

- ``$(call if_changed,xzkern)`` is for compressing the kernel image.
  It runs the script scripts/xz_wrap.sh which uses arch-optimized
  options and a big LZMA2 dictionary.

- ``$(call if_changed,xzkern_with_size)`` is like ``xzkern`` above but
  this also appends a four-byte trailer containing the uncompressed size
  of the file. The trailer is needed by the boot code on some archs.

- Other things can be compressed with ``$(call if_needed,xzmisc)``
  which will use no BCJ filter and 1 MiB LZMA2 dictionary.

Notes on compression options
============================

Since the XZ Embedded supports only streams with CRC32 or no integrity
check, make sure that you don't use some other integrity check type
when encoding files that are supposed to be decoded by the kernel.
With liblzma from XZ Utils, you need to use either ``LZMA_CHECK_CRC32``
or ``LZMA_CHECK_NONE`` when encoding. With the ``xz`` command line tool,
use ``--check=crc32`` or ``--check=none`` to override the default
``--check=crc64``.

Using CRC32 is strongly recommended unless there is some other layer
which will verify the integrity of the uncompressed data anyway.
Double checking the integrity would probably be waste of CPU cycles.
Note that the headers will always have a CRC32 which will be validated
by the decoder; you can only change the integrity check type (or
disable it) for the actual uncompressed data.

In userspace, LZMA2 is typically used with dictionary sizes of several
megabytes. The decoder needs to have the dictionary in RAM:

Annotation

Implementation Notes