reuse.extract module - reuse 6.1.2 documentation

Skip to content
Back to top
View this page

# reuse.extract module

Utilities related to the extraction of REUSE information out of files.

reuse.extract.get_encoding_module() → ModuleType

  Get the module used to detect the encodings of files.

reuse.extract.set_encoding_module(name: Literal['magic', 'charset_normalizer', 'chardet']) → ModuleType

  Set the module used to detect the encodings of files, and return the module.

reuse.extract.CHUNK_SIZE = 65536

  Default chunk size for reading files.

reuse.extract.LINE_SIZE = 1024

  Default line size for reading files.

reuse.extract.HEURISTICS_CHUNK_SIZE = 2048

  Default chunk size used to heuristically detect file type, encoding, et cetera.

class reuse.extract.FilterBlock(text: str, in_ignore_block: bool)

  Bases: `NamedTuple`

  A simple tuple that holds a block of text, and whether that block of text is in an ignore block.

  text: str

    Alias for field number 0

  in_ignore_block: bool

    Alias for field number 1

reuse.extract.filter_ignore_block(text: str, in_ignore_block: bool = False) → FilterBlock

  Filter out blocks beginning with REUSE_IGNORE_START and ending with REUSE_IGNORE_END to remove lines that should not be treated as copyright and licensing information.

  Parameters:

    - text – The text out of which the ignore blocks must be filtered.

    - in_ignore_block – Whether the text is already in an ignore block. This is useful when you parse subsequent chunks of text, and one chunk does not close the ignore block.

  Returns:

    A `FilterBlock` tuple that contains the filtered text and a boolean that signals whether the ignore block is still open.

reuse.extract.extract_reuse_info(text: str) → ReuseInfo

  Extract REUSE information from a multi-line text block.

  Raises:

    - ExpressionError – if an SPDX expression could not be parsed.

    - ParseError – if an SPDX expression could not be parsed.

reuse.extract.detect_encoding(chunk: bytes) → str | None

  Find the encoding of the bytes chunk, and return it as normalised name. See `encodings.normalize_encoding()`. If no encoding could be found, return `None`.

  If the chunk is empty or the encoding of the chunk is ASCII, `'utf_8'` is returned.

reuse.extract.detect_newline(chunk: bytes, encoding: str = 'ascii') → str

  Return one of `'\n'`, `'\r'` or `'\r\n'` depending on the line endings used in chunk. Return `os.linesep` if there are no line endings.

reuse.extract.reuse_info_of_file(fp: BinaryIO, chunk_size: int = 65536, line_size: int = 1024) → ReuseInfo

  Read from fp to extract REUSE information. It is read in chunks of chunk_size, additionally reading up to line_size until the next newline.

  This function decodes the binary data into UTF-8 and removes REUSE ignore blocks before attempting to extract the REUSE information.

reuse.extract.contains_reuse_info(text: str) → bool

  The text contains REUSE info.