<pre class="metadata">
Title: Compression Streams
Shortname: compression
Level: none
Status: CG-DRAFT
Group: wicg
ED: https://wicg.github.io/compression/
Editor: Canon Mukai, Google
Editor: Adam Rice, Google
Abstract:
  This document defines a set of JavaScript APIs to compress and decompress
  streams of binary data.
Repository: wicg/compression
Indent: 2
Markup Shorthands: markdown yes
Boilerplate: omit conformance
</pre>
<pre class="link-defaults">
spec:streams; type:interface; text:ReadableStream
</pre>
<pre class="anchors">
urlPrefix: http://www.ecma-international.org/ecma-262/6.0/index.html; spec: ECMASCRIPT-6.0
  type: dfn
    text: fulfilled; url: sec-promise-objects
    text: rejected; url: sec-promise-objects
    text: pending; url: sec-promise-objects
    text: resolved; url: sec-promise-objects
    text: settled; url: sec-promise-objects
</pre>

# Introduction #    {#introduction}

*This section is non-normative.*

The APIs specified in this specification are used to compress and decompress streams of data. They support "deflate" and "gzip" as compression algorithms. They are widely used in web developers.

# Conformance #  {#conformance}

As well as sections marked as non-normative, all authoring guidelines,
diagrams, examples, and notes in this specification are non-normative.
Everything else in this specification is normative.

The key words *MUST* and *SHOULD* are to be interpreted as described in
[[!RFC2119]].

This specification defines conformance criteria that apply to a single product:
the user agent that implements the interfaces that it contains.

Conformance requirements phrased as algorithms or specific steps may be
implemented in any manner, so long as the end result is equivalent. (In
particular, the algorithms defined in this specification are intended to be
easy to follow, and not intended to be performant.)

Implementations that use ECMAScript to implement the APIs defined in this
specification MUST implement them in a manner consistent with the ECMAScript
Bindings defined in the Web IDL specification [[!WebIDL]], as this
specification uses that specification and terminology.

# Terminology #  {#terminology}

A chunk is a piece of data. In the case of CompressionStream and DecompressionStream, the output chunk type is Uint8Array. They accept any {{BufferSource}} type as input.

A stream represents an ordered sequence of chunks. The terms {{ReadableStream}} and {{WritableStream}} are defined in [[!WHATWG-STREAMS]].

A <dfn>compression context</dfn> is the internal state maintained by a compression or decompression algorithm. The contents of a <a>compression context</a> depend on the format, algorithm and implementation in use. From the point of view of this specification, it is an opaque object. A <a>compression context</a> is initially in a start state such that it anticipates the first byte of input.

# Supported formats # {#supported-formats}

: `deflate`
:: "ZLIB Compressed Data Format" [[!RFC1950]]

   * This format is referred to as "deflate" for consistency with HTTP Content-Encodings. See [[RFC7230]] section 4.2.2.
   * Implementations must be "compliant" as described in [[!RFC1950]] section 2.3.
   * Field values described as invalid in [[!RFC1950]] must not be created by CompressionStream, and are errors for DecompressionStream.
   * The only valid value of the `CM` (Compression method) part of the `CMF` field is 8.
   * The `FDICT` flag is not supported by these APIs, and will error the stream if set.
   * The `FLEVEL` flag is ignored by DecompressionStream.
   * It is an error for DecompressionStream if the `ADLER32` checksum is not correct.
   * It is an error if there is additional input data after the `ADLER32` checksum.

: `gzip`
:: "GZIP file format" [[!RFC1952]]

   * Implementations must be "compliant" as described in [[!RFC1952]] section 2.3.1.2.
   * Field values described as invalid in [[!RFC1952]] must not be created by CompressionStream, and are errors for DecompressionStream.
   * The only valid value of the `CM` (Compression Method) field is 8.
   * The `FTEXT` flag must be ignored by DecompressionStream.
   * If the `FHCRC` field is present, it is an error for it to be incorrect.
   * The contents of any `FEXTRA`, `FNAME` and `FCOMMENT` fields must be ignored by DecompressionStream, except to verify that they are terminated correctly.
   * The contents of the `MTIME`, `XFL` and `OS` fields must be ignored by DecompressionStream.
   * It is an error if `CRC32` or `ISIZE` do not match the decompressed data.
   * A `gzip` stream may only contain one "member".
   * It is an error if there is additional input data after the end of the "member".

# Interface Mixin `GenericTransformStream` #  {#generic-transform-stream}

The {{GenericTransformStream}} interface mixin represents the concept of a transform stream in IDL. It is not a TransformStream, though it has the same interface and it delegates to one.

<pre class="idl">
interface mixin GenericTransformStream {
  readonly attribute ReadableStream readable;
  readonly attribute WritableStream writable;
};
</pre>

An object that includes {{GenericTransformStream}} has an associated <dfn>transform</dfn> of type TransformStream.

## Attributes ##  {#outgoing-stream-attributes}

: <dfn attribute for="GenericTransformStream">readable</dfn>
:: The `readable` attribute's getter, when invoked, must return this object's transform \[[readable]].
: <dfn attribute for="GenericTransformStream">writable</dfn>
:: The `writable` attribute's getter, when invoked, must return this object's transform \[[writable]].

# Interface `CompressionStream` #  {#compression-stream}

<pre class="idl">
[Exposed=(Window,Worker)]
interface CompressionStream {
  constructor(DOMString format);
};
CompressionStream includes GenericTransformStream;
</pre>

A {{CompressionStream}} has an associated <dfn for=CompressionStream>format</dfn> and <a>compression context</a> <dfn for=CompressionStream>context</dfn>.

The {{CompressionStream}}(*format*) constructor, when invoked, must run these steps:
    1. If *format* is unsupported in CompressionStream, then throw a TypeError.
    1. Let *cs* be a new CompressionStream object.
    1. Set *cs*'s <a for=CompressionStream>format</a> to *format*.
    1. Let *startAlgorithm* be an algorithm that takes no arguments and returns nothing.
    1. Let *transformAlgorithm* be an algorithm which takes a *chunk* argument and runs the <a>compress and enqueue a chunk</a> algorithm with *cs* and *chunk*.
    1. Let *flushAlgorithm* be an algorithm which takes no argument and runs the <a>compress flush and enqueue</a> algorithm with *cs*.
    1. Let *transform* be the result of calling <a abstract-op>CreateTransformStream</a>(*startAlgorithm*, *transformAlgorithm*, *flushAlgorithm*).
    1. Set *cs*'s <a>transform</a> to *transform*.
    1. Return *cs*.

The <dfn>compress and enqueue a chunk</dfn> algorithm, given a CompressionStream object *cs* and a *chunk*, runs these steps:
    1. If *chunk* is not a {{BufferSource}} type, then return <a>a promise rejected with</a> a TypeError.
    1. Let *buffer* be the result of compressing *chunk* with *cs*'s <a for=CompressionStream>format</a> and <a for=CompressionStream>context</a>.
    1. Let *controller* be *cs*'s transform.\[[TransformStreamController]].
    1. If *buffer* is empty, return <a>a promise resolved with</a> undefined.
    1. Split *buffer* into one or more non-empty pieces and convert them into Uint8Arrays.
    1. For each Uint8Array *array*, call <a abstract-op>TransformStreamDefaultControllerEnqueue</a>(*controller*, *array*).
    1. Return <a>a promise resolved with</a> undefined.

The <dfn>compress flush and enqueue</dfn> algorithm, which handles the end of data from the input ReadableStream object, given a CompressionStream object *cs*, runs these steps:

    1. Let *buffer* be the result of compressing an empty input with *cs*'s <a for=CompressionStream>format</a> and <a for=CompressionStream>context</a>, with the finish flag.
    1. If *buffer* is empty, return <a>a promise resolved with</a> undefined.
    1. Split *buffer* into one or more non-empty pieces and convert them into Uint8Arrays.
    1. For each Uint8Array *array*, call <a abstract-op>TransformStreamDefaultControllerEnqueue</a>(*controller*, *array*).
    1. Return <a>a promise resolved with</a> undefined.


# Interface `DecompressionStream` #  {#decompression-stream}

<pre class="idl">
[Exposed=(Window,Worker)]
interface DecompressionStream {
  constructor(DOMString format);
};
DecompressionStream includes GenericTransformStream;
</pre>

A {{DecompressionStream}} has an associated <dfn for=DecompressionStream>format</dfn> and <a>compression context</a> <dfn for=DecompressionStream>context</dfn>.

The {{DecompressionStream}}(*format*) constructor, when invoked, must run these steps:
    1. If *format* is unsupported in DecompressionStream, then throw a TypeError.
    1. Let *ds* be a new DecompressionStream object.
    1. Set *ds*'s <a for=DecompressionStream>format</a> to *format*.
    1. Let *startAlgorithm* be an algorithm that takes no arguments and returns nothing.
    1. Let *transformAlgorithm* be an algorithm which takes a *chunk* argument and runs the <a>decompress and enqueue a chunk</a> algorithm with *ds* and *chunk*.
    1. Let *flushAlgorithm* be an algorithm which takes no argument and runs the <a>decompress flush and enqueue</a> algorithm with *ds*.
    1. Let *transform* be the result of calling <a abstract-op>CreateTransformStream</a>(*startAlgorithm*, *transformAlgorithm*, *flushAlgorithm*).
    1. Set *ds*'s <a>transform</a> to *transform*.
    1. Return *ds*.

The <dfn>decompress and enqueue a chunk</dfn> algorithm, given a DecompressionStream object *ds* and a *chunk*, runs these steps:
    1. If *chunk* is not a {{BufferSource}} type, then return <a>a promise rejected with</a> a TypeError.
    1. Let *buffer* be the result of decompressing *chunk* with *ds*'s <a for=DecompressionStream>format</a> and <a for=DecompressionStream>context</a>. If this results in an error, then return <a>a promise rejected with</a> a TypeError.
    1. Let *controller* be *ds*'s <a>transform</a>.\[[TransformStreamController]].
    1. If *buffer* is empty, return <a>a promise resolved with</a> undefined.
    1. Split *buffer* into one or more non-empty pieces and convert them into Uint8Arrays.
    1. For each Uint8Array *array*, call <a abstract-op>TransformStreamDefaultControllerEnqueue</a>(*controller*, *array*).
    1. Return <a>a promise resolved with</a> undefined.

The <dfn>decompress flush and enqueue</dfn> algorithm, which handles the end of data from the input ReadableStream object, given a DecompressionStream object *ds*, runs these steps:

    1. Let *buffer* be the result of decompressing an empty input with *ds*'s <a for=DecompressionStream>format</a> and <a for=DecompressionStream>context</a>, with the finish flag.
    1. If the end of the compressed input has not been reached, return <a>a promise rejected with</a> a TypeError.
    1. If *buffer* is empty, return <a>a promise resolved with</a> undefined.
    1. Split *buffer* into one or more non-empty pieces and convert them into Uint8Arrays.
    1. For each Uint8Array *array*, call <a abstract-op>TransformStreamDefaultControllerEnqueue</a>(*controller*, *array*).
    1. Return <a>a promise resolved with</a> undefined.


# Privacy and Security Considerations #  {#privacy-security}

The API doesn't add any new privileges to the web platform.

However, web developers have to pay attention to the situation when attackers can get the length of the data. If so, they may be able to guess the contents of the data.

# Examples #  {#examples}

## Gzip-compress a stream ##  {#example-gzip-compress-stream}

<pre class="example" highlight="js">
const compressedReadableStream
    = inputReadableStream.pipeThrough(new CompressionStream('gzip'));
</pre>

## Deflate-compress an ArrayBuffer to a Uint8Array ##  {#example-deflate-compress}

<pre class="example" highlight="js">
async function compressArrayBuffer(in) {
  const cs = new CompressionStream('deflate');
  const writer = cs.writable.getWriter();
  writer.write(in);
  writer.close();
  const out = [];
  const reader = cs.readable.getReader();
  let totalSize = 0;
  while (true) {
    const { value, done } = await reader.read();
    if (done)
      break;
    out.push(value);
    totalSize += value.byteLength;
  }
  const concatenated = new Uint8Array(totalSize);
  let offset = 0;
  for (const array of out) {
    concatenated.set(array, offset);
    offset += array.byteLength;
  }
  return concatenated;
}
</pre>

## Gzip-decompress a Blob to Blob ##  {#example-gzip-decompress}

<pre class="example" highlight="js">
function decompressBlob(blob) {
  const ds = new DecompressionStream('gzip');
  const decompressionStream = blob.stream().pipeThrough(ds);
  return new Response(decompressedStream).blob();
}
</pre>

# Acknowledgments #  {#acknowledgments}
The editors wish to thank Domenic Denicola and Yutaka Hirano, for their support.
