Module De.DefSource

Sourcetype dst = [
  1. | `Channel of out_channel
  2. | `Buffer of Buffer.t
  3. | `Manual
]

The type for output destinations. With a `Manual destination the client must provide output storage with dst. With `String or `Channel destination the client can safely discard `Flush case (with assert false).

Sourcetype dynamic

The type for DEFLATE DYNAMIC block.

Sourcetype kind =
  1. | Flat
    (*

    A Flat len block is a non-compressed block of len byte(s). After a flat block, output is aligned on bytes.

    *)
  2. | Fixed
    (*

    A Fixed block is a compressed block by a precomputed Huffman tree. Any symbols can be encoded with this kind of block - encode should never return `Block with it.

    *)
  3. | Dynamic of dynamic
    (*

    A Dynamic h block is a compressed block by an Huffman tree represented by h. It allows to encode a subset of symbols (or any symbols).

    *)

The type for DEFLATE header block.

Sourcetype block = {
  1. kind : kind;
  2. last : bool;
}

The type for DEFLATE block.

Sourcetype encode = [
  1. | `Await
  2. | `Flush
  3. | `Block of block
]

The type for user action into encoder:

  • `Await is expected until encode returns `Ok.
  • `Flush asks to encoder to produce output as long as it can.
  • `Block block asks to encoder to produce a new block block - if current block is last block (see block), encode raises an Invalid_argument.
Sourceval dynamic_of_frequencies : literals:literals -> distances:distances -> dynamic

dynamic_of_frequencies ~literals ~distances is a DEFLATE DYNAMIC block header computed from given frequencies. According frequencies, dynamic_of_frequencies makes a Huffman tree which provides smaller representation for symbols which frequency is upper than 0 (others symbols are not a part of resulted Huffman tree). At the end, a dynamic Huffman tree is able to encode a subset of symbols.

If all frequencies are upper than 0, resulted dynamic Huffman tree is able to encode any symbols.

Sourcetype encoder

The type for DEFLATE encoders.

Sourceval encoder : dst -> q:Queue.t -> encoder

encoder dst ~q is an encoder that outputs to dst.

Internal queue.

encoder needs a side-channel about compressed inputs. To pass compression values to encoder, we use a queue q. Length of it can be a bottleneck where a small one will let encode to emit too many `Flush (which is commonly associated to a syscall). We recommend a queue as large as output buffer.

Sourceval encode : encoder -> encode -> [ `Ok | `Partial | `Block ]

encode e v is:

  • `Partial iff e has a `Manual destination and needs more output storage. The client must use dst to provide a new buffer and then call encode with `Await until `Ok is returned.
  • `Ok when the encoder is ready to encode a new encode action.
  • `Block when the encoder reachs a Queue.cmd which can not be encoded with the current block. The client must respond with `Block block where block is a new block able to encode current Queue.cmd.

How to signal end of flow?

End of flow is characterized by a block where last = true. Then, the client must emit into the queue q Queue.eob.

Limitation.

The encoder must manipulate an output buffer of, at least, 2 bytes. If it's not the case, encode does nothing - and it tells you nothing more than it did nothing. Depending on what you do, a loop can infinitely call encode without any updates until the given output still has less than 2 bytes.

Sourceval dst : encoder -> bigstring -> int -> int -> unit

dst e s j l provides e with l bytes available to write, starting at j in s. This byte range is read by calls to encode with e until `Flush is returned.

Sourceval dst_rem : encoder -> int

dst_rem e is how many bytes it remains in given output buffer.

Sourceval bits_rem : encoder -> int

bits_rem e is how many bits it remains in given output buffer. A DEFLATE flow is not necessary aligned on bytes. The client can call bits_rem only when he reachs `End case. Otherwise, we raises an Invalid_argument.

Sourcemodule Ns : sig ... end